Self-supervised Pretraining of Visual Features in the Wild

Facebook also published its ultimate SElf-supERvised (SEER) model.

- They pretrained it on a 1B random, unlabeled and uncurated Instagram images 👀.
- SEER outperformed SOTA self-supervised systems, reaching 84.2% top-1 accuracy on ImageNet.
- SEER also outperformed SOTA supervised models on downstream tasks, including low-shot, object detection, segmentation, and image classification.
- When trained with just 10% of the ImageNet, SEER still achieved 77.9% top-1 accuracy on the full data set. When trained with just 1% of the annotated ImageNet examples, SEER achieved 60.5% top-1 accuracy.
- SEER is based on recent RegNet achitecture . Under comparable training settings and flops, the RegNet models outperform the popular EfficientNet l models while being up to 5x faster on GPUs.

📝 Paper
📖 Blogpost
⚙️ I guess the source code will be published as a part of VISSL soon.