#08 Summary. Masked Autoencoder is all you need for any modality.
#deeplearning #ml

⚡️ Briefly
To solve complicated tasks machine learning algorithm should understand data and extract good features from it. Usually training generalizing models requires a lot of annotated data. However it is expensive and in some cases impossible.

Masked Autoencoder technique allows to train model on unlabeled data and obtain surprisingly good feature representation for all common modalities.

🔎 Contents:
- Explanation of MAE approach
- Recipe for all domains
- Crazy experimental results for all types of data

👉 Summary [ link ]

Papers:
BERT : text
MAE : image
M3MAE : image + text
MAE that listen : audio spectrograms
VideoMAE : video

Looking forward to your comments and suggestions!

Next time - Mind blowing paper from Meta AI about speech reconstruction from noninvasive brain signals. 🔥🔥🔥