The Real Reason PCA Works: Variance as Signal

Students memorize PCA as “dimensionality reduction.”
But the deeper insight is: PCA assumes variance = information.

If a direction in the data has high variance, PCA considers it meaningful.
If variance is small, PCA considers it noise.

This is not always true in real systems.

PCA fails when:
➖important signals have low variance
➖noise has high variance
➖relationships are nonlinear

That’s why modern methods (autoencoders, UMAP, t-SNE) outperform PCA on many datasets.