⚔️⚡️🤖MDS vs PCA or what is better to use when reducing data dimensionality
Multidimensional Scaling (MDS) and Principal Component Analysis (PCA) are two popular data analysis techniques that are widely used in statistics, machine learning, and data visualization. Both methods aim to compress the information contained in multidimensional data and present it in a form convenient for analysis. Despite similarities in their goals, MDS and PCA have significant differences in their approach and applicability.
Principal Component Analysis (PCA) is a statistical technique used to reduce the dimensionality of data. It looks for linear combinations of the original variables, called principal components, that explain the largest amount of variance in the data.
Benefits of PCA:
1. Eliminate multicollinearity: PCA can be used to eliminate multicollinearity inherent in a set of original variables. It allows you to combine related variables into principal components, eliminating information redundancy.
2. Computational speed: PCA is usually quite efficient in terms of computational costs.
3. Resilience to noise: PCA exhibits greater resilience to noise in the data. In PCA, the dominant components (principal components) explain most of the variance in the data, while the less significant components can represent noise. This allows PCA to better separate signals and noise, which is especially useful when analyzing data with a low signal-to-noise ratio.
Disadvantages of PCA:
1. Linear relationship: PCA is based on the assumption of a linear relationship between variables. In the case of a non-linear relationship, PCA may not detect an important data structure.
2. Loss of interpretability: after projecting the data onto the space of principal components, their interpretation can be difficult, since the new variables are linear combinations of the original variables.
3. Sensitivity to outliers: PCA can be sensitive to the presence of outliers in the data, as they can strongly influence the distribution of principal components.

Multidimensional scaling (MDS) is a data visualization technique that seeks to preserve the relative distances between features in the original data when projected into low-dimensional space.
Advantages of MDS:
1. Accounting for non-linear relationships: MDS does not require the assumption of a linear relationship between variables and can detect non-linear relationships in the data.
2. Preserve relative distances: MDS strives to preserve the relative distances between objects in the source data when rendered. This allows you to detect a data structure that may be lost in the process of dimensionality reduction.
3. Interpretability: MDS makes it relatively easy to interpret low-dimensional projections of data because they preserve the relative distances of the original data.
Disadvantages of MDS:
1. Computational Complexity: MDS can be computationally complex when dealing with large datasets, especially when accurate relative distances between all pairs of objects need to be maintained.
2. Dependency on the metric: MDS requires the definition of a distance metric between objects. Choosing the wrong metric can lead to skewed results.

Thus, PCA and MDS are quite effective data analysis tools. PCA is widely used to reduce data dimensionality and reveal structure in linearly dependent variables, while MDS provides the ability to preserve relative distances and detect non-linear relationships between objects. The choice between these methods depends on the specifics of the data and the goals of the analysis. Where dimensionality reduction and principal component detection is required, PCA may be the preferred choice, while MDS is recommended for visualizing and maintaining relative distances in data.