Principal Component Analysis (PCA)

Spectral Decomposition & Variance Maximization

Principal Components $\mathbf{v}_1$ (Red) and $\mathbf{v}_2$ (Blue) overlayed on centered data.

Eigenvalue Spectrum: For centered data $\mathbf{X}$, we decompose the covariance matrix $\mathbf{\Sigma}$: $$ \mathbf{\Sigma} \mathbf{v}_i = \lambda_i \mathbf{v}_i $$ $\lambda_i$ represents the variance explained by the $i$-th component.

Scree Plot (Explained Variance)

0%
PC1
0%
PC2