Size of PCA decorrelation matrix?

2017-09-09 08:39:15

I have a $P\times K$ matrix $\mathbf X$ with $K$ random vectors as columns (with the respective means subtracted from each entry). My goal is to decorrelate the columns of $\mathbf X$ via PCA to obtain the transformed matrix $\mathbf Y$ of the same size as $\mathbf X$.

I proceed as follows: I form the $P \times P$ covariance matrix $\mathbf C=\frac{1}{K}E\big[\mathbf X \mathbf X^\top\big]$. Let the eigendecomposition of the covariance matrix be $\mathbf \Sigma=\mathbf U\mathbf \Lambda \mathbf U^T$ where $\mathbf U$ contains the eigenvectors and $\mathbf \Lambda = \text{diag}\{\lambda_i,\cdots,\lambda_{P}\}$ is the eigenvalue matrix of $\mathbf

\Sigma$. The eigenvalues correspond to the variances. To decorrelate the columns of $\mathbf X$, we multiply this matrix on the left by $\mathbf

\Lambda^{-1/2} \mathbf U^T$:

$$\mathbf Y = \mathbf \Lambda^{-1/2} \mathbf E^T \mathbf X,$$

so that the transformed data $\mathbf Y$ will have an identity covariance matrix.

In the