Why is the eigenvector in PCA taken to be unit norm?

In deriving the eigenvectors for PCA, the vector is subject to the condition that it should be of unit length. Why is this so?

Answer

The main aim of Principal Component Analysis (PCA) is to look for the directions on Rp that maximize the variance of the projected random vector X=(X1,,Xp). Specifically, the first PC can be defined as the unit vector v(1)Rp such that
v(1)=argmax

If you allow vectors that are not of unit norm in the maximization problem, then you will not get a proper solution, since variance of the projection can become arbitrarily large as long as the norm of the vector increases. For example, if w=\lambda v, with v,w\in\mathbb{R}^p and \lambda\to\infty, then

\mathbb{V}\mathrm{ar}\big[w^TX\big]=\lambda^2\mathbb{V}\mathrm{ar}\big[v^TX\big]\to\infty\quad (\text{if }\mathbb{V}\mathrm{ar}\big[v^TX\big]\neq0).
This is the reason why you need an standardization of unit norm to constraint the search and avoid improper solutions.

Attribution
Source : Link , Question Author : Suhas Lohit , Answer Author : epsilone

Leave a Comment