What is the difference between regular PCA and probabilistic PCA?

I know regular PCA does not follow probabilistic model for observed data. So what is the basic difference between PCA and PPCA? In PPCA latent variable model contains for example observed variables y, latent (unobserved variables x) and a matrix W that does not has to be orthonormal as in regular PCA. One more difference that I can think of regular PCA only provide principal components, where PPCA provides the probabilistic distribution of the data.

Could someone please through more light on the differences between PCA and PPCA?


The goal of PPCA is not to give better results than PCA, but to permit a broad range of future extensions and analysis. The paper states some of the advantages clearly in the introduction, ie/eg:

“the definition of a likelihood measure enables a comparison with other probabilistic techniques, while facilitating statistical testing and permitting the application of Bayesian models”.

Bayesian models in particular are enjoying a huge renaissance lately, eg VAE, “auto-encoding variational Bayes”, https://arxiv.org/abs/1312.6114 . Extension of PCA to be usable in variational frameworks and similar has the potential for another researcher to say ‘Oh hey, what if I do … ?’

Source : Link , Question Author : Vendetta , Answer Author : Hugh Perkins

Leave a Comment