Meaning of “reconstruction error” in PCA and LDA

I am implementing PCA, LDA, and Naive Bayes, for compression and classification respectively (implementing both an LDA for compression and classification).

I have the code written and everything works. What I need to know, for the report, is what the general definition of reconstruction error is.

I can find a lot of math, and uses of it in the literature… but what I really need is a bird’s eye view / plain word definition, so I can adapt it to the report.

Answer

For PCA what you do is that you project your data on a subset of your input space. Basically, everything holds on this image above: you project data on the subspace with maximum variance. When you reconstruct your data from the projection, you’ll get the red points, and the reconstruction error is the sum of the distances from blue to red points: it indeed corresponds to the error you’ve made by projecting your data on the green line. It can be generalized in any dimension of course!

enter image description here

As pointed out in the comments, it does not seem that simple for LDA and I can’t find a proper definition on the internet. Sorry.

Attribution
Source : Link , Question Author : Chris , Answer Author : Vince.Bdn

Leave a Comment