Deriving total (within class + between class) scatter matrix

I was fiddling with PCA and LDA methods and I am stuck at a point, I have a feeling that it is so simple that I can’t see it.

Within-class (SW) and between-class (SB) scatter matrices are defined as:

SW=Ci=1Nt=1(xitμi)(xitμi)T

SB=Ci=1N(μiμ)(μiμ)T

Total scatter matrix ST is given as:

ST=Ci=1Nt=1(xitμ)(xitμ)T=SW+SB

where C is number of classes and N is number of samples x are samples, μi is ith class mean, μ is overall mean.

While trying to derive ST I came up to a point where I had:

(xμi)(μiμ)T+(μiμ)(xμi)T

as a term. This needs to be zero, but why?


Indeed:

ST=Ci=1Nt=1(xitμ)(xitμ)T=Ci=1Nt=1(xitμi+μiμ)(xitμi+μiμ)T=SW+SB+Ci=1Nt=1[(xitμi)(μiμ)T+(μiμ)(xitμi)T]

Answer

If you assume

1NNt=1xit=μi

Then

Ci=1Nt=1(xitμi)(μiμ)T=Ci=1(Nt=1(xitμi))(μiμ)T=0

and formula holds. You deal with the second term in the similar way.

Attribution
Source : Link , Question Author : nimcap , Answer Author : mpiktas

Leave a Comment