Law of total variance as Pythagorean theorem

Assume X and Y have finite second moment. In the Hilbert space of random variables with second finite moment (with inner product of T1,T2 defined by E(T1T2), ||T||2=E(T2)), we may interpret E(Y|X) as the projection of Y onto the space of functions of X.

We also know that Law of Total Variance reads
Var(Y)=E(Var(Y|X))+Var(E(Y|X))

Is there a way to interpret this law in terms of the geometric picture above? I have been told that the law is the same as Pythagorean Theorem for the right-angled triangle with sides Y,E(Y|X),YE(Y|X). I understand why the triangle is right-angled, but not how the Pythagorean Theorem is capturing the Law of Total Variance.

Answer

I assume that you are comfortable with regarding the right-angled triangle as meaning that E[YX] and YE[YX] are uncorrelated random variables.
For uncorrelated random variables A and B,
var(A+B)=var(A)+var(B),
and so if we set A=YE[YX] and B=E[YX] so that A+B=Y, we get
that
var(Y)=var(YE[YX])+var(E[YX]).
It remains to show that var(YE[YX]) is the same as
E[var(YX)] so that we can re-state (2) as
var(Y)=E[var(YX)]+var(E[YX])
which is the total variance formula.

It is well-known that the expected value of the random variable E[YX] isE[Y],
that is, E[E[YX]]=E[Y]. So we see that
E[A]=E[YE[YX]]=E[Y]E[E[YX]]=0,
from which it follows that var(A)=E[A2], that is,
var(YE[YX])=E[(YE[YX])2].
Let C denote the random variable (YE[YX])2 so that we can
write that var(YE[YX])=E[C].
But,
E[C]=E[E[CX]] where
E[CX]=E[(YE[YX])2|X].
Now, given that X=x, the conditional distribution of Y has mean E[YX=x]
and so
E[(YE[YX=x])2|X=x]=var(YX=x).
In other words, E[CX=x]=var(YX=x) so that
the random variable E[CX] is just var(YX).
Hence,
E[C]=E[E[CX]]=E[var(YX)],
which upon substitution into (5) shows that
var(YE[YX])=E[var(YX)].
This makes the right side of (2) exactly what we need and so we have proved
the total variance formula (3).

Attribution
Source : Link , Question Author : renrenthehamster , Answer Author : Dilip Sarwate

Leave a Comment