Assume X and Y have finite second moment. In the Hilbert space of random variables with second finite moment (with inner product of T1,T2 defined by E(T1T2), ||T||2=E(T2)), we may interpret E(Y|X) as the projection of Y onto the space of functions of X.

We also know that Law of Total Variance reads

Var(Y)=E(Var(Y|X))+Var(E(Y|X))Is there a way to interpret this law in terms of the geometric picture above? I have been told that the law is the same as Pythagorean Theorem for the right-angled triangle with sides Y,E(Y|X),Y−E(Y|X). I understand why the triangle is right-angled, but not how the Pythagorean Theorem is capturing the Law of Total Variance.

**Answer**

I assume that you are comfortable with regarding the right-angled triangle as meaning that E[Y∣X] and Y−E[Y∣X] are *uncorrelated* random variables.

For uncorrelated random variables A and B,

var(A+B)=var(A)+var(B),

and so if we set A=Y−E[Y∣X] and B=E[Y∣X] so that A+B=Y, we get

that

var(Y)=var(Y−E[Y∣X])+var(E[Y∣X]).

It remains to show that var(Y−E[Y∣X]) is the same as

E[var(Y∣X)] so that we can re-state (2) as

var(Y)=E[var(Y∣X)]+var(E[Y∣X])

which is the total variance formula.

It is well-known that the expected value of the random variable E[Y∣X] isE[Y],

that is, E[E[Y∣X]]=E[Y]. So we see that

E[A]=E[Y−E[Y∣X]]=E[Y]−E[E[Y∣X]]=0,

from which it follows that var(A)=E[A2], that is,

var(Y−E[Y∣X])=E[(Y−E[Y∣X])2].

Let C denote the random variable (Y−E[Y∣X])2 so that we can

write that var(Y−E[Y∣X])=E[C].

But,

E[C]=E[E[C∣X]] where

E[C∣X]=E[(Y−E[Y∣X])2|X].

Now, *given* that X=x, the conditional distribution of Y has mean E[Y∣X=x]

and so

E[(Y−E[Y∣X=x])2|X=x]=var(Y∣X=x).

In other words, E[C∣X=x]=var(Y∣X=x) so that

the *random variable* E[C∣X] is just var(Y∣X).

Hence,

E[C]=E[E[C∣X]]=E[var(Y∣X)],

which upon substitution into (5) shows that

var(Y−E[Y∣X])=E[var(Y∣X)].

This makes the right side of (2) exactly what we need and so we have proved

the total variance formula (3).

**Attribution***Source : Link , Question Author : renrenthehamster , Answer Author : Dilip Sarwate*