# Law of total variance as Pythagorean theorem

Assume $X$ and $Y$ have finite second moment. In the Hilbert space of random variables with second finite moment (with inner product of $T_1,T_2$ defined by $E(T_1T_2)$, $||T||^2=E(T^2)$), we may interpret $E(Y|X)$ as the projection of $Y$ onto the space of functions of $X$.

We also know that Law of Total Variance reads

Is there a way to interpret this law in terms of the geometric picture above? I have been told that the law is the same as Pythagorean Theorem for the right-angled triangle with sides $Y, E(Y|X), Y-E(Y|X)$. I understand why the triangle is right-angled, but not how the Pythagorean Theorem is capturing the Law of Total Variance.

I assume that you are comfortable with regarding the right-angled triangle as meaning that $E[Y\mid X]$ and $Y - E[Y\mid X]$ are uncorrelated random variables.
For uncorrelated random variables $A$ and $B$,

and so if we set $A = Y - E[Y\mid X]$ and $B = E[Y\mid X]$ so that $A+B = Y$, we get
that

It remains to show that $\operatorname{var}(Y-E[Y\mid X])$ is the same as
$E[\operatorname{var}(Y\mid X)]$ so that we can re-state $(2)$ as

which is the total variance formula.

It is well-known that the expected value of the random variable $E[Y\mid X]$ is$E[Y]$,
that is, $E\biggr[E[Y\mid X]\biggr] = E[Y]$. So we see that

from which it follows that $\operatorname{var}(A) = E[A^2]$, that is,

Let $C$ denote the random variable $(Y-E[Y\mid X])^2$ so that we can
write that
But,
$E[C] = E\biggr[E[C\mid X]\biggr]$ where
$E[C\mid X] = E\biggr[(Y-E[Y\mid X])^2{\bigr\vert} X\biggr].$
Now, given that $X = x$, the conditional distribution of $Y$ has mean $E[Y\mid X=x]$
and so

In other words, $E[C\mid X = x] = \operatorname{var}(Y\mid X = x)$ so that
the random variable $E[C\mid X]$ is just $\operatorname{var}(Y\mid X)$.
Hence,

which upon substitution into $(5)$ shows that

This makes the right side of $(2)$ exactly what we need and so we have proved
the total variance formula $(3)$.