How to find variance between multidimensional points?

Suppose I have a matrix X which is n by p, i.e. it has n observations, with each observation in p-dimensional space.

How do I find the variance of these n observations?

In the case where p = 1, I just need to use the regular variance formula. What about the cases where p > 1.


For a p-dimensional random variable X=(X1,,Xp), we have the following definition of the variance:

Var\left( X \right) = E\left[ {\left( {X – EX} \right){{\left( {X – EX} \right)}^\intercal}} \right] = \left( {\begin{array}{*{20}{c}}
{Var\left( {{X_1}} \right)}& \ldots &{Cov\left( {{X_1},{X_p}} \right)} \\
\vdots & \ddots & \vdots \\
{Cov\left( {{X_p},{X_1}} \right)}& \ldots &{Var\left( {{X_p}} \right)}
\end{array}} \right)

That is, the variance of a random vector is defined as the matrix which stores all the variances on the main diagonal and the covariances between the different components in the other elements. The sample p \times p covariance matrix would then be calculated by plugging in the sample analogs for the population variables:

\frac{1}{{n – 1}}\left( {\begin{array}{*{20}{c}}
{\sum\limits_{i = 1}^n {{{\left( {{X_{i1}} – {{\bar X}_{\cdot1}}} \right)}^2}} }& \ldots &{\sum\limits_{i = 1}^n {\left( {{X_{i1}} – {{\bar X}_{\cdot1}}} \right)\left( {{X_{ip}} – {{\bar X}_{\cdot p}}} \right)} } \\
\vdots & \ddots & \vdots \\
{\sum\limits_{i = 1}^n {\left( {{X_{ip}} – {{\bar X}_{\cdot p}}} \right)\left( {{X_{i1}} – {{\bar X}_{\cdot 1}}} \right)} }& \ldots &{\sum\limits_{i = 1}^n {{{\left( {{X_{ip}} – {{\bar X}_{\cdot p}}} \right)}^2}} }
\end{array}} \right)

where {X_{ij}} denotes the ith observation for feature j and {{\bar X}_{ \cdot j}} the sample mean of the jth feature. To sum up, the variance of a random vector is defined as the matrix containing the individual variances and covariances. It therefore suffices to calculate the sample variances and covariances for all the vector components individually.

Source : Link , Question Author : statnub , Answer Author : Philipp Burckhardt

Leave a Comment