How can I represent R squared in matrix form?

This question is a follow-up to a prior question.


Basically, I wanted to study under what conditions when we regress the residuals to x1, we will get R2 of 20%.

As a first step to attack this problem, my question is, how do I express R2 in matrix form?

Then I will try to express “R2 of regressing residuals to x1” using matrix form.

Also, how can I add regression weights into the expression?

Answer

We have
R2=1e2i(yiˉy)2=1ee˜y˜y,
where ˜y is a vector y demeaned.

Recall that ˆβ=(XX)1Xy, implying that e=yXˆβ=yX(XX)1Xy. Regression on a vector of 1s, written as l, gives the mean of y as the predicted value and residuals from that model produce demeaned y values; ˜y=yˉy=yl(ll)1ly.

Let H=X(XX)1X and let M=l(ll)1l, where l is a vector of 1’s. Also, let I be an identity matrix of the requisite size. Then we have

R2=1ee˜y˜y=1y(IH)(IH)yy(IM)(IM)y=1y(IH)yy(IM)y,

where the second line comes from the fact that H and M (and I) are idempotent.

In the weighted case, let Ω be the weighting matrix used in the OLS objective function, eΩe. Additionally, let Hw=XΩ1/2(XΩX)1Ω1/2X and Mw=lΩ1/2(lΩl)1Ω1/2l. Then,
R2=1yΩ1/2(IHw)Ω1/2yyΩ1/2(IMw)Ω1/2y,

Attribution
Source : Link , Question Author : Luna , Answer Author : Charlie

Leave a Comment