Converting standardized betas back to original variables

I realise this is probably a very simple question but after searching I can’t find the answer I am looking for.

I have a problem where I need to standardize the variables run the (ridge regression) to calculate the ridge estimates of the betas.

I then need to convert these back to the original variables scale.

But how do I do this?

I found a formula for the bivariate case that

β=ˆβSxSy.

This was given in D. Gujarati, Basic Econometrics, page 175, formula (6.3.8).

Where β are the estimators from the regression run on the standardized variables and ˆβ is the same estimator converted back to the original scale, Sy is the sample standard deviation of the regressand, and Sx is the sample standard deviation.

Unfortunately the book doesn’t cover the analogous result for multiple regression.

Also I’m not sure I understand the bivariate case? Simple algebraic manipulation gives the formula for ˆβ in the original scale:

ˆβ=βSySx

It seems odd to me that the ˆβ that were calculated on variables which are already deflated by Sx, has to be deflated by Sx again to be converted back? (Plus why are the mean values not added back in?)

So, can someone please explain how to do this for a multivariate case ideally with a derivation so that I can understand the result?

Answer

For the regression model using the standardized variables, we assume the following form for the regression line

E[Y]=β0+kj=1βjzj,

where zj is the j-th (standardized) regressor, generated from xj by subtracting the sample mean ˉxj and dividing by the sample standard deviation Sj:
zj=xjˉxjSj

Carrying out the regression with the standardized regressors, we obtain the fitted regression line:

ˆY=ˆβ0+kj=1ˆβjzj

We now wish to find the regression coefficients for the non-standardized predictors. We have

ˆY=ˆβ0+kj=1ˆβj(xjˉxjSj)

Re-arranging, this expression can be written as

ˆY=(ˆβ0kj=1ˆβjˉxjSj)+kj=1(ˆβjSj)xj

As we can see, the intercept for the regression using the non-transformed variables is given by ˆβ0kj=1ˆβjˉxjSj. The regression coefficient of the j-th predictor is ˆβjSj.

In the presented case, I have assumed that only the predictors had been standardized. If one also standardizes the response variable, transforming the covariate coefficients back to the original scale is done by using the formula from the reference you gave. We have:

E[Y]ˆySy=β0+kj=1βjzj

Carrying out the regression, we get the fitted regression equation

ˆYscaled=ˆYunscaledˉySy=ˆβ0+kj=1ˆβj(xjˉxjSj),

where the fitted values are on the scale of the standardized response. To unscale them and recover the coefficient estimates for the untransformed model, we multiply the equation by Sy and bring the sample mean of the y to the other side:

ˆYunscaled=ˆβ0Sy+ˉy+kj=1ˆβj(SySj)(xjˉxj).

The intercept corresponding to the model in which neither the response nor the predictors have been standardized is consequently given by
ˆβ0Sy+ˉykj=1ˆβjSySjˉxj,
while the covariate coefficients for the model of interest can be obtained by multiplying each coefficient with Sy/Sj.

Attribution
Source : Link , Question Author : Baz , Answer Author : Philipp Burckhardt

Leave a Comment