I just need a simple explanation of what exactly ridge regression is so I can have a decent intuitive understanding of it. I understand it’s about applying some sort of penalty to the regression coefficients… but beyond that I’m a little confused about how it is different from other kinds of regression which implement penalties. In what case should you use ridge regression as opposed to some other kind of regression?
Ridge Regression is a remedial measure taken to alleviate multicollinearity amongst regression predictor variables in a model. Often predictor variables used in a regression are highly correlated. When they are, the regression coefficient of any one variable depend on which other predictor variables are included in the model, and which ones are left out. (So the predictor variable does not reflect any inherent effect of that particular predictor on the response variable, but only a marginal or partial effect, given whatever other correlated predictor variables are included in the model). Ridge regression adds a small bias factor to the variables in order to alleviate this problem. Hope that helps.