If Mean Squared Error = Variance + Bias^2. Then How can the Mean Squared Error be lower than the Variance

I was reading the Introduction to Statistical Learning. Here it is shown that:- MSE formulae

In a later example, the train and test MSE are plotted. I wanted to know if both the bias^2 and variance are positive quantities then how can MSE be lower than the Variance.
enter image description here

Answer

That’s called overfitting. The apparent MSE on the training data is lower than the variance, but this was only achieved by making a model overly complicated so that it could follow random fluctuations art individual data points (“chasing noise”). Once you try to predict on new data MSE is much worse. I.e. the real MSE of predictions from the model is not lower than the variance.

Attribution
Source : Link , Question Author : Debabrot Bhuyan , Answer Author : Björn

Leave a Comment