# Invariance of results when scaling explanatory variables in logistic regression, is there a proof?

There is a standard result for linear regression that the regression coefficients are given by

$$β=(XTX)−1XTy\mathbf{\beta}=(\mathbf{X^T X})^{-1}\mathbf{X^T y}$$

or

$$(XTX)β=XTy(\mathbf{X^T X})\mathbf{\beta}=\mathbf{X^T y} \tag{2}\label{eq2}$$

Scaling the explanatory variables does not affect the predictions. I have tried to show this algebraically as follows.

The response is related to the explanatory variables via the matrix equation
$$y=Xβ\mathbf{y}=\mathbf{X \beta} \tag{3}\label{eq3}$$

$$X\mathbf{X}$$ is an $$n×(p+1)n \times (p+1)$$ matrix of n observations on p explanatory variables. The first column of $$X\mathbf{X}$$ is a column of ones.

Scaling the explanatory variables with a $$(p+1)×(p+1)(p+1) \times (p+1)$$ diagonal matrix $$D\mathbf{D}$$, whose entries are the scaling factors
$$Xs=XD \mathbf{X^s} = \mathbf{XD} \tag{4}\label{eq4}$$

$$Xs\mathbf{X^s}$$ and $$βs\mathbf{\beta^s}$$ satisfy $$(2)\eqref{eq2}$$:

$$(DTXTXD)βs=DTXTy(\mathbf{D^TX^T XD})\mathbf{\beta^s} =\mathbf{D^TX^T y}$$

so

$$XTXDβs=XTy\mathbf{X^T XD}\mathbf{\beta^s} =\mathbf{X^T y}$$

$$⇒Dβs=(XTX)−1XTy=β\Rightarrow \mathbf{D \beta^s} = (\mathbf{X^T X)^{-1}}\mathbf{X^T y}=\mathbf{\beta}$$

$$⇒βs=D−1β\Rightarrow \mathbf{\beta^s}=\mathbf{D}^{-1}\mathbf{\beta} \tag{5}\label{eq5}$$

This means if an explanatory variable is scaled by $$did_i$$ then the regression coefficient $$βi\beta_i$$is scaled by $$1/di1/d_i$$ and the effect of the scaling cancels out, i.e.
considering predictions based on scaled values, and using $$(4),(5),(3)\eqref{eq4},\eqref{eq5},\eqref{eq3}$$

$$ys=Xsβs=XDD−1β=Xβ=y\mathbf{y^s}=\mathbf{X^s \beta^s} = \mathbf{X D D^{-1}\beta}=\mathbf{X \beta}=\mathbf{y}$$
as expected.

Now to the question.

For logistic regression without any regularization, it is suggested, by doing regressions with and without scaling the same effect is seen

fit <- glm(vs ~ mpg, data=mtcars, family=binomial)

print(fit)

Coefficients:
(Intercept)          mpg
-8.8331       0.4304

mtcars$$mpg <- mtcars$$mpg * 10

fit <- glm(vs ~ mpg, data=mtcars, family=binomial)

print(fit)

Coefficients:
(Intercept)          mpg
-8.83307      0.04304


When the variable mpg is scaled up by 10, the corresponding coefficient is scaled down by 10.

1. How could this scaling property be proved (or disproved ) algebraically for logistic regression?

I found a similar question relating to the effect on AUC when regularization is used.

1. Is there any point to scaling explanatory variables in logistic regression, in the absence of regularization?

$$ℓ(β|y)∝∏i(exp(x′iβ)1+exp(x′iβ))yi(11+exp(x′iβ))1−yi \ell(\beta|y) \propto \prod_i\left(\frac{\exp(x_i'\beta)}{1+\exp(x_i'\beta)}\right)^{y_i}\left(\frac{1}{1+\exp(x_i'\beta)}\right)^{1-y_i}$$