IntroductionIn forecasts combination one of the popular solutions is based on the application of some information criterion. Taking for example Akaike criterion $AIC_j$ estimated for the model $j$, one could compute the differences of $AIC_j$ from $AIC^* = \min_j{AIC_j}$ and then $RP_j = e^{(AIC^*-AIC_j)/2}$ could be interpreted as the relative probability of model $j$ to be the true one. The weights then are defined as

$$w_j = \frac{RP_j}{\sum_j RP_j}$$

ProblemA difficulty that I try to overcome is that the models are estimated on the differently transformed response (endogenous) variables. For example, some models are based on annual growth rates, another – on quarter-to-quarter growth rates. Thus the extracted $AIC_j$ values are not directly comparable.

Tried solutionSince all that matters is the difference of $AIC$s one could take the base model’s $AIC$ (for example I tried to extract

`lm(y~-1)`

the model without any parameters) that is invariant to the response variable transformations and then compare the differences between the $j$th model and the base model $AIC$. Here however it seems the weak point remains – the differenceisaffected by the transformation of the response variable.

Concluding remarksNote, the option like “estimate all the models on the same response variables” is possible, but very time consuming. I would like to search for the quick “cure” before going to the painful decision if there is no other way to resolve the problem.

**Answer**

I think one of the most reliable methods for comparing models is to cross-validate out-of-sample error (e.g. MAE). You will need to un-transform the exogenous variable for each model to directly compare apples to apples.

**Attribution***Source : Link , Question Author : Dmitrij Celov , Answer Author : Zach*