How to test an interaction effect with a non-parametric test (e.g. a permutation test)?

I have two categorical / nominal variables. Each of them can take only two distinct values (so, I have 4 combinations in total).

Each combination of values comes with a set of numerical values. So, I have 4 sets of numbers. To make it more concrete, let us say that I have male / female and young / old as the nominal variables and I have weight as the dependent numerical “output”.

I know that transition from male to female does change the average weight and these changes are statistically significant. So, I can calculate a gender factor. The same is applicable to the age variable. I do know that transition from young to old does change the average weight and I can calculate the corresponding age factor.

Now, what I really want to see if the data proves that transition from young-females to old-males is more that combination of gender- and age-factors. In other words, I want to know if data prove that there are “2D effects” or, in other words, that age- and gender-effects are not independent. For example, it might be that becoming old for males increase the weight by factor 1.3 and for female the corresponding factor is 1.1.

Of course I can calculate the two mentioned factors (age factor for males and age factor for females) and they are different. But I want to calculate the statistical significance of this difference. How real is this difference.

I would like to do a non-parametric test, if possible. Is it possible to do what I want to do by mixing the four sets, shuffling them, re-splitting and calculating something.

Answer

There are nonparametric tests for interaction. Roughly speaking, you replace the observed weights by their ranks and treat the resulting data set as heteroskedastic ANOVA. Look e.g. at “Nonparametric methods in factorial designs” by Brunner and Puri (2001).

However, the kind of nonparametric interaction you are interested in cannot be shown in this generality. You said:

In other words, I want to know if data prove that there are “2D effects” or, in other words, that age- and gender-effects are not independent. For example, it might be that becoming old for males increase the weight by factor 1.3 and for female the corresponding factor is 1.1.

The latter is impossible. Nonparametric interaction must involve a sign change, i.e. growing old increases males’ weight but decreases females weight. Such a sign change remains even if you monotonically transform the weights.
But you can choose a monotonous transformation on the data that maps the weight increase by factor 1.1 as close as you want to 1.3. Of course, you will never show a difference to be significant if it can be as close as you want.

If you really are interested in interactions without sign change, you should stick to usual parametric analysis. There, monotonous transformations that “swallow the difference” aren’t allowed. Of course, this is again something to keep in mind by modeling and interpreting your statistics.

Attribution
Source : Link , Question Author : Roman , Answer Author : Horst Grünbusch

Leave a Comment