Intuition behind Box-Cox transform

For features that are heavily skewed, the Transformation technique is useful to stabilize variance, make the data more normal distribution-like, improve the validity of measures of association.

I am really having trouble understanding the intuition behind Box-Cox transform. I mean how to configure data transform method for both square root and log transform and estimating lambda.

Could anyone explain in simple words (and maybe with an example) what is the Intuition behind Box-Cox transform

Answer

The design goals of the family of Box-Cox transformations of non-negative data were these:

  1. The formulas should be simple, straightforward, well understood, and easy to calculate.

  2. They should not change the middle of the data much, but affect the tails more.

  3. The family should be rich enough to induce large changes in the skewness of the data if necessary: this means it should be able to contract or extend one tail of the data while extending or contracting the other, by arbitrary amounts.

Let’s consider the implications of each in turn.

1. Simplicity

Linear transformations–those of the form xαx+β for constants α and β–merely change the scale and location of data; they cannot change the shape of their distribution. The next simplest formula is to consider power transformations, of the form xxλ for (nonzero) constant λ.

2. Stability

A power transformation enjoys the nice property that rescaling the data results in rescaling their powers. That is, multiplying the data x by some positive scale factor α results in multiplying xλ by αλ. OK, it’s not the same scale factor, but it is still just a rescaling.

In light of this, let’s always standardize any batch of data (x1,x2,,xn) by rescaling it to place its center (perhaps its median) at 1. Specifically, this replaces each xi by xi divided by the middle value of all the x‘s. This won’t change the shape of the data distribution–it really amounts to choosing a suitable unit of measurement for expressing the values. For those who like formulas, let μ be the median of the batch. We will be studying the transformations

x(x/μ)λ1λ=μλλxλ+1λ=αxλ+β

for various λ. The effects of α and β (which depend on λ and μ) on xλ do not change the shape of the distribution of the xλi. In this sense, the Box-Cox transformations of the standardized data really are just the power transformations.

Because we have made 1 the central value of the batch, design crition 2–“stability”–requires that different values of the power λ have relatively little effect on values near 1.

Let’s look at this in a little more detail by examining what a power does to numbers near 1. According to the Binomial Theorem, if we write x as x=1+ϵ (for fairly small ϵ), then approximately

(1+ϵ)λ=1+λϵ+Something×ϵ2.

Ignoring ϵ2 as being truly tiny, this tells us that

Taking a power λ of a number x near 1 is a nearly linear function that changes the distance between x and 1 by a factor λ.

In light of this, we can match the effects of different possible λ by means of a compensating division of the distance by λ. That is, we will use

BCλ(x)=xλ1λλ=xλ1λ.

The numerator is the (signed) distance between the power transform of x and the power transform of the middle of the data (1); the denominator adjusts for the expansion of x1 by the factor λ when taking the power. BCλ is the Box-Cox transformation with parameter λ.

By means of this construction, we guarantee that when x is close to a typical value of its batch of data, BCλ(x) will approximately be the same value (and close to zero) no matter what λ might be (within reason, of course: extreme values of λ can do extreme things).

3. Flexibility

We have many possible values of λ to choose from. How do they differ?

This can be explored by graphing the Box-Cox transformations for various λ. Here is a set of graphs for λ{1,1/2,0,1/2,1,2}. (For the meaning of λ=0, see Natural Log Approximation elsewhere on this site.)

Figure 1

The solid black line graphs the Box-Cox transformation for λ=1, which is just xx1. It merely shifts the center of the batch to 0 (as do all the Box-Cox transformations). The upward curving pink graph is for λ=2. The downward curving graphs show, in order of increasing curvature, the smaller values of λ down to 1.

The differing amounts and directions of curvature provide the desired flexibility to change the shape of a batch of data.

For instance, the upward curving graph for λ=2 exemplifies the effect of all Box-Cox transformations with λ exceeding 1: values of x above 1 (that is, greater than the middle of the batch, and therefore out in its upper tail) are pulled further and further away from the new middle (at 0). Values of x below 1 (less than the middle of the batch, and therefore out in its lower tail) are pushed closer to the new middle. This “skews” the data to the right, or high values (rather strongly, even for λ=2).

The downward curving graphs, for λ<1, have the opposite effect: they push the higher values in the batch towards the new middle and pull the lower values away from the new middle. This skews the data to the left (or lower values).

The coincidence of all the graphs near the point (1,0) is a result of the previous standardizations: it constitutes visual verification that choice of λ makes little difference for values near the middle of the batch.

Finally, let’s look at what different Box-Cox transformations do to a small batch of data.

Figure 2

Transformed values are indicated by the horizontal positions. (The original data look just like the black dots, shown at λ=1, but are located +1 units to the right.) The colors correspond to the ones used in the first figure. The underlying gray lines show what happens to the transformed values when λ is smoothly varied from 1 to +2. It’s another way of appreciating the effects of these transformations in the tails of the data. (It also shows why the value of λ=0 makes sense: it corresponds to taking values of λ arbitrarily close to 0.)

Attribution
Source : Link , Question Author : Pluviophile , Answer Author : whuber

Leave a Comment