Gamma distribution different derivations

According to this link –
the waiting time of the kth event in a poisson process is gamma distributed. Also, we know that if we square n normal random variables, we get a gamma rv (a particular case, the chi-squared). Apparently, two unrelated paths have led us to the same distribution.

But are they really unrelated? Is there a way to connect them (is the waiting time for the kth event in a poisson process the sum of squares of n normals in some intuitive way)? Should one even expect to find such connections when the same distribution is encountered through unrelated paths?


Well, for one thing, one cannot derive a gamma distribution by assuming a discrete non-negative integer variable and then saying that now, all of a sudden let us remove the concept of whole counting numbers of events and allow all reals. For example, there is no such thing as a transcendental event like rolling a six-sided die and getting π rather than a whole number from 1 to 6. In specific, one cannot say without fear of contradiction that a gamma function is a generalized factorial, because that puts the “cart before the horse.” However, one can say that a factorial is a special case of a gamma function, that discards negative reals, and only considers non-negative integers. So the thought experiment error is like saying that a gamma distribution is a generalized exponential distribution, where it is more accurate to say that an exponential distribution is a degenerate case of a gamma distribution.

In order to make this clear consider that we would not call y=bx+c a quadratic equation. It is a linear equation. Sure, it is a degenerate or trivialized case of a quadratic, i.e., y=ax2+bx+c for a=0. What a linear equation lacks is a square term, i.e., a linear equation does not have the same form as a quadratic because it lacks a quadratic term. However, linear equations also lack a cubic term, an xπ term and, any other nonlinear function one cares to consider, such that calling a linear equation a quadratic is only true in one out of a transfinite number of other possible imaginary circumstances. So too an exponential distribution can arise not only as a degenerate or trivialized gamma distribution, (as x0=1) but in myriad other cases as well. So one has to say that a gamma distribution is one of a very large number of generalizations of an exponential distribution, e.g., see generalized exponential distribution.

What is the danger in that you ask? Well for one thing, sloppy thinking leads to mistakes, and there are many of them, including the “derivation” alluded to above, that a gamma distribution arises from a Poisson distribution. Let me be clear, that is not the case. Rather, the Poisson distribution is a special case of a gamma distribution. Another example arises when examining tail heaviness. The correct method for comparing tail heaviness is to contrast so-called “survival” functions (more accurately called complementary cumulative distribution functions, ccdf’s, 1F(x), when continuous) and the correct procedure for this is given as examples on this site at Which has the heavier tail, lognormal or gamma?. Now when our unchecked inductive urge is let run free, mistakes are made, for example, by stating that various tail heaviness can be classified into groups, where one of the groups is functions of “exponential tail heaviness”. The mistake (applying L’Hôpital’s rule to non-indeterminate forms) is replicated throughout the literature, and is documented as a mistake in the Relative Tail Heaviness Appendix section of an article, which relates that “For continuous functions, pdf binary comparison through survival function ratios avoids false attribution, for example, classifying the GD (sic, gamma distribution) as having an ED (sic, exponential distribution) terminal tail, whereas in fact, the exponential has tail heaviness that is within the GD tail heaviness range…”, and “…Even when two functions are in the same category of tail heaviness their range of heavinesses might not overlap, which may seem counterintuitive, but implies once again that only binary tail heaviness comparisons make sense.”

Now finally, we have said that a gamma distribution does not arise from a Poisson distribution, so then, from what does a gamma distribution arise? One thing should be clear, it does not arise from an Erlang distribution despite incorrect claims to that effect, e.g., see this mistake, which is yet another attempt to use uncritical inductive thinking. Rather, it may arise from a gamma Lévy process, where the difference in approach is that a Lévy process begins with a real number variable treatment (i.e., not integer), and where no claim is made that it has to arise only in that context.

The next question relates to the Chi-Squared distribution, and again the treatment in the OP’s link is deficient. Chi-squared implementations, unlike the text of the OP’s link, have ν, degrees of freedom, to be any positive real number. Thus, there is an actual relationship between the gamma distribution and the Chi-squared distribution for continuous variables. Thus, the understanding that v corresponds to a whole number of normal distributions is not actually a derivation of Chi-squared, just an example special case application of it. That is, in the OP’s link:

” Let now consider the special case of the gamma distribution that plays an important role in statistics. Let X have a gamma distribution with θ=2 and α=r/2, where r is a positive integer. “

If we drop the unnecessary r\in\mathbb{Z+}, and require instead that r\in\mathbb{R+}, then the substitutions above yield a more general than integer df derivation of Chi-squared. However, the obverse is not true, that is, Chi-squared does not provide a derivation of a gamma distribution because the number of parameters has decreased by 1 from setting θ=2, and generalizing Chi-squared does not imply only a gamma distribution but other things as well, for example, see Generalized Chi-squared distribution .

The final question is should we expect distributions to be related through alternative paths, the answer to which is yes, we should. For example, see this answer which shows how general interrelationships are between distributions and how, in specific, gamma distributions relate to other distributions.

Source : Link , Question Author : ryu576 , Answer Author : Carl

Leave a Comment