How should you express a negative binomial distribution in an exponential family form?

When X is X1,…Xn, how do you express the following negative binomial distribution in an exponential family form?


f(k;r,p)\equiv\text{Pr}(X=k)=\binom{k+r-1}{k}(1-p)^{r}p^k~~~~~\text{for}~k=0,1,2,\ldots

Answer

Warning: The negative binomial distribution has several alternative formulations for which the formulas below change.

A distribution f(x;\theta) belongs to an exponential family if it can be represented in the form:


f(x;\theta)=h(x)\exp\left[\eta(\theta)T(x)-A(\theta)\right]

The value \eta is called the canonical (natural) parameter of the family, T(x) is a sufficient statistic for p, A(\theta) is called the log-partition function (it’s a normalization factor and sometimes called log-normalizer or cumulant generating function), and h(x) is an arbitrary function called base measure or carrier measure which is 1 in many cases (e.g. exponential distribution, gamma distribution, Bernoulli distribution, …). If the negative binomial distribution with known parameter r (if r is unknown, the negative binomial family is not an exponential family) has the following distribution:


f(k;r,p)=\binom{k+r-1}{k}(1-p)^{r}p^k~~~~~\text{for}~k=0,1,2,\ldots

Then it can be rewritten in exponential form as:


\begin{align}
f(k;r,p) &=\binom{k+r-1}{k}\exp\left[\ln(p^{k}(1-p)^{r})\right] \\
&=\binom{k+r-1}{k}\exp\left[k\ln(p) + r\ln(1-p)\right] \\
\end{align}

So the parameter \theta of the distribution is p (i.e. \theta=p) and the natural parameter is \eta=\ln(p), the sufficient statistic for p is T(k)=k (i.e. T(k)=\sum X_{i}, the sample sum), the log-partition function is A(\eta)=-r\ln(1-p) and h(k)=\binom{k+r-1}{k}. See for example the Wikipedia page for a nice overview of the theory and many common distributions. Nice references can also be found here and here.

Attribution
Source : Link , Question Author : Python_R , Answer Author : Scortchi – Reinstate Monica

Leave a Comment