For an arbitrary continuous random variable, say X, is its differential entropy always less than ∞? (It’s ok if it’s −∞.) If not, what’s the necessary and sufficient condition for it to be less than ∞?

**Answer**

I thought about this question some more and managed to find a counter-example, thanks also to the Piotr’s comments above. The answer to the first question is no – the differential entropy of a continuous random variable (RV) is **not** always less than ∞. For example, consider a continuous RV X whose pdf is

f(x)=log(2)xlog(x)2

for x>2.

It’s not hard to verify that its differential entropy is infinite. It grows quite slowly though (approx. logarithmically).

For the 2nd question, I am **not** aware of a simple necessary and sufficient condition. However, one partial answer is as follows. Categorize a continuous RV into one of the following 3 Types based on its support, i.e.

Type 1: a continuous RV whose support is bounded, i.e. contained in [a,b].

Type 2: a continuous RV whose support is half-bounded, i.e. contained in [a,∞) or (−∞,a]

Type 3: a continuous RV whose support is unbounded.

Then we have the following –

For a Type 1 RV, its entropy is always less than ∞, unconditionally.

For a Type 2 RV, its entropy is less than ∞, if its mean (μ) is finite.

For a Type 3 RV, its entropy is less than ∞, if its variance (σ2) is finite.

The differential entropy of a Type 1 RV is less than that of the corresponding uniform distribution, i.e. log(b−a), a Type 2 RV, that of the exponential distribution, i.e. 1+log(|μ−a|), and a Type 3 RV, that of the Gaussian distribution, i.e. 12log(2πeσ2).

Note that for a Type 2 or 3 RV, the above condition is *only a sufficient condition*. For example, consider a Type 2 RV with f(x)=3x2

for x>3. Clearly, its mean is infinite, but its entropy is 3.1 nats. Or consider a Type 3 RV with f(x)=9|x|3

for |x|>3. Its variance is infinite, but its entropy is 2.6 nats. So it would be great, if someone can provide a complete or more elegant answer for this part.

**Attribution***Source : Link , Question Author : syeh_106 , Answer Author : syeh_106*