Central limit theorem versus law of large numbers

The central limit theorem states that the mean of i.i.d. variables, as $N$ goes to infinity, becomes normally distributed.

This raises two questions:

  1. Can we deduce from this the law of large numbers? If the law of large numbers says that the mean of a sample of a random variable’s values equals the true mean $\mu$ as $N$ goes to infinity, then it seems even stronger to say that (as the central limit says) that the value becomes $\mathcal N(\mu, \sigma)$ where $\sigma$ is the standard deviation. Is it fair then to say that central limit implies the law of large numbers?
  2. Does the central limit theorem apply to linear combination of variables?

Answer

The OP says

The central limit theorem states that the mean of i.i.d. variables, as N goes to infinity, becomes normally distributed.

I will take this to mean that it is the OP’s belief that for i.i.d. random
variables $X_i$ with mean $\mu$ and standard deviation $\sigma$, the
cumulative distribution function $F_{Z_n}(a)$ of
$$Z_n = \frac{1}{n} \sum_{i=1}^n X_i$$
converges to the cumulative distribution function of $\mathcal N(\mu,\sigma)$,
a normal random
variable with mean $\mu$ and standard deviation $\sigma$. Or, the
OP believes minor re-arrangements of this formula, e.g. the distribution
of $Z_n – \mu$ converges to the distribution of $\mathcal N(0,\sigma)$,
or the distribution of $(Z_n – \mu)/\sigma$ converges to the distribution of $\mathcal N(0,1)$, the standard normal random variable. Note as an example
that these statements imply that
$$P\{|Z_n – \mu| > \sigma\} = 1 – F_{Z_n}(\mu + \sigma) + F_{Z_n}((\mu + \sigma)^-) \to 1-\Phi(1)+\Phi(-1) \approx 0.32$$
as $n \to \infty$.

The OP goes on to say

This raises two questions:

  1. Can we deduce from this the law of large numbers? If the law of large numbers says that the mean of a sample of a random variable’s values equals the true mean μ as N goes to infinity, then it seems even stronger to say that (as the central limit says) that the value becomes N(μ,σ) where σ is the standard deviation.

The weak law of large numbers says that for i.i.d. random variables $X_i$
with finite mean $\mu$, given any $\epsilon > 0$,
$$P\{|Z_n – \mu| > \epsilon\} \to 0 ~~ \text{as}~ n \to \infty.$$
Note that it is not necessary to assume that the standard deviation is
finite.

So, to answer the OP’s question,

  • The central limit theorem as stated by the OP does not imply
    the weak law of large numbers. As $n \to \infty$, the OP’s
    version of the central limit theorem says that
    $P\{|Z_n-\mu| > \sigma\} \to 0.317\cdots$ while
    the weak law says that $P\{|Z_n-\mu| > \sigma\} \to 0$

  • From a correct statement of the central limit theorem, one can
    at best deduce only a restricted form of the weak law of large numbers
    applying to random variables with finite mean and standard
    deviation. But the weak law of large numbers also holds for random
    variables such as Pareto random variables with finite means but
    infinite standard deviation.

  • I do not understand why saying that the sample mean converges
    to a normal random variable with nonzero standard deviation is
    a stronger statement than saying that the sample mean converges
    to the population mean, which is a constant (or a random variable
    with zero standard deviation if you like).

Attribution
Source : Link , Question Author : user9097 , Answer Author : Dilip Sarwate

Leave a Comment