## Limits of binomial distribution

### Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)

We know that as $n \to \infty$, the binomial distribution $B(n, p)$, with fixed $p$, after appropriate normalization, converges to a normal distribution. If $p = c/n$ for some constant $c$, then it converges to the Poisson distribution.

What happens for intermediate cases, say when $p = c n^{-\alpha}$ for fixed $c$ and $0 < \alpha < 1$. After appropriate normalization, what is the limiting distribution of $B(n,p)$?

-

## 5 Answers

Assume the distribution of $X_n$ is binomial $(n,p)$. Then, for every real number $t$, $$\mathrm E(\mathrm e^{\mathrm it(X_n-np)})=\mathrm e^{\mathrm itnp}(1-p+p\mathrm e^{\mathrm it})^n.$$ Assuming that $p\to0$ when $n\to\infty$, standard limited expansions yield $$\mathrm E(\mathrm e^{\mathrm it(X_n-np)})=\exp\left(-\tfrac12npt^2(1-p)+O(npt^3)\right).$$ Let us assume that $np\to\infty$ when $n\to\infty$ and choose $t=s/\sqrt{np}$ for a given $s$. Then $npt^2(1-p)\to s^2$ and $npt^3=s^3/\sqrt{np}\to0$, hence $$\mathrm E(\mathrm e^{\mathrm is(X_n-np)/\sqrt{np}})\to\mathrm e^{-s^2/2}.$$ Thus, $(X_n-np)/\sqrt{np}$ converges in distribution to the standard gaussian distribution when $n\to\infty$, as soon as $p\to0$ and $np\to\infty$.

-

### You can accept an answer to one of your own questions by clicking the check mark next to it. This awards 15 reputation points to the person who answered and 2 reputation points to you.

I believe with a fairly standard scaling and shifting, we can recover a normal limit for any $0 < \alpha < 1$ by a standard application of the Lindeberg–Feller theorem.

Define the triangular array of random variables $X_{n,m}$, $1 \leq m \leq n$ such that each row contains iid elements distributed as $\mathrm{Bernoulli}(cn^{-\alpha})$ random variables. Then $S_n = \sum_{m=1}^n X_{n,m}$ is $B(n,p)$ with $p = c n^{-\alpha}$.

Now, shift and rescale by taking $Y_{n,m} = n^{-(1-\alpha)/2} (X_{n,m} - c n^{-\alpha})$. Note that $\mathbb E Y_{n,m} = 0$ and $\mathbb E Y_{n,m}^2 = n^{-1} c (1 - c n^{-\alpha})$.

We need only check the Lindeberg–Feller conditions:

1. $\sum_{m=1}^n \mathbb E Y_{n,m}^2 = c (1-cn^{-\alpha}) \to c > 0$, and
2. For all $\epsilon > 0$, $\lim_{n\to\infty} \sum_{m=1}^n \mathbb E |Y_{n,m}|^2 1_{(|Y_{n,m}|^2 > \epsilon)} = 0$.

The second one follows since $1_{(|Y_{n,m}|^2 > \epsilon)} = 0$ for all sufficiently large $n$ since $|Y_{n,m}|^2 \leq n^{-(1-\alpha)} (1+c)^2$ almost surely.

Hence $n^{-(1-\alpha)/2} (S_n - c n^{1-\alpha}) \;\xrightarrow{\;d\;}\; \mathcal N(0,c)$.

To see what fails in the case where $\alpha = 1$, note that the indicator function $1_{(|Y_{n,m}|^2 > \epsilon)}$ will no longer go to zero almost surely. Therefore the second condition will fail because there will be a contribution of $c n^{-1} (1-c/n)^2$ in expectation for each of the $n$ terms, which, when added up yields (obviously) a nonzero limit.

-

The total variation distance between $B(n,p)$ and Poisson($np$) goes to 0 as $n\to\infty$ whenever $p\to 0$. See http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=2086772 for example. This means that the Poisson and normal ranges overlap considerably and you don't need anything else.

-

1
I disagree with the last part of your last sentence, which puts undue significance in the result recalled in the first sentence. For example, if $np\to0$, the distance between Poisson $(np)$ and a Dirac mass at $0$ ALSO goes to $0$ hence in this regime, the result simply says that $B(n,p)$ converges to a Dirac mass at $0$. Similarly, if $np\to\infty$, Poisson $(np)$ is itself (when shifted by $np$ and rescaled by $\sqrt{np}$) approximately standard gaussian hence in this regime, the result is simply an unusual way of stating that a CLT holds. – Didier Piau Feb 14 2012 at 10:30
I don't see your point. If $np\to 0$, the explicit bounds on the tdv given in the linked paper show more than just accumulation at 0. However, there are stronger results in that case to be sure (very easily obtained). For $np\to\infty$, you are right that scaled Poisson is close to normal, but that's what "the Poisson and normal ranges overlap considerably" means. The OP asked what happens between the ranges where binomial is like Poisson and where binomial is like normal, and the correct answer is that there is nothing between them. – Brendan McKay Feb 14 2012 at 19:10
My point is that in these two regimes, the Poisson distribution is a red herring, for a Dirac distribution or for a gaussian one. (I read your comment by chance, please use @ when commenting to someone else than the author of the post.) – Didier Piau Feb 15 2012 at 0:48
@Didier Piau: Depending on the way $p$ varies as a function of $n$ (except for $p=1-o(1/n)$), $B(n,p)$ is well approximated by either the Poisson or Normal distribution. The details depend on the distance measure, but for most measures (total variation distance being one) the Poisson approximation remains better than the normal well into the range where the binomial is asymptotically normal. For tiny $p$, the Poisson approximation is better than the Dirac approximation. Do you disagree with any of these statements? – Brendan McKay Feb 15 2012 at 9:08

$B(n,p)$ for $n$ very large and $p$ very small is approximately a Poisson distribution with $\lambda=pn$. Normalizing by a constant gives you approximately a constant times the Poisson distribution. You might as well take your constant to be $1/\lambda$. In this case, as $\lambda \to \infty$, the variance, $1/\lambda$, will go to $0$, and the random variable becomes constant.

-
1. Let $X_1,X_2,...,X_n$ be independent and identically distributed random variables such that $E(X_i) = 0, Var(X_i) = 1$ and denote $W=n^{-1/2}\sum X_i$. By using Stein's method, we have that $$d_W(\mathcal{L}(W),N(0,1)) \leq \frac{5 E|X_1|^3}{n^{1/2}},$$ where $d_W$ is the Wasserstein metric.

2. Any binomial distribution $B(n, p)$ is the sum of $n$ independent Bernoulli trials $B(1, p)$, so you can consider $X_i$ as a normalized Bernoulli random variable and apply the above inequality.

-
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 82, "mathjax_display_tex": 4, "mathjax_asciimath": 0, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "github"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "english_confidence": 0.9231011271476746, "perplexity_flag": "head"}