STAT 400

Mon. March 9th, 2020

Approximation of Binomial Distribution

Proposition. Let XX follow bin(n,p)\text{bin}(n,p).
If npnp and n(1−p)n(1-p) are large enough, then XX approximately follows a normal distribution N(np, np(1−p))N(np,\ np(1-p)).

In other words, if YY follows N(np, np(1−p))N(np,\ np(1-p)), and aa is a positive integer, then P(X≤a)=B(a;n,p)≈P(Y≤a+0.5)=P(Z≤a+0.5−npnp(1−p))=Φ(a+0.5−npnp(1−p)).\begin{aligned} P(X\le a)&=B(a;n,p)\\ &\approx P(Y\le a+0.5)\\ &=P(Z\le \frac{a+0.5-np}{\sqrt{np(1-p)}})\\ &=\Phi(\frac{a+0.5-np}{\sqrt{np(1-p)}}). \end{aligned} _(The 0.5 in the equation is for continuity correction, since the binomial distribution is discrete but normal distributions are continuous.)

Rule of thumb: If np>10np>10 and n(1−p)>10n(1-p)>10, this approximation can be used.

Ex. A fair coin is tossed 400 times. What is the probability that the number of heads is between 180 and 220?

Let XX be the number of heads; XX follows bin(400,0.5)\text{bin}(400, 0.5).
We want to find P(180≤X≤220)=P(X≤220)−P(X≤179).P(180\le X\le 220)=P(X\le 220)-P(X\le 179).

We don’t want to have a binomial table that’s hundreds of pages long for many values of nn all the way up to 400, so we can instead approximate this value with a standard normal table.

P(X≤220)−P(X≤179)≈Φ(220+0.5−200100)−Φ(179+0.5−200100)=Φ(2.05)−Φ(−2.05)=0.96.\begin{aligned} P(X\le 220)-P(X\le 179)&\approx \Phi(\frac{220+0.5-200}{\sqrt{100}})-\Phi(\frac{179+0.5-200}{\sqrt{100}})\\ &=\Phi(2.05)-\Phi(-2.05)\\ &=0.96. \end{aligned}

Exponential Distribution

Definition. XX follows an exponential distribution with parameter λ>0\lambda > 0 if its pdf is of the form f(x;λ)={λe−λxx>00otherwisef(x;\lambda)=\begin{cases} \lambda e^{-\lambda x} &x>0\\ 0&\text{otherwise} \end{cases} In this case, we say XX follows Exp(λ)\text{Exp}(\lambda).

CDF

F(x;λ)={1−e−λxx>00otherwiseF(x;\lambda)=\begin{cases} 1-e^{-\lambda x}&x>0\\ 0&\text{otherwise} \end{cases}

Expected Value

E(X)=∫0∞xλe−λxdx=1λE(X2)=∫0∞x2λe−λxdx=2λ2\begin{aligned} E(X)&=\int_0^{\infin} x\lambda e^{-\lambda x}dx=\frac{1}{\lambda}\\ E(X^2)&=\int_0^{\infin} x^2\lambda e^{-\lambda x}dx=\frac{2}{\lambda^2} \end{aligned}

Variance

V(X)=E(X2)−[E(X)]2=1λ2.V(X)=E(X^2)-[E(X)]^2=\frac{1}{\lambda^2}.

Relation to Poisson Process

We are interested in studying this distribution because of its relation to the Poisson process.

Proposition. Suppose X(s,t)X(s,t) is the number of arrivals between time ss and time tt, and that XX is a Poisson process with parameter α\alpha, i.e. X(s,t)X(s,t) follows Pois(α(t−s))\text{Pois}(\alpha(t-s)).

Let TiT_i be the time of the iith arrival. Then Ti+1−TiT_{i+1}-T_i follows exp(α)\text{exp}(\alpha).
(Here, Ti+1−TiT_{i+1}-T_i represents the duration of time between two consecutive arrivals.)