Basic univariate distributions and generating samples: formula sheet

Key formulas

Bernoulli

P(X = 1) = p, P(X = 0) = q = 1 − p; E[X] = p; Var(X) = pq

Building block for the binomial and geometric.

Binomial pmf

P(X = x) = C(n, x) p^x q^(n−x), x = 0, 1, …, n

C(n, x) = n! ÷ (x!(n − x)!). Trials are independent with constant p.

Binomial mean and variance

E[X] = np; Var(X) = npq

Variance is less than the mean, since q < 1.

Poisson pmf

P(X = x) = e^(−μ) μ^x ÷ x!, x = 0, 1, 2, …

For a rate λ per unit time over t units, use μ = λt.

Poisson mean and variance

E[X] = μ; Var(X) = μ

Equal mean and variance is a signature of the Poisson.

Poisson recursion

P(X = x + 1) = P(X = x) × μ ÷ (x + 1)

Fast way to build a table of probabilities.

Geometric (trials version)

P(X = x) = q^(x−1) p, x = 1, 2, 3, …; E[X] = 1/p; Var(X) = q/p²

X is the trial number of the first success.

Geometric (failures version)

P(X = x) = q^x p, x = 0, 1, 2, …; E[X] = q/p; Var(X) = q/p²

X is the number of failures before the first success. Variance is unchanged by the shift.

Geometric tail

Trials version: P(X > x) = q^x. Failures version: P(X ≥ x) = q^x

Handy for 'at least' and 'more than' questions.

Negative binomial (failures version)

P(X = x) = C(x + k − 1, k − 1) p^k q^x, x = 0, 1, 2, …; E[X] = kq/p; Var(X) = kq/p²

X is the number of failures before the k-th success.

Negative binomial (trials version)

P(N = n) = C(n − 1, k − 1) p^k q^(n−k), n = k, k+1, …; E[N] = k/p; Var(N) = kq/p²

N is the trial on which the k-th success occurs.

Poisson approximation to binomial

Bin(n, p) ≈ Poisson(np)

Use when n is large and p is small. A common guide is n ≥ 50 and p ≤ 0.1 or so, but it is only a rule of thumb.

Sum of independent Poissons

If X ~ Poisson(μ1) and Y ~ Poisson(μ2) are independent, X + Y ~ Poisson(μ1 + μ2)

Also true for binomials with the same p: Bin(n1, p) + Bin(n2, p) = Bin(n1 + n2, p).

Normal N(μ, σ²)

f(x) = [1 ÷ (σ√(2π))] exp(−(x − μ)² ÷ (2σ²)); mean = μ; variance = σ²; MGF = exp(μt + σ²t²/2)

The second parameter is the variance, not the standard deviation. Standardise with Z = (X − μ) ÷ σ. Use P(Z < −z) = 1 − Φ(z).

Lognormal (ln X ~ N(μ, σ²))

E[X^k] = exp(kμ + k²σ²/2); mean = exp(μ + σ²/2); variance = exp(2μ + σ²) × (exp(σ²) − 1); median = exp(μ)

Get moments from the normal MGF: E[X^k] = E[exp(kY)] with Y ~ N(μ, σ²). Do not use the MGF of X itself.

Exponential(λ)

f(x) = λe^(−λx); F(x) = 1 − e^(−λx); S(x) = e^(−λx); mean = 1/λ; variance = 1/λ²; MGF = λ ÷ (λ − t) for t < λ

Memoryless: P(X > s + t | X > s) = P(X > t) = e^(−λt).

Gamma(α, λ)

f(x) = λ^α x^(α−1) e^(−λx) ÷ Γ(α); mean = α/λ; variance = α/λ²; MGF = (λ ÷ (λ − t))^α for t < λ

Γ(α) = (α − 1)Γ(α − 1); Γ(n) = (n − 1)! for whole n; Γ(1/2) = √π. A sum of n independent Exp(λ) variables is Gamma(n, λ). Gamma variables with the same λ add: Gamma(α1, λ) + Gamma(α2, λ) = Gamma(α1 + α2, λ) if independent.

Chi-square χ²(ν)

Gamma(ν/2, 1/2); mean = ν; variance = 2ν; Z1² + … + Zν² ~ χ²(ν)

Used for the distribution of the sample variance: (n − 1)S² ÷ σ² ~ χ²(n − 1) for a normal sample.

t and F distributions

t(ν) = Z ÷ √(χ²(ν) ÷ ν); F(m, n) = [χ²(m) ÷ m] ÷ [χ²(n) ÷ n]; 1 ÷ F(m, n) ~ F(n, m)

The t has mean 0 for ν > 1 and variance ν ÷ (ν − 2) for ν > 2. The F has mean n ÷ (n − 2) for n > 2.

Beta(a, b)

f(x) = [Γ(a + b) ÷ (Γ(a)Γ(b))] x^(a−1) (1 − x)^(b−1), 0 < x < 1; mean = a ÷ (a + b); variance = ab ÷ ((a + b)²(a + b + 1))

Beta(1, 1) is the uniform distribution on (0, 1).

Pareto(α, λ)

f(x) = αλ^α ÷ (λ + x)^(α+1), x > 0; F(x) = 1 − (λ ÷ (λ + x))^α; mean = λ ÷ (α − 1) for α > 1; variance = αλ² ÷ ((α − 1)²(α − 2)) for α > 2

E[X^k] exists only for k < α. It has no MGF, because the tail is too heavy.

Weibull(c, γ)

F(x) = 1 − exp(−c x^γ), x > 0; f(x) = cγ x^(γ−1) exp(−c x^γ); mean = c^(−1/γ) Γ(1 + 1/γ)

With γ = 1 this is Exponential(c). Check the parameterisation in the Tables before using it.

Continuous uniform U(a, b) density and CDF

f(x) = 1 ÷ (b − a) for a < x < b; F(x) = (x − a) ÷ (b − a) for a ≤ x ≤ b

Outside (a, b) the density is 0. Probabilities are lengths divided by (b − a).

Continuous uniform mean and variance

E[X] = (a + b) ÷ 2; Var(X) = (b − a)² ÷ 12

The mean is the midpoint. The variance depends only on the width.

Discrete uniform on 1, 2, ..., n

P(X = k) = 1 ÷ n; E[X] = (n + 1) ÷ 2; Var(X) = (n² − 1) ÷ 12

If the values run a, a+1, ..., b, use n = b − a + 1 and shift the mean by a − 1. The variance is unchanged by shifting.

Uniform MGF

M(t) = (e^(bt) − e^(at)) ÷ ((b − a)t), t ≠ 0

M(0) = 1.

Exponential as gamma

Exponential(λ) = Gamma(1, λ)

Gamma(α, λ) has mean α ÷ λ and variance α ÷ λ². Check the Tables for the parameterisation.

Chi-square as gamma

χ²(ν) = Gamma(ν ÷ 2, 1 ÷ 2); mean ν, variance 2ν

Sum of independent χ² variables is χ² with the degrees of freedom added.

Sum of independent exponentials

X₁ + ... + Xₙ ~ Gamma(n, λ) for iid Exponential(λ)

Gives the waiting time to the nth event in a Poisson process.

Normal approximation to binomial

X ~ Bin(n, p) ≈ N(np, np(1 − p))

Use when n is large and p is not near 0 or 1.

Continuity correction

P(X ≤ k) ≈ P(Y < k + 0.5); P(X ≥ k) ≈ P(Y > k − 0.5); P(X = k) ≈ P(k − 0.5 < Y < k + 0.5)

Y is the normal approximation. Apply it for integer-valued X.

Standard normal square

If Z ~ N(0, 1), then Z² ~ χ²(1)

Sum of squares of n independent standard normals is χ²(n).

Congruential generator

X(n+1) = (a × X(n) + c) mod m; U(n) = X(n) ÷ m

Seed X(0) is given. If c = 0 it is called multiplicative. Values of X lie in 0 to m − 1, so U lies in [0, 1).

Inverse transform rule

X = F⁻¹(U), U ~ Uniform(0,1)

Then X has CDF F. Proof: P(X ≤ x) = P(U ≤ F(x)) = F(x).

Discrete version

X = smallest x such that F(x) ≥ U

Use the cumulative probability table. Check the order of the values, from smallest to largest.

Exponential(λ)

x = −ln(1 − u) ÷ λ

Also valid: x = −ln(u) ÷ λ, since 1 − U is uniform. λ is the rate, so the mean is 1/λ.

Uniform(a, b)

x = a + (b − a) × u

A direct linear transform of u.

Pareto(α, λ)

F(x) = 1 − (λ ÷ (λ + x))^α, so x = λ × [(1 − u)^(−1/α) − 1]

This is the two-parameter Pareto used in the IAI tables. Check the form of F given in the question.

Weibull(c, γ)

F(x) = 1 − exp(−c × x^γ), so x = [−ln(1 − u) ÷ c]^(1/γ)

Check the parametrisation given in the question or in the Tables.

Normal(μ, σ²)

x = μ + σ × Φ⁻¹(u)

Φ⁻¹ is read from the normal tables. There is no closed form.

Envelope condition

f(x) ≤ c·g(x) for all x with f(x) > 0, c ≥ 1

g must be a density you can simulate from. g(x) must be positive wherever f(x) is positive.

Acceptance test

Accept Y if U ≤ f(Y) ÷ (c·g(Y))

U ~ U(0,1), independent of Y. Y is drawn from g.

Acceptance probability

P(accept) = 1/c

Holds when both f and g are properly normalised densities.

Expected number of attempts

E[N] = c

N is geometric with success probability 1/c. Each attempt uses one Y and one U.

Best constant

c = max over x of f(x) ÷ g(x)

The smallest valid c gives the highest efficiency.

Box-Muller

Z1 = √(−2 ln U1) · cos(2πU2); Z2 = √(−2 ln U1) · sin(2πU2)

U1, U2 independent U(0,1). Z1, Z2 are independent N(0,1).

Polar method

V1 = 2U1 − 1, V2 = 2U2 − 1, W = V1² + V2². If W < 1 (and W > 0): Z1 = V1·√(−2 ln W ÷ W), Z2 = V2·√(−2 ln W ÷ W)

If W ≥ 1, reject the pair and redraw. About π/4 of pairs are accepted.

Normal scaling

X = μ + σZ

Turns a standard normal Z into N(μ, σ²). σ is the standard deviation.

Simulated mean

x̄ = (1 ÷ n) Σ xᵢ

Each xᵢ is the value of the quantity of interest from simulation i. Runs must be independent.

Standard error of the mean

SE = σ ÷ √n, estimated by s ÷ √n

s is the sample standard deviation with divisor n − 1. Use s when σ is unknown.

Approximate confidence interval for the mean

x̄ ± z × s ÷ √n

z = 1.645 for 90%, 1.96 for 95%, 2.576 for 99%. Based on the Central Limit Theorem for large n.

Number of simulations for margin of error E

n ≥ (z × σ ÷ E)²

Round up. Use a pilot estimate of σ. E is in the same units as the mean.

Standard error of a simulated probability

SE = √(p̂(1 − p̂) ÷ n)

p̂ is the proportion of runs in which the event occurs. Use a pilot p̂ or p̂ = 0.5 for a cautious n.

Number of simulations for a probability

n ≥ z² × p(1 − p) ÷ E²

The worst case is p = 0.5, giving the largest n.

Quick revision

• Binomial(n, p): mean np, variance np(1 − p).
• Poisson(λ): mean = variance = λ.
• Geometric: counts trials or failures until the first success, so check which definition the question uses.
• Exponential(λ): mean 1/λ, variance 1/λ², CDF F(x) = 1 − e^(−λx).
• Gamma(α, λ) with α = 1 is the exponential distribution.
• If X is lognormal, then ln X is normal.
• Inverse transform: if U ~ U(0,1), then X = F⁻¹(U) has CDF F.
• For the exponential, X = −ln(1 − U) ÷ λ, and −ln(U) ÷ λ works too since 1 − U is also U(0,1).
• Acceptance-rejection needs a density g and a constant c with f(x) ≤ c·g(x) for all x.
• The acceptance probability is 1/c, so a smaller c is more efficient.
• Standard error of a simulated mean = s ÷ √n.
• To halve the standard error, you need four times as many simulations.

Common mistakes

• Mixing up the two versions of the geometric or negative binomial. Fix: Check the stated range of X. If it starts at 1, X counts trials and the mean is 1/p. If it starts at 0, X counts failures and the mean is q/p.
• Using the binomial when trials are not fixed, or when p changes. Fix: Ask: is n fixed and are trials independent with the same p? If you stop at a success, it is geometric or negative binomial.
• Treating the second parameter of N(μ, σ²) as the standard deviation, and dividing by 25 instead of 5 (or the reverse). Fix: Write σ² = value, then σ = √value on the next line, before you standardise.
• Giving the lognormal mean as e^μ. Fix: Use exp(μ + σ²/2) for the mean. Remember the mean is larger than the median because of the right skew.
• Using (b − a)² ÷ 12 as the standard deviation. Fix: Take the square root for the standard deviation: (b − a) ÷ √12.
• Using n = b − a for a discrete uniform on a, ..., b. Fix: Use n = b − a + 1 for the number of values.
• Choosing the wrong value at the boundary in a discrete simulation, for example taking the largest x with F(x) ≤ u. Fix: Always use: the smallest x with F(x) ≥ u. Write the cumulative column and compare u to each entry from the top.
• Forgetting to divide by m, so the 'uniform' number is a whole number such as 6 instead of 0.375. Fix: Compute u = X ÷ m every time. A uniform number must lie in (0,1). If yours does not, stop and check.
• Choosing c so that c·g(x) is below f(x) somewhere. Fix: Find the maximum of f ÷ g properly, using calculus if needed. Then check the tails and endpoints.
• Using c < 1 or forgetting that c ≥ 1. Fix: Remember that acceptance probability is 1/c, and c ≥ 1 because both f and g integrate to 1.

Exam tips

• Write down which version of the geometric or negative binomial you are using before any calculation. Examiners reward a clear statement of assumptions.
• In MCQs, check whether the question gives a rate per year or per month before using the Poisson pmf. Many wrong options come from an unscaled mean.
• For 'show that' questions on the mean or variance of the Poisson, be ready to sum the series using the expansion of e^μ. Know E[X(X − 1)] = μ² as the key step.
• For Paper B, know the R functions dbinom, pbinom, dpois, ppois, dgeom and dnbinom. Note that R's geometric and negative binomial count failures, not trials.
• Use the Poisson approximation only when the question allows it or when n is large and p small. State the parameter np clearly.
• Write the parameters and their form in the first line of every answer. Examiners award marks for stating the distribution and parameters correctly.
• In written questions, derive a moment from the MGF or the integral only if asked to show it. Otherwise quote the Tables result and apply it with clear working.
• Lognormal questions often ask for both a probability and a moment. Show ln x, z and Φ(z) on separate lines so that one slip does not cost every mark.