Generating functions: formula sheet

Key formulas

Definition

M_X(t) = E[e^(tX)]

Must be finite for t in an open interval containing 0. M_X(0) = 1.

Moments from the MGF

E[X^r] = M_X^(r)(0) = d^r M_X(t)/dt^r at t = 0

Needs the MGF to exist around 0.

Mean and variance

E[X] = M'(0); Var(X) = M''(0) − (M'(0))²

M''(0) is E[X²], not the variance.

Linear transformation

M_{aX+b}(t) = e^(bt) M_X(at)

Use for scaled and shifted variables.

Sum of independent variables

M_{X+Y}(t) = M_X(t) × M_Y(t)

Independence is required. For n independent copies, M_{ΣX}(t) = (M_X(t))^n.

Uniqueness

M_X(t) = M_Y(t) on an open interval around 0 ⇒ X and Y have the same distribution

Used to identify distributions.

Series form

M_X(t) = Σ E[X^r] t^r ÷ r!

Read moments from the coefficient of t^r, multiplied by r!.

Definition

M_X(t) = E[e^(tX)]

Valid for t in an interval around 0 where the expectation is finite.

Moments from MGF

E[X^k] = M_X^(k)(0)

The k-th derivative at t = 0. So E[X] = M'(0) and Var(X) = M''(0) − (M'(0))².

Linear transformation

M_(aX+b)(t) = e^(bt) M_X(at)

Use this to standardise or shift.

Independent sum

M_(X+Y)(t) = M_X(t) M_Y(t)

Only for independent X and Y.

Binomial(n, p)

M(t) = (1 − p + p e^t)^n

Valid for all t. Bernoulli is the case n = 1.

Poisson(λ)

M(t) = exp(λ(e^t − 1))

Valid for all t.

Geometric (number of trials, support 1, 2, ...)

M(t) = p e^t ÷ (1 − (1 − p) e^t)

Valid for e^t (1 − p) < 1. Check which definition of the geometric the question uses.

Exponential(λ), mean 1/λ

M(t) = λ ÷ (λ − t)

Valid for t < λ.

Gamma(α, λ), mean α/λ

M(t) = (λ ÷ (λ − t))^α

Valid for t < λ. The rate parameter λ is used here, so check how the question defines it.

Normal(μ, σ²)

M(t) = exp(μt + σ²t² ÷ 2)

Valid for all t.

Uniform(a, b)

M(t) = (e^(tb) − e^(ta)) ÷ (t(b − a)), t ≠ 0

M(0) = 1.

Definition of the CGF

K(t) = ln M(t) = ln E[e^(tX)]

Valid where the MGF exists in an interval around t = 0. Note K(0) = ln 1 = 0.

Cumulants

κ_r = K^(r)(0), the r-th derivative of K at t = 0

Equivalently, K(t) = κ1 t + κ2 t²/2! + κ3 t³/3! + ...

Mean

E[X] = κ1 = K'(0)

Same as M'(0), because M(0) = 1.

Variance

Var(X) = κ2 = K''(0)

Easier than M''(0) − (M'(0))².

Third central moment

E[(X − μ)³] = κ3 = K'''(0)

This equality holds for the third cumulant only, together with the first two.

Skewness

γ1 = κ3 ÷ κ2^(3/2) = K'''(0) ÷ (K''(0))^(3/2)

Positive means a longer right tail. Zero for symmetric distributions with finite third moment.

Independent sums

K_{X+Y}(t) = K_X(t) + K_Y(t)

Requires X and Y to be independent. Cumulants then add.

Linear change

K_{aX+b}(t) = bt + K_X(at)

Shifting by b changes only κ1. Higher cumulants scale by a^r.

Definition

G(s) = E[s^X] = Σ s^x P(X = x), x = 0, 1, 2, …

X must take non-negative integer values. Valid at least for |s| ≤ 1.

Total probability

G(1) = 1

Use this as a quick check on any PGF you derive.

Probabilities from the PGF

P(X = k) = G⁽ᵏ⁾(0) ÷ k!

G⁽ᵏ⁾ is the k-th derivative. Equivalently, P(X = k) is the coefficient of sᵏ. So P(X = 0) = G(0).

Mean

E[X] = G′(1)

First factorial moment.

Factorial moments

E[X(X − 1)…(X − k + 1)] = G⁽ᵏ⁾(1)

Assumes the moment is finite.

Variance

Var(X) = G″(1) + G′(1) − [G′(1)]²

Since E[X(X − 1)] = G″(1) and E[X²] = G″(1) + G′(1).

Independent sum

If X and Y are independent, G_{X+Y}(s) = G_X(s) G_Y(s)

Needs independence.

Linear change

G_{a+X}(s) = sᵃ G_X(s) for integer a ≥ 0

Shifts the distribution by a.

Link to MGF

M_X(t) = G_X(eᵗ)

Valid where both exist.

Standard PGFs

Binomial(n, p): (1 − p + ps)ⁿ. Poisson(λ): e^{λ(s − 1)}. Geometric on 0, 1, 2, … with P(X = x) = p(1 − p)ˣ: p ÷ (1 − (1 − p)s)

Check which geometric definition the question uses.

MGF definition

M_X(t) = E[e^(tX)]

Valid for t in an interval around 0 where the expectation is finite.

Sum of independent variables

M_(X₁+…+Xₙ)(t) = M_X₁(t) × … × M_Xₙ(t)

Needs independence. If also identically distributed, the result is [M_X(t)]ⁿ.

Linear transformation

M_(aX+b)(t) = e^(bt) × M_X(at)

Use it for means and scaled variables.

Moments from the MGF

E[Xᵏ] = M_X⁽ᵏ⁾(0)

The kth derivative at t = 0. Variance = M''(0) − [M'(0)]².

PGF of a count

G_N(s) = E[s^N]

G_N(s) = M_N(ln s).

MGF of a compound sum

M_S(t) = G_N(M_X(t)) = M_N(ln M_X(t))

S = X₁ + … + X_N, Xs iid and independent of N.

Compound Poisson MGF

M_S(t) = exp{λ[M_X(t) − 1]}

N ~ Poisson(λ), since G_N(s) = exp{λ(s − 1)}.

Compound mean and variance

E[S] = E[N]E[X]; Var(S) = E[N]Var(X) + Var(N)(E[X])²

For compound Poisson, Var(S) = λE[X²].

Quick revision

• MGF: M(t) = E[e^(tX)], valid where the expectation is finite for t near 0.
• E[Xʳ] = the r-th derivative of M(t) at t = 0.
• Independent X and Y: M(X+Y)(t) = M_X(t) M_Y(t).
• For Y = aX + b: M_Y(t) = e^(bt) M_X(at).
• CGF: K(t) = ln M(t); K′(0) = mean and K″(0) = variance.
• Exponential with rate λ: M(t) = λ ÷ (λ − t), for t < λ.
• Poisson with mean λ: M(t) = exp(λ(eᵗ − 1)).
• Normal(μ, σ²): M(t) = exp(μt + σ²t²÷2).
• PGF: G(s) = E[s^N]; G(1) = 1 and E[N] = G′(1).
• Var(N) = G″(1) + G′(1) − [G′(1)]².
• Compound sum S: E[S] = E[N]E[X] and Var(S) = E[N]Var(X) + Var(N)(E[X])².
• If the MGF exists near 0, it identifies the distribution uniquely.

Common mistakes

• Treating M''(0) as the variance. Fix: Always subtract the square of the mean: Var = M''(0) − (M'(0))².
• Multiplying MGFs for dependent variables. Fix: Check independence first and state it. Without it, M_{X+Y} ≠ M_X M_Y in general.
• Forgetting the validity range, such as writing the exponential MGF without t < λ. Fix: Always state the range. It costs one line and earns a mark in derivations.
• Mixing up rate and scale for the exponential and gamma. Fix: Read the question's definition first. Write the pdf before the MGF. Check M'(0) gives the correct mean.
• Using E[X²] as the variance, or forgetting to subtract the mean squared when working from the MGF. Fix: With the CGF, K''(0) is already the variance. Do not subtract anything.
• Assuming κ4 equals the fourth central moment. Fix: Remember κ4 = μ4 − 3σ⁴. Only κ1, κ2 and κ3 follow the simple pattern.
• Using G″(1) as E[X²] or as the variance Fix: G″(1) = E[X(X − 1)]. Then Var(X) = G″(1) + G′(1) − [G′(1)]².
• Differentiating at s = 0 for moments, or at s = 1 for probabilities Fix: Probabilities: evaluate at s = 0 and divide by k!. Moments: evaluate at s = 1.
• Multiplying MGFs when the variables are not independent. Fix: Write 'by independence' before every product. If the variables are dependent, the product rule does not hold in general.
• Writing the compound MGF as M_N(M_X(t)). Fix: The correct form is G_N(M_X(t)) = M_N(ln M_X(t)). Check by setting t = 0: M_X(0) = 1, so M_S(0) = G_N(1) = 1.

Exam tips

• Always state the range of t for which the MGF is finite. Written papers often give a mark for it.
• Quote independence explicitly before multiplying MGFs, and quote uniqueness when you name a distribution.
• Use K(t) = ln M(t) to save time on variance questions, but show it clearly as a method.
• Check M(0) = 1 as a quick error test before differentiating.
• In computer-based papers, you can verify analytic moments by simulation, but show the formula and working in the written paper.
• Learn the standard MGFs cold. MCQs often give an MGF and ask for a mean, variance or probability, so recognition saves time.
• In written derivations, show the key step: the remaining integral or sum is a full density or pmf, so it equals 1. State the validity range of t.
• For sums, state independence explicitly before multiplying MGFs, and state uniqueness before naming the result.