Confidence intervals and prediction intervals: formula sheet

Key formulas

Definition of a 100(1 − α)% confidence interval

P(L(X) < θ < U(X)) = 1 − α

Holds for all θ. The probability is over the random sample, not over θ.

Pivotal quantity

g(X, θ) has a distribution that does not depend on θ

It must contain θ and must be computable once θ is substituted. A statistic with no θ in it is not a pivot.

Probability statement from a pivot

P(a < g(X, θ) < b) = 1 − α

Choose a and b from the known distribution, usually leaving α/2 in each tail.

Normal mean, σ known

X̄ ± z(1 − α/2) × σ ÷ √n

Pivot: (X̄ − μ) ÷ (σ/√n) ~ N(0,1). For 95%, z = 1.96.

Normal mean, σ unknown

X̄ ± t(n−1, 1 − α/2) × S ÷ √n

Pivot: (X̄ − μ) ÷ (S/√n) ~ t(n−1). Needs normal data.

Normal variance

((n − 1)S² ÷ χ²(upper), (n − 1)S² ÷ χ²(lower))

Pivot: (n − 1)S² ÷ σ² ~ χ²(n−1). Upper percentile gives the lower limit. The interval is not symmetric.

Large-sample interval from MLE

θ̂ ± z(1 − α/2) × se(θ̂)

Approximate. Relies on asymptotic normality of the MLE.

Mean, σ known

x̄ ± z(1−α/2) × σ ÷ √n

Use when the population is normal (or n is large) and σ² is known. For 95%, z = 1.96. For 99%, z = 2.576. For 90%, z = 1.645.

Mean, σ unknown

x̄ ± t(n−1, 1−α/2) × s ÷ √n

Use when the population is normal and σ² is estimated by s². Degrees of freedom = n − 1. Here s² = Σ(x − x̄)² ÷ (n − 1).

Variance

( (n − 1)s² ÷ χ²(n−1, 1−α/2) , (n − 1)s² ÷ χ²(n−1, α/2) )

Here χ²(n−1, p) is the point with probability p below it. The larger value goes under the lower limit. The population must be normal.

Standard deviation

( √lower limit for σ² , √upper limit for σ² )

Take square roots of the variance interval limits.

Pivotal quantities

(X̄ − μ) ÷ (σ/√n) ~ N(0,1); (X̄ − μ) ÷ (S/√n) ~ t(n−1); (n − 1)S² ÷ σ² ~ χ²(n−1)

These are the starting points of each interval. They hold for a random sample from a normal distribution.

Sample size for a mean, σ known

n ≥ ( z(1−α/2) × σ ÷ E )²

E is the required half-width. Always round n up to the next whole number.

Pooled variance

S_p² = [(n − 1)Sx² + (m − 1)Sy²] ÷ (n + m − 2)

Use only when the two population variances are assumed equal. n and m are the sample sizes.

Independent samples, pooled t interval for μx − μy

(x̄ − ȳ) ± t(n+m−2, 1−α/2) × S_p × √(1/n + 1/m)

Degrees of freedom are n + m − 2. Normal populations with equal unknown variances.

Paired samples interval for μd

d̄ ± t(n−1, 1−α/2) × s_d ÷ √n, where dᵢ = xᵢ − yᵢ

n is the number of pairs. s_d is the sample standard deviation of the differences.

Known variances, or large samples

(x̄ − ȳ) ± z(1−α/2) × √(σx²/n + σy²/m)

For large samples with unknown variances, replace σ² by the sample variances and use the normal point as an approximation.

Interval for the variance ratio σx²/σy²

(sx²/sy²) ÷ F(α/2; n−1, m−1) to (sx²/sy²) × F(α/2; m−1, n−1)

F(α/2; a, b) is the upper α/2 point with a numerator and b denominator degrees of freedom. Normal populations.

Reciprocal property of F

F(1−α/2; a, b) = 1 ÷ F(α/2; b, a)

Lets you find the lower point from tables that list only upper points.

Interval for one proportion

p̂ ± z × √(p̂(1 − p̂) ÷ n), where p̂ = x ÷ n

z is the 1 − α/2 point of N(0,1). For 95%, z = 1.96. For 90%, z = 1.645. For 99%, z = 2.576. Needs large n.

Interval for difference of two proportions

(p̂1 − p̂2) ± z × √(p̂1(1 − p̂1) ÷ n1 + p̂2(1 − p̂2) ÷ n2)

Samples must be independent. The variances add, even though you are taking a difference.

Fisher information

I(θ) = −E[d²l/dθ²] = E[(dl/dθ)²]

For n independent observations, I(θ) = n × I₁(θ), where I₁ is the information from one observation.

Asymptotic distribution of the MLE

θ̂ ≈ N(θ, 1 ÷ I(θ)) for large n

Holds under regularity conditions. The variance equals the Cramér-Rao lower bound.

Asymptotic interval from the MLE

θ̂ ± z ÷ √I(θ̂)

Evaluate the information at the estimate θ̂.

Poisson mean

λ̂ = x̄, I(λ) = n ÷ λ, interval: x̄ ± z × √(x̄ ÷ n)

For a Poisson count over n exposure units, use the total count divided by n.

Function of a parameter (delta method)

g(θ̂) ≈ N(g(θ), [g′(θ)]² ÷ I(θ))

Use it for an interval for a function of θ. Requires g′(θ) ≠ 0.

Variance of prediction error (normal sample)

Var(Y − X̄) = σ² (1 + 1/n)

Y is a new observation independent of the sample. The 1 is for Y and the 1/n is for X̄.

Prediction interval, σ known

x̄ ± z × σ × √(1 + 1/n)

z is the standard normal point, e.g. 1.96 for 95% two-sided.

Prediction interval, σ unknown

x̄ ± t(n−1) × s × √(1 + 1/n)

Use the t point with n − 1 degrees of freedom and s² = Σ(xᵢ − x̄)² ÷ (n − 1). Assumes a normal population.

Regression: prediction interval at x₀

ŷ₀ ± t(n−2) × σ̂ × √(1 + 1/n + (x₀ − x̄)² ÷ Sxx)

Simple linear regression, σ̂² = residual sum of squares ÷ (n − 2), Sxx = Σ(xᵢ − x̄)².

Regression: confidence interval for mean response at x₀

ŷ₀ ± t(n−2) × σ̂ × √(1/n + (x₀ − x̄)² ÷ Sxx)

Same as above without the 1. Use it to compare with the prediction interval.

Quick revision

• A pivotal quantity is a function of the data and the parameter whose distribution is completely known and free of the parameter.
• A 95% confidence interval means 95% of intervals built this way would contain the true parameter. The parameter itself is fixed.
• Normal mean, variance known: x̄ ± z × σ ÷ √n.
• Normal mean, variance unknown: x̄ ± t(n−1) × s ÷ √n.
• Normal variance: (n−1)s² ÷ χ²(upper) to (n−1)s² ÷ χ²(lower), with n−1 degrees of freedom.
• The variance interval is not symmetric about s², and you take the square root of its limits to get one for σ.
• Two independent normal means with equal variances use a pooled variance and n₁ + n₂ − 2 degrees of freedom.
• Paired data: take differences and use a one-sample t interval on them.
• The ratio of two variances uses the F distribution, with s₁²/s₂² as the base.
• Large-sample proportion: p̂ ± z × √(p̂(1 − p̂) ÷ n).
• Prediction interval for a new normal observation: x̄ ± t × s × √(1 + 1/n).
• A prediction interval is always wider than the confidence interval for the mean, and it does not shrink to zero as n grows.

Common mistakes

• Saying there is a 95% probability that the true parameter lies in the calculated interval. Fix: Treat θ as fixed. The 95% describes the method over repeated samples. Say: the procedure captures θ in 95% of samples.
• Using a z value when σ is unknown and the sample is small. Fix: If you use S, use the t distribution with n − 1 degrees of freedom. The t value is larger, so the interval is wider.
• Using z instead of t when σ is estimated from the sample. Fix: Check whether σ is given as a known value. If only s is available, use t with n − 1 degrees of freedom.
• Using n instead of n − 1 as the degrees of freedom. Fix: Write 'df = n − 1' as soon as you start. One degree of freedom is used up in estimating the mean.
• Using the independent-samples formula on paired data Fix: Ask whether the observations are linked. If so, take differences and use a one-sample t interval with n − 1 degrees of freedom. Pairing removes variation between subjects.
• Using n − 1 or m − 1 as the degrees of freedom for the pooled t Fix: The pooled t has n + m − 2 degrees of freedom, because two sample means are estimated.
• Subtracting the variances when finding the interval for p1 − p2. Fix: For independent samples, Var(p̂1 − p̂2) = Var(p̂1) + Var(p̂2). Always add.
• Forgetting to divide by n inside the square root, or using p̂(1 − p̂) as the variance of p̂. Fix: The variance of p̂ is p(1 − p)/n. Write it out with the n before you compute.
• Using s√(1/n) instead of s√(1 + 1/n). Fix: Ask what is being predicted. If it is a single new value, add the 1 inside the square root.
• Using the normal point when σ is estimated. Fix: If s replaces σ, use the t distribution with n − 1 degrees of freedom (n − 2 in simple regression).

Exam tips

• Write the pivot and its distribution first. Examiners award marks for naming it and for showing it does not depend on the parameter.
• In interpretation questions, mention repeated sampling and say that the interval is random before the data are observed.
• State assumptions such as normality and independence. Marks are often lost when they are missing.
• For variance intervals, check the lower limit is below the upper limit and that both sit around, not on, S² in a skewed way.
• In the computer-based paper, show the formula and the quantile function you used, such as qt or qchisq, with the degrees of freedom, before the result.
• Always state the assumption: the sample comes from a normal distribution. Examiners award marks for it, and variance intervals depend on it heavily.
• Show the pivotal quantity and its distribution before substituting. In written answers, the method marks are for this step.
• Write the table values you use, with the degrees of freedom. This lets you earn marks even if you misread a later number.