Bayesian Credible Intervals: Construction and Interpretation

Understand Bayesian Credible Intervals

Key rules to remember

Posterior (proportional form)

f(θ | x) ∝ f(x | θ) × f(θ)

Identify the posterior's family by looking at the kernel, then use known quantiles of that distribution.

Credible interval definition

P(a < θ < b | x) = ∫ from a to b of f(θ | x) dθ = 1 − α

This is a probability statement about θ given the observed data.

Equal-tailed interval

P(θ < a | x) = α/2 and P(θ > b | x) = α/2

a and b are the α/2 and 1 − α/2 posterior quantiles.

HPD interval

Set {θ : f(θ | x) ≥ k}, with k chosen so that its posterior probability is 1 − α

Shortest interval of that probability. For a unimodal posterior it is a single interval with f(a | x) = f(b | x).

Normal posterior interval

μ₁ ± z(1−α/2) × σ₁, where θ | x ~ N(μ₁, σ₁²)

For 95%, z = 1.96. Equal-tailed and HPD coincide here.

Normal–normal posterior

Prior N(μ₀, σ₀²), data mean x̄ from n observations with known variance σ²: posterior precision = 1/σ₀² + n/σ²; posterior mean = (μ₀/σ₀² + n x̄/σ²) ÷ posterior precision

Precision is 1 ÷ variance. Posterior variance = 1 ÷ posterior precision.

How to solve Bayesian Credible Intervals questions

1. 1Write down the prior and the likelihood. State any assumptions, such as known variance or independent observations.
2. 2Multiply them and keep only the terms involving θ to get the posterior kernel.
3. 3Identify the posterior distribution and its parameters (for example, gamma, beta or normal).
4. 4Decide the interval type. Use equal-tailed unless the question asks for HPD.
5. 5Find the required quantiles from the tables or from R (qgamma, qbeta, qnorm). For a normal posterior use mean ± z × standard deviation.
6. 6State the interval and interpret it: the posterior probability that θ lies in it is 1 − α, given the data and the prior.
7. 7If asked, compare with the classical confidence interval: the parameter is fixed there and the 1 − α describes the long-run behaviour of the method.

Quickest way: Normal posterior shortcut

Common mistakes in Bayesian Credible Intervals

• Interpreting a 95% confidence interval as a 95% probability that θ is in the interval.

  The wording is similar to the credible interval and the natural reading sounds right.

  Fix: Say the 95% describes the long-run coverage of the method. Reserve the probability statement about θ for credible intervals.

• Using the equal-tailed interval when the question asks for HPD, or assuming they are always equal.

  They coincide for symmetric unimodal posteriors, so students assume it holds in general.

  Fix: Check the shape of the posterior. If it is skewed, the HPD interval is shorter and has equal density at both ends, not equal tail probabilities.

• Adding variances instead of precisions when combining a normal prior and normal data.

  Students mix this up with the variance of a sum.

  Fix: Add precisions (1/variance). Then invert to get the posterior variance.

• Forgetting that the data mean has variance σ²/n, not σ².

  Students use the single-observation variance in the likelihood.

  Fix: Use n/σ² as the data precision when working with x̄ from n observations.

• Taking quantiles of the prior or the likelihood instead of the posterior.

  The prior is written first and is easy to pick up by accident.

  Fix: Name the posterior distribution with its parameters before looking up any quantile.

• Not stating the interpretation in words.

  Students focus on the calculation.

  Fix: End every answer with one sentence: given the data and prior, the posterior probability that θ lies in (a, b) is 1 − α.

Worked examples