Actuarial Statistics · Bayesian inference: priors, posteriors, loss functions and credible intervals

Loss Functions and Bayes Estimators: Posterior Mean, Median and Mode

Q: Which loss function gives the posterior mean, median and mode?

Quadratic (squared error) loss gives the posterior mean. Absolute error loss gives the posterior median. All-or-nothing (0-1) loss gives the posterior mode.

Q: How do I find the Bayes estimate under squared error loss?

Find the posterior distribution of the parameter, then compute its mean. For a standard posterior, use the known mean formula, such as α/λ for a Gamma distribution.

Q: What is expected posterior loss?

It is the average of the loss function over the posterior distribution of the parameter, for a given estimate. The Bayes estimator is the estimate that makes this smallest.

Q: Is the Bayes estimator always the mean?

No. It is the mean only under quadratic loss. A different loss function gives a different summary of the same posterior.

Updated 11 October 2026 · Fact-checked

A Bayes estimator is the value that minimises expected posterior loss. Under quadratic (squared error) loss it is the posterior mean. Under absolute error loss it is the posterior median. Under all-or-nothing (0-1) loss it is the posterior mode. Find the posterior first, then take the matching summary.

Understand Loss Functions and Bayes Estimators

In Bayesian inference you end up with a posterior distribution for the parameter θ. But often you must report one number, a point estimate. A loss function L(θ̂, θ) says how costly it is to report θ̂ when the true value is θ.

The true θ is unknown, so you cannot minimise the loss directly. Instead you average the loss over the posterior. This gives the expected posterior loss: E[L(θ̂, θ) | x]. The Bayes estimator is the value of θ̂ that makes this as small as possible.

The choice of loss function decides which summary of the posterior you get. Quadratic loss, L = (θ̂ − θ)², punishes large errors heavily. Its minimiser is the posterior mean. Absolute loss, L = |θ̂ − θ|, punishes errors in proportion to their size. Its minimiser is the posterior median. All-or-nothing loss gives a loss of 0 if you are exactly right and a fixed loss otherwise. For a continuous θ, it is taken as the limit of a loss that is 0 when |θ̂ − θ| is within a small ε and 1 otherwise. Its minimiser is the posterior mode.

Why does squared error give the mean? Expand E[(θ̂ − θ)²] = Var(θ | x) + (θ̂ − E[θ | x])². The first term does not depend on θ̂. The second is smallest, at zero, when θ̂ = E[θ | x].

In the exam, the loss function is usually named in the question. Your job is to find the posterior, spot its distribution, and read off the mean, median or mode. Mean and mode are often easy from standard results. The median usually needs solving an equation.

Key rules to remember

Expected posterior loss: E[L(θ̂, θ) | x] = ∫ L(θ̂, θ) f(θ | x) dθ; For a discrete θ, replace the integral with a sum over the posterior probabilities. The Bayes estimator minimises this.
Quadratic (squared error) loss: L(θ̂, θ) = (θ̂ − θ)² → θ̂ = E[θ | x]; The Bayes estimator is the posterior mean.
Absolute error loss: L(θ̂, θ) = |θ̂ − θ| → θ̂ = m, where P(θ ≤ m | x) = 0.5; The Bayes estimator is the posterior median.
All-or-nothing (0-1) loss: L(θ̂, θ) = 0 if θ̂ = θ, 1 otherwise → θ̂ = mode of f(θ | x); For a continuous θ, the mode arises as the limit of a loss that is 0 when |θ̂ − θ| < ε and 1 otherwise, as ε → 0.
Decomposition under quadratic loss: E[(θ̂ − θ)² | x] = Var(θ | x) + (θ̂ − E[θ | x])²; Shows why the mean is optimal. The minimum expected loss is the posterior variance.
Posterior up to proportionality: f(θ | x) ∝ f(x | θ) × f(θ); You only need the shape in θ. Drop constants that do not involve θ.
Gamma mean and mode: For Gamma(α, λ) with mean α/λ: mode = (α − 1)/λ, valid for α ≥ 1; Useful for Poisson-Gamma and Exponential-Gamma posteriors.

How to solve Loss Functions and Bayes Estimators questions

Use this order for any question asking for a Bayes estimate under a stated loss function.

1Write down the likelihood f(x | θ) for the data given.
2Write down the prior f(θ). Multiply to get f(θ | x) ∝ f(x | θ) f(θ). Keep only terms involving θ.
3Identify the posterior as a standard distribution if possible, and state its parameters.
4Read the loss function in the question. Quadratic means mean, absolute means median, all-or-nothing means mode.
5Compute that summary. Use the standard formula for mean or mode. For the median, solve P(θ ≤ m | x) = 0.5.
6If the question asks for the expected posterior loss, compute it. Under quadratic loss at the Bayes estimate it equals the posterior variance.
7State the final estimate with units or context, and note your assumptions.

Quickest way: Match the loss to the summary

When to use it: Use in multiple-choice questions and any time the posterior is a standard distribution.

Name the posterior, for example Gamma(α, λ) or Beta(a, b).
Quadratic: write the mean. Absolute: write the median. All-or-nothing: write the mode.
For Gamma(α, λ), mean = α/λ and mode = (α − 1)/λ. For Beta(a, b), mean = a/(a + b) and mode = (a − 1)/(a + b − 2) when a, b > 1.
For the median, if the posterior is symmetric the median equals the mean. Otherwise solve the equation or use a simple cdf, such as an exponential or a power form.
Check the answer lies inside the range of θ.

Common mistakes in Loss Functions and Bayes Estimators

Giving the posterior mean when the question asks for absolute loss or all-or-nothing loss.
The mean is the most familiar estimate, so students default to it.
Fix: Underline the loss function first. Write the pairing mean, median, mode next to the working.
Using the prior mean or the maximum likelihood estimate instead of the posterior summary.
Students skip the step of updating the prior with the data.
Fix: Always derive the posterior first. A Bayes estimator is a function of the posterior.
Using the Gamma mode as α/λ.
It is confused with the mean.
Fix: Remember mean = α/λ and mode = (α − 1)/λ. Differentiate the log posterior if unsure.
Working out the median of a skewed posterior by assuming it equals the mean.
True only for symmetric distributions.
Fix: Set the cdf equal to 0.5 and solve. If the cdf is simple, this is quick.
Keeping constants in the posterior and then getting the wrong normalising constant or mode.
Students are unsure which terms matter.
Fix: Keep only factors containing θ. Compare the kernel with a standard distribution to name it.
Stating that the minimum expected loss under quadratic loss is zero.
Students mix up the loss at the estimate with the average loss.
Fix: The minimum expected loss is the posterior variance, which is positive unless the posterior is a single point.

Worked examples

Example 1

Claim numbers in a year are Poisson with mean θ. The prior for θ is Gamma(α = 3, λ = 2), with mean α/λ. In one year you observe 5 claims. Find the Bayes estimate of θ under (a) quadratic loss and (b) all-or-nothing loss.

Show the solution

Likelihood: f(x | θ) ∝ θ^5 e^(−θ).
Prior: f(θ) ∝ θ^(3−1) e^(−2θ) = θ² e^(−2θ).
Posterior: f(θ | x) ∝ θ^(5+2) e^(−3θ) = θ^7 e^(−3θ).
This is Gamma(α = 8, λ = 3).
(a) Quadratic loss gives the posterior mean = α/λ = 8/3.
(b) All-or-nothing loss gives the posterior mode = (α − 1)/λ = 7/3.

Answer: (a) 8/3 ≈ 2.667. (b) 7/3 ≈ 2.333.

Example 2

The posterior density of a parameter θ is f(θ | x) = 3θ² for 0 < θ < 1. Find the Bayes estimate under (a) quadratic loss, (b) absolute loss, and (c) the expected posterior loss at the quadratic-loss estimate.

Show the solution

(a) Posterior mean: E[θ] = ∫ θ × 3θ² dθ from 0 to 1 = 3/4.
(b) The cdf is F(m) = m³. Set m³ = 0.5, so m = 0.5^(1/3) ≈ 0.7937.
(c) Under quadratic loss the minimum expected loss is Var(θ | x).
E[θ²] = ∫ θ² × 3θ² dθ = 3/5.
Var = 3/5 − (3/4)² = 0.6 − 0.5625 = 0.0375.

Answer: (a) 0.75. (b) 0.5^(1/3) ≈ 0.794. (c) 0.0375.

Exam tips

Circle the loss function in the question. Marks are often lost by using the wrong summary.
Show the posterior kernel and its named distribution. Examiners give marks for this even if later arithmetic slips.
For the median, write P(θ ≤ m | x) = 0.5 explicitly, then solve.
When asked why a loss function leads to an estimator, use the variance decomposition for quadratic loss as a short proof.
In multiple-choice questions, work out the posterior parameters carefully. Distractors are often the mean given when the mode was asked for.

Practice questions from Bayesian inference: priors, posteriors, loss functions and credible intervals

Loss Functions and Bayes Estimators: frequently asked questions

Which loss function gives the posterior mean, median and mode?

Quadratic (squared error) loss gives the posterior mean. Absolute error loss gives the posterior median. All-or-nothing (0-1) loss gives the posterior mode.

How do I find the Bayes estimate under squared error loss?

Find the posterior distribution of the parameter, then compute its mean. For a standard posterior, use the known mean formula, such as α/λ for a Gamma distribution.

What is expected posterior loss?

It is the average of the loss function over the posterior distribution of the parameter, for a given estimate. The Bayes estimator is the estimate that makes this smallest.

Is the Bayes estimator always the mean?

No. It is the mean only under quadratic loss. A different loss function gives a different summary of the same posterior.