FRM Exam Part I · Hypothesis Testing

Chebyshev's Inequality and VaR Backtesting Hypothesis Tests

Updated 11 October 2026 · Fact-checked

Chebyshev's inequality says that for any distribution with a finite mean and variance, the probability of falling k or more standard deviations from the mean is at most 1/k², for k > 0. Backtesting applies hypothesis tests to VaR exceptions: you count exceptions and test whether the count fits the stated confidence level.

Understand Chebyshev's Inequality and Backtesting Applications

Sometimes you know a mean and a standard deviation but not the shape of the distribution. Chebyshev's inequality still gives a limit on tail probability. It is a bound, not an estimate. It holds for any distribution with a finite mean and variance.

The bound is P(|X − μ| ≥ kσ) ≤ 1/k², for k > 0. At k = 2 the bound is 25%. At k = 3 it is about 11.1%. For a normal distribution the true probabilities are about 4.6% and 0.27%. Chebyshev is loose because it must work for every distribution. It is only informative for k > 1, since for k ≤ 1 the bound is 1 or more.

The empirical rule (68-95-99.7) is different. It applies only to a normal distribution, and it gives approximate probabilities, not bounds. Use Chebyshev when the shape is unknown or fat-tailed. Use the empirical rule only when normality is given.

Backtesting checks whether a VaR model is reliable. At confidence level c, an exception is a day when the loss exceeds VaR. If the model is right, each day has exception probability p = 1 − c, and days are independent. The number of exceptions x in T days is then binomial with parameters T and p.

The hypothesis test sets H0: the model is correct (exception probability equals p). You reject if x is too high (model understates risk) or too low (model is too conservative). Two errors matter. Type I rejects a correct model. Type II accepts a bad model. Lowering one raises the other for a fixed sample size. With a normal approximation, z = (x − pT) ÷ √(p(1 − p)T). The Basel traffic-light approach is a supervisory use of the same idea, with green, yellow and red zones based on exception counts.

Key formulas to remember

Chebyshev's inequality: P(|X − μ| ≥ kσ) ≤ 1/k²; Valid for any distribution with finite mean and variance, k > 0. Gives an upper bound only.
One-sided tail bound (Cantelli): P(X − μ ≥ kσ) ≤ 1 ÷ (1 + k²); Tighter when you need only one tail. Do not just halve 1/k²; that is not a valid general rule.
Minimum mass within k standard deviations: P(|X − μ| < kσ) ≥ 1 − 1/k²; Complement of the Chebyshev bound. At k = 2 at least 75%.
Exception probability: p = 1 − c; For a 99% VaR, p = 1%. Expected exceptions = p × T.
Binomial probability of x exceptions: P(X = x) = C(T, x) × p^x × (1 − p)^(T − x); Use for exact backtest probabilities.
Normal approximation z-score: z = (x − pT) ÷ √(p(1 − p)T); Compare with the critical z (e.g. 1.96 two-sided at 5%). Less reliable when pT is small.
Kupiec unconditional coverage (POF) test statistic: LR = −2 ln[(1 − p)^(T − x) × p^x] + 2 ln[(1 − x/T)^(T − x) × (x/T)^x]; Compared with a chi-squared distribution with 1 degree of freedom; 5% critical value is 3.84.

How to solve Chebyshev's Inequality and Backtesting Applications questions

Decide first whether the question is a distribution-free bound or a backtest. Then follow the steps.

1Identify what is given: mean and standard deviation only (Chebyshev), or VaR confidence level, sample size and exception count (backtest).
2For Chebyshev, convert the distance from the mean into k = distance ÷ σ.
3Apply the bound 1/k², and state that it is a maximum probability. Use 1 − 1/k² for the minimum probability inside the interval.
4For a backtest, compute p = 1 − c and the expected exceptions pT.
5State H0 (model correct) and the tails: too many exceptions means risk is understated; too few means the model is too conservative.
6Compute z = (x − pT) ÷ √(p(1 − p)T), or use the binomial or Kupiec statistic if the question gives it.
7Compare with the critical value and decide whether to reject H0.
8Interpret in words: model rejected or not, and what error type risk applies.

Quickest way: Fast checks for Chebyshev and backtests

When to use it: Use when you have about two minutes per question and the numbers are simple.

For Chebyshev, find k, square it and invert. Then check the question asks for 'at most' or 'at least'.
If the question mentions a normal distribution, Chebyshev is not what is asked. Use normal values.
For backtests, compute expected exceptions pT first. If x is close to it, the model is usually not rejected.
Estimate the standard deviation √(p(1 − p)T) and see how many of them x is from pT. Beyond about 2 usually means rejection at 5%.
With the Kupiec LR, compare to 3.84 at 5%, and skip the algebra if the statistic is given.

Common mistakes in Chebyshev's Inequality and Backtesting Applications

Treating the Chebyshev bound as the actual probability.
The result looks like a probability, so students quote 25% at k = 2 as the answer.
Fix: Say 'at most 25%'. The true value can be much smaller.
Applying Chebyshev with k ≤ 1 and expecting a useful answer.
Students plug in without checking k.
Fix: If k ≤ 1 the bound is 1 or more and says nothing useful.
Halving the bound for one tail.
It is assumed the distribution is symmetric, as with the normal.
Fix: Chebyshev is two-sided and the distribution may be skewed. Halving is valid only if symmetry is given. Cantelli gives a general one-sided bound.
Confusing Chebyshev with the empirical rule.
Both talk about standard deviations from the mean.
Fix: The empirical rule needs normality and gives approximate values. Chebyshev works for any distribution and gives a bound.
Using p = c instead of p = 1 − c in the backtest.
The confidence level is the number in the question.
Fix: Exception probability is 1 minus the confidence level. A 99% VaR has p = 0.01.
Rejecting only when exceptions are too high.
Students focus on risk understatement.
Fix: Too few exceptions also signals a problem, usually an overly conservative model that ties up capital. A two-sided test catches both.

Worked examples

Example 1

A portfolio's daily return has a mean of 0.05% and a standard deviation of 1.5%. The distribution is unknown. Using Chebyshev's inequality, what is the maximum probability that a daily return differs from the mean by 4.5 percentage points or more?

Show the solution

Distance = 4.5%, σ = 1.5%.
k = 4.5 ÷ 1.5 = 3.
Bound = 1/k² = 1/9.
1/9 ≈ 0.1111.

Answer: At most about 11.1%.

Example 2

A bank reports a 99% one-day VaR and backtests it over 250 trading days. It records 8 exceptions. Using the normal approximation, test H0 that the model is correct at the 5% two-sided level (critical z = 1.96).

Show the solution

p = 1 − 0.99 = 0.01.
Expected exceptions = pT = 0.01 × 250 = 2.5.
Standard deviation = √(0.01 × 0.99 × 250) = √2.475 ≈ 1.573.
z = (8 − 2.5) ÷ 1.573 = 5.5 ÷ 1.573 ≈ 3.50.
3.50 > 1.96, so reject H0.
Exceptions are far above expectation, so the model probably understates risk. The normal approximation is rough when pT is as small as 2.5, but a z of 3.5 makes the conclusion clear.

Answer: z ≈ 3.50, so reject H0. The VaR model likely understates risk.

Exam tips

Read whether the question asks 'at most' or 'at least'. It decides between 1/k² and 1 − 1/k².
If a question gives a normal distribution, do not use Chebyshev unless it asks for the bound itself.
In backtests, always compute p = 1 − c and pT before anything else.
Know the logic of Type I and Type II errors: a stricter rejection threshold lowers Type I risk but raises Type II risk.
Remember the 3.84 chi-squared critical value (1 degree of freedom, 5%) for Kupiec-style tests.

Practice questions from Hypothesis Testing

Chebyshev's Inequality and Backtesting Applications: frequently asked questions

When should I use Chebyshev's inequality instead of the empirical rule?

Use Chebyshev when the distribution is unknown or fat-tailed, since it needs only a finite mean and variance. Use the empirical rule only when the distribution is normal. The empirical rule gives approximate probabilities, while Chebyshev gives a bound.

Why is Chebyshev's bound so loose?

It must hold for every possible distribution, including the worst ones. A normal distribution has far less tail mass than the bound allows. The price of generality is a weaker result.

How do you backtest VaR with hypothesis testing?

Count the days when losses exceeded VaR. Under the null, the count is binomial with exception probability 1 − c. Compare the count with the expected number using a z-score or the Kupiec likelihood ratio, and reject if it is too far out.

What does it mean if a VaR model has too few exceptions?

It suggests the model is too conservative. Reported risk is higher than needed, so capital may be used inefficiently. A two-sided test flags this as well as too many exceptions.