Skip to content

FRM Part II · FRM Exam Part II · Case Study: Model Risk and Model Validation

A validator benchmarks an internal pricing model against a vendor model for 100 exotic trades. The mean absolute price difference is 0.4% of notional, but for the 10 trades with the longest maturities the difference averages 3.0%. What is the most appropriate validation response?

The validator should investigate the long-maturity divergence and consider limits or reserves meanwhile. A small aggregate difference hides a 3.0% gap on the long-dated trades. Benchmarking shows differences, not which model is right, so discarding outliers or simply swapping models would be unjustified.

  1. AAccept the model, because the overall mean difference is small
  2. BReplace the internal model with the vendor model without further analysis
  3. CInvestigate the long-maturity divergence, as the aggregate hides a material weakness, and consider limits or adjustments until resolvedCorrect
  4. DDiscard the 10 long-dated trades as outliers

Explanation

Aggregate averages can mask concentrated errors. The weighted overall figure is (90 x about 0.07% + 10 x 3.0%)/100 is roughly 0.4%, consistent with the data, yet the long-dated subset is materially off. Benchmarking does not prove which model is right, so root cause must be found and compensating controls such as model reserves or limits applied.

Did you get it right without looking?

One question tells you little. A timed set on Case Study: Model Risk and Model Validation shows your real accuracy, how long you take and where you lose marks.

Practise Case Study: Model Risk and Model ValidationFree diagnostic mock

More Case Study: Model Risk and Model Validation questions