Skip to content

FRM Part II · FRM Exam Part II · Case Study: Model Risk and Model Validation

During validation of a loan-loss model, a validator finds that the model performs well on its development sample but the reviewer is unsure whether it generalises. The validator re-runs the model on data from a later period not used in estimation and compares predicted losses with realised losses. Within the SR 11-7 validation framework, this activity is best classified as:

This is outcomes analysis. Comparing model predictions with realised losses on data not used in estimation is out-of-sample backtesting, which SR 11-7 places under outcomes analysis, distinct from conceptual soundness review and from ongoing monitoring or process verification activities.

  1. AConceptual soundness evaluation of theory and assumptions
  2. BOutcomes analysis, including out-of-sample backtestingCorrect
  3. CGovernance review of board reporting lines
  4. DProcess verification of data-feed controls only

Explanation

SR 11-7 lists three core validation elements: evaluation of conceptual soundness, ongoing monitoring (including process verification and benchmarking), and outcomes analysis (comparing outputs with actual outcomes, such as backtesting). Comparing predictions with realised losses on out-of-sample data is outcomes analysis.

Did you get it right without looking?

One question tells you little. A timed set on Case Study: Model Risk and Model Validation shows your real accuracy, how long you take and where you lose marks.

Practise Case Study: Model Risk and Model ValidationFree diagnostic mock

More Case Study: Model Risk and Model Validation questions