Skip to content

FRM Part II · FRM Exam Part II · Case Study: Model Risk and Model Validation

A bank's validation team is asked to review a newly built retail probability-of-default model. Which activity best represents 'conceptual soundness' assessment, as opposed to outcomes analysis?

Conceptual soundness review examines the model's theory, assumptions, variable selection and developmental evidence. Comparing predictions with realized defaults or counting exceedances is outcomes analysis, and rerunning code tests implementation, so only the review of theoretical basis and design choices fits conceptual soundness.

  1. AComparing predicted default rates with realized default rates over the last eight quarters
  2. BReviewing the model's theoretical basis, variable selection, assumptions and supporting developmental evidenceCorrect
  3. CRe-running the production code to confirm that the same inputs return identical outputs
  4. DCalculating the number of VaR exceedances over a 250-day window

Explanation

Conceptual soundness evaluation examines design, theory, assumptions, data and variable choices, and the documentation supporting them. Comparing predicted to realized results and counting exceedances are outcomes analysis (backtesting). Re-running code checks implementation, not conceptual design.

Did you get it right without looking?

One question tells you little. A timed set on Case Study: Model Risk and Model Validation shows your real accuracy, how long you take and where you lose marks.

Practise Case Study: Model Risk and Model ValidationFree diagnostic mock

More Case Study: Model Risk and Model Validation questions