Skip to content

FRM Part II · FRM Exam Part II · Case Study: Model Risk and Model Validation

A bank's validation team is reviewing a newly developed retail credit scorecard. Which activity best represents 'conceptual soundness' assessment, as opposed to outcomes analysis?

Conceptual soundness review examines the model's theoretical basis, variable choices, assumptions and fitness for its intended use. Comparing predictions with actual results, holdout testing and monitoring input drift are outcomes analysis or ongoing monitoring, which test performance rather than design.

  1. AComparing predicted default rates with realized default rates over the past twelve months
  2. BReviewing the model's theoretical basis, variable selection and assumptions against the intended use and available literatureCorrect
  3. CRunning the model on a holdout sample to count misclassifications
  4. DTracking monthly changes in the model's input data distribution

Explanation

Conceptual soundness evaluates design, theory, assumptions, data and methodology before relying on results. Comparing predictions to realized outcomes, holdout tests and input monitoring are outcomes analysis or ongoing monitoring activities, not review of the model's conceptual design.

Did you get it right without looking?

One question tells you little. A timed set on Case Study: Model Risk and Model Validation shows your real accuracy, how long you take and where you lose marks.

Practise Case Study: Model Risk and Model ValidationFree diagnostic mock

More Case Study: Model Risk and Model Validation questions