FRM Exam Part II · Case Study: Model Risk and Model Validation

Model Validation Process and Techniques for FRM Part 2

Updated 11 October 2026 · Fact-checked

Model validation is an independent check that a model is sound and fit for its use. SR 11-7 sets three core elements: evaluation of conceptual soundness, ongoing monitoring, and outcomes analysis, which includes backtesting. Benchmarking compares the model with alternatives. In exam questions, match each technique to the weakness it exposes.

Understand Model Validation Process and Techniques

A model turns inputs and assumptions into estimates a bank uses to decide. Model risk is the chance of loss or bad decisions because the model is wrong or misused. Model validation is how a bank tests for that risk. SR 11-7 (the US supervisory guidance on model risk management) describes validation as the set of activities that check a model performs as intended and is fit for purpose.

SR 11-7 names three core elements of validation. Evaluation of conceptual soundness reviews the design, theory, data, assumptions and documentation, and checks developmental testing such as sensitivity analysis. Ongoing monitoring confirms the model is still appropriate as products, markets and exposures change, and that it is implemented and used correctly. Outcomes analysis compares model outputs with actual results. Backtesting is one form of outcomes analysis.

Benchmarking compares the model with other models or estimates, such as an alternative methodology or a vendor model. It is useful when outcomes are hard to observe. Differences are a signal to investigate, not proof that the model is wrong.

A key distinction is verification versus validation. Verification checks the model is built and coded as intended, with correct implementation and calculations. Validation asks the wider question: is the model conceptually sound and suitable for its use? A model can be verified perfectly and still be a poor model.

Validation must be done by people with independence from development and with enough competence and influence to give effective challenge. Validation is also risk-based: higher-impact or more complex models get deeper work and more frequent review. Finally, limits matter. Validation should identify limitations and assumptions, and the findings should be reported to governance.

Key formulas to remember

Three core elements of validation (SR 11-7): Validation = conceptual soundness + ongoing monitoring + outcomes analysis; Backtesting sits inside outcomes analysis. Benchmarking is a supporting comparison tool.
Backtest exception rate: Exception rate = number of exceptions ÷ number of observations; Compare with the expected rate, 1 − confidence level. For 99% VaR, expect 1%.
Expected exceptions: Expected exceptions = N × (1 − c); N is the number of days and c the VaR confidence level. A large gap from this value suggests model weakness.
Verification vs validation: Verification: built as intended? Validation: fit for purpose?; Verification is narrower and focuses on implementation and calculation accuracy.

How to solve Model Validation Process and Techniques questions

Use this approach for any question on validation techniques or process.

1Identify what the question describes: design review, live tracking, comparison to actuals, or comparison to another model.
2Map it to the element: conceptual soundness, ongoing monitoring, outcomes analysis (backtesting), or benchmarking.
3Check whether the issue is implementation (verification) or suitability for use (validation).
4If numbers are given, compute the expected count or rate and compare with the observed one.
5Decide what the result implies: acceptable, needs investigation, or model change. Remember differences are signals.
6Check independence, effective challenge and risk-based depth against SR 11-7 principles.
7Choose the option that is precise and does not overstate what the technique proves.

Quickest way: Match the technique to the question

When to use it: Use when a multiple-choice item lists several validation activities and asks which fits.

Reviewing theory, assumptions, data: conceptual soundness.
Tracking performance over time or after changes: ongoing monitoring.
Model output versus actual results: outcomes analysis or backtesting.
Model versus an alternative model or estimate: benchmarking.
Checking code or calculations: verification, not full validation.

Common mistakes in Model Validation Process and Techniques

Treating verification and validation as the same thing
Both sound like checking the model.
Fix: Verification checks implementation against intent. Validation also tests conceptual soundness and fitness for use.
Saying backtesting is a separate element from outcomes analysis
Backtesting is the better-known term.
Fix: Treat backtesting as one type of outcomes analysis under SR 11-7.
Assuming benchmarking proves which model is right
Agreement between models feels like confirmation.
Fix: Benchmarks only show differences to investigate. Both models can share the same flaw.
Thinking passing a backtest means the model is sound
A good backtest looks like full evidence.
Fix: Backtests have limited power and say little about assumptions or use. Conceptual review and monitoring are still needed.
Letting the developer validate the model
Developers know the model best.
Fix: Validation needs independence from development and use, plus the standing to give effective challenge.

Worked examples

Example 1

A bank backtests a 99% one-day VaR over 250 trading days and records 7 exceptions. Compute the expected number and exception rate, and say which validation element this belongs to.

Show the solution

Expected exceptions = 250 × (1 − 0.99) = 250 × 0.01 = 2.5.
Observed exception rate = 7 ÷ 250 = 0.028, or 2.8%.
Expected rate is 1%, so the observed rate is nearly three times higher.
Comparing model output with actual P&L is outcomes analysis, with backtesting as the technique.

Answer: Expected exceptions are 2.5 and the observed rate is 2.8% versus 1%. This is outcomes analysis (backtesting), and the gap calls for investigation of the model.

Example 2

A validator finds that a pricing model's code correctly implements the documented formula, but the formula assumes constant volatility for a product with a strong volatility smile. Which statement is most accurate?

Show the solution

Code matching the documented formula shows verification is satisfactory.
The constant volatility assumption does not suit the product, which is a design issue.
Design, assumptions and fit to the product are assessed in conceptual soundness review.
So implementation is fine, but the model fails validation for this use unless limits or adjustments are made.

Answer: Verification passes, but conceptual soundness fails for this use. The model is not validated as fit for purpose without remediation or documented limitations.

Exam tips

Learn the three SR 11-7 elements by name and place backtesting under outcomes analysis.
Read the stem for the evidence type: theory, live tracking, actual results or another model.
Watch for options that say a technique proves the model correct. Those are usually wrong.
Remember independence, effective challenge and risk-based depth as supporting principles.
For backtest numbers, compute N × (1 − c) first and compare.

Practice questions from Case Study: Model Risk and Model Validation

Model Validation Process and Techniques in other exams

The same ground in other exams, if you are preparing for more than one or want another angle on it.

Model Validation Process and Techniques: frequently asked questions

What are the key components of model validation under SR 11-7?

SR 11-7 describes evaluation of conceptual soundness, ongoing monitoring, and outcomes analysis. Backtesting is part of outcomes analysis. Benchmarking is a common supporting technique.

What is the difference between model verification and model validation?

Verification checks the model is implemented and calculated as intended. Validation is broader and asks whether the model is conceptually sound and suitable for its intended use.

How does benchmarking differ from backtesting?

Backtesting compares model outputs with realised outcomes. Benchmarking compares the model with other models or estimates. Benchmarking is helpful when outcomes are scarce or hard to observe.

Who should perform model validation?

People independent of model development and use, with enough knowledge and standing to challenge the model effectively. Depth of work should reflect the model's risk and importance.