Supervisory Guidance on Model Risk Management: formula sheet

Key formulas

Definition of a model (SR 11-7)

Model = inputs → processing (theory and assumptions) → quantitative estimates → reports

Three components: input, processing and reporting. The output must be a quantitative estimate.

Definition of model risk

Model risk = potential adverse consequences from decisions based on incorrect or misused model outputs

Consequences include financial loss, poor decisions and reputational damage.

Two sources of model risk

Model risk = fundamental errors (design, implementation) + inappropriate use

Error means the model is wrong for its stated purpose. Misuse means a sound model is applied wrongly.

Drivers of the level of model risk

Higher complexity, input uncertainty, breadth of use and impact → higher model risk

Used to decide how much validation and control effort a model needs.

Sources of model risk (SR 11-7)

Model risk = fundamental errors in the model + incorrect or inappropriate use

A sound model used outside its intended purpose still creates model risk.

Core development elements

Purpose → Design, theory and data → Testing → Implementation → Use and monitoring → Documentation

Use this as a checklist to find the missing step in a case question.

Developmental testing set

Accuracy, robustness and stability; sensitivity and stress tests; outcomes analysis and benchmarking

Testing must cover the range of conditions and the actual intended use.

Documentation standard

Detailed enough that a knowledgeable third party could understand and replicate the model

It should cover purpose, theory, data, assumptions, limitations, testing and controls.

Three core elements of validation (SR 11-7)

Validation = conceptual soundness + ongoing monitoring + outcomes analysis

Back-testing belongs to outcomes analysis. Benchmarking is used in ongoing monitoring and outcomes analysis.

Effective challenge requirements

Competence + independence + influence (incentives and standing)

All three are needed. Challenge can come from validators, internal audit or other informed staff.

Back-test comparison

Forecast vs realised outcome over time

Tests accuracy against what actually happened. Benchmarking instead compares against another model.

Validation frequency

Periodic, risk-based, plus event-driven review

SR 11-7 gives no fixed interval. Higher risk and material change mean more frequent review.

Model risk definition (SR 11-7)

Model risk = adverse consequences from decisions based on incorrect or misused model outputs

Two sources: fundamental errors in the model, and incorrect or inappropriate use.

Board responsibility

Board = sets risk appetite + ensures framework + reviews reports

Board does not validate or approve every model.

Senior management responsibility

Senior management = policies + procedures + roles + compliance + validation resources + reporting to board

Day-to-day implementation of the framework.

Internal audit responsibility

Internal audit = independent assessment of the whole framework, including validation activity

Checks that validation is done and policies are followed; does not replace validation.

Model inventory content

Inventory = all models (incl. vendor) + purpose + owner + developer + validation status + limitations + uses

Must be comprehensive and kept current.

Documentation standard

Documentation detailed enough for an informed third party to understand the model

Covers theory, assumptions, data, limitations and testing.

Model risk (SR 11-7)

Model risk = risk of adverse consequences from decisions based on incorrect or misused model outputs

Two sources: fundamental errors, and incorrect or inappropriate use.

Vendor model principle

Vendor model = same SR 11-7 standards as an in-house model

Responsibility cannot be outsourced. The bank must understand, validate and monitor the model.

Materiality-based tiering (rule of thumb)

Tier ↑ as exposure, decision importance and model uncertainty ↑

Higher tier means more frequent, deeper validation and tighter oversight. SR 11-7 requires a risk-based approach but does not set fixed tiers.

Aggregate view

Aggregate model risk ≠ simple sum of individual model risks

Consider shared inputs, assumptions and dependencies between models, and offsetting or compounding errors.

Quick revision

• Model risk is the potential for adverse consequences from decisions based on incorrect or misused model outputs.
• Two sources: fundamental errors in the model, and incorrect or inappropriate use.
• A model is a quantitative method that turns input data into quantitative estimates, using statistical, economic, financial or mathematical theories.
• Model risk rises with greater complexity, more uncertainty, broader use and larger potential impact.
• Validation has three core elements: conceptual soundness, ongoing monitoring, and outcomes analysis.
• Backtesting compares model forecasts with actual results and is part of outcomes analysis.
• Effective challenge needs competence, influence and incentives, and independence from model development.
• Validation should be done by staff independent of development and use, with enough standing to challenge.
• The board and senior management set direction and oversee model risk; they do not build the models.
• A model inventory and clear documentation are core governance controls.
• Vendor models still need validation, and the bank must understand their limits even when the code is proprietary.
• Limits and sound judgment are tools to manage model risk, and models should be used within their intended purpose.

Common mistakes

• Saying model risk comes only from coding or mathematical errors. Fix: Remember the second source: a correct model used inappropriately also creates model risk.
• Treating misuse as the same as error. Fix: Error is a flaw in the model against its design objective. Misuse is applying it outside its intended purpose or conditions.
• Thinking model risk comes only from coding or math errors. Fix: Remember the two sources: fundamental errors and incorrect or inappropriate use.
• Treating a good back-test as proof the model is sound. Fix: Back-testing is one test. Robustness, sensitivity, assumption checks and benchmarking are also needed.
• Treating back-testing and benchmarking as the same Fix: Back-testing compares to realised outcomes. Benchmarking compares to another model or estimate.
• Saying validation is only statistical testing Fix: Conceptual soundness review of theory, data and assumptions is a core element, not optional.
• Saying internal audit validates models Fix: Validation challenges the model. Audit evaluates whether the validation process and wider framework work and are followed.
• Giving the board a hands-on role in approving each model Fix: The board sets appetite, ensures the framework and reviews reports. Senior management handles detailed implementation.
• Thinking vendor validation replaces the bank's own validation. Fix: Vendor evidence is an input only. The bank must still test on its own data, portfolio and use.
• Saying proprietary code means a vendor model is exempt from SR 11-7. Fix: Opacity increases the need for outcomes analysis, benchmarking, sensitivity tests and contingency plans.

Exam tips

• Expect short scenario questions that ask you to label a failure as error or misuse. Decide using design purpose versus actual use.
• Learn the SR 11-7 wording: a model is a quantitative method that turns input data into quantitative estimates, and model risk comes from incorrect or misused outputs.
• Reject options saying model risk can be eliminated or that it is only a developer issue.
• Remember the drivers: complexity, input uncertainty, breadth of use and potential impact. They set how much control a model needs.
• Watch for options that confuse model risk with market or credit risk. Model risk is about the decision made from the output.
• Questions are case-based. Locate the lifecycle stage first, then match the remedy.
• Watch for stems where testing looks good but data or use is the real flaw.
• Expect absolute wording traps: eliminate, guarantee, always.