Skip to content

FRM Part I · FRM Exam Part I · Enterprise Risk Management and Future Trends

A risk team backtests a machine-learning default model. It performs very well on the data used to build it but poorly on new data after an economic regime change. Which diagnosis and response are most appropriate?

The pattern indicates overfitting made worse by a regime change. The appropriate response is out-of-sample validation, regular recalibration and challenger models. Simplifying further treats underfitting, ignoring results is unjustified, and using more data from the old regime repeats the problem.

  1. AThe model is underfit; remove variables and simplify it further
  2. BThe model is likely overfit and exposed to regime change; use out-of-sample validation, regular recalibration and challenger modelsCorrect
  3. CThe model is unbiased; the poor results are purely random noise and should be ignored
  4. DThe model should be replaced with one that uses only more historical data from the same regime

Explanation

Strong in-sample but weak out-of-sample performance signals overfitting, and a regime change worsens it because patterns learned no longer hold. Out-of-sample testing, recalibration and challenger models address this. Simplifying further treats underfitting, which is not the issue, and ignoring the results is unjustified.

Did you get it right without looking?

One question tells you little. A timed set on Enterprise Risk Management and Future Trends shows your real accuracy, how long you take and where you lose marks.

Practise Enterprise Risk Management and Future TrendsFree diagnostic mock

More Enterprise Risk Management and Future Trends questions