Skip to content

FRM Part II · FRM Exam Part II · Credit Scoring and Rating

A bank's rating model was developed on data from an economic expansion. A validator finds that the model's AUC on the development sample is 0.88, while on a recent out-of-time sample covering a downturn it is 0.74, and the average predicted PD is well below the realized default rate. Which interpretation and action is most appropriate?

The sharp AUC drop out of time points to overfitting or changed population characteristics, and PDs below realized defaults show weak calibration too. The validator should investigate both discrimination and calibration and consider redevelopment, rather than excusing the result as cyclical or a sample-size artifact.

  1. AThe model shows likely overfitting or population instability, so it should be investigated and possibly redeveloped, and the PD calibration should be reviewedCorrect
  2. BDiscriminatory power has fallen but calibration is fine, so only the rank ordering needs adjusting
  3. CThe result is expected from a through-the-cycle model, so no action is needed
  4. DThe drop in AUC proves the validation sample is too small and should be ignored

Explanation

A material AUC fall out of time signals weaker discrimination, suggesting overfitting or a shift in the population. Predicted PDs below realized defaults show a calibration failure as well. Both dimensions need review. Dismissing it as through-the-cycle behavior does not explain the poor ranking performance.

Did you get it right without looking?

One question tells you little. A timed set on Credit Scoring and Rating shows your real accuracy, how long you take and where you lose marks.

Practise Credit Scoring and RatingFree diagnostic mock

More Credit Scoring and Rating questions