Skip to content

FRM Part I · FRM Exam Part I · External and Internal Credit Ratings

A validation report notes that a bank's internal rating system has a high accuracy ratio on its development sample but a much lower accuracy ratio when applied to a later period's portfolio with a different mix of borrowers. Which interpretation is most appropriate?

The pattern suggests overfitting or poor stability of the rating model, so validation should use out-of-sample and out-of-time testing. A high accuracy ratio in the development data does not show calibration, and the sharp decline later signals weak generalization.

  1. AThe system likely suffers from overfitting or instability, so out-of-sample and out-of-time validation is neededCorrect
  2. BThe system is well calibrated because discriminatory power is high in the development sample
  3. CThe drop proves that the through-the-cycle philosophy was applied incorrectly
  4. DThe result is irrelevant because accuracy ratios are not comparable across samples

Explanation

A large fall in discriminatory power outside the development sample is a classic sign of overfitting or lack of stability, which is why out-of-sample and out-of-time tests are required. Accuracy ratio does not measure calibration, and a drop does not specifically show a through-the-cycle error. Accuracy ratios can be compared only with caution because they depend on portfolio composition, but the decline is still informative.

Did you get it right without looking?

One question tells you little. A timed set on External and Internal Credit Ratings shows your real accuracy, how long you take and where you lose marks.

Practise External and Internal Credit RatingsFree diagnostic mock

More External and Internal Credit Ratings questions