Skip to content

FRM Part I · FRM Exam Part I · Machine Learning and Prediction

A quantitative analyst fits a very flexible model to 500 observations. It achieves a training mean squared error of 0.2 but a test mean squared error of 1.9 on held-out data. A simpler model has training MSE of 0.9 and test MSE of 1.0. Which conclusion is best supported?

The flexible model is overfitting: its training error is very low but its test error is much higher. Since prediction quality is judged out of sample, the simpler model with test MSE of 1.0 beats 1.9 and should be preferred.

  1. AThe flexible model is underfitting and has high bias
  2. BThe flexible model is overfitting and the simpler model should be preferred for predictionCorrect
  3. CBoth models are equally good because training errors are low
  4. DThe simpler model is overfitting because its training error is higher

Explanation

A large gap between low training error and much higher test error signals overfitting (high variance). Out-of-sample performance is the relevant criterion, and the simpler model's test MSE of 1.0 is lower than 1.9. Higher training error alone does not imply overfitting.

Did you get it right without looking?

One question tells you little. A timed set on Machine Learning and Prediction shows your real accuracy, how long you take and where you lose marks.

Practise Machine Learning and PredictionFree diagnostic mock

More Machine Learning and Prediction questions