Skip to content

IAI Actuarial Core Principles · Risk Modelling and Survival Analysis · Elementary principles of machine learning

An actuary fits a flexible model to 200 motor claims and finds it has a very small error on those 200 records but a much larger error on 100 further records held back. Which is the most appropriate diagnosis and response?

The model is overfitting. It fits noise in the training records, so error is low there but high on unseen data. The sensible response is to regularise or simplify the model and judge it on held-out data, not training error.

  1. AUnderfitting; make the model simpler
  2. BOverfitting; use regularisation or a simpler model and judge it on held-out dataCorrect
  3. CBias in the test set; discard the held-out records and rely on training error
  4. DOverfitting; add more parameters so training error falls further
  5. Underfitting; train longer on the same 200 records until test error equals zero

Explanation

Low training error with high error on unseen data is the signature of overfitting: the model has fitted noise. The remedy is to reduce complexity or regularise, and to assess performance on held-out data. Adding parameters would worsen the problem, and training error is not a reliable guide to predictive performance.

Did you get it right without looking?

One question tells you little. A timed set on Elementary principles of machine learning shows your real accuracy, how long you take and where you lose marks.

Practise Elementary principles of machine learningFree diagnostic mock

More Elementary principles of machine learning questions