Skip to content

IAI Actuarial Core Principles · Risk Modelling and Survival Analysis · Elementary principles of machine learning

A model is fitted with a penalty on the sum of squared coefficients, with penalty weight λ. Training error is 4.0 at λ=0 and validation error is 9.0. As λ is increased from 0 to a moderate value, which pattern is most typical when the model was originally overfitting?

Training error typically rises slightly while validation error falls. The penalty shrinks coefficients, adding a little bias but cutting variance, which helps an overfitted model generalise, until λ becomes so large that the model begins to underfit.

  1. ATraining error rises slightly and validation error falls, as variance is reduced at the cost of a little biasCorrect
  2. BTraining error falls and validation error rises, as bias is reduced
  3. CBoth training and validation errors fall continuously for every λ
  4. DBoth errors stay unchanged because the penalty affects only the intercept
  5. Training error falls sharply and validation error is unchanged

Explanation

A ridge penalty shrinks coefficients, so the fit to training data worsens slightly (more bias) but variance falls. For an overfitted model this usually lowers validation error until λ becomes too large. Training error cannot fall when a constraint is tightened on the same objective's fit term.

Did you get it right without looking?

One question tells you little. A timed set on Elementary principles of machine learning shows your real accuracy, how long you take and where you lose marks.

Practise Elementary principles of machine learningFree diagnostic mock

More Elementary principles of machine learning questions