Skip to content

IAI Actuarial Core Principles · Risk Modelling and Survival Analysis · Elementary principles of machine learning

A data scientist at a Mumbai insurer standardises all predictors using the mean and standard deviation of the full dataset, then splits the data into training and test sets and reports test accuracy. What is the main problem with this procedure?

The problem is data leakage. Scaling with the mean and standard deviation of the whole dataset lets test-set information influence training, so the reported test performance may be overstated. Scaling parameters should come from training data only.

  1. AInformation from the test set leaks into training, so test performance may be overstatedCorrect
  2. BStandardisation should never be applied before fitting a model
  3. CThe test set is too small to be useful after standardisation
  4. DStandardising makes the training error larger than the test error
  5. Standardisation causes underfitting, so accuracy is understated

Explanation

Using statistics computed from the full dataset lets information about the test observations influence the training inputs. This is data leakage, and the test result is no longer a clean estimate of performance on new data. Scaling parameters should be derived from the training set only and then applied to the test set.

Did you get it right without looking?

One question tells you little. A timed set on Elementary principles of machine learning shows your real accuracy, how long you take and where you lose marks.

Practise Elementary principles of machine learningFree diagnostic mock

More Elementary principles of machine learning questions