Skip to content

ACCA Strategic Professional · Strategic Business Leader · Machine learning, AI and robotics

Orion Logistics develops a deep learning model that predicts delivery delays. It achieves 99% accuracy on the historical data used to build it, but accuracy falls to 70% when used on new live shipments. The data scientist says the model captured noise in the training set. Which conclusion and response is most appropriate?

The model is overfitted: it learned noise in the training data, so it performs well historically but poorly on new shipments. The response is to validate on unseen data, use more varied data and simplify or regularise the model. Underfitting would show weak results on training data too.

  1. AThe model is underfitted; the company should simplify it further and use less data
  2. BThe model is overfitted; the company should test on separate unseen data, use more varied data and simplify or regularise the modelCorrect
  3. CThe model is unsupervised; the company should add labels to the live data only
  4. DThe model shows the black box problem has been solved because accuracy was high initially

Explanation

High accuracy on training data with poor performance on new data is the classic sign of overfitting, where noise is learned as if it were signal. Remedies include validating on separate hold-out data, broader training data and reducing complexity. Underfitting would show poor performance on both training and new data, so simplifying further would worsen matters.

Did you get it right without looking?

One question tells you little. A timed set on Machine learning, AI and robotics shows your real accuracy, how long you take and where you lose marks.

Practise Machine learning, AI and roboticsFree diagnostic mock

More Machine learning, AI and robotics questions