Skip to content

FRM Part I · FRM Exam Part I · Machine Learning and Prediction

A fraud model is evaluated on 10,000 transactions, of which 100 are fraudulent. A naive model that labels every transaction as non-fraud is compared to a real model with 90% accuracy that catches 80 of the frauds. Which statement is most accurate?

The naive model achieves 99% accuracy yet zero recall, since it never identifies a fraud. With highly imbalanced classes, accuracy is a poor measure, and metrics such as recall, precision or F1 better reflect usefulness.

  1. AThe naive model has 99% accuracy but zero recall, so accuracy is a poor metric hereCorrect
  2. BThe naive model has 99% accuracy and perfect recall
  3. CThe real model is worse because its accuracy is lower, so it should be chosen less
  4. DThe naive model has 1% accuracy because it detects no fraud

Explanation

The naive model is correct on 9,900 of 10,000 transactions, giving 99% accuracy, but it catches none of the 100 frauds, so recall is 0. With imbalanced classes accuracy misleads; recall, precision or F1 are better. The other options misstate these values.

Did you get it right without looking?

One question tells you little. A timed set on Machine Learning and Prediction shows your real accuracy, how long you take and where you lose marks.

Practise Machine Learning and PredictionFree diagnostic mock

More Machine Learning and Prediction questions