FRM Part I · FRM Exam Part I · Machine-Learning Methods
A model predicting credit losses achieves a training mean squared error of 0.5 and a validation mean squared error of 4.8 on held-out data. Which conclusion and remedy is most consistent with these results?
The pattern indicates overfitting, since training error is very low while validation error is far higher. Remedies include regularization or a simpler model. Underfitting would instead produce high error on both the training and validation samples.
- AUnderfitting; add more complexity to the model
- BOverfitting; apply regularization or simplify the modelCorrect
- CGood fit; the gap shows the validation set is too small
- DUnderfitting; reduce the size of the training set
Explanation
A very low training error combined with a much higher out-of-sample error signals overfitting: the model has learned noise in the training data. Regularization, reducing complexity or gathering more data helps. Underfitting would show high error on both samples.
Did you get it right without looking?
One question tells you little. A timed set on Machine-Learning Methods shows your real accuracy, how long you take and where you lose marks.
More Machine-Learning Methods questions
- A logistic regression for default has the form ln(p/(1-p)) = -3.0 + 0.5 x, where x is the debt-to-income ratio expressed in percentage point…
- In a KNN classifier, an analyst moves from K = 1 to K = 25 on a noisy credit dataset. Which is the expected effect?
- A bank has transaction records for 200,000 corporate clients with no predefined categories. The risk team applies k-means to group clients w…
- A ridge regression with one standardized predictor and no intercept has the OLS slope estimate of 1.20, where the predictor's sum of squares…
- A data scientist standardizes a feature using the training set, which has a mean of 40 and a standard deviation of 8. A test observation has…
- A deep neural network for credit scoring achieves very low training error but much higher validation error. Which single action is most dire…