CFA Level II Exam · Machine Learning

Overview of Machine Learning and Overfitting for CFA Level II

Updated 6 October 2026 · Fact-checked

Machine learning lets algorithms find patterns in data without being explicitly programmed with rules. Supervised learning uses labeled targets, unsupervised learning finds structure without targets, and deep learning uses multilayer neural networks. Overfitting means the model fits noise in training data and fails on new data. You control it with validation, cross-validation and simpler models.

Understand Overview of Machine Learning and Overfitting

Machine learning (ML) uses algorithms that learn patterns from data and then make predictions or find structure. Unlike a classic regression where you pick the form, ML often lets the data decide the form. In a vignette, you are usually asked to identify the type of learning, spot a fitting problem, or pick a remedy.

Supervised learning uses labeled data: inputs (features) and a known target. If the target is continuous, it is regression. If the target is categorical, it is classification. Unsupervised learning has no target. It finds structure, such as grouping similar firms (clustering) or reducing many variables to a few (dimension reduction). Deep learning uses neural networks with many hidden layers and handles complex tasks like image and speech recognition. Reinforcement learning learns by trial and error to maximize a reward. Deep learning and reinforcement learning can be used with or without labels, so read the vignette.

Overfitting happens when a model is too complex for the data. It learns noise as if it were signal. It looks excellent in the training sample but performs poorly out of sample. Underfitting is the opposite: the model is too simple and misses real patterns. Overfit models have low bias but high variance. Underfit models have high bias and low variance.

To detect overfitting, split the data. The training sample fits the model. The validation sample tunes it and compares versions. The test sample gives a final, unbiased check of out-of-sample performance. When data is scarce, use k-fold cross-validation: split the data into k equal parts, train on k − 1 parts, validate on the remaining one, repeat k times so each part is used once as validation, then average the results.

To reduce overfitting, use simpler models, penalize complexity (regularization), prune trees, or use more data. The goal is a model that generalizes, not one that wins in-sample.

Key formulas to remember

Generalization error decomposition: Out-of-sample error = bias error + variance error + base error; Bias is from a model too simple. Variance is from a model too sensitive to the training sample. Base error is random noise that no model removes.
Overfitting signature: Low training error + high validation/test error; Underfitting shows high error in both training and validation samples.
k-fold cross-validation: Each of k folds is the validation set once; training uses k − 1 folds; average the k results; Used when data is limited. Each observation is used for validation exactly once.
Learning type rule: Labeled target → supervised; no target → unsupervised; Continuous target = regression; categorical target = classification.

How to solve Overview of Machine Learning and Overfitting questions

Use this sequence for any item on ML types or fitting problems.

1Find the data description in the vignette. Is there a labeled target variable?
2If a target exists, classify it as supervised. Decide regression (continuous target) or classification (categorical target). If none exists, it is unsupervised (clustering or dimension reduction).
3Check whether the model uses many layers of neurons. If so, it is deep learning.
4Compare performance across samples. Low training error but high validation error means overfitting. High error in both means underfitting.
5Map the problem to bias and variance: overfit equals high variance, underfit equals high bias.
6Pick the remedy: for overfitting, simplify, regularize, prune or add data; for underfitting, add complexity or features.
7For sample questions, assign roles: train to fit, validate to tune, test for final unbiased evaluation. Use k-fold if data is small.

Quickest way: Target, then gap

When to use it: Use when you have about a minute and the question asks for a learning type or a fitting diagnosis.

Ask: is there a labeled target? Yes means supervised, no means unsupervised.
Compare training and validation errors. A big gap means overfitting; both high means underfitting.
Match the answer: overfit means high variance and simplify; underfit means high bias and add complexity.

Common mistakes in Overview of Machine Learning and Overfitting

Calling clustering a supervised method because groups are the output.
The output looks like labels.
Fix: Supervised needs labels in the training data. Clustering creates groups without labels, so it is unsupervised.
Saying an overfit model has high bias.
Mixing up which error comes from which problem.
Fix: Overfit means low bias, high variance. Underfit means high bias, low variance.
Judging a model by its training performance.
High in-sample fit feels like success, as in regression R-squared.
Fix: Judge by out-of-sample performance on validation and test samples.
Using the test sample to tune the model.
Confusing validation and test roles.
Fix: Tune on validation data. Keep the test sample untouched for a final unbiased check.
Thinking k-fold means the model is trained on only one fold.
Misreading k and k − 1.
Fix: Train on k − 1 folds, validate on one, repeat k times, then average.

Worked examples

Example 1

Vignette: An analyst has data on 600 listed companies. Each record has ten financial ratios and a label showing whether the company defaulted within two years. She builds a model to predict default for new companies. Separately, she groups the same companies by ratio profile with no use of the default label. Q1: What type of learning is the default model? Q2: What type is the grouping exercise?

Show the solution

Q1: The default model uses a labeled target (default or not).
The target is categorical, so this is supervised learning, specifically classification.
Q2: The grouping uses no label and finds structure in the ratios.
That is unsupervised learning, specifically clustering.

Answer: Q1: Supervised learning (classification). Q2: Unsupervised learning (clustering).

Example 2

Vignette: A fund builds a complex model to forecast returns. On the training sample, the error rate is 4%. On the validation sample, it is 31%. A simpler model has 18% training error and 20% validation error. Q1: What problem does the complex model show? Q2: Which model should the fund prefer, and why? Q3: With 5-fold cross-validation, how many times is each fold used for validation, and how many folds train each run?

Show the solution

Q1: The complex model has a very low training error and a very high validation error. That gap signals overfitting, with high variance.
Q2: The simpler model has validation error of 20% versus 31%. Out-of-sample performance matters, so prefer the simpler model.
Q3: In k-fold cross-validation each fold is the validation set exactly once. With k = 5, each run trains on k − 1 = 4 folds.

Answer: Q1: Overfitting (high variance). Q2: The simpler model, because its validation error is lower (20% vs 31%). Q3: Each fold validates once; each run trains on 4 folds.

Exam tips

Start every item by asking whether the data has a labeled target. This settles most classification questions.
Quote the training-validation gap in your reasoning. Exam options often differ only in bias vs variance wording.
Remember the sample roles: train, validate, test. A trap option lets the test sample guide tuning.
Do not assume deep learning is always supervised or always unsupervised. Use the vignette detail.
In k-fold questions, count carefully: k runs, k − 1 training folds each.

Overview of Machine Learning and Overfitting in other exams

The same ground in other exams, if you are preparing for more than one or want another angle on it.

Overview of Machine Learning and Overfitting: frequently asked questions

What is the difference between supervised and unsupervised learning?

Supervised learning trains on data with a labeled target and predicts that target for new data. Unsupervised learning has no target and looks for structure, such as clusters or a smaller set of factors.

What is the bias-variance tradeoff?

Simple models have high bias and low variance, so they underfit. Complex models have low bias and high variance, so they overfit. You look for the complexity that gives the lowest out-of-sample error.

How does k-fold cross-validation work?

You split the data into k equal folds. You train on k − 1 folds and validate on the remaining fold, repeating until each fold has been the validation set once. You then average the k results to estimate performance.

How do you reduce overfitting?

Use a simpler model, regularize or penalize complexity, prune trees, collect more data, and use cross-validation to check out-of-sample performance.