Skip to content

CFA Level II Exam · Backtesting and Simulation

Backtesting in Portfolio Management: Basics and Purpose

Updated 7 October 2026 · Fact-checked

Backtesting applies an investment strategy or risk model to historical data to see how it would have performed. You define the rules, run them on past data, then compare results with expectations or a benchmark. It informs portfolio construction, but past results do not guarantee future results.

Understand Backtesting Basics and Purpose

Backtesting means running a rule-based strategy or risk model on historical data as if you had followed it in the past. You ask: if I had applied these rules then, what return, risk and drawdown would I have seen?

The idea is simple. A strategy is a set of rules, such as buy the cheapest 20% of stocks by price-to-book each year and rebalance annually. You apply the rules to past data, record the hypothetical portfolio, and measure its results. Those results are compared with a benchmark or with what the model predicted.

In portfolio construction, backtesting has two main uses. First, it screens strategies and factor ideas before capital is committed. Second, it checks risk models. For example, a model that says daily losses should exceed a threshold only 1% of days can be compared with how often losses actually exceeded it.

Backtesting has limits. It shows what would have happened, not what will happen. The result depends on the sample period, the data quality and the assumptions about costs. It is also not a true out-of-sample test if you tuned the rules after looking at the same data. Exam questions usually ask you to judge whether a backtest is credible and how it should be used.

Key formulas to remember

Backtest purpose (rule)
Strategy rules + historical data → hypothetical performance → compare with benchmark or expectation
This is the core logic. Backtesting is a process, not a calculation.
Excess return versus benchmark
Active return = Backtested portfolio return − Benchmark return
Use the same period and the same return basis (for example, both net of costs).
Risk-model exceedance rate
Exceedance rate = Number of days actual loss exceeds VaR ÷ Number of days tested
Compare it with the stated tail probability. A 1% VaR should be exceeded on roughly 1% of days.
In-sample versus out-of-sample
Fit or tune on in-sample data; judge on out-of-sample data
Results from data used to design the rules are biased upward.

How to solve Backtesting Basics and Purpose questions

Use this sequence for any backtesting question in an item set. Read the vignette for the strategy, data and assumptions before looking at the options.

  1. 1Identify the purpose: is the backtest testing a return strategy or checking a risk model such as VaR?
  2. 2Find the rules in the vignette: signal, universe, rebalancing frequency and holding period.
  3. 3Find the data details: sample period, data source, whether it is in-sample or out-of-sample, and whether costs are included.
  4. 4Compute any required figure, such as active return or exceedance rate, using the vignette numbers over the same period.
  5. 5Compare the result with the benchmark or the expected value, and say whether it supports the strategy or model.
  6. 6Check credibility: look for tuning on the same data, a short or unusual period, or ignored costs.
  7. 7Choose the option that matches both the number and the caveat. Reject options that claim backtests prove future performance.

Quickest way: Three-check shortcut for backtesting items

When to use it: Use it when time is short and the question asks whether a backtest supports a strategy or what it is for.

  1. Purpose check: is it testing a strategy or validating a risk model?
  2. Data check: is the result out-of-sample, and are costs included?
  3. Claim check: eliminate any option saying the backtest guarantees or proves future results.
  4. If a number is needed, divide or subtract using the vignette figures for the same period.

Common mistakes in Backtesting Basics and Purpose

  • Treating a strong backtest as proof of future performance

    A good historical number feels like evidence of skill.

    Fix: Remember it only shows hypothetical past results. The future can differ from the sample.

  • Counting in-sample results as validation

    Candidates overlook that the rules were tuned on the same data.

    Fix: Look for words like optimized, tuned or selected. Real validation needs data not used to design the rules.

  • Ignoring transaction costs and implementation limits

    Vignettes often give gross returns, so they look final.

    Fix: Check whether costs, taxes and liquidity were included. If not, the result is overstated.

  • Confusing backtesting with simulation

    Both are used in risk and portfolio planning.

    Fix: Backtesting uses actual historical data. Simulation generates many hypothetical paths from assumptions or models.

  • Misreading exceedance results for a VaR model

    Candidates forget to compare the actual frequency with the stated probability.

    Fix: Compute exceedances ÷ days and compare with the tail probability. Much higher means the model understates risk.

Worked examples

Example 1

A quantitative team backtests a value strategy over 10 years. Each year it buys the cheapest quintile of stocks by price-to-book and rebalances annually. The backtest shows an annualized return of 11.4% versus 9.0% for the benchmark. The team chose the cheapest-quintile cut-off after testing several cut-offs on the same 10 years. Returns are before transaction costs. Q1: What is the active return? Q2: Is the result a reliable out-of-sample validation? Q3: What is the best use of the result?

Show the solution
  1. Q1: Active return = 11.4% − 9.0% = 2.4% a year.
  2. Q2: The cut-off was chosen after testing on the same 10 years, so the result is in-sample and biased upward. It is not reliable out-of-sample validation.
  3. Q2 also: returns are before costs, which further overstates implementable performance.
  4. Q3: The result is a hypothesis to test further, for example on data not used in design and net of costs. It is not proof.

Answer: Q1: 2.4% a year. Q2: No, it is in-sample and gross of costs. Q3: Treat it as a candidate to be validated out-of-sample net of costs.

Example 2

A risk manager backtests a one-day 1% VaR model over 500 trading days. Actual losses exceeded the VaR estimate on 15 days. Q1: How many exceedances would you expect? Q2: What is the actual exceedance rate? Q3: What does this suggest about the model?

Show the solution
  1. Q1: Expected exceedances = 1% × 500 = 5 days.
  2. Q2: Actual exceedance rate = 15 ÷ 500 = 3%.
  3. Q3: The actual rate is three times the stated 1% tail probability. The model appears to understate risk.

Answer: Q1: 5 days. Q2: 3%. Q3: The model likely understates risk and should be reviewed.

Exam tips

  • Read the vignette for how the rules were chosen. Tuning on the same data is the usual trap.
  • If an option says a backtest proves or guarantees future returns, eliminate it.
  • For VaR backtests, compute exceedances ÷ days and compare with the stated tail probability.
  • Check whether figures are gross or net of costs before comparing strategies.
  • Keep backtesting (real past data) separate from simulation (generated scenarios).

Backtesting Basics and Purpose in other exams

The same ground in other exams, if you are preparing for more than one or want another angle on it.

Backtesting Basics and Purpose: frequently asked questions

What is backtesting in portfolio management?

It is testing an investment strategy or risk model on historical data to see how it would have performed. You apply fixed rules to past data and measure return and risk. The aim is to screen ideas and check models before relying on them.

What are the basic steps to backtest a strategy?

Define the rules, choose the data and sample period, run the rules on the past data, and measure performance against a benchmark. Then check for bias, costs and whether the result holds out-of-sample. Finally, decide how much weight the result deserves.

Does a good backtest mean the strategy will work?

No. A backtest shows hypothetical past results only. Results can be inflated by tuning on the same data, a favourable period or ignored costs.

How is backtesting different from simulation?

Backtesting uses the actual historical record, giving one path. Simulation generates many possible paths from historical or model-based assumptions. They answer related but different questions.