FRM Exam Part I · Simulation and Bootstrapping

Limitations of Bootstrapping and Simulation in FRM Part I

Updated 11 October 2026 · Fact-checked

Bootstrapping resamples observed data with replacement and assumes the data are i.i.d. and representative. It fails with dependent data, outliers, tiny samples and unseen extremes. Monte Carlo simulation draws from an assumed model, so it carries model risk and sampling error. Bootstrapping depends on data; simulation depends on the model.

Understand Limitations of Bootstrapping and Simulation

Both methods create many artificial samples so you can study how a statistic behaves. You use them when a formula for the standard error or the distribution is hard to derive. The difference is where the artificial samples come from.

Monte Carlo simulation draws random numbers from a model you specify, such as a normal distribution with a chosen mean, volatility and correlation. It does not need historical data, except to calibrate the inputs. Its main weakness is model risk: if the assumed distribution, parameters or dependence structure are wrong, the results are wrong, however many draws you run. More draws reduce sampling error, not model error. It can also be slow and costly for complex models.

Bootstrapping draws from the observed data itself. You sample observations with replacement to form a new sample of the same size, compute the statistic, and repeat many times. It makes no distributional assumption, which is its strength. But it assumes the observations are independent and identically distributed (i.i.d.) and that the sample represents the true population.

That assumption is where it breaks. If data are dependent, such as autocorrelated returns or volatility clustering, resampling single observations destroys the time structure and understates or misstates the uncertainty. If the sample has outliers, they can be drawn repeatedly or not at all, so results swing. If the sample is small or misses extreme events, the bootstrap cannot produce a loss worse than the worst one observed. Bootstrapping also struggles with statistics that depend on the extreme tail, such as the maximum. When a regime change occurs, past data no longer describe the future.

One remedy for dependence is the block bootstrap, which resamples blocks of consecutive observations so that short-range dependence is kept. Another is to fit a time-series model and bootstrap its residuals.

Key formulas to remember

Bootstrap resampling rule: Draw n observations with replacement from the original n observations; compute the statistic; repeat B times; Each draw is from the empirical distribution. The bootstrap standard error is the standard deviation of the B statistics.
Bootstrap i.i.d. condition: Observations independent and identically distributed, and sample representative of the population; If either part fails, the bootstrap can be unreliable.
Simulation sampling error: Standard error of a simulated mean ≈ s ÷ √N; Quadrupling the number of simulations N roughly halves the standard error. It does not fix model risk.
Bootstrap standard error: SE = √[ Σ(θ*ᵢ − θ̄*)² ÷ (B − 1) ]; θ*ᵢ is the statistic from resample i and θ̄* is the average across B resamples.

How to solve Limitations of Bootstrapping and Simulation questions

Use this method for any question asking which method is better, why a result is unreliable, or what a limitation is.

1Identify the method in the question: simulation from an assumed model, or resampling from observed data.
2Name the source of the artificial data: a specified distribution for simulation, the empirical sample for bootstrap.
3List what the method assumes: a correct model and parameters for simulation; i.i.d. and representative data for bootstrap.
4Check the data described: autocorrelation, volatility clustering, outliers, a short sample or a regime change.
5Match the problem to the limitation: model risk, sampling error, broken dependence, outlier distortion, or no tail beyond the sample.
6Decide whether more draws help. They cut sampling error but not model risk or unrepresentative data.
7Pick the remedy if asked: block bootstrap, residual bootstrap, a better model, or more data.
8Choose the option that fits the stated facts, not the one that sounds broadly true.

Quickest way: Two-question test: model or data?

When to use it: Use it when options are close and you have under a minute.

Ask: does the question describe an assumed distribution? If yes, think model risk and sampling error.
Ask: does it describe observed returns being resampled? If yes, think i.i.d. and representativeness.
If returns are serially dependent or volatility clusters, the answer is that ordinary bootstrap fails; block bootstrap is the fix.
If the issue is accuracy of a simulated estimate, more simulations help; if the model is wrong, they do not.

Common mistakes in Limitations of Bootstrapping and Simulation

Saying more Monte Carlo draws fix a wrong model
Students link a large N with accuracy in general.
Fix: More draws shrink sampling error only. A misspecified distribution or correlation stays wrong.
Claiming bootstrapping needs a normality assumption
It is confused with parametric methods.
Fix: Standard bootstrapping is nonparametric. Its key assumptions are i.i.d. data and a representative sample.
Thinking bootstrap works fine for autocorrelated returns
Resampling looks harmless because it is random.
Fix: Resampling single observations breaks the time order. Use a block bootstrap or bootstrap model residuals.
Believing bootstrap can generate losses worse than any in the data
Students treat resampling as creating new information.
Fix: Draws come only from observed values, so the bootstrap cannot exceed the worst observation in the sample.
Resampling without replacement
It mirrors ordinary sampling.
Fix: Bootstrap samples are drawn with replacement. Without it you only reorder the same data.
Saying outliers are harmless because resampling averages them out
Averaging over many resamples sounds like smoothing.
Fix: An outlier can appear several times in a resample or not at all, which makes the statistic unstable.

Worked examples

Example 1

An analyst bootstraps 250 daily returns, drawing single returns with replacement, to estimate the standard error of the 10-day average return. The returns show strong positive autocorrelation. Which statement is most accurate? A) The standard error is reliable because resampling is random. B) The standard error is likely understated, so a block bootstrap is more suitable. C) The standard error is reliable if B is increased to 100,000. D) The method fails only because returns are not normal.

Show the solution

Identify the method: ordinary bootstrap resampling single observations.
Check the assumption: it needs i.i.d. data. The returns are positively autocorrelated, so this fails.
Effect: resampling single days removes the positive dependence. With positive autocorrelation, true variability of multi-day averages is larger than under independence.
So the bootstrap standard error is likely too low.
Remedy: resample blocks of consecutive days to keep the dependence.
Option A ignores dependence. Option C: more resamples reduce Monte Carlo noise in the bootstrap, not the bias from broken dependence. Option D is wrong because the bootstrap needs no normality.

Answer: B

Example 2

A risk team estimates a portfolio loss quantile in two ways: (1) Monte Carlo from a multivariate normal model fitted to calm-period data, and (2) a bootstrap of the same calm-period returns. A crisis follows with far larger losses. Explain the limitation each method shows and what increasing the number of trials achieves.

Show the solution

Monte Carlo: results depend on the assumed multivariate normal and its calibrated volatilities and correlations. Normality understates fat tails, and calm-period correlations may rise in a crisis. This is model risk.
Bootstrap: draws come only from calm-period observations, so no loss larger than the worst in that sample can appear. The sample is not representative of crisis conditions.
Both methods inherit the weakness of their inputs: assumed model for one, historical data for the other.
Increasing trials in either method reduces sampling error in the estimate. It does not add crisis behaviour that the model or data lack.

Answer: Monte Carlo suffers model risk from wrong distribution and correlation assumptions. Bootstrap suffers from an unrepresentative sample that cannot produce losses beyond those observed. More trials only reduce sampling error in both.

Exam tips

Frame every answer as model risk for simulation and data dependence for bootstrap. This pairing is the usual testing point.
Remember that more trials reduce sampling error only. Distractors often claim they fix model error.
When the stem mentions autocorrelation, clustering or time series, look for block bootstrap or residual bootstrap.
Be careful with absolute words like always and never. Bootstrapping works well for i.i.d. data, so it is not simply bad.
Expect conceptual questions with no calculation. Read the stem for clues about outliers, small samples and regime shifts.

Practice questions from Simulation and Bootstrapping

Limitations of Bootstrapping and Simulation: frequently asked questions

When does the bootstrap fail?

It fails mainly when data are not i.i.d., such as with autocorrelation or volatility clustering. It is also unreliable with strong outliers, very small samples, or samples that miss extreme events or a regime change. It cannot create outcomes outside the observed data.

What is the difference between Monte Carlo simulation and bootstrapping?

Monte Carlo draws from a distribution or model that you specify, so it is exposed to model risk. Bootstrapping draws with replacement from observed data, so it needs no distribution but depends on the data being i.i.d. and representative.

What are the disadvantages of Monte Carlo simulation?

Results depend on the assumed model and parameters, so errors in distribution, volatility or correlation carry through. It can be computationally expensive, and results have sampling error that falls only as the number of trials grows.

How do you fix bootstrapping with time-series dependence?

Use a block bootstrap, which resamples blocks of consecutive observations to keep short-range dependence. Another option is to fit a time-series model and bootstrap the residuals.