Skip to content

FRM Part II · FRM Exam Part II · Advances in Artificial Intelligence: Implications for Capital Markets Activities

A trading firm builds a model to forecast intraday price moves and selects the best of 500 candidate model configurations by their performance on a single validation period. The selected configuration shows an impressive Sharpe ratio on that period. Which concern is most relevant for a risk manager assessing the result?

The key concern is data-snooping, or multiple testing. Picking the best of 500 configurations on one validation period makes the top Sharpe ratio likely to reflect chance, so it is overstated. A risk manager should require testing on untouched out-of-sample data or adjust for the number of trials.

  1. AData-snooping: the best of many trials may reflect luck, so the Sharpe ratio is likely overstatedCorrect
  2. BConcept drift is impossible here because the validation period was used for selection
  3. CLook-ahead bias is eliminated because multiple configurations were tested
  4. DHeteroskedasticity, which biases the Sharpe ratio downward

Explanation

Choosing the maximum across many trials on the same data inflates the apparent performance through multiple testing, so the selected result is biased upward. A fresh, untouched out-of-sample test or a multiple-testing adjustment is needed. Testing many configurations does not remove look-ahead bias, and using the validation period does not rule out drift.

Did you get it right without looking?

One question tells you little. A timed set on Advances in Artificial Intelligence: Implications for Capital Markets Activities shows your real accuracy, how long you take and where you lose marks.

Practise Advances in Artificial Intelligence: Implications for Capital Markets ActivitiesFree diagnostic mock

More Advances in Artificial Intelligence: Implications for Capital Markets Activities questions