IAI Actuarial Core Principles · Risk Modelling and Survival Analysis

Markov Chains for IAI CS2: Study Guide

A Markov chain is a stochastic process where the next state depends only on the current state, not on the past. To solve questions, write the transition matrix, use Chapman-Kolmogorov (P^(n+m) = P^(n) P^(m)) for multi-step probabilities, and solve πP = π with Σπ = 1 for the long-run distribution.

What this chapter covers

This chapter in CS2 (Risk Modelling and Survival Analysis) deals with Markov chains: processes that move between a set of states at discrete times, where the future depends only on the present state. This is the Markov property. You describe the whole process with a one-step transition matrix P, where entry p_ij = P(X_(n+1) = j | X_n = i) and each row sums to 1.

You first learn to set up and use the matrix. Multi-step probabilities come from matrix powers, via the Chapman-Kolmogorov equations. Then you study the structure of the chain: communicating classes, irreducibility, periodicity, recurrence and transience. These decide whether a stationary distribution exists and whether the chain settles down to it. The chapter ends with applications, mainly the no claims discount (NCD) model used in motor insurance, and random walks.

The chapter is the first step into the Stochastic processes part of the CS2 syllabus. The ideas carry straight into continuous-time Markov jump processes and the Markov multiple-state models used in survival analysis. If you are shaky here, those later chapters become much harder. It also links to CM1 ideas on multiple-state models and decrements, and to the way insurers model movement between states such as healthy, sick and dead.

Stochastic processes carries a large share of the 2026 CS2 syllabus, and Markov chains are its foundation. Questions are very predictable in form: build a matrix, compute a two- or three-step probability, classify states, find a stationary distribution, or analyse an NCD system. These are methodical and the marks are easy to collect if your working is clean. The same skills also feed into the computer-based Paper B, where you may simulate or compute with a transition matrix in R, so one chapter pays off in both papers.

Markov chains: topics in the order to study them

1Stochastic Processes and the Markov PropertyYou need the definitions of state space, time set and the Markov property before any matrix work makes sense.
2Transition Probabilities and Chapman-Kolmogorov EquationsOnce you can build P and take its powers, you can answer the most common numerical questions.
3Classification of States and Stationary DistributionsThis needs the matrix skills above, because you test communication, periodicity and long-run behaviour using P.
4Applications: No Claims Discount and Random WalksApplications combine everything before them, so do them last to practise full exam-style questions.

How to prepare Markov chains

Markov chains reward practice more than reading. Work in small blocks of 30 to 40 minutes that you can fit around work, and always write out the steps as you would in the exam.

Learn the definitions precisely: state space, Markov property, time-homogeneous chain, transition matrix. Write the Markov property in your own words and in notation.
Practise building transition matrices from short word descriptions. Check every row sums to 1 before moving on.
Do many multi-step problems. Compute P², P³ by hand for 3×3 matrices, and also practise finding a single entry without the full matrix. Use Chapman-Kolmogorov to split a long step into shorter ones.
Draw the transition diagram for every chain you meet. Use it to find communicating classes, closed classes, absorbing states and periods before doing any algebra.
Solve for stationary distributions by writing πP = π, using Σπ_i = 1, and dropping one redundant equation. Then check your answer satisfies all the equations.
Work through NCD questions fully: set up the states, write the matrix, find the stationary distribution and compute expected premium or discount in the long run.
Finish with past-paper style questions under time. Then repeat the key calculations in R so you are ready for Paper B.

Common mistakes in Markov chains

Using a matrix whose rows do not sum to 1, or reading rows and columns the wrong way round.
Fix: Fix the convention: row = current state, column = next state. Check every row sums to 1 before you calculate anything.
Multiplying matrices in the wrong order or computing P² entry by entry by squaring each entry.
Fix: Use row-by-column multiplication. For one entry, take the row of the starting state and the column of the target state, then multiply and add.
Forgetting the initial distribution when asked for an unconditional probability.
Fix: Use the row vector: distribution at time n = (initial distribution) × P^n. Only use a single row if the start state is certain.
Claiming a stationary distribution is the limiting distribution without checking the conditions.
Fix: State the conditions. A periodic chain can have a stationary distribution but p_ij^(n) will not converge to it.
Misclassifying states by ignoring the diagram, for example calling a state recurrent when the chain can leave its class and never return.
Fix: Draw the diagram. Closed classes in a finite chain are recurrent. States in a class that can be left are transient.
Finding the stationary distribution but forgetting to normalise, or using all equations including the redundant one.
Fix: Drop one equation, solve for the ratios, then apply Σπ_i = 1. Finally verify with one of the original equations.

Last-day revision: Markov chains

Markov property: P(X_(n+1) = j | X_n = i, X_(n-1), ..., X_0) = P(X_(n+1) = j | X_n = i).
Each row of the transition matrix P sums to 1, and every entry is between 0 and 1.
Chapman-Kolmogorov: p_ij^(n+m) = Σ_k p_ik^(n) p_kj^(m), so P^(n+m) = P^(n) P^(m).
For a time-homogeneous chain, the n-step matrix is P^n.
State j is accessible from i if p_ij^(n) > 0 for some n ≥ 0. States communicate if each is accessible from the other.
A chain is irreducible if all states form one communicating class.
The period of state i is the greatest common divisor of all n with p_ii^(n) > 0. Period 1 means aperiodic.
A state is recurrent if the chain returns to it with probability 1, and transient otherwise. In a finite chain, a closed class is recurrent.
A stationary distribution π satisfies πP = π with Σπ_i = 1 and π_i ≥ 0.
A finite irreducible chain has a unique stationary distribution. If it is also aperiodic, p_ij^(n) → π_j as n → ∞.
An absorbing state has p_ii = 1.
Simple random walk: steps +1 with probability p and -1 with probability 1 - p. It is irreducible with period 2 on the integers.

Markov chains practice questions

Markov chains: frequently asked questions

What is the Markov property in simple terms?

It means the future depends only on where the process is now, not on how it got there. In notation, the conditional probability of the next state given the whole history equals the conditional probability given only the current state.

How do I find a stationary distribution?

Write πP = π as a set of linear equations and add Σπ_i = 1. One equation from πP = π is redundant, so drop it and solve the rest. Check the answer in the equation you dropped.

Do I need to know Markov chains for Paper B as well?

Yes, it helps. Paper B is computer-based and uses R, and transition matrices, matrix powers and simulation of chains are natural tasks. Practise the same calculations by hand and in R.

How is the no claims discount model solved?

Treat each discount level as a state and write the transition matrix from the claim and no-claim rules. Then use P^n for short-term questions or the stationary distribution for long-run average discount or premium.

Which topic in this chapter should I master first?

Start with the Markov property and building the transition matrix. Every later topic, from Chapman-Kolmogorov to NCD, depends on setting up and using that matrix correctly.