IAI Actuarial Core Principles · Risk Modelling and Survival Analysis

Markov Processes for CS2 Risk Modelling and Survival Analysis

A Markov process is a stochastic process where the future depends only on the present state, not on how you got there. To solve questions, define the state space, write the transition matrix or generator, then apply Chapman-Kolmogorov, stationary distribution or Kolmogorov equations as the question needs.

What this chapter covers

This chapter covers stochastic processes whose future depends only on the current state. You start with the Markov property and the idea of a state space and time set. You then study discrete-time Markov chains, where a transition matrix drives everything. After that you move to continuous time, where a generator matrix and Kolmogorov equations replace the matrix of one-step probabilities.

The chapter sits inside the Stochastic processes part of the CS2 syllabus, which carries a 25% weighting for 2026. That weighting is shared with other process topics, so do not assume Markov processes alone carry it. The same ideas run into the Survival models part: a multi-state model is a Markov jump process, and the two-state alive-dead model is the basic survival model written in this language.

It also links to other work in the paper. The no-claims discount and similar insurance examples are Markov chains. Simulation and fitting of chains can appear in the Paper B computer-based exam, where you may use R to compute powers of a matrix or a stationary distribution. So you need both the theory and the working.

Markov processes give you a reliable source of marks because the questions follow set patterns: build the matrix, find n-step probabilities, classify states, find a stationary distribution, and set up or solve Kolmogorov equations. Written questions often reward method and clear statement of assumptions, so careful working earns marks even when arithmetic slips. The chapter also supports multi-state and survival questions, so time spent here pays off in more than one part of the paper. Multiple-choice questions on it tend to test definitions and quick calculations, which you can master with practice.

Markov processes: topics in the order to study them

1Stochastic Processes and the Markov PropertyYou need the language of state space, time set and the Markov property before any matrix or equation makes sense.
2Discrete-Time Markov Chains and Transition ProbabilitiesTransition matrices and the Chapman-Kolmogorov equations are the simplest setting to practise the Markov idea, and they feed every later topic.
3Classification of States and Stationary DistributionsOnce you can handle the matrix, you study long-run behaviour, which needs communicating classes, periodicity, recurrence and stationary distributions.
4Markov Jump Processes and Kolmogorov EquationsThis extends the discrete ideas to continuous time, using transition rates, holding times and the forward and backward equations.
5Multi-State Models and ApplicationsIt comes last because it applies jump processes to real settings such as survival, illness and disability, and so needs everything before it.

How to prepare Markov processes

Treat this chapter as a skills chapter. You learn the ideas once, then drill the calculation types until they are automatic.

Write the Markov property in your own words, and list the state space and time set for five examples, such as no-claims discount and alive-dead.
Practise building transition matrices from a verbal description. Check that every row sums to 1 before you do anything else.
Compute n-step probabilities by hand for small matrices, and again in R, so you can handle both Paper A and Paper B styles.
Classify states for several chains by drawing the transition diagram first. Then solve for the stationary distribution using πP = π and Σπᵢ = 1, and state whether it is a limiting distribution.
For jump processes, practise forming the generator matrix, checking that each row sums to 0, and writing the forward and backward equations from a diagram.
Work multi-state models such as alive-sick-dead. Set up the rates, write the equations and state your assumptions clearly.
Finish with timed past-paper questions. Review each one by checking notation, stated assumptions and whether you answered the exact question asked.

Common mistakes in Markov processes

Using a transition matrix whose rows do not sum to 1, or a generator whose rows do not sum to 0.
Fix: Always compute the row sums straight after building the matrix. For a generator, set each diagonal entry to minus the sum of the other entries in its row.
Claiming a stationary distribution is the limiting distribution without checking conditions.
Fix: State whether the chain is irreducible and aperiodic before saying the distribution is the long-run one. A periodic chain can have a stationary distribution but no limit.
Misclassifying states by looking only at the matrix, not the diagram.
Fix: Draw the transition diagram every time. Mark which states can reach which, then identify classes, closed classes and absorbing states.
Mixing up forward and backward Kolmogorov equations, or confusing rates with probabilities.
Fix: Remember that forward puts the generator on the right and backward on the left. Never read a rate as a probability; convert using the holding-time and jump-probability rules.
Not stating assumptions in multi-state or application questions.
Fix: Say whether the process is time-homogeneous, which states are absorbing, and that transitions follow the Markov property. These statements often carry marks.
Doing matrix powers by hand when a shortcut or the computer exam makes it unnecessary.
Fix: Look for structure first, such as absorbing states or a two-state form. In Paper B, use R matrix multiplication and show the code and output clearly.

Last-day revision: Markov processes

Markov property: given the present state, the future is independent of the past.
Every row of a transition matrix sums to 1; every row of a generator matrix sums to 0.
Chapman-Kolmogorov: P(m+n) = P(m) × P(n), so n-step matrix is Pⁿ for a time-homogeneous chain.
States that communicate form a class; a chain with one class is irreducible.
A state is recurrent if return is certain, transient otherwise.
Period of a state is the greatest common divisor of possible return times; period 1 means aperiodic.
A stationary distribution π satisfies π = πP with Σπᵢ = 1.
An irreducible, aperiodic finite chain has a unique stationary distribution that is also the limiting distribution.
In a jump process, holding time in state i is exponential with rate λᵢ = −qᵢᵢ.
Jump probabilities from i to j are qᵢⱼ ÷ λᵢ for j ≠ i.
Kolmogorov forward: P′(t) = P(t)A. Backward: P′(t) = A P(t), where A is the generator.
Stationary distribution of a jump process solves πA = 0 with Σπᵢ = 1.

Markov processes practice questions

Markov processes: frequently asked questions

How much of CS2 is Markov processes?

Markov processes sit within the Stochastic processes part of the syllabus, which has a 25% weighting for 2026. That figure covers other process topics as well, so do not read it as the weight of this chapter alone. Treat it as a core area because multi-state models also link to Survival models.

What is the difference between a Markov chain and a Markov jump process?

A Markov chain moves at fixed, discrete time steps and is described by a transition matrix. A Markov jump process can change state at any time and is described by transition rates in a generator matrix. Both satisfy the Markov property.

How do I find a stationary distribution quickly?

Write πP = π for a chain, or πA = 0 for a jump process, and add the condition that the entries of π sum to 1. Solve the equations, dropping one redundant equation if needed. Then check the answer by substituting back.

Will Markov processes be tested in Paper B?

It can be, since Paper B is a computer-based exam using R. Typical tasks include matrix powers, stationary distributions and simulation. Practise the R steps as well as the written method, and show your working and results clearly.