Risk Modelling and Survival Analysis · Markov processes

Multi-State Models and Applications in Actuarial Work

Updated 11 October 2026 · Fact-checked

A multi-state model is a Markov jump process where a life moves between states such as healthy, sick and dead. Transition intensities μij drive everything. You find probabilities with Kolmogorov equations and estimate each intensity by maximum likelihood as observed transitions ÷ total time spent in the starting state.

Understand Multi-State Models and Applications

A multi-state model describes a life as being in one of a few states at any time. Examples: alive and dead; healthy, sick and dead; active, disabled and dead. A life can move between states at any moment, so time is continuous.

The model is Markov. This means the future depends only on the current state, not on how the life got there. Under this assumption, the chance of moving from state i to state j over a tiny time h is about μij(t) × h. The rate μij(t) is the transition intensity (force of transition). It plays the same role as the force of mortality in the two-state model.

From the intensities you build the transition probabilities pij(t, t+s): the probability of being in state j at time t+s given state i at time t. They satisfy the Kolmogorov forward equations and the Chapman-Kolmogorov equations. In simple models you can solve them directly. In harder ones you use the integrated form or a numerical method such as Euler's.

For estimation, you observe lives over a period and record two things: the number of transitions from i to j, and the total time each life spends in state i (the waiting time). With constant intensities, the likelihood is a product of exponential terms. Maximising it gives a simple estimator: transitions ÷ time spent. This is used for disability income insurance: the intensities for sickness, recovery and death feed premium and reserve calculations.

Key rules to remember

Definition of transition intensity: μij(t) = lim h→0 of P(X(t+h) = j | X(t) = i) ÷ h, for i ≠ j; Holds for i ≠ j. Gives P(move i→j in time h) ≈ h × μij(t).
Total force of leaving a state: μi(t) = Σ over j ≠ i of μij(t); Waiting time in state i is exponential with this rate if intensities are constant.
Probability of staying in a state: pii-bar(t, t+s) = exp(−∫ from t to t+s of μi(u) du); This is staying continuously in i. It is not the same as pii(t, t+s), which allows leaving and returning.
Kolmogorov forward equations: ∂/∂s pij(t, t+s) = Σ over k ≠ j of [pik(t, t+s) × μkj(t+s)] − pij(t, t+s) × μj(t+s); Inflow into j minus outflow from j. Initial condition: pij(t, t) = 1 if i = j, else 0.
Kolmogorov backward equations: ∂/∂t pij(t, t+s) = μi(t) × pij(t, t+s) − Σ over k ≠ i of [μik(t) × pkj(t, t+s)]; Less often used in exams, but you should recognise it.
Integrated form (healthy to sick): pHS(t, t+s) = ∫ from 0 to s of pHH-bar(t, t+u) × μHS(t+u) × pSS-bar(t+u, t+s) du; Valid when no return from sick to healthy. Stay healthy, jump at time u, stay sick.
MLE of constant intensity: μ̂ij = Nij ÷ Vi; Nij = number of i→j transitions; Vi = total time spent in state i by all lives.
Variance of the MLE: Var(μ̂ij) ≈ μij ÷ E[Vi], estimated by μ̂ij ÷ Vi = Nij ÷ Vi²; Asymptotic result. Used for approximate confidence intervals: μ̂ ± 1.96 × √(μ̂² ÷ Nij).
Log-likelihood (constant intensities): ln L = Σ over i≠j of [Nij × ln μij] − Σ over i of [μi × Vi]; Differentiate with respect to each μij separately. Terms for different transitions separate.

How to solve Multi-State Models and Applications questions

Use this method for any multi-state question, whether it asks for probabilities, equations or estimation.

1Draw the state diagram. Label every state and put each intensity on its arrow. Mark which transitions are impossible (for example, no arrow out of dead).
2Check the Markov assumption and whether intensities are constant or depend on age or duration. State this assumption in your answer.
3Decide what is asked: a probability, an equation, an expected value, or an estimate. This chooses your tool.
4For probabilities, first look for a direct route: staying in a state uses exp(−∫ μi), and a single jump uses the integrated form. Use Kolmogorov only if the path is awkward.
5For Kolmogorov equations, write inflow minus outflow for the target state. Include every state that can feed it, and give the initial conditions.
6For estimation, tabulate for each lives-record the time spent in each state and the transitions made. Add up to get Nij and Vi.
7Compute μ̂ij = Nij ÷ Vi. If asked, give the variance Nij ÷ Vi² and a confidence interval.
8Check the answer: probabilities between 0 and 1, rows summing to 1, intensities positive, and units in years.

Quickest way: Fast route for exam time pressure

When to use it: Use when the question gives constant intensities or observed data and asks for a probability or estimate.

Sketch the diagram in ten seconds and write each intensity on it.
For estimation, go straight to μ̂ij = Nij ÷ Vi. Do not re-derive unless the question says 'derive'.
For a probability with constant intensities, add the outgoing rates from the start state to get μi, then use exp(−μi × t) for staying put.
For a one-jump route, write the integral as: stay, jump at u, stay again. For constant rates, integrate the exponentials directly.
If only a Kolmogorov equation is asked, write 'inflow − outflow' per state and stop; do not solve unless told to.

Common mistakes in Multi-State Models and Applications

Confusing pii(t, t+s) with the probability of staying continuously in state i.
Both look similar, but pii allows leaving and returning, while the exponential only counts uninterrupted stay.
Fix: Use the exponential only when no return is possible or when the question says 'remains continuously'. Otherwise solve Kolmogorov.
Leaving out an inflow or outflow term in the Kolmogorov equation.
Students focus on the arrows they like and forget the total exit rate from the target state.
Fix: For each state, list all arrows in and all arrows out from the diagram before writing the equation.
Using the number of lives instead of total time as the denominator for the MLE.
It feels natural to divide by the lives observed.
Fix: Always divide by total time in the starting state. Intensity is a rate per unit time.
Counting time in the wrong state, such as including time in the sick state when estimating a healthy-to-dead intensity.
Records mix time spent in different states per life.
Fix: Split each life's record by state. Vi only counts time spent in state i.
Forgetting the Markov and constant-intensity assumptions in the answer.
Students treat them as obvious.
Fix: Write the assumptions in one line. Exam marks are awarded for stating them.
Adding the intensities into a probability, for example treating μHS × t as the probability of becoming sick.
The small-h approximation is mistaken for an exact formula.
Fix: Use h × μ only for very small h. For longer periods use the exponential or integral form.

Worked examples

Example 1

In a healthy-sick-dead model, constant intensities are: healthy to sick 0.10, healthy to dead 0.02, sick to healthy 0.40, sick to dead 0.08 (per year). A life is healthy now. Find the probability the life is still continuously healthy after 2 years. Give the answer to 4 decimal places.

Show the solution

Total force of leaving healthy: μH = 0.10 + 0.02 = 0.12.
Probability of staying continuously healthy for 2 years = exp(−0.12 × 2).
This equals exp(−0.24).
exp(−0.24) = 0.7866 (to 4 decimal places).

Answer: 0.7866

Example 2

A study follows lives in a healthy-sick-dead model. In total, the lives spent 800 years healthy and 100 years sick. There were 64 healthy-to-sick transitions, 16 healthy-to-dead, 30 sick-to-healthy and 10 sick-to-dead. Assuming constant intensities, estimate all four intensities and find the standard error of the healthy-to-sick estimate.

Show the solution

Use μ̂ij = Nij ÷ Vi with VH = 800 and VS = 100.
Healthy to sick: 64 ÷ 800 = 0.08.
Healthy to dead: 16 ÷ 800 = 0.02.
Sick to healthy: 30 ÷ 100 = 0.30.
Sick to dead: 10 ÷ 100 = 0.10.
Variance of the healthy-to-sick estimate ≈ NHS ÷ VH² = 64 ÷ 640,000 = 0.0001.
Standard error = √0.0001 = 0.01.

Answer: μ̂HS = 0.08, μ̂HD = 0.02, μ̂SH = 0.30, μ̂SD = 0.10 per year. Standard error of μ̂HS is 0.01.

Exam tips

Draw the diagram first, even if one is given. It stops you missing inflow and outflow terms in Kolmogorov equations.
In estimation questions, set out a small table of Nij and Vi. It earns method marks even if arithmetic slips.
State your assumptions: Markov property and constant intensities over the period.
Paper B may ask you to code the estimation or solve Kolmogorov equations numerically. Know the Euler step: p(t+h) ≈ p(t) + h × (inflow − outflow).
For disability income questions, link the intensities to the premium or reserve only after the probabilities are correct. Show the probability step clearly.

Practice questions from Markov processes

Multi-State Models and Applications: frequently asked questions

What is the difference between a multi-state model and a Markov jump process?

A multi-state model is the application: states and allowed transitions for a life or policy. A Markov jump process is the mathematics behind it, with continuous time and a finite set of states. Most IAI multi-state models are time-homogeneous or have intensities that depend only on age.

How do I estimate transition intensities from data?

Count the transitions from state i to j and the total time spent in state i. With constant intensities, the maximum likelihood estimate is the count divided by the total time. The estimator is approximately normal with variance Nij ÷ Vi².

When should I use Kolmogorov equations instead of the integral form?

Use the integral form when the path is simple, such as no return from sick to healthy. Use Kolmogorov when states can be revisited or when a numerical solution is requested. Euler's method then gives approximate probabilities.

How is the healthy-sick-dead model used in disability income insurance?

The intensities give the probabilities of being healthy or sick at future times. Premiums are benefits paid while sick, discounted with those probabilities, set against premiums paid while healthy. Reserves use the same probabilities from the valuation date.