Actuarial Mathematics for Modelling · Valuing cashflows contingent on multiple transition events

Multiple State Models and Markov Processes in Actuarial Mathematics

Q: What is the Markov property in a multiple state model?

It says that, given the state a life is in now, the probability of future moves does not depend on the past path. Only the current state, and the current age or time, matter. You should state it as an assumption whenever you use a Markov model.

Q: What are the Chapman-Kolmogorov equations in simple words?

They say that to go from state i to state j over s + t years, you sum over every state k the life could be in at time s. For each k, multiply the probability of reaching k by the probability of going from k to j in the remaining time. Remember to start the second factor at age x + s.

Q: How is a multiple decrement model different from a general multiple state model?

A multiple decrement model has one starting state and several exit states, and no one returns after leaving. A general multiple state model can allow movement back, for example sick to healthy. The decrement model is a special case.

Q: What is the difference between a transition intensity and a transition probability?

A probability is a number between 0 and 1 over a stated period. An intensity is an instantaneous rate and can be larger than 1. For a very short period h, the probability of moving is approximately h times the intensity.

Updated 11 October 2026 · Fact-checked

A multiple state model describes a life moving between a finite set of states, such as healthy, sick and dead. It is Markov if the future depends only on the current state, not on the path taken. To solve questions, draw the state diagram, list the intensities, and use the Chapman-Kolmogorov equations to combine transition probabilities over time.

Understand Multiple State Models and Markov Processes

A multiple state model has a finite state space and describes which state a life occupies at each time. A life insurance policy is the simplest case: two states, alive and dead. Health and disability cover needs more, for example healthy, sick and dead. Each arrow on the state diagram is a possible transition.

The model is built on a Markov process. The Markov property says that, given the state you are in now, the past does not change the probability of what happens next. In symbols, P[X(t) = j | X(s) = i and the whole history before s] = P[X(t) = j | X(s) = i]. Only the current state (and the current age or time) matters, not how long you have been there or where you came from.

The transition probability t p_x^ij is the probability that a life in state i at age x is in state j at age x + t. Note that j can equal i. Be careful with one point: t p_x^ii allows the life to leave state i and come back, while the holding (waiting) probability t p_x^īi (written with a bar over the second i) means the life stays in i for the whole period without leaving at all.

The model is driven by transition intensities μ_x^ij, the instantaneous rate of moving from i to j at age x. For small h, h p_x^ij ≈ h × μ_x^ij when i ≠ j. If intensities are constant, the model is time-homogeneous and the calculations simplify a lot.

A multiple decrement model is a special case. There is one starting state (for example active employee) and several exit states (withdrawal, retirement, death). Once a life leaves, it never returns. Setting up the state space correctly is the first and most mark-earning step. If the real process depends on duration in a state, the plain Markov model is not suitable unless you add states to capture that duration.

Key rules to remember

Markov property: P[X(t) = j | X(s) = i, X(r) = x(r) for all r < s] = P[X(t) = j | X(s) = i]; State this in words too: the future depends only on the present state, not on the history.
Transition probability: t p_x^ij = P[X(x + t) = j | X(x) = i]; j may equal i. It includes paths that leave i and return.
Probabilities sum to one: Σ over all states j of t p_x^ij = 1; Use this to find a missing probability or to check your answers.
Chapman-Kolmogorov equations: s+t p_x^ij = Σ over k of ( s p_x^ik × t p_{x+s}^kj ); Sum over every state k the life could be in at the intermediate time. The second factor uses age x + s.
Transition intensity: μ_x^ij = lim as h→0 of ( h p_x^ij ÷ h ), for i ≠ j; Defined for different states. It is a rate, not a probability, and can exceed 1.
Holding probability: t p_x^īi = exp( − ∫ from 0 to t of Σ over j≠i of μ_{x+s}^ij ds ); Probability of remaining continuously in state i. With constant intensities it is exp(−t × total intensity out of i).
Probability of a first move to j: P[in j at time t, having first left i directly to j] = ∫ from 0 to t of ( s p_x^īi × μ_{x+s}^ij × (t−s) p_{x+s}^jj ) ds; Condition on the time s of leaving i. If j is absorbing, the last factor is 1. In general t p_x^ij also includes routes that go from i to j via other states, so add those paths. The integral equals t p_x^ij only when j can be reached from i by this direct route alone.

How to solve Multiple State Models and Markov Processes questions

Use this order for any question on multiple state models. It keeps the working clear and shows the examiner you understand the structure.

1Read the policy or situation and list the states. Mark which are absorbing, such as dead or withdrawn.
2Draw the state diagram. Add an arrow for every transition that is possible and label it with its intensity or probability. Leave out impossible transitions.
3State the assumptions: Markov property, constant or age-dependent intensities, and whether the model is time-homogeneous.
4Decide what is asked: a holding probability, a transition probability over a period, or a probability of a path.
5If the answer needs a path, split the period at the intermediate time or states and apply Chapman-Kolmogorov. Sum over all possible intermediate states.
6For leaving a state, compute the holding probability from the total intensity out of that state. For ending in a given state, integrate over the time of leaving if needed.
7Check that the probabilities from the starting state sum to 1 and that each lies between 0 and 1.
8Write the answer clearly with the notation, for example 2 p_x^HD = 0.064, and a short statement in words.

Quickest way: Matrix and holding-probability shortcut

When to use it: Use it when you are given one-step transition probabilities, or constant intensities, and need probabilities over one or two periods in a small state space.

Write the one-step transition matrix with rows as the starting state. Check every row sums to 1.
For a two-step probability from i to j, multiply row i by column j: sum of p(i,k) × p(k,j) over all k.
Compute all two-step probabilities from the one starting state, then check they sum to 1.
With constant intensities, the probability of staying put for t is exp(−t × sum of intensities leaving the state). Do this first, then subtract from 1 for the chance of having left.
For the probability of ending in an absorbing state j with constant intensities, use (μ^ij ÷ total intensity out of i) × (1 − exp(−t × total)). This works only when j is absorbing and a life in state i can leave only to states from which it cannot return, with all intensities constant. Do not use it if j is not absorbing.

Common mistakes in Multiple State Models and Markov Processes

Confusing t p_x^ii with the holding probability t p_x^īi.
The notation looks almost the same, and both describe a life that is in state i at the end.
Fix: Ask whether leaving and returning is allowed. If yes, use t p_x^ii. If the life must stay in i throughout, use the holding probability with the bar.
Treating intensities as probabilities, or adding them to 1.
Intensities such as 0.05 look like probabilities.
Fix: Intensities are rates per year. Use them only inside exp(−…) or integrals, or as h × μ for a very small h.
Leaving out an intermediate state in Chapman-Kolmogorov.
Students include only the 'obvious' path and forget routes such as staying healthy, then moving.
Fix: Write the sum over k with every state listed. For three states there are three terms, even if one is zero.
Using the same age in both factors of Chapman-Kolmogorov.
With constant intensities or a homogeneous table it does not matter, so the habit forms.
Fix: Write s p_x^ik × t p_{x+s}^kj. The second factor starts at age x + s.
Letting the model move out of an absorbing state, or showing a transition that cannot happen.
The diagram is drawn from the story without checking, for example allowing dead to healthy.
Fix: Mark absorbing states and give them no outgoing arrows. Then t p^DD = 1 and t p^Dj = 0 for j ≠ D.
Applying the Markov model when the transition rate depends on time spent in the state.
Students forget the Markov property is a real assumption, not a given.
Fix: Say that the assumption is needed. If duration matters, such as recovery chances falling with the length of sickness, mention adding states or a different model.

Worked examples

Example 1

A health insurance model has three states: H (healthy), S (sick) and D (dead). D is absorbing. The Markov property holds, and the transition probabilities over one year are the same each year: from H, 0.90 to H, 0.07 to S, 0.03 to D. From S, 0.30 to H, 0.60 to S, 0.10 to D. A life is healthy at time 0. Find the probability that the life is in each state after two years and check the results.

Show the solution

Write the transition matrix P with rows H, S, D. Row H = (0.90, 0.07, 0.03). Row S = (0.30, 0.60, 0.10). Row D = (0, 0, 1). Each row sums to 1.
By Chapman-Kolmogorov with s = t = 1 and a time-homogeneous model, 2 p^HH = p^HH p^HH + p^HS p^SH + p^HD p^DH = 0.90 × 0.90 + 0.07 × 0.30 + 0.03 × 0 = 0.81 + 0.021 = 0.831.
2 p^HS = p^HH p^HS + p^HS p^SS + p^HD p^DS = 0.90 × 0.07 + 0.07 × 0.60 + 0 = 0.063 + 0.042 = 0.105.
2 p^HD = p^HH p^HD + p^HS p^SD + p^HD p^DD = 0.90 × 0.03 + 0.07 × 0.10 + 0.03 × 1 = 0.027 + 0.007 + 0.030 = 0.064.
Check: 0.831 + 0.105 + 0.064 = 1.000.

Answer: After two years the probabilities are: healthy 0.831, sick 0.105, dead 0.064. They sum to 1.

Example 2

An employer models its staff with four states: 0 (active), 1 (withdrawn), 2 (retired through ill health) and 3 (dead). States 1, 2 and 3 are absorbing. The Markov property holds and the transition intensities from state 0 are constant: μ^01 = 0.06, μ^02 = 0.02, μ^03 = 0.01 per year. For an active employee, find (a) the probability of still being active after 3 years and (b) the probability of having left by withdrawal within 3 years.

Show the solution

Draw the diagram: one arrow from 0 to each of 1, 2 and 3, and no arrows out of states 1, 2 and 3.
Total intensity out of state 0 = 0.06 + 0.02 + 0.01 = 0.09 per year.
(a) The holding probability is 3 p^0̄0 = exp(−0.09 × 3) = exp(−0.27). Since there is no return to state 0, this equals 3 p^00. exp(−0.27) = 0.76338.
(b) Condition on the time s of leaving. 3 p^01 = ∫ from 0 to 3 of exp(−0.09 s) × 0.06 ds, because state 1 is absorbing so the later factor is 1.
Evaluate: 0.06 × (1 − exp(−0.27)) ÷ 0.09 = (2/3) × (1 − 0.76338) = (2/3) × 0.23662 = 0.15775.
Check: the three exit probabilities share 0.23662 in proportion 6 : 2 : 1, so withdrawal gets 6/9 of it, which agrees.

Answer: (a) 0.7634 (b) 0.1577

Exam tips

Start every written answer with a state diagram and a one-line statement of the Markov assumption. These are quick marks and they organise your thinking.
In MCQs, check first whether the question asks for t p^ii or the holding probability. The two options often differ only by the return paths.
Always run the sum-to-1 check on your probabilities. It catches arithmetic and missed-path errors in seconds.
Keep notation exact: superscripts for states, subscript for age, and a bar for holding probabilities. Say which intensities are constant.
For Paper B work in R or Excel, set up the transition matrix cleanly, label states, and show the matrix multiplication or formula you used.

Practice questions from Valuing cashflows contingent on multiple transition events

Multiple State Models and Markov Processes: frequently asked questions

What is the Markov property in a multiple state model?

It says that, given the state a life is in now, the probability of future moves does not depend on the past path. Only the current state, and the current age or time, matter. You should state it as an assumption whenever you use a Markov model.

What are the Chapman-Kolmogorov equations in simple words?

They say that to go from state i to state j over s + t years, you sum over every state k the life could be in at time s. For each k, multiply the probability of reaching k by the probability of going from k to j in the remaining time. Remember to start the second factor at age x + s.

How is a multiple decrement model different from a general multiple state model?

A multiple decrement model has one starting state and several exit states, and no one returns after leaving. A general multiple state model can allow movement back, for example sick to healthy. The decrement model is a special case.

What is the difference between a transition intensity and a transition probability?

A probability is a number between 0 and 1 over a stated period. An intensity is an instantaneous rate and can be larger than 1. For a very short period h, the probability of moving is approximately h times the intensity.