Risk Modelling and Survival Analysis · Markov chains

Transition Probabilities and Chapman-Kolmogorov Equations Explained

Updated 11 October 2026 · Fact-checked

A Markov chain's one-step transition matrix P holds the probabilities of moving between states in one step. The Chapman-Kolmogorov equations say the n-step matrix is the matrix power P^n. To find an n-step probability, multiply P by itself n times (or use a path split) and read off the required entry.

Understand Transition Probabilities and Chapman-Kolmogorov Equations

A Markov chain is a process that moves between a set of states. The key idea is the Markov property: the next state depends only on the current state, not on how you got there. Given today's state, the past adds no information.

In a time-homogeneous chain, the probability of moving from state i to state j in one step is the same at every time. We write it p_ij = P(X_{n+1} = j | X_n = i). Put all these into a square matrix P. This is the one-step transition matrix. Every entry is between 0 and 1 and every row sums to 1, because from state i you must go somewhere.

The n-step transition probability p_ij^(n) = P(X_{m+n} = j | X_m = i) is the chance of being in j after n steps, starting from i. To get from i to j in n steps, you must be in some state k after a smaller number of steps. The Chapman-Kolmogorov equations add up over all possible k. This gives p_ij^(m+n) = Σ_k p_ik^(m) × p_kj^(n).

In matrix form this is P^(m+n) = P^(m) P^(n). So the n-step matrix is simply P multiplied by itself n times. If you only need one entry, you do not need the whole matrix. Add up the probabilities of the paths, or multiply a row vector by P repeatedly.

If the starting state is random, with distribution row vector π0, then the distribution after n steps is π0 P^n. The order of multiplication matters: matrices do not commute in general, and the row vector goes on the left.

Key rules to remember

One-step transition probability: p_ij = P(X_{n+1} = j | X_n = i); Holds for all n if the chain is time-homogeneous. Each row of P sums to 1.
Markov property: P(X_{n+1} = j | X_n = i, X_{n-1} = i_{n-1}, ..., X_0 = i_0) = P(X_{n+1} = j | X_n = i); Future depends only on the present state.
Chapman-Kolmogorov equations: p_ij^(m+n) = Σ_k p_ik^(m) × p_kj^(n); Sum over all states k. Valid for any non-negative integers m and n.
Matrix form: P^(n) = P^n; Entry (i, j) of P^n is the n-step probability from i to j. Use for time-homogeneous chains.
Distribution after n steps: π_n = π_0 P^n; π_0 is a row vector of the initial distribution.
Row-sum check: Σ_j p_ij = 1 for each i; Use to check P and also to check P^n.

How to solve Transition Probabilities and Chapman-Kolmogorov Equations questions

Use this method for any question on one-step or n-step transition probabilities.

1Define the states clearly and write the one-step matrix P. Check each row sums to 1.
2Identify the start state (or initial distribution) and the end state, and the number of steps n.
3Decide whether you need the full matrix P^n or only one entry. For small n, one entry via a path split is quicker.
4Apply Chapman-Kolmogorov: split n as m + (n − m) and sum over the intermediate states k.
5For P^n with larger n, compute by repeated squaring, for example P^4 = (P^2)^2, or multiply a row vector by P step by step.
6If the start is random, multiply the initial row vector by P^n and read off the required state.
7Check that the row sums of any computed matrix are 1 and that answers lie between 0 and 1.
8State the answer as a probability, with the assumption of time-homogeneity if you used it.

Quickest way: Row vector times P, step by step

When to use it: When you need one n-step probability from a known start state and n is small (about 2 to 4), or on a calculator-only MCQ.

Write the start state as a row vector, for example (1, 0, 0) if you start in state 1.
Multiply it by P to get the distribution after one step. This is just the first row of P.
Multiply the result by P again for each extra step. Do not square the whole matrix.
After n steps, read off the entry for the target state.
For n = 2 to a single target, use p_ij^(2) = Σ_k p_ik p_kj directly, which is one short sum.

Common mistakes in Transition Probabilities and Chapman-Kolmogorov Equations

Multiplying entries of P together element by element instead of doing matrix multiplication.
Squaring each entry looks like 'P squared'.
Fix: Use row-by-column multiplication. Entry (i, j) of P² is Σ_k p_ik p_kj.
Multiplying in the wrong order, such as P π0 instead of π0 P.
Column-vector habits from other maths courses.
Fix: In this course the distribution is a row vector on the left: π_n = π_0 P^n. Check the dimensions.
Forgetting to sum over all intermediate states in Chapman-Kolmogorov.
Students follow only the most obvious path.
Fix: List every state k, including k = i and k = j, and add every term, including those with zero probability if you wish to be safe.
Adding probabilities of paths instead of multiplying along each path.
Confusing the 'and' and 'or' rules.
Fix: Multiply along a path (step 1 then step 2). Add across different paths.
Using a transition matrix that has rows not summing to 1, or reading columns as rows.
Matrix is built from a description with rows and columns swapped, or an arithmetic slip.
Fix: Check that each row sums to 1 before you start and again after computing P^n. Row = from, column = to.
Applying P^n when the chain is not time-homogeneous.
Assuming the matrix is constant without checking.
Fix: If the matrix changes with time, multiply the specific matrices in order: P(0) P(1) ... P(n−1). State the assumption you use.

Worked examples

Example 1

A Markov chain has states 1, 2 and 3 with one-step transition matrix P with rows: (0.5, 0.3, 0.2), (0.2, 0.6, 0.2), (0.1, 0.3, 0.6). Find the probability that the chain is in state 3 after two steps, given it starts in state 1.

Show the solution

We need p_13^(2) = Σ_k p_1k p_k3, using row 1 of P and column 3 of P.
Row 1 is (0.5, 0.3, 0.2). Column 3 is (0.2, 0.2, 0.6).
Compute: 0.5 × 0.2 = 0.10.
0.3 × 0.2 = 0.06.
0.2 × 0.6 = 0.12.
Sum: 0.10 + 0.06 + 0.12 = 0.28.

Answer: p_13^(2) = 0.28

Example 2

A no-claims-discount system has two states: A (no discount) and B (discount). A policyholder in A moves to B with probability 0.7 and stays in A with probability 0.3. A policyholder in B stays in B with probability 0.9 and moves to A with probability 0.1. The policyholder starts in A. Find the probability of being in B after three years.

Show the solution

P has rows A: (0.3, 0.7) and B: (0.1, 0.9). Each row sums to 1.
Start vector π0 = (1, 0).
After one step: π1 = (0.3, 0.7).
After two steps: A = 0.3 × 0.3 + 0.7 × 0.1 = 0.09 + 0.07 = 0.16. B = 0.3 × 0.7 + 0.7 × 0.9 = 0.21 + 0.63 = 0.84. So π2 = (0.16, 0.84).
After three steps: B = 0.16 × 0.7 + 0.84 × 0.9 = 0.112 + 0.756 = 0.868.
Check: A = 0.16 × 0.3 + 0.84 × 0.1 = 0.048 + 0.084 = 0.132. Then 0.132 + 0.868 = 1.

Answer: P(in B after three years) = 0.868

Exam tips

Write the matrix with row and column labels. Most lost marks come from reading the wrong row or column.
In written questions, quote the Chapman-Kolmogorov equation before you use it, then show each product term.
For n = 2 or 3 to a single target state, the row-vector method is faster and less error-prone than finding the full P^n.
Always do a row-sum check on your result. It takes seconds and catches arithmetic slips.
State the time-homogeneity assumption when you use P^n. If the question gives different matrices for different years, multiply them in time order.

Practice questions from Markov chains

Transition Probabilities and Chapman-Kolmogorov Equations: frequently asked questions

What do the Chapman-Kolmogorov equations say in simple words?

To go from state i to state j in m + n steps, the chain must be in some state k after m steps. The equations add up the probability of going i to k in m steps, then k to j in n steps, over all k. In matrix form this is P^(m+n) = P^(m) P^(n).

How do I find an n-step transition probability?

Find the (i, j) entry of P^n. For small n, use a sum over intermediate states. For one start state, multiply the start row vector by P repeatedly, which is quicker than finding the whole matrix.

Do the rows or columns of a transition matrix sum to 1?

The rows sum to 1. Row i lists the probabilities of moving from state i to each state. Columns do not need to sum to 1.

Does Chapman-Kolmogorov need a time-homogeneous chain?

The equation itself holds for any Markov chain if you use the correct time-dependent matrices. The simple form P^n needs time-homogeneity, meaning the same P at every step. Exam questions usually say when this applies.