Risk Modelling and Survival Analysis · Markov processes

Markov Jump Processes and Kolmogorov Equations Explained

Updated 11 October 2026 · Fact-checked

A Markov jump process is a continuous-time process with discrete states, where the future depends only on the present state. You describe it by transition rates μij. Holding times are exponential. Transition probabilities solve the Kolmogorov forward or backward equations, written using the generator matrix A.

Understand Markov Jump Processes and Kolmogorov Equations

A Markov jump process moves between a finite or countable set of states, and it can jump at any moment in continuous time. Examples are Healthy, Sick and Dead in a health model, or the number of claims so far. The Markov property says that, given the current state, the past does not affect the future.

The process is described by transition rates (also called transition intensities). For i ≠ j, μij is defined by P(X(t+h) = j | X(t) = i) = h·μij + o(h) as h → 0. So over a very short time h, the chance of a jump from i to j is about μij × h. In a survival model, μ for Alive to Dead is just the force of mortality. In an exam you are often told that the rates are constant. If they depend on age or time, the process is time-inhomogeneous.

If you stay in state i, the holding time is the time until the next jump. It has an exponential distribution with parameter λi = Σ(j≠i) μij, the total rate of leaving i. When the jump happens, the process goes to state j with probability μij ÷ λi. This is the jump chain. The holding time and the destination are independent.

The generator matrix A collects all the rates. Off the diagonal, Aij = μij. On the diagonal, Aii = −λi. Every row sums to zero. Probabilities pij(t) = P(X(t) = j | X(0) = i) form the matrix P(t). They satisfy the Kolmogorov equations. The forward equations follow by conditioning on the state just before time t+h. The backward equations follow by conditioning on what happens just after time 0.

The Poisson process with rate λ is a special case. It has states 0, 1, 2, ... and the only possible jump is from n to n+1 at rate λ. Holding times are exponential with mean 1/λ. The number of events in time t is Poisson with mean λt.

Key rules to remember

Transition rate definition: P(X(t+h) = j | X(t) = i) = h·μij + o(h), for i ≠ j; Also pii(h) = 1 − h·λi + o(h), where o(h)/h → 0.
Total rate of leaving state i: λi = Σ(j≠i) μij; This is the parameter of the exponential holding time in state i.
Holding time distribution: Ti ~ Exp(λi); P(Ti > t) = e^(−λi t); E[Ti] = 1 ÷ λi; Valid for time-homogeneous (constant rate) processes. Var(Ti) = 1 ÷ λi².
Jump probabilities: P(next state is j | jump from i) = μij ÷ λi; These probabilities sum to 1 over j ≠ i.
Generator matrix: Aij = μij (i ≠ j); Aii = −λi; each row sums to 0; Check the row sums every time you build A.
Kolmogorov forward equations: d/dt pij(t) = Σ(k≠j) pik(t)·μkj − pij(t)·λj; matrix form: P′(t) = P(t)·A; Rate into j from other states, minus rate out of j. Starting state i is fixed.
Kolmogorov backward equations: d/dt pij(t) = Σ(k≠i) μik·pkj(t) − λi·pij(t); matrix form: P′(t) = A·P(t); Here the end state j is fixed and you vary the starting state.
Solution for constant rates: P(t) = exp(tA), with P(0) = I; Useful for two-state models and for checking answers.
Occupancy probability: pii-bar(t) = P(stay in i throughout [0, t]) = e^(−λi t); Different from pii(t), which allows leaving and returning.
Poisson process: P(N(t) = n) = e^(−λt)·(λt)^n ÷ n!; inter-arrival times ~ Exp(λ); Increments over disjoint intervals are independent and stationary.

How to solve Markov Jump Processes and Kolmogorov Equations questions

Use this routine for most questions on jump processes, whether you are asked for rates, holding times or Kolmogorov equations.

1Draw the state diagram. Label each state and write the rate on every arrow.
2Write the generator matrix A. Fill in off-diagonal rates, then set each diagonal entry to minus the sum of the rest of the row.
3Identify what is asked: a holding time, a jump probability, an occupancy probability, a pij(t), or an equation.
4For holding times, use λi, the total rate out of state i, and the exponential distribution. For the next state, use μij ÷ λi.
5For an equation, decide forward or backward. Forward: fix the starting state i, look at who feeds j. Backward: fix the end state j, look at where i can go first.
6For each term write inflow minus outflow. Outflow uses the full rate λ of the state you are in, not one rate.
7Solve if asked. For two states, use the equations directly. For one-way chains, solve in order from the first state onward, using an integrating factor.
8Check: probabilities lie in [0, 1], rows of P(t) sum to 1 and rows of A sum to 0.

Quickest way: Write inflow minus outflow from the diagram

When to use it: When you are asked to write down or use the Kolmogorov equations for a small model and have little time.

Sketch the arrows and rates.
For forward equations for pi·(t), take each target state j: rate in = sum of pik(t) × μkj over arrows into j; rate out = pij(t) × total rate leaving j.
For backward equations, take start state i: sum of μik × (pkj(t) − pij(t)) over arrows leaving i.
For a state with no way back, solve it first. For example, in a model where the process can only leave state 0, p00(t) = e^(−λ0 t) straight away.
Then substitute to get the next state.

Common mistakes in Markov Jump Processes and Kolmogorov Equations

Using the wrong diagonal entry in the generator, such as −μij for one arrow rather than the total rate out.
Students focus on one arrow and forget other exits from the state.
Fix: Set Aii = −(sum of all other entries in row i). Confirm every row sums to zero.
Mixing up forward and backward equations.
Both look similar, and the matrix forms P′ = PA and P′ = AP differ only by order.
Fix: In forward equations the rates μkj carry the target state j. In backward equations the rates μik carry the starting state i. Check which index the rate shares with the probability.
Using the rate μij as the holding time parameter instead of λi.
Students forget that the holding time ends at any jump, not just a particular one.
Fix: Always use λi = Σ(j≠i) μij. Use μij ÷ λi only for the destination.
Treating pii(t) as the probability of staying in state i all the time.
The notation suggests staying, but pii(t) includes leaving and coming back.
Fix: Use e^(−λi t) for continuous stay. Use pii(t) only for being in i at time t.
Confusing a rate with a probability, for example writing P(jump in a year) = μ.
A rate looks like a probability, but it can exceed 1 and has units of per time.
Fix: For small h use P ≈ μh. For a full period with a constant total rate λ, use 1 − e^(−λt).
Forgetting boundary conditions P(0) = I when solving.
Students solve the differential equation and leave the constant undetermined.
Fix: Use pii(0) = 1 and pij(0) = 0 for i ≠ j to fix constants.

Worked examples

Example 1

A health model has states 1 (Healthy), 2 (Sick) and 3 (Dead). Constant rates per year: μ12 = 0.2, μ13 = 0.05, μ21 = 0.4, μ23 = 0.1. (a) Write the generator matrix. (b) Find the expected holding time in state 1 and the probability that the first jump from state 1 is to Dead. (c) Find the probability that a healthy life stays healthy for 2 years continuously.

Show the solution

Total rate out of state 1: λ1 = 0.2 + 0.05 = 0.25. Total out of state 2: λ2 = 0.4 + 0.1 = 0.5. State 3 is absorbing so its row is zero.
Generator matrix A, rows in order 1, 2, 3: row 1 = (−0.25, 0.2, 0.05); row 2 = (0.4, −0.5, 0.1); row 3 = (0, 0, 0). Each row sums to zero.
Holding time in state 1 is exponential with parameter 0.25, so the mean is 1 ÷ 0.25 = 4 years.
Probability the first jump is to Dead = μ13 ÷ λ1 = 0.05 ÷ 0.25 = 0.2.
Probability of staying in state 1 for 2 years = e^(−0.25 × 2) = e^(−0.5) = 0.6065.

Answer: (a) A = [[−0.25, 0.2, 0.05], [0.4, −0.5, 0.1], [0, 0, 0]]. (b) Mean holding time 4 years; probability the first jump is to Dead is 0.2. (c) e^(−0.5) ≈ 0.6065.

Example 2

A two-state model has states 0 (Alive) and 1 (Dead) with constant force of mortality μ = 0.02 per year and no recovery. (a) Write the Kolmogorov forward equation for p00(t) and solve it. (b) Hence find the probability that a life alive now is dead within 10 years, to 4 decimal places.

Show the solution

Rates: μ01 = 0.02, μ10 = 0. Total rate out of state 0 is λ0 = 0.02.
Forward equation for p00(t): d/dt p00(t) = p01(t)·μ10 − p00(t)·λ0 = 0 − 0.02·p00(t), since μ10 = 0.
So d/dt p00(t) = −0.02·p00(t). Separate variables: p00(t) = C·e^(−0.02t).
Boundary condition p00(0) = 1 gives C = 1, so p00(t) = e^(−0.02t).
Probability of dying within 10 years = 1 − p00(10) = 1 − e^(−0.2).
e^(−0.2) = 0.818731, so 1 − 0.818731 = 0.181269.

Answer: p00(t) = e^(−0.02t). Probability of death within 10 years = 1 − e^(−0.2) ≈ 0.1813.

Exam tips

Always draw the state diagram first. Most lost marks come from missing an arrow or mixing up indices.
State clearly whether you are writing forward or backward equations. Examiners award marks for the right form even if you do not solve it.
Show the generator matrix and check rows sum to zero. It is quick and catches errors.
For MCQs on holding times, compute λi as the total exit rate before anything else.
In Paper B, you may be asked to compute exp(tA) or solve the equations numerically. Set out the matrix and method, then give the result with units and sensible rounding.

Practice questions from Markov processes

Markov Jump Processes and Kolmogorov Equations: frequently asked questions

What is the generator matrix of a Markov jump process?

It is the matrix A of transition rates. Off-diagonal entries are the rates μij, and each diagonal entry is minus the sum of the other entries in its row. Every row sums to zero. For constant rates, P(t) = exp(tA).

Why is the holding time exponential?

The Markov property means the time already spent in a state tells you nothing about the time still to come. Only the exponential distribution has this memoryless property. Its parameter is the total rate of leaving the state.

When do I use the forward and when the backward equations?

Either gives the same answer. Forward equations fix the starting state and track where the process ends, and are the usual choice for multi-state models. Backward equations fix the end state and are useful for some theory proofs. Use whichever the question asks for.

How do I get transition rates from the force of mortality?

In a model with Alive and Dead, the rate from Alive to Dead is the force of mortality μx at that age. For small h, the chance of dying within h is about μx·h. In multi-state models, other transitions such as illness or recovery have their own rates, which may depend on age.

How is the Poisson process a Markov jump process?

It has states 0, 1, 2, ... and the only transition is from n to n+1 at constant rate λ. Holding times are Exp(λ), and N(t) is Poisson with mean λt.