Risk Modelling and Survival Analysis · Stochastic processes

Markov Jump Processes and Kolmogorov Forward and Backward Equations

Updated 11 October 2026 · Fact-checked

A Markov jump process is a continuous-time process with a discrete state space. It stays in a state for an exponential holding time, then jumps. Transition rates form the generator matrix A. The Kolmogorov equations are P'(t) = P(t)A (forward) and P'(t) = A P(t) (backward). Solve them for P(t).

Understand Markov Jump Processes and Kolmogorov Equations

A Markov jump process X(t) moves between a finite or countable set of states, and time is continuous. It has the Markov property: given the present state, the future does not depend on the past path. Examples are healthy-sick-dead models and no-claims-discount models in continuous time.

The process is described by transition rates (also called transition intensities). For i ≠ j, the rate μ_ij is the limit of P(X(t+h) = j | X(t) = i) ÷ h as h → 0. In other words, P(X(t+h) = j | X(t) = i) = μ_ij h + o(h). The rate out of state i is λ_i = Σ over j ≠ i of μ_ij.

The holding time in state i is exponential with parameter λ_i, so its mean is 1/λ_i. When the process leaves state i, it jumps to state j with probability μ_ij ÷ λ_i. The sequence of states visited is a discrete-time Markov chain called the jump chain. If λ_i = 0, state i is absorbing.

The generator matrix A collects all the rates. Off-diagonal entries are a_ij = μ_ij. Diagonal entries are a_ii = −λ_i. Every row sums to zero. This is a useful check.

Let p_ij(t) = P(X(t) = j | X(0) = i). Conditioning on what happens in the first small interval gives the backward equations. Conditioning on what happens in the last small interval gives the forward equations. Both give the same P(t). For a time-homogeneous process, P(t) = exp(tA) with P(0) = I.

Key rules to remember

Transition rate: μ_ij = lim (h → 0) P(X(t+h) = j | X(t) = i) ÷ h, for i ≠ j; Equivalent to p_ij(h) = μ_ij h + o(h).
Total rate of leaving a state: λ_i = Σ (j ≠ i) μ_ij; Holding time in state i is Exponential(λ_i), mean 1/λ_i.
Generator matrix: a_ij = μ_ij (i ≠ j); a_ii = −λ_i; each row sums to 0; Use the row-sum check to catch errors.
Jump probabilities: P(next state is j | leaving i) = μ_ij ÷ λ_i; Defined when λ_i > 0.
Kolmogorov forward equations: d/dt p_ij(t) = Σ (k ≠ j) p_ik(t) μ_kj − p_ij(t) λ_j; matrix form P'(t) = P(t) A; Condition on the state just before time t+h. Sum over the end state's inflows and outflow.
Kolmogorov backward equations: d/dt p_ij(t) = Σ (k ≠ i) μ_ik p_kj(t) − λ_i p_ij(t); matrix form P'(t) = A P(t); Condition on the first small step from the start state.
Matrix solution: P(t) = exp(tA) = Σ (n ≥ 0) (tA)^n ÷ n!, with P(0) = I; Valid for time-homogeneous processes.
Probability of staying in a state: P(X(s) = i for all s in [0, t] | X(0) = i) = exp(−λ_i t); This is not the same as p_ii(t), which allows leaving and returning.
Occupancy probability, inhomogeneous case: P(stay in i over [s, t]) = exp(−∫ from s to t of λ_i(u) du); Use when rates depend on time.

How to solve Markov Jump Processes and Kolmogorov Equations questions

Use this method for most questions on jump processes and Kolmogorov equations.

1Define the states and write down every transition rate given in the question. Check units (per year, per month).
2Build the generator matrix A. Put −λ_i on the diagonal and check that every row sums to zero.
3Identify what is asked: a holding time, a jump probability, an occupancy probability, or p_ij(t).
4For holding times, use the exponential with parameter λ_i. For next-state questions, use μ_ij ÷ λ_i.
5For p_ij(t), pick forward or backward. Choose the one that gives the simplest differential equation, usually the one involving the fewest unknown functions.
6Write the equation with the correct initial condition, such as p_ii(0) = 1 and p_ij(0) = 0 for j ≠ i. Solve it, often with an integrating factor, starting with absorbing or simple states.
7Check the result: probabilities lie in [0, 1], rows of P(t) sum to 1, and p_ij(0) matches the initial condition.
8State the answer with the notation, assumptions and units.

Quickest way: Rates, then simplest equation

When to use it: Use when time is short and you need p_ij(t) for a small model, such as two or three states.

Write A at once. Row sums of zero confirm it.
If the question is about staying in one state, skip Kolmogorov and use exp(−λ_i t).
For a two-state model with rates a (1 → 2) and b (2 → 1), use p_11(t) = b/(a+b) + a/(a+b) × exp(−(a+b)t). This is a standard result you can quote only if you can derive it quickly; otherwise derive it from the equations.
For a model with an absorbing state, solve the equation for the non-absorbing states first, then get the absorbing probability as 1 minus the others.
Check P(0) = I and the row sums before moving on.

Common mistakes in Markov Jump Processes and Kolmogorov Equations

Putting +λ_i on the diagonal of the generator matrix, or getting rows that do not sum to zero.
Students copy the rates into the matrix and forget the diagonal is the negative of the total outflow.
Fix: Set a_ii = −Σ of the other entries in the row. Always check the row sum.
Mixing up forward and backward equations.
Both look similar and both use A. The order of P(t) and A differs.
Fix: Forward: P'(t) = P(t)A, condition on the last step, rates act on the end state j. Backward: P'(t) = A P(t), condition on the first step, rates act on the start state i.
Using p_ii(t) as the probability of staying in state i throughout [0, t].
Both describe being in state i at time t.
Fix: p_ii(t) allows leaving and returning. Staying throughout has probability exp(−λ_i t).
Using μ_ij as the jump probability.
Rates and probabilities are confused.
Fix: Jump probability is μ_ij ÷ λ_i. Rates can exceed 1; probabilities cannot.
Forgetting initial conditions when solving, so the constant of integration is wrong.
Students focus on the differential equation and rush the final step.
Fix: Write p_ij(0) = δ_ij (1 if i = j, else 0) before solving and use it to fix the constant.
Mixing time units, such as a rate per year with a time in months.
Questions often give data in mixed units.
Fix: Convert everything to one unit before computing.

Worked examples

Example 1

A two-state Markov jump process has states 1 (healthy) and 2 (sick). The rate from 1 to 2 is 0.2 per year and the rate from 2 to 1 is 0.6 per year. (a) Write the generator matrix. (b) Find the mean time spent in state 2 on each visit. (c) Find the probability that a healthy life stays healthy throughout the next 2 years.

Show the solution

(a) λ_1 = 0.2 and λ_2 = 0.6. So A has rows (−0.2, 0.2) and (0.6, −0.6). Each row sums to 0.
(b) Holding time in state 2 is Exponential(0.6). Mean = 1 ÷ 0.6 = 1.6667 years.
(c) Probability of staying in state 1 for 2 years = exp(−0.2 × 2) = exp(−0.4).
exp(−0.4) = 0.6703.

Answer: (a) A = [[−0.2, 0.2], [0.6, −0.6]]. (b) 5/3 years, about 1.667 years. (c) exp(−0.4) ≈ 0.670.

Example 2

For the same process as above, derive p_11(t) using the Kolmogorov forward equations and find p_11(2).

Show the solution

Let a = 0.2 and b = 0.6. Forward equation: d/dt p_11(t) = p_12(t) × b − p_11(t) × a.
Since p_12(t) = 1 − p_11(t), we get d/dt p_11(t) = b(1 − p_11) − a p_11 = b − (a + b) p_11.
Write this as d/dt p_11 + 0.8 p_11 = 0.6. The integrating factor is exp(0.8t).
Then d/dt [p_11 exp(0.8t)] = 0.6 exp(0.8t). Integrating: p_11 exp(0.8t) = 0.75 exp(0.8t) + C.
So p_11(t) = 0.75 + C exp(−0.8t). Since p_11(0) = 1, C = 0.25.
p_11(t) = 0.75 + 0.25 exp(−0.8t). Check: this matches b/(a+b) = 0.75 and a/(a+b) = 0.25.
At t = 2: exp(−1.6) = 0.2019. So p_11(2) = 0.75 + 0.25 × 0.2019 = 0.75 + 0.0505 = 0.8005.

Answer: p_11(t) = 0.75 + 0.25 exp(−0.8t), so p_11(2) ≈ 0.8005.

Exam tips

Always write the generator matrix first, and show the row-sum check. It earns marks and prevents errors.
Say which equation you use (forward or backward) and why. State the initial conditions explicitly.
Distinguish staying throughout a period (exponential) from being in the state at the end (p_ii(t)). Examiners test this often.
In MCQs, check units and whether the question asks for a rate, a probability or a mean time before calculating.
In Paper B, you may be asked to compute exp(tA) numerically or simulate a jump process. Write the method and formulas, then the result.

Practice questions from Stochastic processes

Markov Jump Processes and Kolmogorov Equations: frequently asked questions

What is the difference between Kolmogorov forward and backward equations?

The forward equations condition on the last small time step, giving P'(t) = P(t)A. The backward equations condition on the first small time step, giving P'(t) = A P(t). For a time-homogeneous process both give P(t) = exp(tA).

How do I get the generator matrix from transition rates?

Put each rate μ_ij in row i, column j for i ≠ j. Set each diagonal entry to minus the sum of the other entries in its row. Every row then sums to zero.

Why is the holding time exponential?

The Markov property means the time already spent in a state tells you nothing about the time left. The exponential distribution is the only continuous distribution with this memoryless property. Its parameter is the total rate of leaving the state.

Which equation should I use in the exam?

Either gives the correct answer. Choose the one that leads to the simplest equations. Often this is the one where the unknown functions are fewest, for example when there is an absorbing state.