Risk Modelling and Survival Analysis · Stochastic processes

How to Find the Stationary Distribution of a Markov Chain

Updated 11 October 2026 · Fact-checked

A stationary distribution π is a probability vector that the chain leaves unchanged: πP = π with Σπ = 1. You find it by solving those equations. It equals the long-run limit of the n-step probabilities only if the chain is irreducible, aperiodic and positive recurrent. Classify the states first.

Understand Stationary Distributions and Long-Term Behaviour

A Markov chain moves between states, and the next state depends only on the current one. The question for long-term behaviour is: after many steps, where is the chain likely to be, and does the starting state still matter?

To answer it, you first classify the states. State j is accessible from i if you can get from i to j in some number of steps with positive probability. If i and j are accessible from each other, they communicate. Communication splits the state space into communicating classes. A chain with one class is irreducible. A class is closed if the chain cannot leave it. An absorbing state is a closed class with one state.

A state is recurrent if the chain returns to it with probability 1, and transient otherwise. In a finite chain, a class is recurrent exactly when it is closed. The period of state i is d = gcd of all n with p_ii(n) > 0. If d = 1 the state is aperiodic. Periodicity is a class property: every state in a communicating class has the same period.

A stationary distribution π satisfies πP = π, with π_j ≥ 0 and Σπ_j = 1. If you start the chain with distribution π, it has distribution π at every later time. A finite irreducible chain has exactly one stationary distribution. If the chain has several closed classes, there are many stationary distributions. If the chain is irreducible but periodic, π still exists but the n-step probabilities do not converge to it.

The limiting distribution is the limit of p_ij(n) as n → ∞. It exists and equals π for every starting state when the chain is irreducible, aperiodic and positive recurrent (all finite irreducible aperiodic chains qualify). So a limiting distribution is always stationary, but a stationary distribution need not be a limiting one. That is the key difference the exam tests.

Key rules to remember

Stationary distribution equations: πP = π, Σ π_j = 1, π_j ≥ 0; Written out: π_j = Σ_i π_i p_ij. One equation is redundant, so replace one of them with Σ π_j = 1.
Chapman-Kolmogorov (n-step): P(n) = Pⁿ, so p_ij(m+n) = Σ_k p_ik(m) p_kj(n); Use it to test whether p_ii(n) > 0 when finding the period.
Period of a state: d_i = gcd{ n ≥ 1 : p_ii(n) > 0 }; d_i = 1 means aperiodic. A self-loop (p_ii > 0) makes the state aperiodic.
Existence and uniqueness: Finite, irreducible chain ⇒ unique π with π_j > 0 for all j; If there is more than one closed class, π is not unique.
Limiting behaviour: Irreducible, aperiodic, positive recurrent ⇒ p_ij(n) → π_j as n → ∞; The limit does not depend on the starting state i.
Mean recurrence time: m_j = 1 ÷ π_j; Expected number of steps to return to state j, for an irreducible positive recurrent chain.
Long-run proportion of time: (1/n) × (number of visits to j in n steps) → π_j; Holds for irreducible positive recurrent chains even if periodic.

How to solve Stationary Distributions and Long-Term Behaviour questions

Use this order for any question on classification, stationary distributions or long-term behaviour.

1Draw the transition diagram from P. Mark every arrow with positive probability.
2Find the communicating classes. State which are closed and which are not. Name any absorbing states. Say whether the chain is irreducible.
3Label each class recurrent or transient. In a finite chain, closed means recurrent and not closed means transient.
4Find the period of each class using gcd{n : p_ii(n) > 0}. Look for self-loops first, then for short cycles.
5Write down πP = π and Σπ_j = 1. Express every π_j in terms of one of them, then use the sum to fix the constant.
6Check your answer by substituting into one equation you did not use.
7State whether p_ij(n) converges. Give the limit only if the chain is irreducible and aperiodic (finite case). Otherwise say what does happen, such as a limit that depends on the start or a cycle.
8If asked, give 1 ÷ π_j as the mean recurrence time, or π_j as the long-run proportion of time.

Quickest way: Fast route to π for a small chain

When to use it: Use for 2 to 4 state chains where P has many zeros, which is typical in CS2 Paper A.

Skip the full diagram. Scan each row and column for zeros to see the cycles.
Set π for one state as x. Use the equation for the column with the fewest non-zero entries to express the other π_j in terms of x.
Chain the equations along the diagram. For a birth-death type chain, balance flows between neighbouring states: π_i p_i,i+1 = π_{i+1} p_{i+1,i}.
Add up all the multiples of x, set the total to 1, and solve for x.
Check with one unused equation. Then state irreducible, aperiodic or not, and conclude on the limit.

Common mistakes in Stationary Distributions and Long-Term Behaviour

Solving πP = π without using Σπ = 1.
The equations πP = π are dependent and give only ratios, so students stop at the ratios.
Fix: Always finish by normalising. Replace one redundant equation with Σπ_j = 1.
Using the columns of P instead of the rows, or solving Pπ = π.
Confusion with eigenvector conventions from other courses.
Fix: π is a row vector multiplying P on the left. The equation for π_j uses column j of P: π_j = Σ_i π_i p_ij.
Saying that a stationary distribution means p_ij(n) converges to it.
Stationary and limiting distributions are treated as the same thing.
Fix: Convergence needs irreducibility and aperiodicity too. A periodic chain has π but p_ii(n) oscillates. A reducible chain may have many π.
Declaring a chain irreducible because every state has an outgoing arrow.
Students check outgoing arrows but not whether every state can be reached from every other.
Fix: Check that you can get from each state to every other state. One absorbing state is enough to make a chain reducible.
Taking the period as the shortest return time, or as 1 whenever some return is possible.
The gcd definition is forgotten.
Fix: List the return lengths n with p_ii(n) > 0 and take their gcd. Returns in 2 and 3 steps give gcd 1. Returns only in 2, 4, 6 steps give period 2.
Calling a non-closed class recurrent, or a closed class transient.
Mixing up the definition with the finite-chain shortcut.
Fix: In a finite chain, closed class means recurrent and non-closed class means transient. State this reason in the answer.

Worked examples

Example 1

A Markov chain on states 1, 2, 3 has transition matrix P with rows (0.5, 0.5, 0), (0.25, 0.5, 0.25) and (0, 0.5, 0.5). (a) Show the chain is irreducible and aperiodic. (b) Find the stationary distribution. (c) State lim p_13(n) and the mean recurrence time of state 1.

Show the solution

(a) From 1 you can reach 2 directly, and from 2 you reach 3. From 3 you reach 2, and from 2 you reach 1. So all states communicate and the chain is irreducible.
Each state has p_ii = 0.5 > 0, so p_ii(1) > 0 and the period is 1. The chain is aperiodic.
(b) πP = π gives: π1 = 0.5π1 + 0.25π2, π2 = 0.5π1 + 0.5π2 + 0.5π3, π3 = 0.25π2 + 0.5π3.
From the first: 0.5π1 = 0.25π2, so π2 = 2π1. From the third: 0.5π3 = 0.25π2, so π3 = 0.5π2 = π1.
Normalise: π1 + 2π1 + π1 = 4π1 = 1, so π1 = 0.25.
Check the middle equation: 0.5(0.25) + 0.5(0.5) + 0.5(0.25) = 0.125 + 0.25 + 0.125 = 0.5 = π2. ✓
(c) The chain is finite, irreducible and aperiodic, so p_13(n) → π3 = 0.25.
Mean recurrence time of state 1 = 1 ÷ π1 = 1 ÷ 0.25 = 4.

Answer: π = (0.25, 0.5, 0.25). lim p_13(n) = 0.25. Mean recurrence time of state 1 is 4 steps.

Example 2

A Markov chain on states 1, 2, 3 has p_12 = 1, p_21 = 0.5, p_23 = 0.5, p_32 = 1, and all other entries zero. (a) Find the period of the chain. (b) Find the stationary distribution. (c) Does p_11(n) converge? Explain.

Show the solution

(a) The chain is irreducible: 1 ↔ 2 and 2 ↔ 3. So all states share one period.
Returns to state 1 are 1→2→1 (2 steps), and also 1→2→3→2→1 (4 steps). Every return takes an even number of steps because the chain alternates between state 2 and the set {1, 3}.
So gcd{2, 4, 6, ...} = 2. The chain has period 2.
(b) πP = π gives: π1 = 0.5π2, π3 = 0.5π2, π2 = π1 + π3.
The third equation holds automatically from the first two. Normalise: 0.5π2 + π2 + 0.5π2 = 2π2 = 1, so π2 = 0.5.
So π1 = 0.25 and π3 = 0.25.
(c) p_11(n) = 0 for odd n. For even n, starting at 1, the chain is at 2 after an odd number of steps, then moves to 1 or 3 with probability 0.5 each. So p_11(n) = 0.5 for every even n.
The sequence 0, 0.5, 0, 0.5, ... does not converge. The average over a cycle is 0.25 = π1, which matches the long-run proportion of time in state 1.

Answer: Period 2. π = (0.25, 0.5, 0.25). p_11(n) alternates between 0 and 0.5, so it does not converge. π exists and gives the long-run proportion of time in each state, but it is not a limiting distribution.

Exam tips

Write the classification before any calculation. Marks are given for stating communicating classes, closed or not, and period, each with a reason.
Always say why a limit exists: irreducible, aperiodic and finite. A bare statement that the chain converges loses marks.
In Paper A MCQs, a quick check of the period or of whether there is more than one closed class often removes two options straight away.
Show the check of your π in one unused equation. It protects against arithmetic slips and shows method.
In a computer-based paper, state that you solve πP = π with the sum constraint, or take a high power of P, and say that the power method works only if the chain is aperiodic and irreducible.

Practice questions from Stochastic processes

Stationary Distributions and Long-Term Behaviour: frequently asked questions

How do I find the stationary distribution of a Markov chain?

Solve πP = π together with Σπ_j = 1. Express each π_j in terms of one unknown using the equations, then use the sum to fix it. Check in one equation you did not use.

What is the difference between stationary and limiting distribution?

A stationary distribution stays unchanged after one step: πP = π. A limiting distribution is the limit of p_ij(n) as n grows. Every limiting distribution is stationary, but a periodic or reducible chain can have stationary distributions without a limit that is the same for every start.

How do I check if a Markov chain is irreducible?

Draw the transition diagram and test whether you can get from every state to every other state. If one state is absorbing, or the states split into groups with no route between them, the chain is reducible.

How do I find the period of a state?

List the numbers of steps n for which p_ii(n) > 0 and take their greatest common divisor. A self-loop gives period 1. Periodicity is the same for every state in a communicating class.

Can a Markov chain have more than one stationary distribution?

Yes, if it has more than one closed communicating class. Each closed class supports its own stationary distribution, and any mixture of them is also stationary. A finite irreducible chain has exactly one.