Risk Modelling and Survival Analysis · Markov processes

Classification of States and Stationary Distributions in Markov Chains

Updated 11 October 2026 · Fact-checked

Classify states by which states can reach each other: communicating classes, closed or open, recurrent or transient, with a period. A stationary distribution π is a probability vector with πP = π and Σπᵢ = 1. Solve those equations. An irreducible, aperiodic finite chain converges to π from any start.

Understand Classification of States and Stationary Distributions

A Markov chain moves between states, and the next state depends only on the current one. To predict where it ends up, first sort the states into groups, then look for a long-run pattern.

State j is accessible from state i if you can get from i to j in some number of steps with positive probability. If i and j are accessible from each other, they communicate. Communication splits the state space into communicating classes. A chain with only one class is irreducible. A class is closed if no transition leads out of it. A closed class with one state is an absorbing state.

A state is recurrent if, starting there, you return with probability 1. It is transient if there is a positive probability you never return. For a transient state, the number of visits is finite, so it is visited only a limited number of times. In a finite chain, a class is recurrent if it is closed and transient if it is not closed. Every state in a class has the same type, so you only test one state.

The period of state i is the greatest common divisor of all n ≥ 1 with pᵢᵢ(n) > 0. If the period is 1, the state is aperiodic. A self-loop (pᵢᵢ > 0) makes a state aperiodic. Periodicity is also shared by every state in a class.

A stationary distribution π is a probability vector that does not change after one step: πP = π. If the chain starts with distribution π, it has distribution π at every time. For a finite irreducible chain, π exists and is unique. If it is also aperiodic, the distribution of Xₙ tends to π whatever the starting state, so π is the long-run distribution. If the chain is periodic, π still exists but the distribution of Xₙ need not converge to it.

Key rules to remember

Stationary distribution equations: π = πP and Σ πⱼ = 1, with πⱼ ≥ 0; Write one equation per state. One equation is redundant, so drop one and use the sum equal to 1 instead.
Period of a state: d(i) = gcd{ n ≥ 1 : pᵢᵢ(n) > 0 }; If d(i) = 1 the state is aperiodic. Any state with pᵢᵢ > 0 has period 1.
Recurrence and transience: Recurrent if P(return to i | X₀ = i) = 1; transient if this is < 1; In a finite chain, closed classes are recurrent and non-closed classes are transient.
Uniqueness: Finite, irreducible chain ⇒ exactly one stationary distribution; If there are several closed classes, there are infinitely many stationary distributions. Each closed class has its own, and any mixture of them also works.
Convergence: Finite, irreducible, aperiodic ⇒ pᵢⱼ(n) → πⱼ as n → ∞ for all i; Every row of Pⁿ tends to π. This needs aperiodicity.
Mean recurrence time: E[return time to i | X₀ = i] = 1 ÷ πᵢ; Valid for a finite irreducible chain.
Visits to a transient state: If the return probability is r < 1, the number of visits (including the start) is geometric with mean 1 ÷ (1 − r); Useful for expected time spent in a transient state.

How to solve Classification of States and Stationary Distributions questions

Use this order for any question on classification or long-run behaviour. Do the classification first, because it tells you whether a stationary distribution is unique and whether the limit exists.

1Draw the transition diagram from the matrix. Mark each arrow with pᵢⱼ > 0 and ignore the exact values for now.
2Find the communicating classes. Check which states can reach each other.
3Mark each class as closed or not closed. In a finite chain, closed means recurrent and not closed means transient.
4Find the period of each class. Look for self-loops first (period 1). Otherwise find the gcd of the lengths of return paths.
5If the chain is irreducible, write πP = π as one equation per state. Express every πⱼ in terms of one of them, then use Σπⱼ = 1 to fix the value.
6If there are several closed classes, find the stationary distribution within each closed class separately. Then find the probability of being absorbed into each class from the starting state.
7State the conclusion in words: whether π is unique, whether Pⁿ converges, and what the long-run probabilities are.
8Check your answer: every πⱼ ≥ 0, the πⱼ add to 1, and substitute back into at least one equation.

Quickest way: Quick route to a stationary distribution

When to use it: Use this when the chain is irreducible with 2 to 4 states and you have limited time. Check it first with the diagram.

Check irreducibility by tracing a cycle through all states.
Set π for one state to a variable, for example π₁ = x.
Use the equations of the states with the simplest columns first. Each equation says that πⱼ equals the sum of πᵢpᵢⱼ over i.
Solve the remaining unknowns as multiples of x.
Add all the multiples and set the total equal to 1 to find x.
Verify with one unused equation. If it fails, you made an arithmetic error.

Common mistakes in Classification of States and Stationary Distributions

Using the column-sum form wrongly: solving Pπ = π instead of πP = π.
Linear algebra courses often use column vectors. Actuarial notation uses a row vector π multiplied on the left of P.
Fix: Write the equation for each state j as πⱼ = Σᵢ πᵢpᵢⱼ. This sums down column j of P, with each term weighted by πᵢ.
Forgetting the condition Σπⱼ = 1 and giving a vector that only satisfies πP = π.
The equations πP = π are homogeneous, so any multiple of a solution also satisfies them. Students stop when the equations are solved.
Fix: Always finish by normalising. The final line of working should be the sum equal to 1.
Claiming that a stationary distribution means Pⁿ converges to it.
Students mix up existence of π with convergence.
Fix: Convergence needs irreducibility and aperiodicity (for a finite chain). A chain that flips between two states with certainty has π = (½, ½) but does not converge.
Calling a state recurrent because the chain can return to it with positive probability.
Positive return probability is confused with certain return.
Fix: Recurrent needs return probability exactly 1. If there is any exit from the class that cannot be reversed, the states are transient. In a finite chain, test whether the class is closed.
Assuming a chain with a transient state has no long-run distribution, or that π is unique when there are two closed classes.
Students apply the irreducible-chain result to chains that are not irreducible.
Fix: Transient states get long-run probability 0. With several closed classes, the long-run distribution depends on the starting state through the absorption probabilities.
Finding the period by looking only at one return path.
Students find one cycle length and stop.
Fix: Period is the gcd of all possible return lengths. If you find return lengths with gcd 1, such as 2 and 3, the state is aperiodic. A self-loop settles it immediately.

Worked examples

Example 1

A chain on states 1, 2, 3 has transition matrix with rows: state 1: (0.5, 0.5, 0); state 2: (0.25, 0.5, 0.25); state 3: (0, 0.5, 0.5). (a) Show the chain is irreducible and aperiodic. (b) Find the stationary distribution. (c) State the limit of P(Xₙ = 3 | X₀ = 1) as n → ∞.

Show the solution

(a) From 1 you can go to 2 (p = 0.5). From 2 you can go to 3 (p = 0.25) and back to 1 (p = 0.25). From 3 you can go to 2. So every state is reachable from every other state, and there is one class: the chain is irreducible.
Each state has a positive self-transition (p₁₁ = 0.5, p₂₂ = 0.5, p₃₃ = 0.5), so each state has period 1. The chain is aperiodic.
(b) Write πP = π. State 1: π₁ = 0.5π₁ + 0.25π₂, so 0.5π₁ = 0.25π₂ and π₂ = 2π₁.
State 3: π₃ = 0.25π₂ + 0.5π₃, so 0.5π₃ = 0.25π₂ and π₃ = 0.5π₂ = π₁.
Use Σπ = 1: π₁ + 2π₁ + π₁ = 4π₁ = 1, so π₁ = 0.25, π₂ = 0.5, π₃ = 0.25.
Check state 2: 0.5π₁ + 0.5π₂ + 0.5π₃ = 0.125 + 0.25 + 0.125 = 0.5 = π₂. Correct.
(c) The chain is finite, irreducible and aperiodic, so pᵢⱼ(n) → πⱼ regardless of the start. The limit is π₃ = 0.25.

Answer: The chain is irreducible and aperiodic. The stationary distribution is π = (0.25, 0.5, 0.25), and P(Xₙ = 3 | X₀ = 1) → 0.25.

Example 2

A chain on states 1, 2, 3, 4 has transition matrix with rows: state 1: (0.5, 0.5, 0, 0); state 2: (0.4, 0.6, 0, 0); state 3: (0.2, 0, 0.5, 0.3); state 4: (0, 0, 0, 1). (a) Find the communicating classes and classify each state. (b) Find the long-run distribution starting from state 3.

Show the solution

(a) States 1 and 2 reach each other, so {1, 2} is a class. No arrows leave it, so it is closed and its states are recurrent. State 4 is absorbing, so {4} is a closed class and state 4 is recurrent.
State 3 can reach states 1 and 4 but nothing can return to 3. So {3} is a class that is not closed, and state 3 is transient. States 1, 2 and 3 have self-loops, as does state 4, so all are aperiodic.
(b) Starting from 3, the chain stays at 3 with probability 0.5 and leaves with probability 0.5. Given that it leaves, it goes to 1 with probability 0.2 ÷ 0.5 = 0.4 and to 4 with probability 0.3 ÷ 0.5 = 0.6. So the probability of ending in class {1, 2} is 0.4 and of ending in state 4 is 0.6.
Find the stationary distribution inside {1, 2}: π₁ = 0.5π₁ + 0.4π₂, so 0.5π₁ = 0.4π₂ and π₁ = 0.8π₂. With π₁ + π₂ = 1: 1.8π₂ = 1, so π₂ = 5/9 and π₁ = 4/9.
Combine: long-run P(state 1) = 0.4 × 4/9 = 8/45. P(state 2) = 0.4 × 5/9 = 10/45 = 2/9. P(state 3) = 0. P(state 4) = 0.6 = 27/45 = 3/5.
Check: 8/45 + 10/45 + 0 + 27/45 = 45/45 = 1.

Answer: {1, 2} is closed and recurrent, {3} is transient, and {4} is absorbing. The long-run distribution from state 3 is (8/45, 2/9, 0, 3/5).

Exam tips

Always classify before you calculate. Examiners give marks for naming the classes, saying closed or not closed, and stating the period.
Write the reason as well as the label. For example, say that {3} is transient because it is not closed and cannot be re-entered once left.
When a chain has more than one closed class, say that the stationary distribution is not unique. Then give the long-run distribution from the specific starting state asked.
In a computer-based paper, solve πP = π in R or Excel by replacing one equation with the sum condition. Show the matrix set-up and the result in the answer.
If the chain is periodic, say explicitly that the limit of Pⁿ does not exist but a stationary distribution does. Do not just write that the chain converges.

Practice questions from Markov processes

Classification of States and Stationary Distributions: frequently asked questions

How do I find the stationary distribution of a Markov chain?

Write πⱼ = Σᵢ πᵢpᵢⱼ for each state j, together with Σπⱼ = 1. Express all πⱼ in terms of one of them, then normalise. Check the answer in an equation you did not use.

What is the difference between recurrent and transient states?

A recurrent state is one you return to with probability 1, so you visit it infinitely often. A transient state has a positive probability of never being returned to, so it is visited only finitely many times. In a finite chain, closed classes are recurrent and the others are transient.

Does every Markov chain have a unique stationary distribution?

No. A finite irreducible chain has exactly one. A chain with two or more closed classes has infinitely many, because each closed class has its own and mixtures also work. Transient states always get probability 0.

When does the long-run distribution equal the stationary distribution?

For a finite chain, this holds when the chain is irreducible and aperiodic. Then every row of Pⁿ tends to π. If the chain is periodic, π still exists but the distribution of Xₙ keeps cycling instead of settling.