Skip to content

IAI Actuarial Core Principles · Risk Modelling and Survival Analysis

Stochastic Processes for CS2: Chapter Guide

A stochastic process is a collection of random variables indexed by time, X(t) or Xₙ. In CS2 you classify processes by state space and time set, then solve Markov chains, Poisson processes and jump processes using transition probabilities, stationary distributions and Kolmogorov equations. Practise setting up the model before calculating.

What this chapter covers

This chapter covers models for quantities that change randomly over time. You start with the basic language: state space, time set, sample path, and the Markov property. Then you move to discrete-time Markov chains, continuous-time Markov jump processes, and the Poisson process as the key counting model.

The chapter has two linked halves. Discrete time uses transition matrices, n-step probabilities (Pⁿ), classification of states and stationary distributions. Continuous time uses transition rates (the generator matrix), the Kolmogorov forward and backward equations, holding times and jump chains. The Poisson process is the simplest continuous-time chain and sits between the two.

It connects to the rest of CS2 directly. Survival models in the paper are two-state or multi-state Markov jump processes, with death or other exits as transitions with rates (forces of transition). The chapter also feeds claim-arrival modelling in risk modelling, and it supports CM1 ideas on multiple-state and decrement models. Time series is a separate chapter, but both use the idea of a process evolving over time. Both Paper A and the computer-based Paper B can test this chapter, so you need hand methods and, where useful, R for matrix powers and simulation.

Stochastic processes carries one of the largest syllabus weightings in CS2, at 25%, the same as survival models, and its ideas run through survival analysis too. Questions are quite structured: set up the states and rates, write the equation, solve it. That makes marks reliable if your method is clean. Multiple-choice questions test definitions and quick calculations, while written questions reward clear setup, correct notation and stated assumptions. Skill here also pays off in the survival models chapter, so effort spent now is used twice.

Stochastic processes: topics in the order to study them

  1. 1Stochastic Processes: Basics and ClassificationYou need the vocabulary of state space, time set and the Markov property before any model makes sense.
  2. 2Markov Chains and Transition ProbabilitiesDiscrete-time chains are the easiest place to learn the Markov property, matrix powers and Chapman–Kolmogorov equations.
  3. 3Stationary Distributions and Long-Term BehaviourIt builds directly on the transition matrix and adds irreducibility, periodicity and the π = πP condition.
  4. 4Poisson ProcessesIt is the simplest continuous-time process, so it prepares you for rates and exponential holding times.
  5. 5Markov Jump Processes and Kolmogorov EquationsIt generalises the Poisson process to many states, using the generator matrix and forward and backward equations.
  6. 6Applications: Multi-State Models and Random WalksIt applies everything above to realistic models such as health and disability states, and to random walks with barriers.

How to prepare Stochastic processes

Treat this chapter as a set of setup skills first and calculation skills second. Most lost marks come from a wrong model, not wrong arithmetic.

  1. Learn the definitions precisely: state space, time set, Markov property, time-homogeneous, irreducible, periodic. Write each in one sentence from memory.
  2. For discrete chains, practise drawing the transition graph from a matrix and writing the matrix from a word problem. Then compute two- and three-step probabilities by hand.
  3. Practise finding stationary distributions by solving π = πP with Σπᵢ = 1. Check that every probability is non-negative and the sum is 1.
  4. Move to continuous time. Memorise that a Poisson process has independent, stationary increments and exponential inter-arrival times, then do compound and thinned examples.
  5. For jump processes, build the generator matrix from rates, check that each row sums to zero, and write both Kolmogorov equations. Solve small two-state cases fully.
  6. Work past-style written questions on multi-state models. Always define states, state assumptions and show the equation before solving.
  7. Use R for matrix powers, simulating chains and checking answers. Practise a few commands so Paper B is not slowed by syntax.

Common mistakes in Stochastic processes

  • Mixing up row and column conventions in transition matrices

    Fix: Remember that rows are the current state and each row sums to 1. Check row sums before using the matrix.

  • Claiming a stationary distribution gives the limit without checking conditions

    Fix: State irreducibility for uniqueness and aperiodicity for convergence. A periodic chain can have a stationary distribution but no limit.

  • Writing a generator matrix whose rows do not sum to zero

    Fix: Set each diagonal entry to the negative sum of the other rates in its row, then check every row.

  • Using the forward and backward equations interchangeably

    Fix: Learn the matrix order: forward is P(t)A, backward is AP(t). Use the one that fits the question's setup.

  • Treating Poisson counts and inter-arrival times as the same thing

    Fix: Counts in time t are Poisson(λt). Waiting times between events are Exp(λ). Write which you need before calculating.

  • Skipping the model setup in written answers

    Fix: Define states, list transitions and rates, and state assumptions first. Marks are given for the setup.

Last-day revision: Stochastic processes

  • Markov property: the future depends on the past only through the present state.
  • n-step transition matrix = Pⁿ, from Chapman–Kolmogorov: p(m+n) = p(m) · p(n).
  • Rows of a transition matrix sum to 1; rows of a generator matrix sum to 0.
  • Stationary distribution: π = πP with Σπᵢ = 1, or πA = 0 for a jump process with generator A.
  • A finite irreducible chain has a unique stationary distribution; a limit also needs aperiodicity.
  • Poisson process with rate λ: N(t) ~ Poisson(λt), with independent and stationary increments.
  • Inter-arrival times of a Poisson process are independent Exp(λ).
  • Holding time in state i of a jump process is Exp(λᵢ), where λᵢ = −aᵢᵢ.
  • Jump probability from i to j is aᵢⱼ ÷ λᵢ for j ≠ i.
  • Kolmogorov forward: dP(t)/dt = P(t)A. Backward: dP(t)/dt = A P(t).
  • Absorbing state: once entered, never left. Random walk with barriers is a chain with absorbing ends.
  • Always state the assumptions used, such as time-homogeneity.

Stochastic processes practice questions

Stochastic processes: frequently asked questions

How much of CS2 is stochastic processes?

The 2026 syllabus weights stochastic processes at 25% of CS2, equal to survival models. It is one of the two largest topics, so it deserves serious time.

Do I need R for this chapter?

It helps, especially in the computer-based Paper B. You can use R to compute matrix powers, find stationary distributions and simulate chains. Paper A still needs hand methods.

Should I learn Markov chains before Poisson processes?

Yes, in most cases. Markov chains teach the Markov property and the matrix thinking. Poisson processes then lead naturally into jump processes and Kolmogorov equations.

How does this chapter link to survival models?

Survival models are Markov jump processes with states such as alive and dead. Transition rates are forces of mortality or other decrements. Understanding this chapter makes multi-state survival questions much easier.