Stochastic processes: formula sheet

Key formulas

Definition of a stochastic process

{X_t : t ∈ T}, with X_t taking values in S

T is the time set and S is the state space. State both whenever you describe a process.

Discrete time set

T = {0, 1, 2, ...} (or any countable set)

Process is observed at separate points, so you write X_n.

Continuous time set

T = [0, ∞) (or any interval)

Process is defined at every instant, so you write X_t.

Simple random walk

X_n = X_0 + Y_1 + Y_2 + ... + Y_n, with Y_i independent and identically distributed

Discrete time. The state space is discrete if the Y_i take integer values.

Counting process

N(t) = number of events in (0, t], N(0) = 0

Continuous time, discrete state space {0, 1, 2, ...}. Sample paths never decrease.

Markov property

P(Xₙ₊₁ = j | Xₙ = i, Xₙ₋₁, ..., X₀) = P(Xₙ₊₁ = j | Xₙ = i)

Future depends only on the present state.

One-step transition probability

pᵢⱼ = P(Xₙ₊₁ = j | Xₙ = i)

For a time-homogeneous chain this does not depend on n.

Row sum condition

Σⱼ pᵢⱼ = 1 for every state i

Rows sum to 1. Columns do not have to.

Chapman-Kolmogorov (elementwise)

pᵢⱼ⁽ᵐ⁺ⁿ⁾ = Σₖ pᵢₖ⁽ᵐ⁾ pₖⱼ⁽ⁿ⁾

Sum over all intermediate states k at the middle time.

Chapman-Kolmogorov (matrix)

P⁽ᵐ⁺ⁿ⁾ = P⁽ᵐ⁾ P⁽ⁿ⁾, so P⁽ⁿ⁾ = Pⁿ

Valid for time-homogeneous chains. For inhomogeneous chains, multiply the matrices for each step in order.

Distribution at time n

πₙ = π₀ Pⁿ

π₀ is a row vector of initial probabilities.

Stationary distribution equations

πP = π, Σ π_j = 1, π_j ≥ 0

Written out: π_j = Σ_i π_i p_ij. One equation is redundant, so replace one of them with Σ π_j = 1.

Chapman-Kolmogorov (n-step)

P(n) = Pⁿ, so p_ij(m+n) = Σ_k p_ik(m) p_kj(n)

Use it to test whether p_ii(n) > 0 when finding the period.

Period of a state

d_i = gcd{ n ≥ 1 : p_ii(n) > 0 }

d_i = 1 means aperiodic. A self-loop (p_ii > 0) makes the state aperiodic.

Existence and uniqueness

Finite, irreducible chain ⇒ unique π with π_j > 0 for all j

If there is more than one closed class, π is not unique.

Limiting behaviour

Irreducible, aperiodic, positive recurrent ⇒ p_ij(n) → π_j as n → ∞

The limit does not depend on the starting state i.

Mean recurrence time

m_j = 1 ÷ π_j

Expected number of steps to return to state j, for an irreducible positive recurrent chain.

Long-run proportion of time

(1/n) × (number of visits to j in n steps) → π_j

Holds for irreducible positive recurrent chains even if periodic.

Transition rate

μ_ij = lim (h → 0) P(X(t+h) = j | X(t) = i) ÷ h, for i ≠ j

Equivalent to p_ij(h) = μ_ij h + o(h).

Total rate of leaving a state

λ_i = Σ (j ≠ i) μ_ij

Holding time in state i is Exponential(λ_i), mean 1/λ_i.

Generator matrix

a_ij = μ_ij (i ≠ j); a_ii = −λ_i; each row sums to 0

Use the row-sum check to catch errors.

Jump probabilities

P(next state is j | leaving i) = μ_ij ÷ λ_i

Defined when λ_i > 0.

Kolmogorov forward equations

d/dt p_ij(t) = Σ (k ≠ j) p_ik(t) μ_kj − p_ij(t) λ_j; matrix form P'(t) = P(t) A

Condition on the state just before time t+h. Sum over the end state's inflows and outflow.

Kolmogorov backward equations

d/dt p_ij(t) = Σ (k ≠ i) μ_ik p_kj(t) − λ_i p_ij(t); matrix form P'(t) = A P(t)

Condition on the first small step from the start state.

Matrix solution

P(t) = exp(tA) = Σ (n ≥ 0) (tA)^n ÷ n!, with P(0) = I

Valid for time-homogeneous processes.

Probability of staying in a state

P(X(s) = i for all s in [0, t] | X(0) = i) = exp(−λ_i t)

This is not the same as p_ii(t), which allows leaving and returning.

Occupancy probability, inhomogeneous case

P(stay in i over [s, t]) = exp(−∫ from s to t of λ_i(u) du)

Use when rates depend on time.

Poisson count

P(N(t) = n) = e^(−λt) (λt)ⁿ ÷ n!

Mean and variance of N(t) both equal λt.

Inter-arrival time

T ~ Exponential(λ); f(t) = λe^(−λt); P(T > t) = e^(−λt); E[T] = 1/λ; Var[T] = 1/λ²

Gaps are independent. Time to the nth event is Gamma(n, λ) with mean n/λ.

Memoryless property

P(T > s + t | T > s) = P(T > t)

Holds for the exponential only, among continuous distributions.

Superposition

N₁(t) + N₂(t) is Poisson process with rate λ₁ + λ₂

Requires independent processes.

Thinning

Type 1 events: rate pλ; type 2 events: rate (1 − p)λ

Each event classified independently. The two resulting processes are independent.

Compound Poisson mean

E[S(t)] = λt E[X]

X is a single claim size.

Compound Poisson variance

Var[S(t)] = λt E[X²]

Uses the second moment, not the variance, of X.

Compound Poisson MGF

M_S(r) = exp{λt (M_X(r) − 1)}

Useful for skewness and for proving closure results.

Third central moment

E[(S − E[S])³] = λt E[X³]

Gives the skewness as λt E[X³] ÷ (λt E[X²])^(3/2).

Markov property (discrete time)

P(X_{n+1} = j | X_n = i, X_{n-1}, ..., X_0) = P(X_{n+1} = j | X_n = i)

Rows of the transition matrix must each sum to 1.

n-step probabilities

P(n) = Pⁿ, and p_ij(m+n) = Σ_k p_ik(m) p_kj(n)

This is the Chapman-Kolmogorov equation. Multiply matrices in the right order.

Stationary distribution

π = πP, with Σ π_i = 1

Use the normalising condition. Check the chain is irreducible and aperiodic if you claim the chain converges to π.

Expected long-run NCD premium

Σ π_i × (full premium) × (1 − discount_i)

Weight each level's premium by its stationary probability.

Transition intensity

P(X_{t+h} = j | X_t = i) = μ_ij × h + o(h), for j ≠ i

Intensities are rates, not probabilities. They can exceed 1.

Holding time in state i

Time in i ~ Exp(λ_i), where λ_i = Σ_{j≠i} μ_ij; mean = 1 ÷ λ_i

For constant intensities. The next state is j with probability μ_ij ÷ λ_i.

Kolmogorov forward equations

d/dt p_ij(t) = Σ_{k≠j} p_ik(t) μ_kj − p_ij(t) Σ_{k≠j} μ_jk

Inflow into j minus outflow from j. The backward equations use the starting state instead.

Probability of staying in a state

p_ii(t) = exp(−λ_i t) when the process cannot leave and return

Not valid if you can come back to i. Then you need the full equations.

Random walk with absorbing barriers, p ≠ q

P(hit 0 before N | start i) = ((q/p)^i − (q/p)^N) ÷ (1 − (q/p)^N)

Here the walk goes up with probability p. The probability of hitting N first is 1 minus this.

Symmetric walk (p = q = ½)

P(hit 0 before N | start i) = 1 − i/N; expected steps to absorption = i(N − i)

Do not use the p ≠ q formula here, as it gives 0 ÷ 0.

Quick revision

• Markov property: the future depends on the past only through the present state.
• n-step transition matrix = Pⁿ, from Chapman–Kolmogorov: p(m+n) = p(m) · p(n).
• Rows of a transition matrix sum to 1; rows of a generator matrix sum to 0.
• Stationary distribution: π = πP with Σπᵢ = 1, or πA = 0 for a jump process with generator A.
• A finite irreducible chain has a unique stationary distribution; a limit also needs aperiodicity.
• Poisson process with rate λ: N(t) ~ Poisson(λt), with independent and stationary increments.
• Inter-arrival times of a Poisson process are independent Exp(λ).
• Holding time in state i of a jump process is Exp(λᵢ), where λᵢ = −aᵢᵢ.
• Jump probability from i to j is aᵢⱼ ÷ λᵢ for j ≠ i.
• Kolmogorov forward: dP(t)/dt = P(t)A. Backward: dP(t)/dt = A P(t).
• Absorbing state: once entered, never left. Random walk with barriers is a chain with absorbing ends.
• Always state the assumptions used, such as time-homogeneity.

Common mistakes

• Calling a process continuous in state because the number of values is large or infinite. Fix: Values like 0, 1, 2, ... are countable, so the state space is discrete. Only intervals of real numbers give continuous state.
• Mixing up time set and state space. Fix: Always write T and S as two separate lines before naming the type.
• Squaring each entry of P instead of multiplying matrices. Fix: Use matrix multiplication: (P²)ᵢⱼ = Σₖ pᵢₖpₖⱼ, row i of P times column j of P.
• Using columns that sum to 1, or reading the matrix as P(from j to i). Fix: In IAI work, row i is the current state and column j is the next state. Rows sum to 1.
• Solving πP = π without using Σπ = 1. Fix: Always finish by normalising. Replace one redundant equation with Σπ_j = 1.
• Using the columns of P instead of the rows, or solving Pπ = π. Fix: π is a row vector multiplying P on the left. The equation for π_j uses column j of P: π_j = Σ_i π_i p_ij.
• Putting +λ_i on the diagonal of the generator matrix, or getting rows that do not sum to zero. Fix: Set a_ii = −Σ of the other entries in the row. Always check the row sum.
• Mixing up forward and backward equations. Fix: Forward: P'(t) = P(t)A, condition on the last step, rates act on the end state j. Backward: P'(t) = A P(t), condition on the first step, rates act on the start state i.
• Using Var[S] = λt Var[X] for a compound Poisson process. Fix: For Poisson N, Var[N] = E[N] = λt, so the result simplifies to λt E[X²]. Always use the second moment.
• Forgetting to change the time unit, for example using a monthly rate for a yearly interval. Fix: Write the unit next to λ and convert the interval length before computing λt.

Exam tips

• Begin every answer on this topic by writing T and S. It earns marks and keeps you organised.
• In multiple-choice questions, check the state space first. The options often differ only in this.
• Learn one standard example for each of the four types. Examiners reuse them: random walk, AR(1), Poisson process, Brownian motion.
• When asked to give an example of a process, add a short reason so the classification is clear.
• Link the classification to later topics. Discrete time and state suggests a Markov chain; continuous time and discrete state suggests a Markov jump process.
• Show the general formula first, such as the Chapman-Kolmogorov sum, then substitute numbers. Method marks depend on it.
• Write out every non-zero path term in two-step questions. It makes errors easy for you and the examiner to spot.
• If a question describes a system with memory, such as a no claims discount depending on the last two years, define new states so the Markov property holds.