Autoregressive (AR) Models: Stationarity, Yule-Walker, ACF and PACF

Understand Autoregressive (AR) Models

Key rules to remember

AR(p) model

X_t = μ + α₁(X_{t-1} − μ) + … + αₚ(X_{t-p} − μ) + e_t

e_t is white noise with mean 0 and variance σ². Written with the mean μ, as in IAI notation.

Stationarity condition

All roots of 1 − α₁z − α₂z² − … − αₚzᵖ = 0 satisfy |z| > 1

For AR(1): |α| < 1. Roots may be complex; use the modulus.

AR(1) ACF

ρ_k = αᵏ for k ≥ 0

Decays geometrically. Alternates in sign if α < 0.

AR(1) variance

γ₀ = σ² ÷ (1 − α²)

Needs |α| < 1.

Yule-Walker recursion for ACF

ρ_k = α₁ρ_{k-1} + α₂ρ_{k-2} + … + αₚρ_{k-p} for k ≥ 1

Use ρ₀ = 1 and ρ_{-k} = ρ_k.

AR(2) Yule-Walker, first two lags

ρ₁ = α₁ + α₂ρ₁ and ρ₂ = α₁ρ₁ + α₂

So ρ₁ = α₁ ÷ (1 − α₂).

AR(2) stationarity triangle

α₁ + α₂ < 1, α₂ − α₁ < 1, |α₂| < 1

All three must hold. This is equivalent to the root condition.

Variance from autocovariances

γ₀ = σ² ÷ (1 − α₁ρ₁ − … − αₚρₚ)

Gives the variance once the ρ values are known.

PACF of AR(p)

φ_kk ≠ 0 for k ≤ p, φ_kk = 0 for k > p; φ₁₁ = ρ₁ and φ_pp = α_p

Cut-off at lag p identifies the order.

How to solve Autoregressive (AR) Models questions

1. 1Write the model in standard form with the mean μ and noise e_t. State that e_t is white noise with variance σ².
2. 2Test stationarity. For AR(1) check |α| < 1. For AR(2) check the three triangle conditions, or find the roots of the characteristic polynomial and check their modulus.
3. 3Write the Yule-Walker equations for k = 1, 2, … using ρ_k = Σ α_i ρ_{k-i}, with ρ₀ = 1 and ρ_{-k} = ρ_k.
4. 4Solve the first p equations for the unknown ρ values (or for the α values if you are given sample ρ).
5. 5Use the recursion to extend to higher lags.
6. 6Find the variance from γ₀ = σ² ÷ (1 − Σ α_i ρ_i) if needed, then γ_k = ρ_k γ₀.
7. 7For identification questions, describe the ACF (decays, never cuts off) and the PACF (cuts off after lag p), and state the order.
8. 8State assumptions and give the final answer with units or a clear conclusion.

Quickest way: Fast route for AR(1) and AR(2)

Common mistakes in Autoregressive (AR) Models

• Using |α| > 1 or the wrong side of the root condition.

  Students confuse the condition on the coefficients with the condition on the roots of the polynomial in z.

  Fix: For AR(1), the root of 1 − αz = 0 is z = 1/α. It must lie outside the unit circle, so |α| < 1. Check which polynomial you are using.

• Checking only α₁ + α₂ < 1 for AR(2).

  Students remember one of the three triangle conditions and stop.

  Fix: Check all three: α₁ + α₂ < 1, α₂ − α₁ < 1 and |α₂| < 1.

• Saying the ACF of an AR process cuts off.

  It is mixed up with the MA(q) process, where the ACF cuts off.

  Fix: Remember: AR has a decaying ACF and a cut-off PACF. MA is the reverse.

• Forgetting ρ_{-k} = ρ_k in the Yule-Walker equations.

  For k = 1 the AR(2) equation involves ρ_{-1}, which looks unfamiliar.

  Fix: Replace ρ_{-1} with ρ₁ and ρ₀ with 1 before solving.

• Using σ² as the variance of X_t.

  σ² is the variance of the noise e_t, and students mix the two.

  Fix: Use γ₀ = σ² ÷ (1 − α²) for AR(1). The process variance is larger than σ² when α ≠ 0.

• Dropping the mean μ when writing the model.

  Many examples are centred at zero, so the mean is forgotten.

  Fix: Write the model in terms of (X_t − μ). The mean of a stationary AR process is μ.

Worked examples