Autoregressive (AR) Models: Mean, Variance and ACF

Understand Autoregressive (AR) Models

Key formulas to remember

AR(1) model

Yt = c + φ Yt−1 + εt, with εt white noise (mean 0, variance σ²)

φ is the persistence. c is an intercept, not the mean.

AR(1) stationarity condition

|φ| < 1

φ = 1 is a unit root (random walk). |φ| > 1 is explosive.

AR(1) mean

μ = c ÷ (1 − φ)

Valid only when |φ| < 1. If there is no intercept, the mean is 0.

AR(1) variance

γ0 = σ² ÷ (1 − φ²)

Always larger than σ² when φ ≠ 0. It grows as |φ| approaches 1.

AR(1) autocovariance and ACF

γ(τ) = φ^τ × γ0 and ρ(τ) = φ^τ

Geometric decay. Alternates in sign if φ < 0.

AR(1) forecast

E[Yt+h | Yt] = μ + φ^h (Yt − μ)

Forecasts revert to the mean μ as h grows.

AR(p) model

Yt = c + φ1 Yt−1 + … + φp Yt−p + εt

The PACF cuts off after lag p. The ACF decays gradually.

AR(p) mean

μ = c ÷ (1 − φ1 − … − φp)

Requires φ1 + … + φp ≠ 1. Stationarity needs the sum to be below 1, but that alone is not sufficient.

AR(p) stationarity

All roots of 1 − φ1z − … − φpz^p = 0 lie outside the unit circle

For AR(2): φ1 + φ2 < 1, φ2 − φ1 < 1 and |φ2| < 1.

Yule-Walker equations

ρ(τ) = φ1 ρ(τ−1) + φ2 ρ(τ−2) + … + φp ρ(τ−p), for τ ≥ 1, with ρ(0) = 1 and ρ(−k) = ρ(k)

For AR(2): ρ1 = φ1 ÷ (1 − φ2) and ρ2 = φ1ρ1 + φ2.

AR(p) variance

γ0 = σ² ÷ (1 − φ1ρ1 − φ2ρ2 − … − φpρp)

For p = 1 this reduces to σ² ÷ (1 − φ²).

How to solve Autoregressive (AR) Models questions

1. 1Write the model in standard form Yt = c + φ1Yt−1 + … + εt. Identify c, each φ and σ². Check whether the question gives the variance or the standard deviation of the shock.
2. 2Check stationarity first. For AR(1), test |φ| < 1. For AR(2), test the three conditions. If the process is not stationary, the mean and variance formulas do not apply.
3. 3Find the mean with μ = c ÷ (1 − Σφ). If the model is written in deviations from the mean, the mean is already given.
4. 4Find the variance. For AR(1), use σ² ÷ (1 − φ²). For AR(p), first get ρ1 … ρp from Yule-Walker, then use σ² ÷ (1 − Σφkρk).
5. 5Get autocorrelations. For AR(1), ρ(τ) = φ^τ. For AR(p), use the Yule-Walker recursion ρ(τ) = Σ φk ρ(τ−k). Multiply by γ0 to get autocovariances.
6. 6For forecasts, use deviations from the mean: E[Yt+h] = μ + φ^h (Yt − μ) for AR(1). For AR(p), iterate one step at a time.
7. 7Sanity check: variance must be at least σ², |ρ| must be at most 1, and the forecast must move toward μ.

Quickest way: Three-number shortcut for AR(1)

Common mistakes in Autoregressive (AR) Models

• Treating the intercept c as the mean of the process.

  In a regression, the intercept looks like the average level. In an AR model the lagged term also contributes.

  Fix: Always use μ = c ÷ (1 − φ) for AR(1), or c ÷ (1 − Σφ) for AR(p).

• Using σ² ÷ (1 − φ) for the variance, or forgetting to square φ.

  It is easy to mix up the mean denominator (1 − φ) with the variance denominator (1 − φ²).

  Fix: Remember: the mean has (1 − φ), the variance has (1 − φ²). Check that variance is at least σ².

• Applying mean and variance formulas without checking stationarity.

  Candidates jump straight to the formulas.

  Fix: Test |φ| < 1 first. If φ = 1 or |φ| > 1, say there is no finite long-run mean or variance in the stationary sense.

• Believing the ACF of an AR model cuts off after lag p.

  Mixing up AR and MA models. An MA(q) ACF cuts off after lag q.

  Fix: AR: ACF decays gradually, PACF cuts off after lag p. MA: ACF cuts off, PACF decays.

• Assuming Σφ < 1 proves an AR(p) process is stationary.

  The sum condition is quick, and it is exactly right for AR(1).

  Fix: For p ≥ 2 the sum condition is necessary, not sufficient. Check that all roots lie outside the unit circle, or use the three AR(2) conditions.

• Losing the sign of φ when it is negative.

  Candidates compute φ^τ using the absolute value.

  Fix: With φ = −0.5, ρ1 = −0.5, ρ2 = +0.25, ρ3 = −0.125. Odd lags are negative, even lags positive.

Worked examples