AR, MA and ARMA Models: Stationarity and Invertibility

Understand AR, MA and ARMA Models

Key rules to remember

AR(p) definition

X_t = μ + α₁(X_{t-1} − μ) + ... + α_p(X_{t-p} − μ) + e_t

Using the backward shift operator B: φ(B)(X_t − μ) = e_t, with φ(B) = 1 − α₁B − ... − α_pB^p.

MA(q) definition

X_t = μ + e_t + β₁e_{t-1} + ... + β_q e_{t-q}

Written as X_t − μ = θ(B)e_t, with θ(B) = 1 + β₁B + ... + β_qB^q.

Stationarity condition

All roots of 1 − α₁z − ... − α_p z^p = 0 satisfy |z| > 1

Applies to the AR part of AR and ARMA. MA processes are always stationary.

Invertibility condition

All roots of 1 + β₁z + ... + β_q z^q = 0 satisfy |z| > 1

Applies to the MA part of MA and ARMA.

AR(1) stationarity and ACF

|α| < 1; ρ_k = α^k

Variance γ₀ = σ² ÷ (1 − α²).

AR(2) stationarity region

α₁ + α₂ < 1, α₂ − α₁ < 1, |α₂| < 1

All three must hold. This is the triangle region.

AR(2) Yule-Walker equations

ρ₁ = α₁ ÷ (1 − α₂); ρ_k = α₁ρ_{k-1} + α₂ρ_{k-2} for k ≥ 2

Use these to find ACF values recursively.

MA(1) ACF

ρ₁ = β ÷ (1 + β²); ρ_k = 0 for k ≥ 2

Variance γ₀ = σ²(1 + β²). Invertible if |β| < 1.

MA(q) variance

γ₀ = σ²(1 + β₁² + ... + β_q²)

Autocovariance at lag k is σ²(β_k + β₁β_{k+1} + ... + β_{q-k}β_q), with β₀ = 1.

ARMA(1,1) ACF

ρ₁ = (1 + αβ)(α + β) ÷ (1 + 2αβ + β²); ρ_k = α ρ_{k-1} for k ≥ 2

Model: X_t = αX_{t-1} + e_t + βe_{t-1}, with |α| < 1 and |β| < 1.

ACF and PACF patterns

AR(p): ACF decays, PACF cuts off after p. MA(q): ACF cuts off after q, PACF decays. ARMA: both decay.

The main identification rule.

How to solve AR, MA and ARMA Models questions

1. 1Write the model in standard form and note the parameters. Remove the mean μ by working with X_t − μ if needed.
2. 2Write the operator form φ(B)(X_t − μ) = θ(B)e_t. Identify p and q.
3. 3Test stationarity using the AR polynomial roots, or the AR(1) and AR(2) inequalities. Say clearly which parameter values are allowed.
4. 4Test invertibility using the MA polynomial roots. Say whether the model is invertible.
5. 5To find autocorrelations, use the Yule-Walker equations for AR, direct covariance sums for MA, or the ARMA recursions. Compute ρ₁ first, then recurse.
6. 6If asked to identify a model from data, compare the sample ACF and PACF patterns with the cut-off or decay rules.
7. 7State the answer with the condition used. Show the working in standard actuarial notation.

Quickest way: Fast checks for conditions and ACF

Common mistakes in AR, MA and ARMA Models

• Saying the roots must lie inside the unit circle.

  Students mix up the polynomial in z with the polynomial in the reciprocal variable, where the condition flips.

  Fix: Write the polynomial as 1 − α₁z − ... and require |z| > 1. Check with AR(1): root z = 1/α, so |α| < 1 is the same as |z| > 1.

• Testing only α₁ and α₂ individually for AR(2).

  The AR(1) rule |α| < 1 feels like it should extend.

  Fix: Use all three conditions: α₁ + α₂ < 1, α₂ − α₁ < 1 and |α₂| < 1.

• Applying a stationarity check to an MA process, or forgetting the invertibility check.

  Students memorise one condition and apply it everywhere.

  Fix: MA is always stationary. Stationarity belongs to the AR part. Invertibility belongs to the MA part. Check each one on its own polynomial.

• Using β rather than β ÷ (1 + β²) as the MA(1) lag 1 autocorrelation.

  Students forget to divide by the variance.

  Fix: Compute γ₁ = σ²β and γ₀ = σ²(1 + β²), then divide. Note ρ₁ never exceeds 0.5 in absolute value.

• Mixing up the ACF and PACF cut-off rules.

  Both patterns look similar and are learnt as a pair.

  Fix: Remember: AR is cut off in the PACF, MA is cut off in the ACF. The 'A' for autoregressive pairs with 'partial'.

• Applying ρ_k = αρ_{k-1} in ARMA(1,1) from k = 1.

  The AR(1) recursion starts at lag 1, so students copy it.

  Fix: In ARMA(1,1) the recursion holds only for k ≥ 2. Find ρ₁ from the full formula first.

Worked examples