ARMA and ARIMA Models: Stationarity, Invertibility and Differencing

Understand ARMA and ARIMA Models

Key rules to remember

Backward shift operator

B X_t = X_{t-1}; B^k X_t = X_{t-k}

B acts on the time index only. B applied to a constant gives the same constant.

ARMA(p,q) model

X_t = μ + φ₁(X_{t-1} − μ) + … + φ_p(X_{t-p} − μ) + e_t + θ₁e_{t-1} + … + θ_q e_{t-q}

e_t is white noise with variance σ². Check which sign convention the question uses for θ.

ARMA in operator form

φ(B)(X_t − μ) = θ(B) e_t, with φ(B) = 1 − φ₁B − … − φ_p B^p and θ(B) = 1 + θ₁B + … + θ_q B^q

Move all X terms to the left and all e terms to the right.

Stationarity condition

All roots of φ(z) = 0 satisfy |z| > 1

Only the AR polynomial matters. Equivalent: roots of the reversed polynomial lie inside the unit circle.

Invertibility condition

All roots of θ(z) = 0 satisfy |z| > 1

Only the MA polynomial matters.

Difference operator

∇ = 1 − B; ∇X_t = X_t − X_{t-1}; ∇^d = (1 − B)^d

∇² X_t = X_t − 2X_{t-1} + X_{t-2}. It is not X_t − X_{t-2}.

ARIMA(p,d,q) model

φ(B)(1 − B)^d X_t = θ(B) e_t

Here the differenced series Y_t = ∇^d X_t is a stationary, invertible ARMA(p,q). The ARIMA process itself is not stationary when d ≥ 1.

AR(1) stationarity

X_t = φX_{t-1} + e_t is stationary if |φ| < 1

φ(z) = 1 − φz has root z = 1/φ.

MA(1) invertibility

X_t = e_t + θe_{t-1} is invertible if |θ| < 1

θ(z) = 1 + θz has root z = −1/θ.

How to solve ARMA and ARIMA Models questions

1. 1Write the model in the form given. Identify p, d and q by reading the lags of X (after any differencing) and of e.
2. 2Rewrite it with the backward shift operator. Put all X_t terms on the left as φ(B)X_t and all e terms on the right as θ(B)e_t.
3. 3If differencing is mentioned or the series is non-stationary, factor out (1 − B)^d. Check that the remaining AR polynomial does not contain further (1 − B) factors.
4. 4Test stationarity: find the roots of φ(z) = 0 and check that each has modulus greater than 1. For ARIMA, test the AR polynomial after removing the (1 − B)^d factor.
5. 5Test invertibility: find the roots of θ(z) = 0 and check that each has modulus greater than 1.
6. 6State the conclusion clearly. For ARIMA say that the differenced series is stationary, but X_t is not.
7. 7If asked for the order, state ARIMA(p,d,q) with the values worked out. If asked to expand, multiply out the operators and convert back to X_{t-k} terms.

Quickest way: Factor and read off the roots

Common mistakes in ARMA and ARIMA Models

• Declaring an ARIMA(p,d,q) process with d ≥ 1 to be stationary because the ARMA part satisfies the root conditions.

  Students check the roots of φ(z) and forget that (1 − B)^d adds roots equal to 1, which lie on the unit circle.

  Fix: Always say: the differenced series ∇^d X_t is stationary, but X_t itself is not stationary when d ≥ 1.

• Testing stationarity using the MA polynomial, or invertibility using the AR polynomial.

  The two conditions look alike and the polynomials are easy to mix up.

  Fix: Remember: AR part gives stationarity, MA part gives invertibility. Write this beside your working.

• Writing ∇²X_t = X_t − X_{t-2}.

  Students read the power as a lag instead of repeated differencing.

  Fix: Expand (1 − B)² = 1 − 2B + B², so ∇²X_t = X_t − 2X_{t-1} + X_{t-2}.

• Taking the root condition as |root| < 1 instead of |root| > 1.

  Confusion with the condition on the coefficient, such as |φ| < 1 for AR(1).

  Fix: The roots of the polynomial in z must lie outside the unit circle. For φ(z) = 1 − φz the root is 1/φ, which has modulus > 1 exactly when |φ| < 1.

• Getting the sign wrong when moving AR terms to the left, so φ(B) = 1 + φ₁B instead of 1 − φ₁B.

  The AR coefficients appear with a plus sign on the right of the model, but change sign when moved across.

  Fix: Move terms carefully: X_t − φ₁X_{t-1} = e_t gives φ(B) = 1 − φ₁B.

• Forgetting to check for common factors in φ(B) and θ(B).

  Students accept the stated orders without simplifying.

  Fix: If the same factor appears on both sides, cancel it. The model then has lower orders, so state the reduced ARMA form.

Worked examples