Forecasting with AR, MA and ARIMA Models

Understand Forecasting with Time Series Models

Key rules to remember

Best forecast

x̂(n, k) = E[X(n+k) | X(n), X(n−1), ...]

Replace future e by 0, future X by forecasts, and keep past X and past e as known values.

AR(1) k-step forecast

x̂(n, k) = μ + α^k (x(n) − μ)

For X(t) − μ = α(X(t−1) − μ) + e(t). It tends to μ as k grows if |α| < 1.

MA(q) forecast

x̂(n, k) = μ for k > q

For k ≤ q, use the known past e terms with the correct θ coefficients, and set future e to 0.

MA(∞) form and ψ weights

X(t) = μ + Σ ψ(j) e(t−j), with ψ(0) = 1

For ARMA(1,1) with parameters α and θ: ψ(1) = α + θ and ψ(j) = α ψ(j−1) for j ≥ 2.

k-step forecast error

e(n, k) = X(n+k) − x̂(n, k) = Σ (j = 0 to k−1) ψ(j) e(n+k−j)

It has mean zero for the best forecast.

Forecast error variance

Var[e(n, k)] = σ² Σ (j = 0 to k−1) ψ(j)²

For AR(1): σ²(1 − α^(2k)) ÷ (1 − α²). For a 1-step forecast it equals σ².

ARIMA(p,1,q) forecasting

Forecast the differenced series Y(t) = X(t) − X(t−1), then x̂(n, k) = x(n) + Σ (j = 1 to k) ŷ(n, j)

For a random walk, x̂(n, k) = x(n) and the error variance is kσ².

Exponential smoothing

m̂(t) = α x(t) + (1 − α) m̂(t−1), and x̂(n, k) = m̂(n) for all k

Equivalent weights on past observations are α(1 − α)^j. A larger α reacts faster.

How to solve Forecasting with Time Series Models questions

1. 1Write the model in the form given. Identify μ (or whether the mean is zero), the parameters and σ². Check if the series is differenced.
2. 2If the model is ARIMA, define the differenced series Y and write the ARMA model for Y. Keep x(n) to add back later.
3. 3List what is known at time n: the latest observations and the latest noise terms e(n), e(n−1), ... Estimate noise terms from the model if they are not given.
4. 4Compute the 1-step forecast by putting time n+1 into the equation. Set e(n+1) to 0 and use known values.
5. 5Move forward step by step. In each new equation, replace unknown future X by the forecast you just found, and future e by 0.
6. 6If the model is ARIMA, add the forecast differences back to x(n) to return to the original series.
7. 7For the error variance, find the ψ weights up to ψ(k−1), then compute σ² Σ ψ(j)². For AR(1) use the closed form.
8. 8State the result with its units, and say if the forecast converges to the mean or the variance grows without bound.

Quickest way: Forecast by recursion, error variance by ψ weights

Common mistakes in Forecasting with Time Series Models

• Forgetting to subtract the mean before applying the AR decay factor.

  Students remember α^k x(n) from zero-mean examples.

  Fix: Always use μ + α^k (x(n) − μ). The decay applies to the deviation from the mean, not to the level.

• Replacing a past noise term e(n) by zero in an MA or ARMA forecast.

  Students remember that noise has mean zero but forget it only applies to future noise.

  Fix: Only e(n+1), e(n+2), ... become zero. Terms up to time n are known and must be kept.

• Summing too many or too few ψ weights in the error variance.

  Confusion between k and k−1 in the sum limits.

  Fix: A k-step error has k terms: ψ(0) to ψ(k−1). Check that the 1-step variance is σ².

• Forecasting an ARIMA model as if the original series were stationary.

  Students forget to difference first, or forget to add the forecast differences back.

  Fix: Forecast the differenced series, then cumulate the forecasts onto x(n). Expect the error variance to grow without bound.

• Using the 2-step forecast formula for AR(1) as x̂ = α x(n) instead of α² (x(n) − μ) + μ.

  Students stop after one step of the recursion.

  Fix: Each extra step multiplies the deviation by α once more. The power of α equals k.

• Thinking exponential smoothing forecasts change with k.

  Students mix it up with AR forecasts that decay.

  Fix: In simple exponential smoothing the forecast for every future step is the same latest smoothed level m̂(n).

Worked examples