Risk Modelling and Survival Analysis · Applications of time series models

Model Fitting, Diagnostics and the Box-Jenkins Method for Time Series

Q: What are the steps of the Box-Jenkins method?

Make the series stationary, identify the order of the ARMA model from the sample ACF and PACF, estimate the parameters, and check the residuals for white noise. If the checks fail, change the model and repeat. Then use the accepted model for forecasting.

Q: How do I choose the order of an ARMA model using the ACF and PACF?

If the PACF cuts off after lag p and the ACF decays, try AR(p). If the ACF cuts off after lag q and the PACF decays, try MA(q). If both decay, try a low-order ARMA and compare models with residual checks and AIC or BIC.

Q: How do I estimate AR parameters with the Yule-Walker equations?

Write the equations ρk = α1ρ(k−1) + … + αpρ(k−p) for k = 1 to p. Replace each ρk with the sample autocorrelation and solve the linear system. For AR(1) the answer is just α̂1 = ρ̂1.

Q: What should residuals look like for a good fit?

They should look like white noise: no trend or pattern in the plot, no significant autocorrelation at any lag, and roughly constant variance. A Ljung-Box test with degrees of freedom m − (p + q) gives a formal check.

Q: Should I use AIC or BIC?

Both reward fit and penalise parameters, and a lower value is better. BIC penalises extra parameters more when n is at least 8, so it often picks a smaller model. If the question names one, use that one.

Updated 11 October 2026 · Fact-checked

The Box-Jenkins method fits an ARIMA model in three repeated stages: identify the order from the sample ACF and PACF, estimate the parameters (Yule-Walker, least squares or maximum likelihood), then check the residuals look like white noise. If they do not, change the order and repeat.

Understand Model Fitting, Diagnostics and Box-Jenkins Method

You have a series of data and want a model that describes it and can forecast it. The Box-Jenkins method gives you a routine for this. It is a loop, not a one-way path.

First, make the series stationary. Plot it. If it has a trend or the sample ACF decays very slowly, difference it (once or twice). Seasonal patterns may need seasonal differencing. Only then do you pick an ARMA(p, q) model for the differenced series, which means an ARIMA(p, d, q) for the original.

Second, identify the order using the sample ACF and PACF. An AR(p) process has a PACF that cuts off after lag p and an ACF that decays. An MA(q) process has an ACF that cuts off after lag q and a PACF that decays. A mixed ARMA shows decay in both, so the plots do not give a clean answer. Sample values are noisy, so treat a spike as significant only if it lies outside roughly ±2/√n.

Third, estimate the parameters. For an AR(p), the Yule-Walker equations link the autocovariances to the coefficients, so you replace theoretical values with sample ones and solve. Least squares and maximum likelihood are the alternatives, and they are needed for models with MA terms.

Finally, check the fit. If the model is adequate, the residuals behave like white noise: no pattern in the plot, no significant residual autocorrelations, roughly constant variance and, if normality is assumed, roughly normal. You can also compare candidate models with information criteria such as AIC and BIC, which reward fit but penalise extra parameters. Prefer the simpler model when fit is similar (parsimony).

Key rules to remember

Yule-Walker equations for AR(p): ρk = α1ρ(k−1) + α2ρ(k−2) + … + αpρ(k−p), for k = 1, 2, …, p; Here ρk is the autocorrelation at lag k and ρ0 = 1. Replace ρk with sample values to estimate α1, …, αp.
Yule-Walker for AR(1): α̂1 = ρ̂1; The estimate of the AR(1) coefficient is the lag-1 sample autocorrelation.
Yule-Walker for AR(2): ρ1 = α1 + α2ρ1 ; ρ2 = α1ρ1 + α2; Solve: α2 = (ρ2 − ρ1²) ÷ (1 − ρ1²), then α1 = ρ1(1 − α2).
Residual variance of AR(p): σ² = γ0(1 − α1ρ1 − α2ρ2 − … − αpρp); γ0 is the variance of the process. For AR(1) this gives σ² = γ0(1 − α1²).
Sample autocorrelation: ρ̂k = Σ(xt − x̄)(x(t+k) − x̄) ÷ Σ(xt − x̄)²; Numerator sums over t = 1 to n−k; denominator over t = 1 to n.
Approximate significance band: ±2 ÷ √n; For white noise, sample ACF and PACF values are approximately N(0, 1/n). Values outside the band suggest a real correlation.
Ljung-Box portmanteau statistic: Q = n(n+2) Σ ρ̂k² ÷ (n−k), k = 1 to m; Compare with a chi-squared distribution. Degrees of freedom are m minus the number of fitted ARMA parameters (p + q). Large Q means the residuals are not white noise.
AIC: AIC = −2 ln L + 2k; k is the number of estimated parameters, L the maximised likelihood. Lower is better.
BIC: BIC = −2 ln L + k ln n; Penalises parameters more heavily than AIC when n ≥ 8, so it tends to choose smaller models.
Identification pattern: AR(p): PACF cuts off after p. MA(q): ACF cuts off after q.; ARMA(p, q): both decay, so use trial models and diagnostics.

How to solve Model Fitting, Diagnostics and Box-Jenkins Method questions

Use this order for any Box-Jenkins question, whether it asks you to pick a model, estimate it or test it.

1Look at the data plot and the sample ACF. A trend or a very slowly decaying ACF means the series is not stationary, so difference it and re-examine. State the d you choose and why.
2Examine the sample ACF and PACF of the stationary series. Note which cuts off and where, using the ±2/√n band. Propose one or two candidate orders (p, q).
3Estimate the parameters. For an AR model, write out the Yule-Walker equations with sample autocorrelations and solve. Estimate the mean first and work with deviations from it if the mean is not zero.
4Estimate the white noise variance σ² from the fitted model and the sample variance.
5Compute the residuals, then check them: plot them, look at their sample ACF, apply a portmanteau (Ljung-Box) test, and note any visible change in variance.
6Compare candidate models using AIC or BIC and the principle of parsimony. Choose the lowest value unless the difference is trivial.
7State your conclusion in words: which model, its fitted equation, and whether the residual checks support it. If not, say what you would change and repeat.

Quickest way: Fast order choice and AR estimation

When to use it: Use this for short-answer or MCQ items that give you sample ACF or PACF values and ask for a model or a parameter estimate.

Find which of the ACF or PACF stops abruptly. PACF stops at lag p means AR(p). ACF stops at lag q means MA(q).
If the series is AR(1), the estimate is simply the lag-1 sample autocorrelation.
If AR(2), use α2 = (ρ2 − ρ1²) ÷ (1 − ρ1²) and then α1 = ρ1(1 − α2).
Check stationarity of the result. For AR(1) you need |α1| < 1. For AR(2) you need α1 + α2 < 1, α2 − α1 < 1 and |α2| < 1.
For model comparison, compute the criterion for each model with the same formula and pick the smallest.

Common mistakes in Model Fitting, Diagnostics and Box-Jenkins Method

Reading the ACF cut-off to choose an AR order
Students remember that one plot cuts off but mix up which model goes with which plot.
Fix: Memorise: PACF cuts off for AR, ACF cuts off for MA. The PACF at lag p is the last coefficient of the fitted AR(p).
Fitting an ARMA model to a non-stationary series
The sample ACF is computed anyway and looks like it has some pattern, so students skip the differencing check.
Fix: Always look at the plot and a slowly decaying ACF first. Difference, then identify the order of the differenced series.
Using Yule-Walker with raw autocovariances in place of autocorrelations, or forgetting that ρ0 = 1
The equations can be written in either form, and mixing them gives wrong numbers.
Fix: Pick one form and stay with it. In the autocorrelation form, ρ0 = 1 and ρ−k = ρk.
Treating one residual autocorrelation outside the band as proof of a bad model
The band is approximate, and about 1 in 20 values will fall outside it by chance even for white noise.
Fix: Look for a pattern or a large spike at a low lag, and support the view with a portmanteau test.
Choosing the model with the highest likelihood or the most parameters
Extra parameters always raise the maximised likelihood, so a better fit looks better.
Fix: Use AIC or BIC, which add a penalty, and pick the smallest value. Note parsimony in your answer.
Using the wrong degrees of freedom in the Ljung-Box test
Students use the number of lags m only.
Fix: Subtract the number of fitted ARMA parameters: degrees of freedom = m − (p + q).

Worked examples

Example 1

A stationary series of n = 100 observations has sample autocorrelations ρ̂1 = 0.60 and ρ̂2 = 0.20. A sample variance of 4 is observed. Assume an AR(2) model for the series after removing the mean. (a) Estimate α1 and α2 using the Yule-Walker equations. (b) Estimate the residual variance σ².

Show the solution

Yule-Walker equations: ρ1 = α1 + α2ρ1 and ρ2 = α1ρ1 + α2.
Solve for α2: α2 = (ρ2 − ρ1²) ÷ (1 − ρ1²) = (0.20 − 0.36) ÷ (1 − 0.36) = −0.16 ÷ 0.64 = −0.25.
Then α1 = ρ1(1 − α2) = 0.60 × 1.25 = 0.75.
Check the second equation: α1ρ1 + α2 = 0.75 × 0.60 − 0.25 = 0.45 − 0.25 = 0.20. This equals ρ2, so the solution is consistent.
Check stationarity: α1 + α2 = 0.50 < 1; α2 − α1 = −1.00 < 1; |α2| = 0.25 < 1. The model is stationary.
Residual variance: σ² = γ0(1 − α1ρ1 − α2ρ2) = 4 × (1 − 0.75 × 0.60 − (−0.25) × 0.20) = 4 × (1 − 0.45 + 0.05) = 4 × 0.60 = 2.4.

Answer: α̂1 = 0.75, α̂2 = −0.25 and σ̂² = 2.4.

Example 2

Two models are fitted to the same n = 50 observations. Model A is an AR(1) with maximised log-likelihood −80. Model B is an AR(2) with maximised log-likelihood −77. Count only the AR coefficients as parameters. Which model do AIC and BIC choose?

Show the solution

Model A has k = 1 parameter and Model B has k = 2.
AIC for A = −2(−80) + 2(1) = 160 + 2 = 162.
AIC for B = −2(−77) + 2(2) = 154 + 4 = 158.
AIC picks the lower value, so it picks Model B.
BIC penalty per parameter is ln 50 = 3.912 (to 3 decimal places).
BIC for A = 160 + 3.912 = 163.912.
BIC for B = 154 + 2 × 3.912 = 154 + 7.824 = 161.824.
BIC also picks the lower value, which is Model B.
The improvement in log-likelihood of 3 is large enough (a drop of 6 in −2 ln L) to outweigh the penalty under both criteria.

Answer: Both AIC (158 against 162) and BIC (about 161.8 against 163.9) choose the AR(2) model.

Exam tips

Written questions usually ask you to justify the model. Name the pattern you see (for example, PACF cuts off after lag 2) and say what it implies, rather than just stating the order.
Show the Yule-Walker equations before substituting numbers, and check your answer by plugging it back into the second equation. This earns method marks and catches slips.
Always state whether the fitted model is stationary, and check the conditions. Examiners reward this.
In computer-based Paper B, use R functions such as acf, pacf, arima and tsdiag, but write down the order chosen and the reason, and interpret the residual output in words.
For comparison questions, say that lower AIC or BIC is better and mention parsimony. Do not forget that fitted parameters reduce the degrees of freedom in portmanteau tests.

Practice questions from Applications of time series models

Model Fitting, Diagnostics and Box-Jenkins Method: frequently asked questions

What are the steps of the Box-Jenkins method?

Make the series stationary, identify the order of the ARMA model from the sample ACF and PACF, estimate the parameters, and check the residuals for white noise. If the checks fail, change the model and repeat. Then use the accepted model for forecasting.

How do I choose the order of an ARMA model using the ACF and PACF?

If the PACF cuts off after lag p and the ACF decays, try AR(p). If the ACF cuts off after lag q and the PACF decays, try MA(q). If both decay, try a low-order ARMA and compare models with residual checks and AIC or BIC.

How do I estimate AR parameters with the Yule-Walker equations?

Write the equations ρk = α1ρ(k−1) + … + αpρ(k−p) for k = 1 to p. Replace each ρk with the sample autocorrelation and solve the linear system. For AR(1) the answer is just α̂1 = ρ̂1.

What should residuals look like for a good fit?

They should look like white noise: no trend or pattern in the plot, no significant autocorrelation at any lag, and roughly constant variance. A Ljung-Box test with degrees of freedom m − (p + q) gives a formal check.

Should I use AIC or BIC?

Both reward fit and penalise parameters, and a lower value is better. BIC penalises extra parameters more when n is at least 8, so it often picks a smaller model. If the question names one, use that one.