Stationary Time Series: formula sheet

Key formulas

Constant mean

E(Yt) = μ for all t

The expected value does not depend on time.

Constant, finite variance

Var(Yt) = γ(0) = σ² < ∞ for all t

Variance is the autocovariance at lag 0.

Autocovariance depends only on lag

γ(k) = Cov(Yt, Yt-k), the same for every t

Depends on k, not on the date t. Also γ(k) = γ(-k).

Autocorrelation function

ρ(k) = γ(k) ÷ γ(0)

Lies between -1 and +1, and ρ(0) = 1.

White noise

E(εt) = 0; Var(εt) = σ²; Cov(εt, εt-k) = 0 for k ≠ 0

A basic covariance stationary process.

AR(1) stationarity condition

Yt = c + φYt-1 + εt is stationary if |φ| < 1

Then the mean is c ÷ (1 - φ) and the ACF is ρ(k) = φ^k.

AR(1) variance

Var(Yt) = σ² ÷ (1 - φ²)

Valid only when |φ| < 1.

Approximate ACF significance band

± 1.96 ÷ √T

Rough 95% band for a sample autocorrelation under white noise, with T observations.

White noise conditions

E(ε_t) = 0; Var(ε_t) = σ²; Cov(ε_t, ε_t-k) = 0 for k ≠ 0

All three must hold. σ² must be finite and the same for every t.

Autocorrelation of white noise

ρ(0) = 1; ρ(k) = 0 for k ≥ 1

Lag 0 is always 1. All other lags are zero in the population.

Gaussian white noise

ε_t ~ i.i.d. N(0, σ²)

Normal plus uncorrelated gives independence.

Variance of a sum of white noise terms

Var(ε_1 + ε_2 + … + ε_n) = nσ²

Holds because covariances are zero. Standard deviation is σ√n.

Variance of a linear combination

Var(aε_t + bε_t-1) = (a² + b²)σ²

Cross term vanishes because Cov(ε_t, ε_t-1) = 0.

Approximate sample ACF band

±1.96 ÷ √T

For a white noise series of length T, about 95% of sample autocorrelations should fall inside this band.

Ljung-Box statistic

Q = T(T + 2) Σ [ρ̂(k)² ÷ (T − k)], k = 1 to m

Under the null of white noise, Q is approximately chi-squared with m degrees of freedom when you test a raw series. When you test the residuals of a fitted ARMA(p,q) model, the degrees of freedom are m − p − q, because the estimated parameters use up degrees of freedom.

AR(1) model

Yt = c + φ Yt−1 + εt, with εt white noise (mean 0, variance σ²)

φ is the persistence. c is an intercept, not the mean.

AR(1) stationarity condition

|φ| < 1

φ = 1 is a unit root (random walk). |φ| > 1 is explosive.

AR(1) mean

μ = c ÷ (1 − φ)

Valid only when |φ| < 1. If there is no intercept, the mean is 0.

AR(1) variance

γ0 = σ² ÷ (1 − φ²)

Always larger than σ² when φ ≠ 0. It grows as |φ| approaches 1.

AR(1) autocovariance and ACF

γ(τ) = φ^τ × γ0 and ρ(τ) = φ^τ

Geometric decay. Alternates in sign if φ < 0.

AR(1) forecast

E[Yt+h | Yt] = μ + φ^h (Yt − μ)

Forecasts revert to the mean μ as h grows.

AR(p) model

Yt = c + φ1 Yt−1 + … + φp Yt−p + εt

The PACF cuts off after lag p. The ACF decays gradually.

AR(p) mean

μ = c ÷ (1 − φ1 − … − φp)

Requires φ1 + … + φp ≠ 1. Stationarity needs the sum to be below 1, but that alone is not sufficient.

AR(p) stationarity

All roots of 1 − φ1z − … − φpz^p = 0 lie outside the unit circle

For AR(2): φ1 + φ2 < 1, φ2 − φ1 < 1 and |φ2| < 1.

Yule-Walker equations

ρ(τ) = φ1 ρ(τ−1) + φ2 ρ(τ−2) + … + φp ρ(τ−p), for τ ≥ 1, with ρ(0) = 1 and ρ(−k) = ρ(k)

For AR(2): ρ1 = φ1 ÷ (1 − φ2) and ρ2 = φ1ρ1 + φ2.

AR(p) variance

γ0 = σ² ÷ (1 − φ1ρ1 − φ2ρ2 − … − φpρp)

For p = 1 this reduces to σ² ÷ (1 − φ²).

MA(1) model

y(t) = μ + ε(t) + θ·ε(t−1), ε ~ white noise (0, σ²)

Mean is μ at every date. Past shocks enter, not past values of y.

MA(1) variance

γ0 = σ²(1 + θ²)

Always larger than σ² unless θ = 0.

MA(1) autocovariance

γ1 = θσ²; γk = 0 for k ≥ 2

Only lag 1 is non-zero.

MA(1) autocorrelation

ρ1 = θ ÷ (1 + θ²); ρk = 0 for k ≥ 2

The largest possible |ρ1| is 0.5, reached at θ = 1 or −1. Sign of ρ1 equals sign of θ.

MA(q) model

y(t) = μ + ε(t) + θ1·ε(t−1) + ... + θq·ε(t−q)

Mean is μ. ACF is zero for all lags above q.

MA(q) variance

γ0 = σ²(1 + θ1² + θ2² + ... + θq²)

Sum of squared weights, with weight 1 on the current shock.

MA(q) autocovariance

γk = σ²(θk + θ1·θ(k+1) + ... + θ(q−k)·θq) for 1 ≤ k ≤ q; γk = 0 for k > q

Add the products of weights that line up on the same shock, with θ0 = 1. For MA(2): γ1 = σ²(θ1 + θ1θ2), γ2 = σ²θ2.

Invertibility condition (MA(1))

|θ| < 1

Then ε(t) = y(t) − μ − θ(y(t−1) − μ) + θ²(y(t−2) − μ) − ... , an AR(∞) form.

MA(1) forecasts

1-step: μ + θ·ε(t); h-step for h ≥ 2: μ

The forecast equals μ for all horizons h > q. For MA(1) that means h ≥ 2. The 1-step forecast (h = q) is not the mean. One-step error variance is σ².

ARMA(1,1) process

Yt = c + φYt-1 + εt + θεt-1

εt is white noise with mean 0 and variance σ². Stationary if |φ| < 1.

ARMA(p,q) process

Yt = c + φ1Yt-1 + … + φpYt-p + εt + θ1εt-1 + … + θqεt-q

p counts AR lags, q counts MA lags.

ARMA(1,1) mean

μ = c ÷ (1 − φ)

Requires φ ≠ 1. The MA term does not change the mean.

ARMA(1,1) variance

γ0 = σ² × (1 + 2φθ + θ²) ÷ (1 − φ²)

Valid for |φ| < 1.

ARMA(1,1) autocorrelations

ρ1 = (1 + φθ)(φ + θ) ÷ (1 + 2φθ + θ²); ρk = φ × ρk-1 for k ≥ 2

Decay at rate φ starts from lag 1, not lag 0.

AR(1) ACF

ρk = φ^k

Geometric decay; PACF is φ at lag 1 and zero afterwards.

MA(1) ACF

ρ1 = θ ÷ (1 + θ²); ρk = 0 for k ≥ 2

PACF decays gradually.

Identification rule

AR(p): PACF cuts off after p | MA(q): ACF cuts off after q | ARMA: both decay

Cutoff means statistically insignificant, not exactly zero, in a sample.

Box-Pierce Q statistic

Q_BP = T × Σ(k=1 to m) ρ̂k²

T = sample size, ρ̂k = sample autocorrelation at lag k. Under the null of no autocorrelation, Q is approximately chi-squared with m degrees of freedom (reduced by the number of estimated ARMA parameters when applied to residuals).

Ljung-Box Q statistic

Q_LB = T(T + 2) × Σ(k=1 to m) [ρ̂k² ÷ (T − k)]

Better small-sample behaviour than Box-Pierce. Same chi-squared reference distribution. Reject the null of white noise if Q exceeds the critical value.

AIC

AIC = ln(σ̂²) + 2k ÷ T

σ̂² = SSR ÷ T, k = number of estimated parameters. Choose the lowest value. Some texts write AIC = −2 ln L + 2k; the ranking is the same.

BIC (SIC)

BIC = ln(σ̂²) + k ln(T) ÷ T

Penalty is larger than AIC's when T ≥ 8 (ln T > 2). Prefers more parsimonious models.

Wold's representation

Yt = μ + Σ(i=0 to ∞) ψi εt−i, with ψ0 = 1 and Σψi² < ∞

Applies to covariance stationary processes. εt is white noise. Any AR or ARMA model that is stationary has such a form.

Lag operator

L Yt = Yt−1; Lⁿ Yt = Yt−n

AR(1): (1 − φL)Yt = c + εt. For |φ| < 1, Yt = (c ÷ (1 − φ)) + Σ φⁱ εt−i.

AR(1) multi-step forecast

E[Yt+h | Yt] = μ + φʰ (Yt − μ), where μ = c ÷ (1 − φ)

Valid for |φ| < 1. The forecast decays geometrically to the mean μ. One-step: Ŷt+1 = c + φYt.

Seasonal dummy model (no intercept)

y(t) = γ1·D1(t) + γ2·D2(t) + ... + γs·Ds(t) + ε(t)

Use s dummies and no intercept. Each γ is the mean of its season.

Seasonal dummy model (with intercept)

y(t) = β0 + β2·D2(t) + ... + βs·Ds(t) + ε(t)

Use s − 1 dummies. β0 is the mean of the omitted base season; each β is the difference from the base. Using s dummies plus an intercept causes perfect multicollinearity (dummy variable trap).

Seasonal AR(1)

y(t) = φ·y(t−s) + ε(t)

Seasonal lag s: 4 for quarterly, 12 for monthly. Needs |φ| < 1 for stationarity.

Seasonal MA(1)

y(t) = ε(t) + θ·ε(t−s)

ACF is nonzero at lag s and zero beyond it.

Seasonal ARMA combined with short-run terms

(1 − φ1·L)(1 − Φ1·L^s)·y(t) = (1 + θ1·L)(1 + Θ1·L^s)·ε(t)

L is the lag operator. Short-run terms capture momentum; seasonal terms capture the repeat at lag s. The product form creates cross terms at lag s+1.

Forecast of seasonal dummy model

ŷ(T+h) = γ of the season that period T+h falls in

Pick the dummy coefficient for the target season.

Quick revision

• Covariance stationary: constant mean, constant variance, autocovariance depends only on the lag.
• White noise: mean 0, constant variance, zero autocorrelation at all lags.
• AR(1): yt = c + φyt−1 + εt; stationary only if |φ| < 1.
• AR(1) mean is c ÷ (1 − φ); variance is σ² ÷ (1 − φ²).
• AR(1) autocorrelation at lag k is φ^k, so it decays geometrically.
• MA(1): yt = μ + εt + θεt−1; always stationary; ACF is zero beyond lag 1.
• MA(1) variance is σ²(1 + θ²).
• AR(p): ACF decays gradually, PACF cuts off after lag p.
• MA(q): ACF cuts off after lag q, PACF decays gradually.
• ARMA: both ACF and PACF decay gradually.
• Information criteria such as AIC and BIC penalise extra parameters; BIC penalises more heavily.
• Seasonality shows as significant autocorrelation at seasonal lags and can be handled with seasonal dummies or seasonal terms.

Common mistakes

• Saying a stationary series has constant values or no randomness. Fix: Stationary means the statistical properties (mean, variance, autocovariance) are stable. The series can still fluctuate a lot around its mean.
• Forgetting that the variance must be finite as well as constant. Fix: Include 'finite' in your definition. It matters for heavy-tailed series.
• Treating white noise as independent in all cases Fix: Remember that independence is stronger. Only for normal variables does uncorrelated imply independent.
• Assuming white noise must be normal Fix: White noise needs only zero mean, constant variance and no autocorrelation. Any distribution with those moments qualifies.
• Treating the intercept c as the mean of the process. Fix: Always use μ = c ÷ (1 − φ) for AR(1), or c ÷ (1 − Σφ) for AR(p).
• Using σ² ÷ (1 − φ) for the variance, or forgetting to square φ. Fix: Remember: the mean has (1 − φ), the variance has (1 − φ²). Check that variance is at least σ².
• Writing the MA(1) variance as σ²(1 + θ) or σ² + θ². Fix: Variance of a sum of uncorrelated terms is the sum of weight² × σ². The current shock has weight 1, so γ0 = σ²(1 + θ²).
• Using ρ1 = θ for an MA(1). Fix: For MA(1), ρ1 = θ ÷ (1 + θ²). Check that |ρ1| ≤ 0.5.
• Swapping the cutoff rules for AR and MA. Fix: Remember: MA cuts off in the ACF (its memory is finite in shocks). AR cuts off in the PACF.
• Claiming ARMA has a cutoff in the ACF or PACF at lag p or q. Fix: In ARMA both functions decay gradually, so neither identifies p and q cleanly.

Exam tips

• Expect questions that give a process equation and ask which option is stationary. Scan for trends and a coefficient of 1 or more first.
• Memorise the AR and MA patterns: PACF cut-off signals AR, ACF cut-off signals MA.
• For AR(1) questions, compute the mean as c ÷ (1 - φ) and the variance as σ² ÷ (1 - φ²) before looking at the options.
• Read carefully whether the question asks for autocovariance or autocorrelation, and divide by γ(0) when needed.
• When a series is not stationary, the usual remedy in answers is differencing or detrending, so be ready to name it.
• Know the hierarchy: Gaussian white noise ⇒ independent white noise ⇒ white noise. Questions often test which implication fails.
• Expect variance-of-a-combination questions. Use squared coefficients times σ² and no covariance terms.
• Remember that white noise is the building block: AR, MA and ARMA models are built from it, and good residuals should look like it.