ARMA models are the linear building blocks of time-series modelling. AR (autoregressive) captures persistence — past values predict current values. MA (moving-average) captures lagged shock effects. ARMA combines both. Box-Jenkins methodology, dominant from the 1970s to the present, builds univariate forecasts around this framework.
AR(p)
X_t = c + φ₁ X_{t-1} + φ₂ X_{t-2} + ... + φ_p X_{t-p} + ε_t
Current value is a linear combination of the previous p values plus innovation. AR(1) with |φ| < 1 is stationary; the ACF decays geometrically as ρ(h) = φ^h. AR(2) gives more interesting dynamics including oscillation (complex roots of the characteristic polynomial).
Stationarity condition for AR(p)
Write 1 - φ₁ z - φ₂ z² - ... - φ_p z^p as the characteristic polynomial. The AR(p) is stationary iff all roots lie outside the unit circle in the complex plane. Equivalently, all roots of z^p - φ₁ z^{p-1} - ... - φ_p = 0 have modulus < 1.
MA(q)
X_t = μ + ε_t + θ₁ ε_{t-1} + θ₂ ε_{t-2} + ... + θ_q ε_{t-q}
Current value depends on current and recent innovations. MA(q) is always stationary (it's a finite linear combination of white noise). Its ACF is non-zero exactly up to lag q and zero beyond — useful for identification.
Invertibility
MA processes are invertible if the MA characteristic polynomial 1 + θ₁ z + ... + θ_q z^q has roots outside the unit circle. Invertibility allows writing the MA as an infinite AR — which is necessary for using past observations to identify innovations.
ARMA(p, q)
X_t = c + φ₁ X_{t-1} + ... + φ_p X_{t-p} + ε_t + θ₁ ε_{t-1} + ... + θ_q ε_{t-q}
AR and MA combined. Stationarity requires AR-side roots outside the unit circle; invertibility requires MA-side roots outside. ARMA(1,1) is the workhorse parsimonious model for many series.
Lag operator
Define L X_t = X_{t-1}. Then AR(p): (1 - φ₁L - ... - φ_p L^p) X_t = ε_t, abbreviated Φ(L) X_t = ε_t. MA(q): X_t = Θ(L) ε_t. ARMA: Φ(L) X_t = Θ(L) ε_t. Compact, abstract, and the natural notation for derivations.
Wold representation
Wold's theorem (1938): every purely non-deterministic stationary process can be written as an infinite MA — a linear combination of current and past innovations. ARMA is therefore not just a convenient parametric family but, in a deep sense, the canonical form of stationary linear processes.
Why ARMA-fit returns rarely beat the mean
Empirical AR/MA coefficients on equity returns are tiny — typically |φ| < 0.1 — and statistically not different from zero. The first-order autocorrelation of daily returns is often near zero by EMH; any pattern shows up only over very specific frequencies or sub-periods. ARMA modelling of returns directly is rarely profitable; ARMA on volatility (via GARCH, Module 7) or on macro variables is where the tools earn their keep.
ARIMA(p, d, q)
When the raw series is non-stationary, first-difference it d times until stationary, then fit an ARMA(p, q) to the differenced series. ARIMA(0,1,0) is the random walk. ARIMA(0,1,1) is the model behind exponential smoothing. The 'I' stands for integrated, denoting d unit roots.
Exercise
An AR(1) process X_t = 0.6 X_{t-1} + ε_t has ε_t ~ WN(0, 1). (1) Is it stationary? (2) Compute its variance, ρ(1), ρ(3). (3) Given X_0 = 2, forecast E[X_1], E[X_2], E[X_3].