State-space models and the Kalman filter

State-space models are a remarkably general framework for time-series modelling. Almost every linear time-series model — ARMA, VAR, time-varying-parameter models, dynamic factor models — can be cast in state-space form. The Kalman filter is the algorithm that performs exact recursive Bayesian inference for the latent state, and it underlies modern signal processing, optimal control, and a large portion of macroeconometric work.

State-space form

State equation:        x_t = F x_{t-1} + w_t,    w_t ~ N(0, Q)
Observation equation:  y_t = H x_t + v_t,        v_t ~ N(0, R)

x_t is the latent state vector (unobserved). y_t is the observed measurement. F propagates the state; H maps state to observations; Q and R are the state and observation noise covariances. The system is linear-Gaussian — both equations are linear and the errors are normal.

Kalman filter equations

Two alternating steps: predict (project state forward using F) and update (correct with new observation y_t).

Predict:
  x̂_{t|t-1} = F x̂_{t-1|t-1}
  P_{t|t-1}  = F P_{t-1|t-1} Fᵀ + Q
Update:
  S_t = H P_{t|t-1} Hᵀ + R
  K_t = P_{t|t-1} Hᵀ S_t⁻¹   (Kalman gain)
  x̂_{t|t} = x̂_{t|t-1} + K_t (y_t - H x̂_{t|t-1})
  P_{t|t} = (I - K_t H) P_{t|t-1}

Smoothing

The filter gives x̂_{t|t} — the best estimate of x_t given observations through time t. The smoother gives x̂_{t|T} — the best estimate of x_t given all observations through time T (including t+1, ..., T). The Rauch-Tung-Striebel (RTS) smoother runs backward through time after a forward Kalman pass; it always reduces variance.

Time-varying parameters

A regression with drifting coefficients fits naturally in state-space form: y_t = x_t' z_t + v_t, where x_t (the coefficients) evolve as a random walk x_t = x_{t-1} + w_t. The Kalman filter estimates the time-varying coefficient path optimally. This is how you'd model a slowly-drifting equity beta or a CAPM with a regime-dependent risk premium.

Dynamic factor models

Many-variable time series often have a small number of common factors driving most variation. Casting as state-space: factors are the latent state, observations are linear combinations of factors plus idiosyncratic noise. Kalman filter / smoother gives the optimal factor estimates. Stock-Watson (1989) coincident-leading indicator index is the classical macro example.

Missing data and irregular sampling

Kalman filter naturally handles missing observations: just skip the update step at missing time points (and continue the predict step). Particularly powerful for mixed-frequency data — combine monthly inflation with quarterly GDP and daily yields in a single coherent state-space model.

Non-linear extensions

• Extended Kalman filter (EKF): linearise around the current state estimate. Works for mild non-linearity.
• Unscented Kalman filter (UKF): propagate sigma points through the non-linearity. More accurate for moderate non-linearity.
• Particle filter (Sequential Monte Carlo): samples from the posterior; arbitrary non-linearity/non-Gaussianity. Computationally heavy.

Likelihood computation

The Kalman filter produces the prediction error y_t - H x̂_{t|t-1} and its variance S_t at every step. The log-likelihood of the data given parameters is:

log L = -(1/2) Σ [log(2π|S_t|) + (y_t - H x̂_{t|t-1})ᵀ S_t⁻¹ (y_t - H x̂_{t|t-1})]

Parameter estimation: numerically maximise this over (F, H, Q, R, ...). This is how Bayesian dynamic models, time-varying VARs, and most modern state-space estimators work.