State Space Model¶

by Professor Throckmorton
for Time Series Econometrics
W&M ECON 408/PUBP 616
Slides

Summary¶

A state space model describes the evolution of observable time series using a set of hidden (unobserved) variables called states.
Applications include
- Econometrics: state-space ARIMA model, Kalman Filter, regime-switching models
- Engineering: GPS tracking, missile guidance, robotics, audio processing
- Biostatistics: Tracking pandemics, i.e., the susceptible, infected, recovered (SIR) model with hidden states

Consider the MA($1$) model $y_t = \mu + \varepsilon_t + \theta \varepsilon_{t-1}$
This maps to a state space model
The State Equation is $x_{t} = 0 \times x_{t-1} + \theta\varepsilon_{t-1}$, which describes how the unobserved state evolves over time.
The Observation Equation is $y_t = \mu + x_{t} + \varepsilon_t$
This equation shows how the observed variable $y_t$ is related to the state variable $x_t$ and the current error term $\varepsilon_t$.
A mapping like this can be done for the ARIMA($p,d,q$) model

The Capital Asset Pricing Model (CAPM) tries to explain how the expected return on a stock is related to market risk.
Observation Equation

$$ R_t = \alpha_t + \beta_t R_{M,t} + \varepsilon_t, \quad \varepsilon_t \sim i.i.d. N(0, \sigma^2_\varepsilon) $$
- $R_t$: stock's excess return (relateive to a risk-free rate)
- $R_{M,t}$: market's excess return
- $\beta_t$: the sensitivity of a stock's return to the market return
State Equations (Time-varying coefficients):

\begin{gather*} \alpha_t = \alpha_{t-1} + \nu_{1,t}, \quad \nu_{1,t} \sim \text{i.i.d.} N(0, \sigma^2_{\nu_1}) \\ \beta_t = \beta_{t-1} + \nu_{2,t}, \quad \nu_{2,t} \sim \text{i.i.d.} N(0, \sigma^2_{\nu_2}) \end{gather*}

State Vector:

$$ \mathbf{x}_t = \begin{bmatrix} \alpha_t \\ \beta_t \end{bmatrix} $$
State Equation:

\begin{gather*} \mathbf{x}_{t} = \mathbf{x}_{t-1} + \boldsymbol{\boldsymbol{\eta}}_t \\ \boldsymbol{\boldsymbol{\eta}}_t = \begin{bmatrix} \nu_{1,t} \\ \nu_{2,t} \end{bmatrix} \end{gather*}
Observation Equation: $R_t = [1 \quad R_{M,t}] \mathbf{x}_t + \varepsilon_t$
This form allows for estimation via the Kalman filter, handling time variation in the intercept and slope of the CAPM regression efficiently.

The Kalman Filter estimates the unobservable states from (noisy) measurements (i.e., time series data).
"The Kalman filter is an algorithm for sequentially updating a linear projection of the system." - Hamilton, Chapter 13
Benefits: allows calculation of exact likelihood function of Gaussian ARMA processes; estimate time-varying parameter VAR
How it works:
1. Update the previous prediction (forecast) using latest observation
2. Predict (forecast) what is expected to happen
3. Repeat this loop for the next period (i.e., loop across time)
The Kalman Filter balances prior knowledge (what we expect) with new observations (what actually happens).

Filtering
- Estimate the current state using past and present data
- e.g., where is my car right now based on past GPS signals?
- Used in real-time applications, like navigation or finance
Smoothing
- Estimate the state at a past time using all available data, including future observations
- e.g., after the trip, refining the car's exact position using its future positions
- Used in post-processing, trend analysis, and historical reconstructions
Filtering is like writing the first draft of an essay, and smoothing is like editing it later when you have a better idea of what you're trying to say.

Simplified state-space model

\begin{gather*} \mathbf{x}_{t+1} = \mathbf{F} \mathbf{x}_t + \boldsymbol{\eta}_t, \quad \boldsymbol{\eta}_t \sim N(0, \mathbf{Q}) \\ \mathbf{y}_t = \mathbf{Z}_t \mathbf{x}_t + \boldsymbol{\varepsilon}_t, \quad \boldsymbol{\varepsilon}_t \sim N(0, \mathbf{H}) \end{gather*}
Let $\hat{\mathbf{x}}_{t+1|t} \equiv E(\mathbf{x}_{t+1} | \mathbf{y}_t, \mathbf{y}_{t-1},\ldots)$ represent a forecast of the state vector at $t+1$ conditional on data through time $t$
Associated with each of these forecasts is a mean squared error (MSE) matrix

$$ \mathbf{P}_{t+1|t} \equiv E[(\mathbf{x}_{t+1} - \hat{\mathbf{x}}_{t+1|t})(\mathbf{x}_{t+1} - \hat{\mathbf{x}}_{t+1|t})'] $$
Assume $\mathbf{F},\mathbf{Q},\mathbf{Z}_t,\mathbf{H}$ are known with certainty. Since a byproduct of the Kalman filter is the likelihood function, we can estimate these parameters by maximum likelihood estimation (MLE).

Updated Innovation (using the observation equation, see Hamilton equation 13.2.2)

$$ \hat{\boldsymbol{\varepsilon}}_t = \mathbf{y}_t - \mathbf{Z}_t \hat{\mathbf{x}}_{t|t-1} $$
Updated variance of $\hat{\boldsymbol{\varepsilon}}_t$ (i.e., $E[(\mathbf{y}_t - \hat{\mathbf{y}}_{t|t-1})(\mathbf{y}_t - \hat{\mathbf{y}}_{t|t-1})']$, see Hamilton equation 13.2.12)

$$ \mathbf{V}_t = \mathbf{Z}_t \mathbf{P}_{t|t-1} \mathbf{Z}_t' + \mathbf{H} $$
Updated state vector (see Hamilton equation 13.2.15) This is the filtered state! (i.e., estimated state vector)

$$ \hat{\mathbf{x}}_{t|t} = \hat{\mathbf{x}}_{t|t-1} + \mathbf{P}_{t|t-1}\mathbf{Z}_t'\mathbf{V}_t^{-1}\hat{\boldsymbol{\varepsilon}}_t $$
A byproduct of the Kalman filter is the log-likelihood function (i.e., the multivariate normal distribution)

$$ \mathcal{L}_t = \log | V_t | + \hat{\boldsymbol{\varepsilon}}_t' V_t^{-1} \hat{\boldsymbol{\varepsilon}}_t $$

Kalman gain (see Hamilton equation 13.2.19)

$$ \mathbf{K}_t = \mathbf{F} \mathbf{P}_{t|t-1} \mathbf{Z}_t' \mathbf{V}_t^{-1} $$
Forecast of state vector (see Hamilton equation 13.2.18)

$$ \hat{\mathbf{x}}_{t+1|t} = \mathbf{F} \hat{\mathbf{x}}_{t|t-1} + \mathbf{K}_t \hat{\boldsymbol{\varepsilon}}_t $$
MSE of forecast (see Hamilton equation 13.2.22)

$$ \mathbf{P}_{t+1|t} = \mathbf{F} \mathbf{P}_{t|t-1}\mathbf{F}' - \mathbf{K}_t \mathbf{Z}_t \mathbf{P}_{t|t-1}\mathbf{F}' + \mathbf{Q} $$