AR Model

by Professor Throckmorton
for Time Series Econometrics
W&M ECON 408/PUBP 616
Slides

AR($p$) Model¶

• The AR($p$) model is: $y_t = \phi_1 y_{t-1} + \phi_2 y_{t-2} + \dots + \phi_p y_{t-p} + \varepsilon_t$

• $\phi_1$, $\phi_2$, $\ldots$, $\phi_p$ are the autoregressive parameters that quantify the influence of past values on the current value

• $\varepsilon_t$ is white noise error term (or innovation) with mean zero and constant variance, $\sigma^2$

AR(1) Model¶

• An AR(1) Model is a special case of the AR($p$)

  $$y_t = \phi y_{t-1} + \varepsilon_t$$

  (1)

• An AR(1) is often used to model cyclical dynamics

• e.g., the fluctuations in an AR(1) series where $\phi = 0.95$ appear much more persistent (i.e., smoother) than those where $\phi = 0.4$

AR(1) for any mean¶

• An AR(1) model is covariance stationary if $|\phi| < 1$

• Suppose $E[y_t] = E[y_{t-1}] \equiv \bar{y}$

• Then $E[y_t] = 0$ since $E[y_t] = \phi E[y_{t-1}] + E[\varepsilon_t] \rightarrow (1-\phi)\bar{y} = 0 \rightarrow \bar{y} = 0$

• Instead consider the AR(1) model with a generic mean: $y_t - \mu = \phi(y_{t-1} - \mu) + \varepsilon_t$

• Then $E[y_t] = \mu$ since

  $$\begin{align*} E[y_t] - \mu &= \phi E[y_{t-1}] -\phi\mu + E[\varepsilon_t] \\ \rightarrow (1-\phi)\bar{y} &= (1-\phi)\mu \\ \rightarrow \bar{y} &= \mu \end{align*}$$

  (2)

• You can rearrange the (any mean) AR(1) model as $y_t = (1-\phi)\mu + \phi y_{t-1} + \varepsilon_t$

AR(1) as MA($\infty$)¶

• The AR(1) model structure is the same at all points in time

  $$\begin{align*} y_t &= \phi y_{t-1} + \varepsilon_t \\ y_{t-1} &= \phi y_{t-2} + \varepsilon_{t-1} \\ y_{t-2} &= \phi y_{t-3} + \varepsilon_{t-2} \end{align*}$$

  (3)

• Combine, i.e., recursively substitute, to get $y_t = \phi^3 y_{t-3} + \phi^2 \varepsilon_{t-2}+ \phi \varepsilon_{t-1} + \varepsilon_t$

• Rinse and repeat to get $y_t = \sum_{i=0}^\infty \phi^i \varepsilon_{t-i}$ (i.e., the MA($\infty$) Model or Wold Representation)

• And an MA($\infty$) is invertible back to an AR(1) so long as $|\phi| < 1$.

• An AR model that can be written as an MA($\infty$) is causal (or exhibits causality).

• That property differentiates it from an invertible MA model, which can be expressed as an AR($\infty$) model.

Variance¶

• Suppose $\mu=0$ without loss of generality.

• Given $Var(\varepsilon_t) = \sigma^2$ and $|\phi| < 1$, then $\gamma(0) \equiv Var(y_t)$

  $$\begin{align*} &= Var\left(\sum_{i=0}^\infty \phi^i \varepsilon_{t-i}\right) \\ &= Var(\varepsilon_t + \phi \varepsilon_{t-1} + \phi^2 \varepsilon_{t-2} + \phi^3 \varepsilon_{t-3} + \phi^4 \varepsilon_{t-4} + \cdots) \\ &= \sigma^2 + \phi^2 \sigma^2 + (\phi^2)^2 \sigma^2 + (\phi^3)^2 \sigma^2 + (\phi^4)^2 \sigma^2 + \cdots \\ &= \sigma^2(1 + \phi^2 + (\phi^2)^2 + (\phi^2)^3 + (\phi^2)^4 + \cdots ) \\ &= \sigma^2/(1-\phi^2) \end{align*}$$

  (4)

  See Geometric Series.

• $Std(y_t) = \sqrt{\sigma^2/(1-\phi^2)}$

Autocorrelation¶

• $\gamma(1) = E[y_t y_{t-1}]$

  $$\begin{align*} &= E[(\phi y_{t-1} + \varepsilon_t)y_{t-1}] \\ &= E[\phi y_{t-1}^2] + E[\varepsilon_t y_{t-1}] \\ &= \phi Var[y_{t-1}] + 0 \\ &= \phi \gamma(0) \\ &= \phi \sigma^2/(1-\phi^2) \end{align*}$$

  (5)

• $\rho(1) = \gamma(1)/\gamma(0) = \phi$

• $\gamma(2) = E[y_t y_{t-2}] = E[(\phi^2y_{t-2} + \varepsilon_t + \phi \varepsilon_{t-1})y_{t-2}] = \phi^2 E[y_{t-2}^2] = \phi^2 \gamma(0)$

• $\rho(2) \equiv \gamma(2)/\gamma(0) = \phi^2$

• In general, $\rho(k) = \phi^k$, which is a very nice looking ACF.

• Thus, for an AR(1) process/model we can refer to $\phi$ as the autocorrelation (AC) parameter/coefficient.

AR Model as DGP¶

Let’s compare a couple different AR models to their underlying white noise.

1. White noise:

   $$y_t = \mu + \varepsilon_t$$

   (6)

2. AR(1), low AC:

   $$y_t = (1-\phi_l)\mu + \phi_l y_{t-1} + \varepsilon_t$$

   (7)

3. AR(1), high AC:

   $$y_t = (1-\phi_h)\mu + \phi_h y_{t-1} + \varepsilon_t$$

   (8)

Estimation Results¶

• Parameter estimates are close to DGP and significantly different from 0

• Residuals have skewness near 0 and kurtosis near 3 so they are probably normally distributed

Modeling UR as AR(1)¶

• The U.S. unemployment rate is very persistent.

• What if we model it with an AR(1)?

• The ACF decays very slow $\rightarrow$ UR might be non-stationary.

• Let’s assume it is stationary for a moment and estimate an AR(1) model

Estimation Results¶

• Estimate of AR coeffecient is very high at 0.9704, but the data is not a random walk

• Residuals are skewed and have excess kurtosis $\rightarrow$ model is missing nonlinearity

Actual vs. Simulated Data¶

• What if we simulated data from AR(1) model setting the parameters to the previous estimates?

• What does a simulation from this estimated model look like?

• This ACF approaches zero at shorter lags compared to the actual data.

• We know an AR(1) model with $\phi = 0.9704$ generates stationary time series.

• Maybe the actual UR isn’t stationary?

Does it fit?¶

• The actual UR is very smooth, but the AR(1) model is not.

• The simulated UR often falls below $2\%$, whereas the actual UR does not.

• The actual UR meets or exceeds $10\%$ a few times, whereas the simulated UR does not.

• The actual UR rises quickly and falls slowly, whereas the simulated UR seems to rise and fall at roughly the same rates.

• This all suggests that the AR(1) model is unable to reproduce some important features of the actual UR.

Partial ACF¶

• Most of the slides in this chapter are about the AR(1) model.

• Suppose the true DGP is an AR model, but we do not know the order.

• When the DGP is a MA model, we can use the ACF to determine the order since it drops off to zero after a certain lag.

• But for AR model, the autocorrelation at higher lags is influenced by the intermediate observations between $t$ and $t-k$, and even for an AR(1) model the ACF tapers slowly.

• Alternatively, Partial ACF (PACF) helps to identify the order of an AR model.

• The PACF at lag $k$ is the correlation between $y_t$ and $y_{t-k}$ that is not explained by observations in between, $\{y_{t-k+1}:y_{t-1}\}$

• The PACF at lag $k$ is denoted as $\phi_{kk}$, and plotting $\phi_{kk}$ against lag $k$ forms the PACF plot.

AR(2) Model¶

• Let’s simulate the following AR(2) model

  $$y_t = (1-\phi_1-\phi_2)\mu + \phi_1 y_{t-1} + \phi_2 y_{t-2} + \sigma \varepsilon_t$$

  (9)

  where $\mu = 2$, $\phi_1 = 0.8$, $\phi_2 = 0.15$, and $\sigma = 0.5$

• We know the DGP and the true order of the AR model. Does the PACF suggest the same order?

• The PACF is significantly different from zero for $k = \{1,2\}$, which indicates the DGP is probably an AR(2).

• So the PACF is good at deterimining order so long as AR coefficients are large enough.

Modeling UR as AR(2)¶

• Is a higher order AR model a good fit for the U.S. unemployment rate?

• Let’s first plot the PACF of the UR to see if we can determine an order.

This suggests that an AR(1) might be the best model.

• What if we ignore the PACF and estimate an AR(2) anyway?

Estimation Results¶

• Both AR coefficients are significantly different from 0

• But the AR(2) is still linear, i.e., normal and symmetric, so the residuals are still skewed and fat-tailed

• PACF did not help to determine that second lag might improve fit

UR First Difference¶

Is it stationary?

• Yes, mean looks constant around 0.

• Yes, volatility looks constant, except for COVID.

• Maybe not, it’s impossible to see if the autocovariance is constant.

• The ACF indicates that the series is probably stationary.

• What does the PACF say?

• The PACF shows that the autocorrelation is not significantly different from zero for all lags.

• What is the best model to estimate? Let’s try MA(1) and AR(1) and compare.

Estimation Results¶

• The MA coefficient estimate is statistically significantly different from 0.

• However, the residuals are still probably heteroskedastic and not normally distributed.

• Thus, a linear/normal model is probably not the best choice.

• The AR coefficient is small/weak but estimate is statistically significantly different from 0.

• Like the MA(1) model, the residuals are still probably heteroskedastic and not normally distributed.

• This again confirms that a linear/normal model is probably not the best choice.