VAR Model

by Professor Throckmorton
for Time Series Econometrics
W&M ECON 408/PUBP 616
Slides

Introduction¶

• A $VAR(p)$ is a system of $n$ linear equations that can be compactly expressed with vector/matrix notation

  $$\begin{gather*} \mathbf{y}_t = \boldsymbol{\mu} + \boldsymbol{\Phi}_1 \mathbf{y}_{t-1} + \boldsymbol{\Phi}_2 \mathbf{y}_{t-2} + \cdots + \boldsymbol{\Phi}_p \mathbf{y}_{t-p} + \boldsymbol{\varepsilon}_t \end{gather*}$$

  (1)

  where bold indicates a vector/matrix, e.g., $\mathbf{y}$ has size $n\times 1$ and $\boldsymbol{\Phi}$ are $n\times n$ square matrices

• Now we assume $E(\boldsymbol{\varepsilon}_t) = \mathbf{0}$ and $E(\boldsymbol{\varepsilon} \boldsymbol{\varepsilon}') = \boldsymbol{\Omega}$, which has size $n\times n$

• $\boldsymbol{\Omega}$ is the covariance matrix and must be a symmetric positive definite matrix (i.e., this is analogous to a scalar SD/variance being positive)

• A $VAR$ is a system of linear equations where each variable is regressed on $p$ of its own lags as well as $p$ lags of each other variable

Growth Model Example¶

• Consider the following (nonlinear) growth model

  $$\begin{gather*} y_t = z_{t-1} k_{t-1}^\alpha \tag{Prod.} \\ k_t = (1-\delta)k_{t-1} + i_{t-1} \tag{Capital} \\ \log z_t = (1-\rho)\log(\bar{z}) + \rho \log z_{t-1} + \varepsilon_t \tag{Tech.} \\ \varepsilon \sim i.i.d.~N(0,\sigma^2) \tag{Shock} \end{gather*}$$

  (2)

• Capital accumulation and technology are (linear) first-order difference questions, and technology is specifically an AR(1)

• Investment could be exogenous, or endogenous to output, e.g., $i_t = sy_t$ where $s$ is the savings rate.

• The production function is nonlinear, but we can linearize it with natural log, e.g., $\log y_t = \log z_{t-1} + \alpha \log k_{t-1}$

Natural log and percent change¶

• It would be nice if all the units of all variables were something easily interpretable, e.g., in percent changes

• Suppose each variable is covariance stationary and has a constant long-run mean (a.k.a. steady state or long-run equilibrium). For any variable in the model, $x_t$, define its mean as $\bar{x}$.

• Note that $\log(x_t/\bar{x}) \approx x_t/\bar{x} - 1 \equiv \hat{x}_t$ (for more detail, see natural logs and percent changes)

• This approximation works so long as the percent changes are relatively small, e.g., 5% or less considering observing a 5% GDP growth rate would be far in the right tail of the distribution of growth rates for most countries.

Linearization¶

• Recall that natural log converts exponents to multiplication and multiplication to addition (awesome!)

• Consider log output, $\log y_t = \log z_t + \alpha \log k_{t-1}$.

• Suppose that each variable has a long-run mean/steady state, then $\log \bar{y} = \log \bar{z} + \alpha \log \bar{k}$.

• Question: How do we express those variables as percent change?

• Answer: Subtract the steady state equation from the dynamic equation (note that $\log x_t - \log \bar{x} = \log(x_t/\bar{x})$)

  $$\begin{gather*} \log y_t = \log z_{t-1} + \alpha \log k_{t-1} \\ - (\log \bar{y} = \log \bar{z} + \alpha \log \bar{k}) \\ \rightarrow \log (y_t/\bar{y}) = \log (z_{t-1}/\bar{z}) + \alpha \log (k_{t-1}/\bar{k}) \\ \rightarrow \hat{y}_t = \hat{z}_{t-1} + \alpha \hat{k}_{t-1} \end{gather*}$$

  (3)

• Now $k_t = (1-\delta)k_{t-1} + i_{t-1}$ is already linear, but the variables are not expressed as percent changes.

• Subtract the steady-state equation from the dynamic equation

  $$\begin{gather*} k_t = (1-\delta)k_{t-1} + i_{t-1} \\ - (\bar{k} = (1-\delta)\bar{k} + \bar{i}) \\ \rightarrow (k_t-\bar{k}) = (1-\delta)(k_{t-1}-\bar{k}) + (i_{t-1}-\bar{i}) \end{gather*}$$

  (4)

• Multiply each term by one in the form of $\bar{x}/\bar{x}$ using the appropriate variable for each term.

  $$\begin{gather*} \frac{\bar{k}}{\bar{k}}(k_t-\bar{k}) = (1-\delta)\frac{\bar{k}}{\bar{k}}(k_{t-1}-\bar{k}) + \frac{\bar{i}}{\bar{i}}(i_{t-1}-\bar{i}) \\ \rightarrow \bar{k}\hat{k}_t = (1-\delta)\bar{k}\hat{k}_{t-1} + \bar{i}\hat{i}_{t-1} \end{gather*}$$

  (5)

• Now it’s linear and all variables are expressed as percent changes.

Linear Growth Model¶

• The linear growth model with endogenous investment

  $$\begin{gather*} \hat{y}_t = \hat{z}_{t-1} + \alpha \hat{k}_{t-1} \\ \hat{k}_t = (1-\delta)\hat{k}_{t-1} + \bar{i}\hat{i}_{t-1}/\bar{k} \\ \hat z_t = \rho \hat z_{t-1} + \varepsilon_t \\ \hat i_t = \hat{y}_{t-1} \\ \varepsilon \sim i.i.d.~N(0,\sigma^2) \end{gather*}$$

  (6)

• This is a linear system of 4 dynamic equations and 4 variables (i.e., unknowns), $\{\hat{y}_t,\hat{k}_t,\hat z_t,\hat i_t \}$ that we can map to a VAR(1), $\mathbf{x}_t = \boldsymbol{\mu} + \boldsymbol{\Phi} \mathbf{x}_{t-1} + \boldsymbol{\varepsilon}_t$

  $$\begin{gather*} \begin{bmatrix} \hat{y}_t \\ \hat{k}_t \\ \hat{z}_t \\ \hat{i}_t \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} + \begin{bmatrix} 0 & \alpha & 1 & 0 \\ 0 & (1-\delta) & 0 & \bar{i}/\bar{k} \\ 0 & 0 & \rho & 0 \\ 1 & 0 & 0 & 0 \\ \end{bmatrix} \begin{bmatrix} \hat{y}_{t-1} \\ \hat{k}_{t-1} \\ \hat{z}_{t-1} \\ \hat{i}_{t-1} \end{bmatrix}+ \begin{bmatrix} 0 \\ 0 \\ \varepsilon_t \\ 0 \end{bmatrix} \end{gather*}$$

  (7)

  where $\varepsilon \sim i.i.d.~N(0,\sigma^2)$

Covariance Matrix¶

• In a VAR($p$), assume $E(\boldsymbol{\varepsilon}_t) = \mathbf{0}$ and $E(\boldsymbol{\varepsilon} \boldsymbol{\varepsilon}') = \boldsymbol{\Omega}$

• In the simple linear growth model, there is only one innovation/shock.

  $$\begin{gather*} \boldsymbol{\varepsilon}_t \equiv \begin{bmatrix} 0 \\ 0 \\ \varepsilon_t \\ 0 \end{bmatrix} \end{gather*}$$

  (8)

  which clearly satisfies $E(\boldsymbol{\varepsilon}_t) = 0$ since all elements are zero or $E(\varepsilon_t) = 0$

• As for the covariance matrix $E(\boldsymbol{\varepsilon}_t\boldsymbol{\varepsilon}_t')=$

  $$\begin{gather*} E\left( \begin{bmatrix} 0 \\ 0 \\ \varepsilon_t \\ 0 \end{bmatrix} \begin{bmatrix} 0 & 0 & \varepsilon_t & 0 \\ \end{bmatrix}\right) = E\left(\begin{bmatrix} 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & \varepsilon_t^2 & 0 \\ 0 & 0 & 0 & 0 \\ \end{bmatrix}\right) = \begin{bmatrix} 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & \sigma^2 & 0 \\ 0 & 0 & 0 & 0 \\ \end{bmatrix} \equiv \boldsymbol{\Omega} \end{gather*}$$

  (9)

Likelihood Function¶

• Since a VAR($p$) is a linear system of equations where the innovations are i.i.d. normally distrubted, the likelihood function is a multivariate normal density function. Maximizing the likelihood is the same as maximizing the log-likelihood,

  $$\begin{gather*} \mathcal{L}(\boldsymbol{\Theta}) = -\left(\dfrac{Tn}{2}\right) \log (2\pi) + \left(\dfrac{T}{2}\right) \log \left| \boldsymbol{\Omega}^{-1} \right| \\ - \left(\dfrac{1}{2}\right) \sum_{t=1}^T (\mathbf{y}_t - \boldsymbol{\mu})' \boldsymbol{\Omega}^{-1} (\mathbf{y}_t - \boldsymbol{\mu}). \end{gather*}$$

  (10)

• It turns out that the ordinary least squares (OLS) estimates are the maximum likelihood estimates (MLE) of the parameters in a VAR($p$).

• You can also use estimated residuals, $\hat{\boldsymbol{\varepsilon}}$, from OLS to get the MLE of the covaraince matrix, $\hat{\boldsymbol{\Omega}} = E(\hat{\boldsymbol{\varepsilon}} \hat{\boldsymbol{\varepsilon}}')$.

• Information criterion are used to select the optimal number of lags, e.g., Akaike Information Criterion (AIC) or Bayesian Information Criterion (BIC).

Structural VAR¶

• A structural VAR($p$) is given by

  $$\begin{gather*} \mathbf{A}_0 \mathbf{y}_t = \mathbf{a}_0 + \mathbf{A}_1 \mathbf{y}_{t-1} + \cdots + \mathbf{A}_p \mathbf{y}_{t-p} + \boldsymbol{\varepsilon}_t, \end{gather*}$$

  (11)

  where $\mathbf{A}_0$ allows for contemporaneous relationships between variables, e.g., if there is an aggregate demand shock, then both GDP growth and the inflation rate might increase simultaneously.

• Let’s assume $\boldsymbol{\varepsilon}_t\sim N(0,\mathbf{I}_n)$, i.e., the multivariate standard normal distribution

• There are $n$ variables, each coefficient matrix $\mathbf{A}_j$ are $n \times n$ square matrices, while $\mathbf{a}_0$, $\mathbf{y}_{t-j}$, and $\boldsymbol{\varepsilon}_t$ are $n \times 1$ column vectors.

Reduced-form VAR¶

• Pre-multiplying by the inverse of $\mathbf{A}_0$ yields a reduced-form $VAR$ model

  $$\begin{gather*} \mathbf{y}_t = \mathbf{b}_0 + \mathbf{B}_1 \mathbf{y}_{t-1} + \cdots + \mathbf{B}_p \mathbf{y}_{t-p} + \boldsymbol{\upsilon}_t, \end{gather*}$$

  (12)

  where $\mathbf{b}_0 = \mathbf{A}_0^{-1}\mathbf{a}_0$ is a $n\times 1$ column vector, $\mathbf{B}_j = \mathbf{A}_0^{-1}\mathbf{A}_j$ are $n\times n$ matrices

• Note: $\mathbf{A}_0^{-1} \mathbf{A}_0 \mathbf{y}_t = \mathbf{I}_n \mathbf{y}_t = \mathbf{y}_t$, so just $\mathbf{y}_t$ remains on the left-hand side

• $\boldsymbol{\upsilon}_t = \mathbf{A}_0^{-1}\boldsymbol{\varepsilon}_t$ is a $n\times 1$ vector of shocks that has a multivariate normal distribution with zero mean and variance-covariance matrix $\boldsymbol{\Omega}$

• Since the shocks are possibly correlated across variables, $\mathbf{y}$ becomes an $n\times 1$ vector of endogenous variables.

Estimation¶

• Estimate a reduced-form VAR($p$)'s parameters with ordinary least squares (OLS) estimator.

• However, there might be contemporaneous relationships between variables, i.e., shocks/residuals to different variables can be correlated, so some restriction on the covariance matrix is necessary for identification of a structural VAR($p$).

• Given OLS estimate of covariance matrix, $\hat{\boldsymbol{\Omega}}$, “structural” shocks are often identified recursively, e.g., with a Cholesky decomposition, $\hat{\boldsymbol{\Omega}} = (\hat{\mathbf{A}}_0^{-1})'\hat{\mathbf{A}}_0^{-1}$, which ensures that $\hat{\mathbf{A}}_0^{-1}$ is a lower-triangular matrix.

• The implication is that the ordering of the variables matter in a structural VAR, so researchers must justify the ordering as part of their identification strategy.

OLS Estimates¶

• A reduced-form VAR($p$) can be written compactly as

  $$\begin{gather*} \mathbf{Y}_{n\times(T-p)} = \mathbf{B}_{n\times(1+np)} \mathbf{X}_{(1+np)\times(T-p)} + \mathbf{U}_{n\times(T-p)} \end{gather*}$$

  (13)

  which is a multivariate linear regression where everything in bold is a matrix defined as

  $$\begin{gather*} \mathbf{Y}_{T-j} = [\mathbf{y}_{p+1-j},\ldots,\mathbf{y}_{T-j}] \\ \mathbf{B} =[\mathbf{b}_0,\mathbf{B}_1,\ldots,\mathbf{B}_p] \\ \mathbf{X} = [\mathbf{1},\mathbf{Y}_{T-1}',\ldots,\mathbf{Y}_{T-p}']' \\ \mathbf{U} = [\boldsymbol{\upsilon}_{p+1},\ldots,\boldsymbol{\upsilon}_T] \end{gather*}$$

  (14)

• Parameters estimates are OLS, $\hat{\mathbf{B}} = \mathbf{Y} \mathbf{X}' \left( \mathbf{X}\mathbf{X}'\right)^{-1}$.

• Residual estimates are $\hat{\mathbf{U}} = \mathbf{Y} - \hat{\mathbf{B}} \mathbf{X}$ and the covariance matrix is $\hat{\boldsymbol{\Omega}} = \hat{\mathbf{U}} \hat{\mathbf{U}}'$

• With $\hat{\boldsymbol{\Omega}}$ in hand, the structural shocks are identified by a Cholesky decomposition, $\hat{\boldsymbol{\Omega}} = (\hat{A}_0^{-1})'\hat{A}_0^{-1}$

Recursive Identification¶

• Suppose the data is 1. Output ($y_t$), 2. Inflation ($\pi_t$), and 3. Interest Rate ($r_t$)

  $$\begin{gather*} \small \begin{bmatrix} y_t \\ \pi_t \\ r_t \end{bmatrix} = \begin{bmatrix} b_{0,1} \\ b_{0,2} \\ b_{0,3} \end{bmatrix} + \begin{bmatrix} b_{1,1} & b_{1,4} & b_{1,7} \\ b_{1,2} & b_{1,5} & b_{1,8} \\ b_{1,3} & b_{1,6} & b_{1,9} \end{bmatrix} \begin{bmatrix} y_{t-1} \\ \pi_{t-1} \\ r_{t-1} \end{bmatrix} + \begin{bmatrix} a_{0,1} & 0 & 0 \\ a_{0,2} & a_{0,4} & 0 \\ a_{0,3} & a_{0,5} & a_{0,6} \end{bmatrix} \begin{bmatrix} \varepsilon_{y,t} \\ \varepsilon_{\pi,t} \\ \varepsilon_{r,t} \end{bmatrix} \end{gather*}$$

  (15)

• Recursive identification, e.g., using a Cholesky decomposition, imposes that

  • Output does not respond immediately to other variables’ shocks

  • Inflation may respond immediately to output shocks, but does not respond to interest rate shocks

  • Interest Rate may respond immediately to both output and inflation shocks

• This ordering reflects these beliefs or theory

  • Firms and consumers don’t instantly change output/expenditure $\rightarrow$ output responds slowly.

  • Prices are sticky $\rightarrow$ inflation adjusts with a lag.

  • Central banks move fast $\rightarrow$ interest rates adjust quickly to new information.

Cholesky Decomposition Example¶

• Here’s an example of a positive definite matrix

  $$\boldsymbol{\Omega} = \begin{bmatrix} 4 & 12 & -16 \\ 12 & 37 & -43 \\ -16 & -43 & 98 \end{bmatrix}$$

  (16)

• Our goal is to find a lower triangular matrix, $\mathbf{L}$, such that $\boldsymbol{\Omega} = \mathbf{L} \mathbf{L}'$. The Cholesky factor is

  $$L = \begin{bmatrix} 2 & 0 & 0 \\ 6 & 1 & 0 \\ -8 & 5 & 3 \end{bmatrix}$$

  (17)

• Verifying $\boldsymbol{\Omega} = \mathbf{L} \mathbf{L}'$

  $$\mathbf{L} \mathbf{L}' = \begin{bmatrix} 2 & 0 & 0 \\ 6 & 1 & 0 \\ -8 & 5 & 3 \end{bmatrix}\begin{bmatrix} 2 & 6 & -8 \\ 0 & 1 & 5 \\ 0 & 0 & 3 \end{bmatrix} = \begin{bmatrix} 4 & 12 & -16 \\ 12 & 37 & -43 \\ -16 & -43 & 98 \end{bmatrix} = A$$

  (18)