Autocovariance of an ARMA(2,1) process – derivation of analytical model for $\gamma( k)$

I need to derive analytic expressions for the autocovariance function $\gamma\left(k\right)$ of an ARMA(2,1) process denoted by:


So, I know that:

$\gamma\left(k\right) = \mathrm{E}\left[y_t,y_{t-k}\right]$

so I can write:

$\gamma\left(k\right) = \phi_1 \mathrm{E}\left[y_{t-1}y_{t-k}\right]+\phi_2 \mathrm{E}\left[y_{t-2}y_{t-k}\right]+\theta_1 \mathrm{E}\left[\epsilon_{t-1}y_{t-k}\right]+\mathrm{E}\left[\epsilon_{t}y_{t-k}\right]$

then, to derive the analytic version of the autocovariance function, I need to substitute values of $k$ – 0, 1, 2 … until I get a recursion that is valid for all $k$ greater than some integer.

Therefore, I substitute $k=0$ and work this through to get:

\gamma \left(0\right) = \mathrm{E}\left[y_t,y_t\right]
= \phi_1 \mathrm{E}\left[y_{t-1}y_t\right] + \phi_2 \mathrm{E}\left[y_{t-2}y_t\right]+\theta_1 \mathrm{E}\left[\epsilon_{t-1}y_t\right]+\mathrm{E}\left[\epsilon_ty_t\right]\\

now I can simplify the first two of these terms, and then substitute for $y_t$ as before:

\gamma\left(0\right) = \phi_1 \gamma\left(1\right) + \phi_2 \gamma\left(2\right)\\
+ \theta_1 \mathrm{E}\left[\epsilon_{t-1} \left(\phi_1 y_{t-1} +\phi_2 y_{t-2} +\theta_1 \epsilon_{t-1} + \epsilon_t \right)\right]\\
+ \mathrm{E}\left[\epsilon_t \left(\phi_1 y_{t-1} +\phi_2 y_{t-2} +\theta_1 \epsilon_{t-1} + \epsilon_t \right)\right]

then I multiply out the eight terms, which are:

+\theta_1\mathrm{E}\left[\epsilon_t\epsilon_{t-1}\right] = \theta_1\mathrm{E}\left[\epsilon_{t}\right]\mathrm{E}\left[\epsilon_{t-1}\right]=0 \\
+\mathrm{E}\left[\left(\epsilon_t\right)^2\right] = \sigma_{\epsilon}^2

So, I am left needing to resolve the four remaining terms. I want to use the same logic for lines 1, 2, 5 and 6 as I used on lines 4 and 7 – for example for line 1:

$\theta_1\phi_1\mathrm{E}\left[\epsilon_{t-1}y_{t-1}\right] = \theta_1\phi_1\mathrm{E}\left[\epsilon_{t-1}\right]\mathrm{E}\left[y_{t-1}\right] = 0$ because $\mathrm{E}\left[\epsilon_{t-1}\right]=0$.

Similarly for lines 2, 5 and 6. But I have a model solution that suggests the expression for $\gamma\left(0\right)$ simplifies to:

$\gamma\left(0\right) = \phi_1\gamma\left(1\right)+\phi_2\gamma\left(2\right) +\theta_1\left(\phi_1+\theta_1\right)\sigma_{\epsilon}^2+\sigma_{\epsilon}^2$

This suggests my simplification as described above would miss the term with the coefficient $\phi_1$ – which under my logic should be 0. Is my logic at fault, or is the model solution I found incorrect?

The worked solution also suggest that “analogously” $\gamma\left(1\right)$ can be found as:

$\gamma\left(1\right) = \phi_1\gamma\left(0\right)+\phi_2\gamma\left(1\right) + \theta_1\sigma_{\epsilon}^2$

and for $k>1$:

$\gamma\left(k\right) = \phi_1\gamma\left(k-1\right)+\phi_2\left(k-2\right)$

I hope the question is clear. Any assistance will be much appreciated. Thank you in advance.

This is a question related to my research, and is not in preparation for any exam or coursework.


If the ARMA process is causal there is a general formula that provides the autocovariance coefficients.

Consider the causal $\text{ARMA}(p,q)$ process
y_t = \sum_{i = 1}^p \phi_i y_{t-1} + \sum_{j = 1}^q \theta_j \epsilon_{t – j} + \epsilon_t,
where $\epsilon_t$ is a white noise with mean zero and variance $\sigma_\epsilon^2$.
By the causality property, the process can be written as
y_t = \sum_{j = 0}^\infty \psi_j \epsilon_{t – j},
where $\psi_j$ denotes the $\psi$-weights.

The general homogeneous equation for the autocovariance coefficients of a causal $\text{ARMA}(p,q)$ process is
\gamma (k) – \phi_1 \gamma (k-1) – \cdots – \phi_p \gamma (k-p) = 0,
\quad k \geq \max (p, q+1),
with initial conditions
\gamma (k) – \sum_{j = 1}^p \phi_j \gamma (k-j)
= \sigma_\epsilon^2 \sum_{j = k}^q \theta_j \psi_{j – k},
\quad 0 \leq k < \max (p, q+1).

Source : Link , Question Author : hydrologist , Answer Author : QuantIbex

Leave a Comment