<p>Let $\epsilon_1, \epsilon_2, ... ,\epsilon_t,...$ denote i.i.d. stochastic variables, each having a standard deviation of $\sigma$.</p>

<p>Let
$$
x_t = \sum_{i = 1}^t \epsilon_i.
$$
Then
$$
cov(x_t,x_{t+h}) =
cov\left(
\sum_{i = 1}^t \epsilon_i,
\sum_{i = 1}^{t+h} \epsilon_i
\right).
$$
Because of the i.i.d. property
$$
cov\left(
\sum_{i = 1}^t \epsilon_i,
\sum_{i = 1}^{t+h} \epsilon_i
\right)
=
\sum_{i = 1}^t cov\left(\epsilon_i,\epsilon_i\right) +
\sum_{i = 1}^t\sum_{j \neq i} cov\left(\epsilon_i,\epsilon_j\right)
$$
and furthermore because of independence
$$
\sum_{i = 1}^t cov\left(\epsilon_i,\epsilon_i\right) +
\sum_{i = 1}^t\sum_{j \neq i} cov\left(\epsilon_i,\epsilon_j\right)
=
\sum_{i = 1}^t \sigma +
\sum_{i = 1}^t\sum_{j \neq i} 0
=
t \cdot \sigma.
$$
Now we have shown that for this time series
$$
cov(x_t,x_{t+h}) =
t \cdot \sigma
$$
depends on $t$ hence it is not covariance stationary.</p>

<p>For a covariance stationary time series, let
$$
z_t = \epsilon_{t-1} + \epsilon_{t}.
$$
By performing the same analyis, you can conlude that for $t &gt; 1$,
$$
cov(z_t,z_{t+h}) =
\left\{
\begin{array}{ll}
\sigma &amp; \text{ if } h = 1 \\
0 &amp; \text{ if } h &gt; 1.
\end{array}
\right.
$$
Since this depends on $h$, but not on $t$, the time series $z_t$ will be covariance stationary (after $t = 1$).</p>

CC BY-SA 4.0
2018-09-19T16:31:49.577Z
Would you be able to maybe give an example or two of how this might come up in applications, rather than more of a math explanation?
M. Damon 
CC BY-SA 4.0
2018-09-19T17:56:41.430Z
Hi: Any stationary ARIMA model is covariance stationary. Take an ARIMA(2,0,0), calculate the covariance and you'll see that it's only a function of h. Same thing for any of them, as long as they are stationary..
mark leeds
CC BY-SA 4.0
2018-09-19T19:01:42.487Z
@markleeds Yes, that's why I chose that as a simple example. If you want to provide an explanation for a general case, write it up as an answer, I'll vote for it!
Giskard
CC BY-SA 4.0
2018-09-20T00:12:49.790Z
@denesp: My apologies. I didn't read your answer carefully.  I imagine that there  must be a general proof but I'm not able to come up with one, atleast  in some reasonable amount of time. ( and maybe not even in an unreasonable amount of time ).  I'll google and  and see if anything is out there.
mark leeds
CC BY-SA 4.0
2018-09-20T00:27:25.317Z
@denesp: I think 4.5, 4.6 and 4.7 of link below is sort of a proof because, since any stationary arima model can be written in form of  a wold decomposition and wold says that any covariance stationary process can be written that way, then, any  stationary arima model is covariance stationary. ( but check me on that. I may be hand waving. statement is true for sure  ).  http://sfb649.wiwi.hu-berlin.de/fedc_homepage/xplore/tutorials/xegbohtmlnode36.html
mark leeds
CC BY-SA 4.0
2018-09-20T01:18:13.003Z
@denesp: I don't want to write it an an answer since the question doesn't ask for one but I think the proof goes like this ( to fix my hand waving in the earlier comment ):  Only covariance stationary processes can be written in the form of a Wold decomposition. All stationary ARIMA models can be written in the form of a Wold decomposition. Therefore, all  stationary ARIMA models are covariance stationary.
mark leeds
CC BY-SA 4.0
2018-09-20T05:06:48.447Z
@denesp: I decided to write an answer pretty unrelated to what we were talking about because my guess is that it is more useful to the person who asked.
mark leeds
CC BY-SA 4.0
2018-09-20T05:29:55.260Z
@markleeds I am not sure. You are basically explaining what stationary means, randomness or covariance does not seem to appear in your story. Perhaps you are right and this is more line with the actual demand.
Giskard
CC BY-SA 4.0
2018-09-20T07:43:28.787Z
@denesp: In my explanation, I'm not explaining what stationary means because you need to specify stationary with respect to what, the mean, the variance, the covariance. In the example, I used covariance structure which is the same as covariance.
mark leeds
CC BY-SA 4.0
2018-09-19T16:06:54.393Z
2018-09-19T16:06:54.393Z
Giskard
<p>@ M Damon: If you're just starting with time series ( actually, even if you're not ) ,  think it's best to forget about Wold Decomp and ARIMA models and all that jibber jabber. I re-read your question and I think the best way to think of covariance stationary is the following.</p>

<p>You're sitting in the audience and there's a stage that you look at. The stage is  such that the beginning of the curtain to the end of the curtain is of length h. You also have the ability to see the h numbers
( picture each number taking up a unit length of space )  that come out ( some stochastic process causes them to arrive on stage ) in the h second interval
so that they perfectly fit on stage from the left side of the stage all the
way to the right.</p>

<p>So, at time <span class="math-container">$t = 1, X_1$</span> arrives on the stage. Then, the <span class="math-container">$X_2$</span> arrives on stage at <span class="math-container">$t = 2$</span>, to the right of <span class="math-container">$X_1$</span> ( from your vantage point ). So, after <span class="math-container">$h$</span> seconds, you have <span class="math-container">$X_1, \ldots X_h$</span> on stage and ordered from left to right. </p>

<p>You are sitting there at time <span class="math-container">$t=h$</span> ( picture time stopping for a moment ) and since the observations are from some stochastic process,  there is a covariance structure between those h elements on stage.  Whatever it is, doesn't matter. Maybe it's <span class="math-container">$\theta^i$</span> where <span class="math-container">$i$</span> is the lag between two <span class="math-container">$X_t$</span> say <span class="math-container">$X_t$</span> and <span class="math-container">$X_{t-i}$</span>.</p>

<p>Next, time starts again and you close your eyes for another <span class="math-container">$h$</span> seconds so that a new set of <span class="math-container">$h$</span> elements arrive on stage and the old ones leave. Now you're at time <span class="math-container">$2h$</span> and you open your eyes. So, it's time <span class="math-container">$t = 2h$</span>, but the covariance structure between the new <span class="math-container">$h$</span> elements on stage does not change from what it was earlier. So, the specific elements themselves don't matter when it comes to the covariance of any two elements. All one needs to know is the distance between any two elements in order to know what the covariance ( or correlation ) of them is. 2h plays no role. All that matters for knowing the covariance of any two elements on stage is their distance. This is the practical meaning of the term "covariance stationary".</p>

<p>Note that stage could have been bigger or smaller. <span class="math-container">$h$</span> was just random number picked.</p>

<p>This assumption is made and is useful because ( informally speaking ) it kind of allows one to estimate model parameters in an unbiased way with only one  set of time series observations. If this assumption was not made, then it would be kind of like having a sample size of <span class="math-container">$n=1$</span> because the stochastic realization is one element. I hope that helps.</p>

CC BY-SA 4.0
2024-03-05T18:09:06.687Z
Thanks for this nice explanation! I'm new to Time Series and want to clarify - isn't covariance always between two series or two datasets? Here you mention "covariance of any 2 elements" - how would you get covariance of just two numbers? What am I missing? Do you mean the corresponding Random Variables?
user2450223
CC BY-SA 4.0
2024-03-06T19:10:39.383Z
@user2450223. Your welcome. I mis-spoke. I should have said covariance at some lagged value.  So, $cov(x_{t},x_{t-h})$ would mean the covariance at lag h. So, really what I meant is the covariance at lag $h$ for any integer value of $h$. My mistake. If you need a good text and tell me your background, I can possibly recommend one.
mark leeds
CC BY-SA 4.0
2024-03-06T19:41:48.930Z
One more thing: Note that you have more than one observation at say lag $i$ because during the first stage, you have $h$ elements. So,  you have $(x_{t}, x_{t-i})$ and $(x_{t+1}, x_{t-i+1})$  and $(x_{t+2}, x_{t-i+2})$ etc. So, for any given set of $h$ observations on the stage, you have many observations at say lag $i$.
mark leeds
CC BY-SA 4.0
2018-09-20T05:00:35.197Z
2018-09-21T05:26:57.733Z
2018-09-21T05:26:57.733Z
mark leeds
<p>I seem to be having trouble in comprehending what it means for a time series to be covariance stationary. Specifically, with the third condition that for any $t,h$ the $cov(x_t,x_{t+h})$ only depends on $h$ and not $t$. </p>

<p>Would anyone have any examples of how a time series might be covariance stationary or any examples of non-covariance stationary time series?</p>

<econometrics><time-series>
Examples of Covariance Stationary Time Series
M. Damon 
# Examples of Covariance Stationary Time Series

(Asked by: username_0 on 2018-09-19)

I seem to be having trouble in comprehending what it means for a time series to be covariance stationary. Specifically, with the third condition that for any $t,h$ the $cov(x\_t,x\_{t+h})$ only depends on $h$ and not $t$.

Would anyone have any examples of how a time series might be covariance stationary or any examples of non-covariance stationary time series?

**Tags:** econometrics, time-series


---

## 2 answers

### Answer by username_1 on 2018-09-19. Score: 1

Let $\epsilon\_1, \epsilon\_2, ... ,\epsilon\_t,...$ denote i.i.d. stochastic variables, each having a standard deviation of $\sigma$.

Let
$$
x\_t = \sum\_{i = 1}^t \epsilon\_i.
$$
Then
$$
cov(x\_t,x\_{t+h}) =
cov\left(
\sum\_{i = 1}^t \epsilon\_i,
\sum\_{i = 1}^{t+h} \epsilon\_i
\right).
$$
Because of the i.i.d. property
$$
cov\left(
\sum\_{i = 1}^t \epsilon\_i,
\sum\_{i = 1}^{t+h} \epsilon\_i
\right)
=
\sum\_{i = 1}^t cov\left(\epsilon\_i,\epsilon\_i\right) +
\sum\_{i = 1}^t\sum\_{j \neq i} cov\left(\epsilon\_i,\epsilon\_j\right)
$$
and furthermore because of independence
$$
\sum\_{i = 1}^t cov\left(\epsilon\_i,\epsilon\_i\right) +
\sum\_{i = 1}^t\sum\_{j \neq i} cov\left(\epsilon\_i,\epsilon\_j\right)
=
\sum\_{i = 1}^t \sigma +
\sum\_{i = 1}^t\sum\_{j \neq i} 0
=
t \cdot \sigma.
$$
Now we have shown that for this time series
$$
cov(x\_t,x\_{t+h}) =
t \cdot \sigma
$$
depends on $t$ hence it is not covariance stationary.

For a covariance stationary time series, let
$$
z\_t = \epsilon\_{t-1} + \epsilon\_{t}.
$$
By performing the same analyis, you can conlude that for $t > 1$,
$$
cov(z\_t,z\_{t+h}) =
\left\{
\begin{array}{ll}
\sigma & \text{ if } h = 1 \\
0 & \text{ if } h > 1.
\end{array}
\right.
$$
Since this depends on $h$, but not on $t$, the time series $z\_t$ will be covariance stationary (after $t = 1$).

> **Comments:**
> **username_0**: Would you be able to maybe give an example or two of how this might come up in applications, rather than more of a math explanation?  
> **username_2**: Hi: Any stationary ARIMA model is covariance stationary. Take an ARIMA(2,0,0), calculate the covariance and you'll see that it's only a function of h. Same thing for any of them, as long as they are stationary..  
> **username_1**: @markleeds Yes, that's why I chose that as a simple example. If you want to provide an explanation for a general case, write it up as an answer, I'll vote for it!  
> **username_2**: @denesp: My apologies. I didn't read your answer carefully.  I imagine that there  must be a general proof but I'm not able to come up with one, atleast  in some reasonable amount of time. ( and maybe not even in an unreasonable amount of time ).  I'll google and  and see if anything is out there.  
> **username_2**: @denesp: I think 4.5, 4.6 and 4.7 of link below is sort of a proof because, since any stationary arima model can be written in form of  a wold decomposition and wold says that any covariance stationary process can be written that way, then, any  stationary arima model is covariance stationary. ( but check me on that. I may be hand waving. statement is true for sure  ).  http://sfb649.wiwi.hu-berlin.de/fedc_homepage/xplore/tutorials/xegbohtmlnode36.html  
> **username_2**: @denesp: I don't want to write it an an answer since the question doesn't ask for one but I think the proof goes like this ( to fix my hand waving in the earlier comment ):  Only covariance stationary processes can be written in the form of a Wold decomposition. All stationary ARIMA models can be written in the form of a Wold decomposition. Therefore, all  stationary ARIMA models are covariance stationary.  
> **username_2**: @denesp: I decided to write an answer pretty unrelated to what we were talking about because my guess is that it is more useful to the person who asked.  
> **username_1**: @markleeds I am not sure. You are basically explaining what stationary means, randomness or covariance does not seem to appear in your story. Perhaps you are right and this is more line with the actual demand.  
> **username_2**: @denesp: In my explanation, I'm not explaining what stationary means because you need to specify stationary with respect to what, the mean, the variance, the covariance. In the example, I used covariance structure which is the same as covariance.  

---

### Answer by username_2 on 2018-09-20. Score: 0

@ M Damon: If you're just starting with time series ( actually, even if you're not ) , think it's best to forget about Wold Decomp and ARIMA models and all that jibber jabber. I re-read your question and I think the best way to think of covariance stationary is the following.

You're sitting in the audience and there's a stage that you look at. The stage is such that the beginning of the curtain to the end of the curtain is of length h. You also have the ability to see the h numbers
( picture each number taking up a unit length of space ) that come out ( some stochastic process causes them to arrive on stage ) in the h second interval
so that they perfectly fit on stage from the left side of the stage all the
way to the right.

So, at time $t = 1, X\_1$ arrives on the stage. Then, the $X\_2$ arrives on stage at $t = 2$, to the right of $X\_1$ ( from your vantage point ). So, after $h$ seconds, you have $X\_1, \ldots X\_h$ on stage and ordered from left to right.

You are sitting there at time $t=h$ ( picture time stopping for a moment ) and since the observations are from some stochastic process, there is a covariance structure between those h elements on stage. Whatever it is, doesn't matter. Maybe it's $\theta^i$ where $i$ is the lag between two $X\_t$ say $X\_t$ and $X\_{t-i}$.

Next, time starts again and you close your eyes for another $h$ seconds so that a new set of $h$ elements arrive on stage and the old ones leave. Now you're at time $2h$ and you open your eyes. So, it's time $t = 2h$, but the covariance structure between the new $h$ elements on stage does not change from what it was earlier. So, the specific elements themselves don't matter when it comes to the covariance of any two elements. All one needs to know is the distance between any two elements in order to know what the covariance ( or correlation ) of them is. 2h plays no role. All that matters for knowing the covariance of any two elements on stage is their distance. This is the practical meaning of the term "covariance stationary".

Note that stage could have been bigger or smaller. $h$ was just random number picked.

This assumption is made and is useful because ( informally speaking ) it kind of allows one to estimate model parameters in an unbiased way with only one set of time series observations. If this assumption was not made, then it would be kind of like having a sample size of $n=1$ because the stochastic realization is one element. I hope that helps.

> **Comments:**
> **username_3**: Thanks for this nice explanation! I'm new to Time Series and want to clarify - isn't covariance always between two series or two datasets? Here you mention "covariance of any 2 elements" - how would you get covariance of just two numbers? What am I missing? Do you mean the corresponding Random Variables?  
> **username_2**: @username_3. Your welcome. I mis-spoke. I should have said covariance at some lagged value.  So, $cov(x_{t},x_{t-h})$ would mean the covariance at lag h. So, really what I meant is the covariance at lag $h$ for any integer value of $h$. My mistake. If you need a good text and tell me your background, I can possibly recommend one.  
> **username_2**: One more thing: Note that you have more than one observation at say lag $i$ because during the first stage, you have $h$ elements. So,  you have $(x_{t}, x_{t-i})$ and $(x_{t+1}, x_{t-i+1})$  and $(x_{t+2}, x_{t-i+2})$ etc. So, for any given set of $h$ observations on the stage, you have many observations at say lag $i$.  

---

