This question follows from my previous question, where Robin answered the question in the case of weak stationary processes. Here, I am asking a similar question for (strong?) stationary processes. I’ll define what this means (the definition can also be found here).

Let X(t) be a stochastic process. We say that X(t) is Nth-order stationary if, for every t1,t2,…,tN we have that the joint cumulative density functions

FX(t1),X(t2),…,X(tN)=FX(t1+τ),X(t2+τ),…,X(tN+τ)

for all τ.This is quite a strong condition, it says that the joint statistics don’t change at all as time shifts.

For example, a 1st order stationary process is such that FX(t1)=FX(t2) for all t1 and t2. That is, the X(t) are all identically distributed. It is quite easy to see that a 1st order stationary process need not be 2nd order stationary. Simply assign a correlation structure to say X(t), X(t+1), X(t+2) that

does notcorrespond to a (symmetric) Toeplitz matrix. That is, in vector form, the covariance matrix of [X(t),X(t+1),X(t+2)] could be given as[σ2abaσ2cbcσ2]

for a,b,c distinct. This is now not 2nd order stationary because E[X(t)X(t+1)]=a and, time shifting by 1 we have E[X(t+1)X(t+2)]=c≠a.

In a similar way (presumably), a process that is 1st and 2nd order stationary need not be 3rd order stationary and this leads to my question:

Does somebody have a nice example of a stochastic process that is both 1st and 2nd order stationary, but not 3rd order stationary?

**Answer**

Here is such an example. **I will describe the process in terms of its sample paths**.

It’s a simple process, it only has four sample paths: ω1, ω2, ω3, ω4. This means that the only possible outcomes are:

..., X(−1)=ωi(−1), X(0)=ωi(0), X(1)=ωi(1), ...

One outcome for every i∈{1,2,3,4}.

The sample paths are defined as follows, for t∈Z:

ω1(t):={1if t is a multiple of 4,0else.

ω2(t):={0if t is a multiple of 4,1else.

ω3(t):=ω1(t−2)

ω4(t):=ω2(t−2)

So they are:

...,1,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,...

...,0,1,1,1,0,1,1,1,0,1,1,1,0,1,1,1,...

...,0,0,1,0,0,0,1,0,0,0,1,0,0,0,1,0,...

...,1,1,0,1,1,1,0,1,1,1,0,1,1,1,0,1,...

Here is a plot of the four possible sample paths:

Each sample path is defined to have probability 1/4. This concludes the definition of the process.

**I will now show that it satisfies the conditions required by the OP**.

For t∈{1,2,3,4} one can see that there are exactly 2 sample paths such that X(t)=1, and exactly 2 sample paths such that X(t)=0. Hence for t∈{1,2,3,4}:

P[X(t)=1]=P[X(t)=0]=1/2.

Since the process is periodic this is true for every t∈Z. This means that the distribution of X(t) does not depend on t and therefore the process is **1st order stationary**.

Likewise, looking at consecutive pairs, for t∈{1,2,3,4} there is exactly 1 sample path with X(t)=1 and X(t+1)=0, exactly 1 sample path with X(t)=0 and X(t+1)=1, exactly 1 sample path with X(t)=0 and X(t+1)=0, and exactly 1 sample path with X(t)=1 and X(t+1)=1. So for t∈{1,2,3,4}:

P[X(t)=1,X(t+1)=0]=P[X(t)=0,X(t+1)=1]=P[X(t)=0,X(t+1)=0]=P[X(t)=1,X(t+1)=1]=1/4.

Again by periodicity, this is true for every t∈Z and the process is **2nd order stationary**.

However, for instance: P[X(0)=0,X(1)=0,X(2)=0]=0, while P[X(1)=0,X(2)=0,X(3)=0]=1/4.

Therefore the process is not **3rd order stationary**

**Attribution***Source : Link , Question Author : Robby McKilliam , Answer Author : Winkelried*