I was wrestling with stationarity in my head for a while… Is this how you think about it? Any comments or further thoughts will be appreciated.
Stationary process is the one which
generates timeseries values such that
distribution mean and variance is kept
constant. Strictly speaking, this is
known as weak form of stationarity or
covariance/mean stationarity.Weak form of stationarity is when the
timeseries has constant mean and
variance throughout the time.Let’s put it simple, practitioners say
that the stationary timeseries is the
one with no trend – fluctuates around
the constant mean and has constant
variance.Covariance between different lags is
constant, it doesn’t depend on
absolute location in timeseries. For
example, the covariance between t and
t1 (first order lag) should always be
the same (for the period from
19601970 same as for the period from
19651975 or any other period).In nonstationary processes there is
no longrun mean to which the series
reverts; so we say that nonstationary
time series do not mean revert. In
that case, the variance depends on
absolute position in timeseries and
variance goes to infinity as time goes
on. Technically speaking,
autocorrelations to not decay with
time, but in small samples they do
disappear – although slowly.In stationary processes, shocks are
temporary and dissipate (lose energy)
over time. After a while, they do not
contribute to the new timeseries
values. For example, something which
happened log time ago (long enough)
such as World War II, had an impact,
but, it the timeseries today is the
same as if World War II never
happened, we would say that shock lost
its energy or dissipated. Stationarity
is especially important as many
classical econometric theories are
derived under the assumptions of
stationarity.A strong form of stationarity is when
the distribution of a timeseries is
exactly the same trough time. In other
words, the distribution of original
timeseries is exactly same as lagged
timeseries (by any number of lags) or
even subsegments of the timeseries.
For example, strong form also suggests
that the distribution should be the
same even for a subsegments
19501960, 19601970 or even
overlapping periods such as 19501960
and 19501980. This form of
stationarity is called strong because
it doesn’t assume any distribution. It
only says the probability distribution
should be the same. In the case of
weak stationarity, we defined
distribution by its mean and variance.
We could do this simplification
because implicitly we assumed normal
distribution, and normal distribution
is fully defined by its mean and
variance or standard deviation. This
is nothing but saying that probability
measure of the sequence (within
timeseries) is the same as that for
lagged/shifted sequence of values
within same timeseries.
Answer
First of all, it is important to note that stationarity is a property of a process, not of a time series. You consider the ensemble of all time series generated by a process. If the statistical properties¹ of this ensemble (mean, variance, …) are constant over time, the process is called stationary. Strictly speaking, it is impossible to say whether a given time series was generated by a stationary process (however, with some assumptions, we can take a good guess).
More intuitively, stationarity means that there are no distinguished points in time for your process (influencing the statistical properties of your observation). Whether this applies to a given process depends crucially on what you consider as fixed or variable for your process, i.e., what is contained in your ensemble.
A typical cause of nonstationarity are timedependent parameters – which allow to distinguish time points by the values of the parameters. Another cause are fixed initial conditions.
Consider the following examples:

The noise reaching my house from a single car passing at a given time is not a stationary process. E.g., the average amplitude² is highest when the car is directly next to my house.

The noise reaching my house from street traffic in general is a stationary process, if we ignore the time dependency of the traffic intensity (e.g., less traffic at night or on weekends). There are no distinguished points in time anymore. While there may be strong fluctuations of individual time series, these vanish when I consider the ensemble of all realisations of the process.

If I we include known impacts on traffic intensity, e.g., that there is less traffic at night, the process is nonstationary again: The average amplitude² varies with a daily rhythm. Every point in time is distinguished by the time of the day.

The position of a single peppercorn in a pot of boiling water is a stationary process (ignoring the loss of water due to evaporation). There are no distinguished points in time.

The position of a single peppercorn in a pot of boiling water dropped in the exact middle at t=0 is not a stationary process, as t=0 is a distinguished point in time. The average position of the peppercorn is always in the middle (assuming a symmetric pot without distinguished directions), but at t=ε (with ε small), we can be sure that the peppercorn is somewhere near the middle for every realisation of the process, while at a later time, it can also be closer to the border of the pot.
So, the distribution of positions changes over time. To give a specific example, the standard deviation grows. The distribution quickly converges to the respective distributions of the previous example and if we only take a look at this process for t>T with a sufficiently high T, we can neglect the nonstationarity and approximate it as a stationary process for all purposes – the impact of the initial condition has faded away.
^{ ¹ For practical purposes, this is sometimes reduced to the mean and the variance (weak stationarity), but I do not consider this helpful to understand the concept. Just ignore weak stationarity until you understood stationarity. ² Which is the mean of the volume, but the standard deviation of the actual sound signal (do not worry too much about this here). }
Attribution
Source : Link , Question Author : user333 , Answer Author : Wrzlprmft