STATE 03 · PROCESSES

Stationarity and spectra

Two languages for the same dependence: lag and frequency.

Weak and strict stationarity

A series is weakly stationary when its mean is constant, its variance is finite and constant, and its autocovariance at a given separation depends only on that separation and not on the absolute time. The weaker word reflects that only the first two moments are constrained. A series is strictly stationary when the whole joint law of any finite block of values is unchanged when the block is shifted in time; the finite-dimensional distributions depend only on the shape of the block.

Strict stationarity does not imply weak stationarity unless the second moments exist, and weak stationarity does not imply strict stationarity. The two notions coincide for Gaussian series, since a Gaussian law is fixed by its mean and its covariance structure alone. Gaussian stationarity is therefore especially clean, and it is the setting most often assumed in elementary treatments.

Autocovariance and autocorrelation

The autocovariance function returns the covariance between values separated by a given lag. At lag zero it is the variance; as the lag grows the autocovariance usually declines, though it may oscillate or change sign. The autocorrelation function divides the autocovariance by the variance so that the result begins at one and lies between minus one and one throughout.

The autocorrelation function is the signature of a series. A slow, smooth decay signals strong persistence; a sharp cut-off after a few lags signals short memory; a pronounced oscillation signals a periodic character. The reader learns to recognise a family from the shape of its signature.

The spectral density

The spectral density describes the same dependence but splits it by frequency rather than by lag. It is the Fourier transform of the autocovariance function, and it reports how much of the variance of the series is attributable to oscillations in a neighbourhood of each frequency. A peak in the spectral density marks a frequency band in which the series carries substantial energy; a flat spectral density marks the featureless case of white noise.

The spectral density is non-negative and periodic, with period two pi, and it integrates over one period to the variance of the series. This integral identity is the spectral restatement of the fact that the total variance is the sum of its parts distributed across frequency. A reader who grasps the identity holds the central accounting rule of spectral analysis.

The periodogram idea

The periodogram is an empirical counterpart to the spectral density, formed by squaring the magnitude of the Fourier transform of a finite record. Its expectation approximates the spectral density, but its variance does not shrink as the record lengthens, so raw periodograms are erratic. Smoothing over neighbouring frequencies restores stability, at the expense of some resolution. The periodogram is therefore best read as an estimate of shape rather than of exact value.

The dynamic models of the next movement, beginning with Markov chains, replace the continuous index of spectra with discrete states.