STATE 03 · PROCESSES

Stochastic processes

A family of variables indexed by time, and the dependence that binds them.

From one variable to a family

A stochastic process is a collection of random variables indexed by a parameter, usually taken to represent time. At each fixed instant the process is an ordinary random variable; considered all at once, the collection is a single object whose law lives on a space of whole trajectories. This shift of viewpoint, from a number to a curve, is the defining move of the subject.

The index set may be discrete, as with a daily observation, or continuous, as with a record taken without gaps. A discrete-index process is often called a series, and a continuous-index process is called a process proper. The two have most of their conceptual furniture in common, though the technical details differ when the index becomes continuous.

White noise

The simplest process is white noise, a family of variables that are uncorrelated, each with the same mean and the same variance. The name suggests a featureless record, and that is exactly the picture: no dependence from one instant to the next. White noise is the building block from which almost every other process is assembled, and it plays the role that the standard Gaussian plays among single variables.

White noise is uncorrelated but need not be independent. Independence is the stronger demand that the joint law of any finite collection factorise. A process of independent variables with a common distribution is white noise, but not every white noise is independent. The distinction matters whenever the variance of a sum is evaluated, since uncorrelatedness alone already suffices for many variance statements.

The random walk

Let the successive values of a random walk be formed by adding one white-noise increment at each step. The value at any time is then a sum of independent increments, and its variance grows in proportion to the number of steps. The walk wanders, and its range of typical values widens without bound as time advances.

The random walk is a first example of a process that is not stationary. Its mean may remain fixed at the starting point, but its variance changes with the index, so the probabilistic character of the family shifts as time goes on. A differenced random walk, by contrast, is exactly white noise, which is why first differences recur throughout the study of series.

Moving-average and autoregressive ideas

Two broad techniques build dependent series from white noise. The moving-average technique forms each new value from a finite, weighted combination of the recent disturbances, so that the memory of a disturbance fades after a fixed number of steps. The autoregressive technique forms each new value from a weighted combination of the previous values, plus a fresh disturbance, so that the memory persists and decays gradually.

Both techniques generate stationary series when their weights are suitably restrained. Both admit a simple dependence structure, and both can be combined into richer models. The pages that follow, beginning with time-series models, study these constructions in detail.