STATE 02 · THE GAUSSIAN

The Gaussian normal law

The bell-shaped law at the centre of mathematical statistics.

The density

The Gaussian law is described by two parameters, a centre μ and a scale σ. Its density is proportional to the exponential of the negative squared standardised deviation, and the constant of proportionality is chosen so that the integral over the whole line equals one. The resulting curve is symmetric about the centre, highest at that point, and falls away on either side in a manner that is neither too fast nor too slow. Its profile is the familiar bell.

The two parameters have plain meanings. The centre μ is the mean and also the median and the mode, since symmetry makes the three coincide. The scale σ is the standard deviation, and its square is the variance. Increasing the scale broadens the curve without changing the total area beneath it, while shifting the centre slides the whole profile along the axis.

Standardisation

Subtracting the mean and dividing by the standard deviation turns any Gaussian variable into the standard Gaussian, whose centre is zero and whose scale is one. This operation of standardisation is the single most useful step in the subject. It converts a whole family of curves into one reference curve, and it allows a single table or a single integral to serve every member of the family. A value expressed in these standard units is often called a z-score.

Standardisation also reveals that the shape of the law does not depend on the parameters at all. The proportions of the area lying within one, two or three standard deviations of the centre are fixed numbers, identical for every Gaussian variable regardless of its centre and scale.

Figure 1. The Gaussian density with its centre and one standard deviation marked.

Figure 2. A stem-and-leaf display beside a histogram of the same values, two views of one sample.

The empirical rule and the tails

For every Gaussian variable, roughly two thirds of the probability lies within one standard deviation of the centre, roughly ninety-five per cent within two, and roughly ninety-nine point seven per cent within three. This summary is often called the empirical rule. Beyond three standard deviations the density falls extremely quickly, and the probability of such extreme values is small but never zero. The Gaussian tails are thin in comparison with many other laws, yet they still extend without bound.

The thinness of the tails is the source of both the usefulness and the caution of the Gaussian family. Observations many standard deviations from the centre are rare under the law, so when they occur in practice a careful analyst pauses and asks whether the Gaussian assumption is the right one.

The central limit theorem

The central limit theorem explains why the Gaussian law is everywhere. Consider a sum of many independent contributions, each small in comparison with the total, and suppose each contribution has a finite variance. Then the standardised sum approaches the standard Gaussian law as the number of contributions grows. The theorem does not require that the contributions themselves be Gaussian; it requires only that none of them dominate the rest.

This explains the ubiquity of the bell curve in measurement, in averaging and in the study of aggregates. A quantity that arises as the combined effect of many small, weakly dependent influences tends to look Gaussian. The theorem is not a universal law of nature, however, and a single large contribution or a heavy-tailed ingredient can spoil the approximation entirely. The next page, bivariate density matrices, extends the Gaussian idea to two variables at once.