STATE 01 · FOUNDATIONS

Continuous densities and transformations

Weight, area and the change of variable.

Density and distribution functions

For a continuous variable the probability of any single point is zero, so the density, not the point value, carries the information. A density is a non-negative function whose integral over the whole line equals one, and the probability of an interval is the integral of the density over that interval. The cumulative distribution function is the integral of the density from the far left up to a chosen point, and it rises from zero to one as the point advances. Where the density is continuous, it is the derivative of the distribution function, so the two accounts carry the same content in different form.

The distribution function is the more fundamental object, because it exists whether or not a density does, and it completely identifies the law. It is non-decreasing, right-continuous, and bounded between zero and one. The density may fail to exist at isolated points without affecting the distribution function at all.

Transformations of a variable

Suppose a new variable is formed by applying a smooth, strictly increasing function to an old one. The distribution function of the new variable is then read off from that of the old by substituting the inverse function. When the map is strictly decreasing the reasoning reverses the order of the arguments. Either way, the transformation of the distribution function needs no extra machinery.

For densities the story is richer, because a change of variable stretches or compresses the horizontal axis and the density must be rescaled to keep its integral equal to one. The adjustment factor is the absolute value of the derivative of the inverse map, sometimes called the Jacobian of the transformation. In several dimensions the Jacobian becomes the absolute value of a scalar factor, and the rescaling keeps the total mass fixed under the change of coordinates.

Common continuous families

The uniform law spreads its weight evenly over an interval. The exponential law describes the waiting time between events of a steady stream and has a single scale parameter. The gamma family extends the exponential and serves as the law of a sum of independent exponential variables. The beta family lives on the unit interval and is flexible enough to model a wide range of shapes. The Gaussian family, treated in its own pages, occupies the centre of the subject.

Each family is described by one or more parameters, and each has its own density, distribution function, mean and variance. Learning to move between these forms is the practical skill of the subject. A reader who can pass from density to distribution function and back, and who can rescale a density under a change of variable, holds the essential craft. The next movement of the reference introduces the Gaussian law itself.