The central limit theorem is a measure of central tendency.

The central limit theorem states that as more and more random samples of a given size are taken from a population, the distribution of the sample means can be approximated by a normal distribution. The more samples, the better the approximation.

Simulation techniques can be used to simulate many samples from a given distribution.

What happens if samples are taken from a population that is normally distributed?

Here is the result of sampling from a normal distribution.

The distribution of the sample means appears to be a normally distributed.

This is to be expected. Notice that the distribution of the sample means is narrower and taller. Since the height of all boxes (in this discrete approximation) is 1.0, the standard deviation of the sample means must be smaller than the standard deviation of the population.

Suppose that a real-world process is modeled by a uniform distribution in the range 30.0 to 60.0.

One example of a uniform distribution is a truly random number generation process within the range of interest, here 30.0 to 60.0. Suppose that 1000 samples are to be taken where each sample consists of 16 values selected at random from the distribution and averaged to get a sample mean. Of course, there are random errors inherent in each sample of 16 values, but these random errors can be minimized by taking more samples, which is why 1000 samples are taken. Then create a bar chart of the distribution of the means of the 1000 samples. What is the distribution of the sampling process?

Here is the result of sampling from a uniform distribution.

The sample means appear to have a bell-shaped distribution. This distribution is called the normal distribution, a measure of central tendency. The central limit theorem states that most sampling distributions can be approximated by a normal distribution, even if the population distribution (in this case, the uniform distribution) is not normally distributed. Thus, the central limit theorem has great importance since it means that the normal distribution has useful applications in practice.

Consider the following exponential distribution.

What happens if samples are taken from this exponential distribution?

Here is a chart of the results of 2000 samples of size 16 from an exponential distribution.

The distribution of the sample means appears to be a normally distributed.

Consider the following discrete distribution.

What happens if samples are taken from this discrete distribution?

Here is the chart of the results of 2000 samples of size 32 from the very non-normal discrete distribution.

The distribution of the sample means appears to be a normally distributed. The standard deviation of the population would be quite large, as the values are only at the extremes (low and high) of the possible values.

Thus, the standard deviation of the sample means appears to be less than the standard deviation of the population. Suppose that the middle values with 0.0 probability are omitted. An intuitive analogy at this point is to compare this distribution with the binomial distribution of a biased coin.

As can be seen, the central limit theorem is important in that it shows that normal distributions can be used to model the distribution of the sample means.