STATISTICAL INFERENCE

RESULTS FROM PROBABILITY THEORY

Since sampling theory (and hence statistical inference) as used in this course relies on the concept of repeated samples, we consider three results from probability theory:

Recall Kerrich's coin tossing experiment-

In 10,000 tosses of a coin you'd expect the number of heads (#heads) to approximately equal the number of tails

so #heads ½ #tosses

Fig. 1 shows that (#heads - ½ #tosses) can become large in absolute terms as the number of tosses increases

Fig. 2 shows that in relative terms

as #tosses increases

You can think of this as

where chance error becomes large in absolute terms but small as % of #tosses as #tosses increases.

Figure 1. Kerrich's coin tossing experiment. A plot of the 'chance error' number of heads - half the number of tosses against the number of tosses. As the number of tosses goes up, the size of the chance error tends to go up. The horizontal axis is not drawn to scale.

Figure 2. The chance error expressed as a percentage of the number of tosses. As the number of tosses goes up, this percentage gets smaller. In other words, the chance error gets smaller relative to the number of tosses. The horizontal axis is not drawn to scale.

Law of Averages

The law of averages does not work by compensation. A run of heads is just as likely to be followed by a head as by a tail because the outcomes of successive tosses are independent events.

We can prove this result using results from the binomial distribution.

Let RV X be the number of heads in n tosses.

Expected value for the proportion of heads = 0.5 with variance (0.5)²/n, which goes to zero as n ->

Law of Large Numbers

If are independent random variables all with the same probability distribution with expected value µ and variance s ² then

is very likely to become very close to µ as n becomes very large.

Coin tossing is a simple example.

Thus the law of averages is an informal version of the law of large numbers

Law of large numbers says that

and var () becomes very small

In fact, using the result var(X+Y) = var(X) + var(Y) if X and Y are independent, then

In particular, if the repeated independent measurements are all Normally distributed, that is

then their mean also has the Normal distribution

The Central Limit Theorem says that is approximately Normally distributed even if the original measurements were not Normally distributed.

Central Limit Theorem

regardless of the shape of the probability distributions of X1, X2, ... .