The law of large numbers and the gambler's fallacy

Red has come up seven times in a row at roulette: now black «must» come up. A number has not been drawn in the lottery for months: it is «due». Reasoning like this is extremely common and leans, wrongly, on a true mathematical result: the law of large numbers. Understanding exactly what that law says is the best way to dismantle the error.

What the law of large numbers states

Consider independent and identical trials, such as tosses of a fair coin. The law of large numbers states that, as the number of trials grows, the relative frequency of an event approaches its probability. More generally, the average of many independent observations with the same distribution approaches the expected value.

After 10 tosses the proportion of heads can easily be 0.3 or 0.7. After 10,000 tosses it will almost certainly be very close to 0.5. This is what makes surveys, insurance, Monte Carlo simulations and statistical process control possible.

What the law does not state

The crucial point is that convergence concerns the proportion, not the absolute number of successes. The difference between heads and tails, in absolute terms, actually tends to grow with the number of tosses, even though it grows more slowly than the number of tosses itself.

The typical deviations shown here are standard deviations. The relative deviation shrinks, but the absolute one grows. There is no mechanism that evens out the counts: an early lead of heads is not «given back», it is simply diluted by an enormous number of subsequent tosses.

The gambler's fallacy

The gambler's fallacy is the belief that, after a series of outcomes of the same kind, the opposite outcome becomes more likely. With independent trials this is false: the coin, the roulette ball or the lottery urn have no memory. After seven consecutive reds, the probability of the next red is exactly the same as always.

The confusion comes from mixing two ideas:

• it is true that a run of eight consecutive reds is rare, seen in advance;
• but once seven reds have already come up, the probability that they become eight is just the probability of a single red.

The past has already happened and does not enter the calculation of the next independent outcome.

The opposite error

There is also the reverse error, the belief that a lucky streak tends to continue: someone who has won three hands in a row is «on a roll». With independent events it is as wrong as the gambler's fallacy. In reality, however, not all events are independent: a tired athlete or a degraded system can genuinely show streaks. Independence must be assessed case by case, neither assumed nor ruled out.

When the fallacy becomes a technical problem

The same error appears outside casinos:

• «The backup has not failed for a year, so it will fail soon»: if failures are independent, a past without errors does not increase future risk, although media wear is a real factor to be considered separately.
• «We had three incidents this month, the next one will be quiet»: nothing automatically compensates for an unlucky run.
• «This test has failed five times in a row, the next run will pass»: if the failure has a cause, repeating without changing anything does not remove it.

Small samples are more extreme

An important consequence of the law of large numbers is that small samples produce more variable results. Tversky and Kahneman made a problem of this kind famous: in which hospital, a large one or a small one, are there more days on which over 60% of the babies born are boys? Many answer that it makes no difference, but it is the small hospital: with few births a day the proportions fluctuate much more.

The same phenomenon explains why:

• rankings of schools, towns or shops often place the smallest ones at both extremes;
• a new product with five reviews may have a perfect or terrible average that says little;
• an A/B test stopped after a few hundred visits often shows striking differences that later vanish.

Before interpreting an extreme result, always ask how many cases it is based on.

A simulation to try

import random

heads = 0
for n in range(1, 100_001):
    heads += random.random() < 0.5
    if n in (10, 100, 1_000, 10_000, 100_000):
        print(n, heads / n, heads - n / 2)

The second column approaches 0.5, the third does not: the absolute gap between heads and half the tosses stays wide or grows. This is the law of large numbers seen in its correct form.

In short

• The law of large numbers concerns proportions and averages, not absolute counts.
• Chance does not compensate for streaks: deviations are diluted, not corrected.
• With independent trials, the past does not change the probability of the next outcome.
• Small samples produce extreme results more often.
• Independence must be assessed in context: neither assumed nor ruled out in advance.