Counter-Intuitive Statistics – Birthdays and Goats, the Monty Hall Problem

The Monty Hall Problem

There’s no arguing with the numbers.  Studies in mathematical statistics often throw up an understanding of reality that defies our natural intuition and seems utterly unlikely.  Nothing illustrates this more clearly than the so-called Monty Hall problem, which caused eminent thinkers, PhD holders, and a host of others to argue for a long time for solution that was in fact completely incorrect.  Read on and see if you would’ve guessed the right choice (if you don’t know it already!)

The wonderful and wacky world of statistics deals in the collection and analysis of data and grew out of studies of games of chance, in which the mathematics of probability were first laid down in the 16th century.  Toss a coin and there’s a 50/50 chance of heads or tails, but also a vanishingly small (but nonzero) probability that the flipping thing might land on its side!

Studies of the whim and chance of the universe can lead to some surprising conclusions - not least the fact that it is needed for the mind-bendingly odd discipline of quantum mechanics in which the behaviour of the electron is described as a probability cloud called the wave function.  This remarkable science doesn’t reflect our uncertainty about the particle, it really does seem to reflect some fundamental nature of the universe in which chance is built in to the fabric of reality.

Take the following question for example… how many people need to be in a room for there to be a 50% chance that two of them share the same birthday?  There’s 365 days a year and so you might image it’s a fairly large number, say 50, or a 100?  The answer is in fact just 23 people!  You crunch the numbers, perhaps by asking the question the other way round, what is the probability that no one shares a birthday? and you find that the maths is clear.  This problem is named the Birthday Paradox and is a good example of how our natural instinct are completely wrong. See the boxed text at the end for an explanation.

The example par excellence of this counter-intuitive statistics is called the Monty Hall Problem, named after a gameshow presenter from the 1970’s.  The challenge runs like this…

You are on a gameshow and there are three doors.  Behind one door is a fabulous prize (let’s say a bound and autographed copy of all my hapidev articles) and behind the other two doors are goats.  You don’t want to pick a goat, they smell, and would chew your furniture.  The gameshow host allows you to pick one door at random.  After you have done so, the host (who knows where the goats are hidden) opens one of the doors you didn’t choose and reveals one of the goats, thereby eliminate one of the losing doors.  With me so far?  Having got rid of a sofa-munching goat, the gameshow host now asks if you want to stay with the original door you picked or swap to the other unopened door, the one you did not originally pick.

The question is this… is it advantageous to swap?  Does it make any difference at this point in the game to change your choice?  Are the chances more, less, or the same for you to swap to the other door?

Most people when confronted with this problem believe that there is no advantage to swapping, that the probability remains 50 / 50 that you originally chose the winning door but, in fact, there is a significant improvement in your odds if you DO decide to swap.  This seems fundamentally wrong but once more, the maths is clear.

There are a few ways to explain it and you can even draw out the “decision tree” of all the different configurations to also prove that you should swap doors.

One way to explain it is that when you first selected your door, at the start of the challenge, there was a 1 in 3 chance that you chose the winning door (1/3rd).  Now that one door has been eliminated, it’s argued that the remaining 2/3rd chance is transferred to the door that you did not originally choose.

Another way to look at is to imagine that you started with 100 doors, but still only one prize and 99 goats!  You choose a door, and the host now opens 98 doors to reveal the goats and then asks you if you want to swap.  In this scaled-up version of the same problem it is much more intuitive that you should change your choice – what were the odds that you got the right door (1 in 100) right off the bat?

If you’re still unconvinced (as I was when I first heard this), then draw out the decision tree of the different possible combinations and count up the number of times you win when switching compared to winning by sticking with your original choice.  See the handy attached figure (the prize here is a car, not my articles) and go wrap your head around that.

So why is out intuition wrong?  Maybe because we overlook the fact that the gameshow host knows where the goats are located and so his influence to change the scenario mid-game means that extra information has effectively been injected into the situation.  Or maybe we are just hard wired to underestimate some laws of chance, such as picking up the phone at the same moment that our intended caller was trying to dial our number.  Given the estimated 12.4 billion phone calls made every day, scenarios like this should seem a lot less surprising.

In a universe that, quantum mechanics clearly tells us, is rooted on uncertainty and chance maybe we need to be cautious concerning our appreciation of just how likely things are.  I am reminded of this, for example, every time I see someone buy a lottery ticket…

To examine the birthday paradox, think of it like this… by asking what’s the chance that everyone’s birthdays are different…

The first person in the room can have any birthday so 365 days out of 365 so 365 / 365 = 1 – there is a 100% chance that their birthday is not the same as they are the only person in the room!

The second person can only have 364 days out of 365 so 364/365 because they must avoid the date of person number 1.

The third person can only have 363 days out of 365 so 363/365 as they avoid the first two birthdays.

So, the chance of all birthdays being different is (1  364/365  363/365 *…) and so on…

And if you do that 23 times you find that the chance of 23 people in the room all having different birthdays is 0.49 i.e., 49% which means that the chance that two of them DO have the same birthday is 51%, well 50.7% to be precise!

Probability of Same Birthday as a Function of Number of People