The other day I was making a list of clients I needed to call. While jotting down their phone numbers, I noticed something.
Three phone numbers in a row ended in "97".
Rather than run out to buy lottery tickets, I asked myself, "What are odds of that happening?"
Fortunately, our old friends basic math and human perception were there to help me out.
Basic math had already come to the rescue when Balloon Boy was all over the news. Thanks to basic math and 40 seconds on Google to find out the lift power of a cubic foot of helium, I rested easy that night knowing there was no way on God's green Earth that Falcon Heene was in that saucer.
A few days later, basic math was there again when Arnold Schwarzenegger wrote his famous love note to Tom Ammiano. That time math needed a little help from psychology, since the problem wasn't just about the odds of Arnold's message appearing at random.
What will math and perception say this time?
You Get The First One Free
At first glance, this is easy. Each phone number could end with one of 100 two digit combinations and repeats are allowed (two numbers, for example, could both end in "81"). So the odds of three "97"s in a row is just 1 in 100 x 100 x 100.
That's literally one in a million. It's a miracle.
But not so fast. Just like with Arnold's love note we have to do a little more work.
The first thing we have to do is knock one factor of 100 off the odds. The first phone number has to end in some two digit combination, so the first number is free. Anything from "00" to "99" would work. It just doesn't matter.
We're already down to 1 in 10,000. Not quite a miracle anymore and we're just getting started.
The Second Ones Free, Too
It gets worse for our miracle because it turns out that the second number is free, too. The second phone number had to match the first one or I won't have ever noticed the coincidence. Having "97" followed by "31" and "28" and then another "97" won't have caught my eye.
So the second number is free, too. Cut two more zeros off the probability. Now it's just 1 in 100.
Nothing to see here, people. Move along.
Bayesian Probability Is Your Friend
The little exercise we just did is a trivial, informal example of Bayesian Probability. Bayesian Probability is the big rival to the Frequentist View of probability.
A frequentist would have looked at the three numbers, multiplied out 100 x 100 x 100 and called it a day.
In Bayesian probability, it's more complicated. Bayesian probability accounts for the assumptions underlying the problem and the knowledge we gain as the problem unfolds. So the Bayesian analyst says:
We know the first number has to end in something and we know the second number has to match the first or we won't even be talking about this.
Given that, what are the odds the third number will match?
The Bayesian answer to a problem can differ radically from the frequentist answer. And it can differ in ways that really screw up our intuition about life.
Monty Hall, the Car and the Goats
Do you remember when Marilyn vos Savant published the Monty Hall problem in her Parade magazine column?
Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?
The answer is that you should switch. If you do, your chances of winning the car jump from 1 in 3 to 2 in 3.
But this answer offends our intuition. Intuition says that knowing a goat is behind a door you didn't pick shouldn't matter. It just reduces the number of doors, so the odds are now 1 in 2, no matter if we switch or not.
It is incredibly hard to over-come this intuition. Despite Marilyn's expanation, 10,000 readers, including 1,000 claiming to have PhDs in statistics, wrote to Parade saying she was wrong.
Another False Intuition About Odds
Let's say you're in a poker game. The game is 5 Card Stud and one of the other players is dealt a Royal Flush in spades. You're playing stud, so there are no hole cards, no wild cards, no draws. Just a straight-up, natural Royal Flush.
If you're in the Old West or your state has liberal carry laws, your hand would probably be reaching for iron as soon as the ace turns up.
But should it?
Well, yes. It should. The odds of being dealt that particular hand are tiny. There are 311,875,200 different 5 card poker hands, so the odds of getting that Royal Flush in Spades is 0.00000032%
[Correction. Here I am writing a diary on how easy it is to screw up probabilities and what do I do? I screw up one of the probabilities.
It's true that there are 311,875,200 different ways to deal a 5 card hand. So the odds of being dealt 10S, JS, QS, KS, AS are 1 in 311,875,200.
But that's not the correct way to calculate the odds of being dealt a Royal Flush in Spades. My error was that a Royal Flush is a Royal Flush regardless of whether it's dealt to you 10, J, Q, K, A or A, K, Q, J, 10 or J, A, 10, K, Q. The order in which the cards are dealt to you doesn't matter.
That means to get the correct odds we have to divide by the number of ways we can be dealt a Royal Flush. That's 5!, so the correct odds are 1 in 2,598,960 or 0.0000384%. Still very tiny, but not as tiny as 0.00000032%]
Here's the interesting and sort-of Bayesian thing. The odds of getting any poker hand are exactly the same. A hand of 2D, 8S, KD, JH, 5C is just as probable as that Royal Flush. It's only because the rules of Poker make the Royal Flush such a valuable hand that we sit up and notice when one comes up.
If the rules of Poker made the Royal Sampler the highest hand, that 2D, 8S, KD, JH, 5C hand might put you in a shooting mode.
This is another thing about probability that people have a hard time with. It' obvious to us that the Royal Flush is a significant hand and the Royal Sampler is a good reason to fold and get another beer.
The odds are exactly the same for both hands, yet people see intent, purpose and cheating behind one of them. This false intuition isn't limited to poker. People have actually sued and won because they bought a lottery ticket with random numbers and got 1, 2, 3, 4, 5. Since that sequence obviously isn't random, the machine must have been rigged.
That's Nice, But Who Cares?
These diaries are mostly just fun with math, physics, probabilities and intuition. But I hope they are more than just fluff and fun.
Developing a mental habit of checking the math (and knowing when the naive interpretation of the math is wrong) is a powerful thing. It saved me a trip to the 7-11 for lottery tickets. Colorado could have saved millions in helicopters and search and rescue if someone had just compared the lifting power of Heene's balloon against the combined weight of Falcon and the balloon.
Of course, that would have cost Fox and CNN millions. So it's a wash.
One More Thing
I wrote this diary because earlier today KAMuston wrote a diary called GOING TO SEE THE ELEPHANT.
I commented in that diary because it explained an obscure lyric in the great James McMurtry song "See The Elephant".
Just one hour later, Hoghead99 saw that comment while listening to the great James McMurtry song "Choctaw Bingo"
What are the odds of that?
Seriously, somebody figure out what are the odds of that.