Remember the Monty Hall probability problem? If not, then you're in for a nerd treat. If you've seen the problem before, can you remember how you felt when you first heard the solution? I think people's reactions to the problem are even more interesting than the underlying mathematics. The Monty Hall problem is underutilized as a personality test.
The actual problem is a minor variation of a game played on the old TV game show "Let's Make a Deal," hosted by Monty Hall. It goes like this: Monty, the host, presents you with three closed doors. Behind one is a car, and behind the other two are goats. You get to choose one door. If you choose the one hiding the car, you win the car as your prize. Because I live in hipster Seattle, I have to clarify that picking a goat means you lose. Delicious cheese aside, goats are disappointments, not prizes.
Here's the twist: After you select a door, Monty opens one of the two unselected doors to reveal a goat. He always reveals a goat; never the car. Now there are two closed doors. Monty asks you: Do you want to switch your choice of door?
If you've never heard this problem before, take a moment to think about it before you read on. Would you switch doors? Why? Does switching change the likelihood of your winning a car?
:: Spoilers Ahead ::
The right answer is yes. You should switch doors. Doing so doubles your chances of winning! Your odds go from 1 in 3 if you stick with your original choice, to 2 in 3 if you switch. I know, I know, this goes against all intuition. I'll walk through an explanation in just a minute.
First, though, take stock of what you're thinking right now. Are you looking for a way to prove that your instinct was actually correct, even if it disagrees with what I just told you? Are you curious, and trying to figure out how my answer could possibly be correct? Are you researching my background to check my credibility, or hunting for other discussions of this problem to verify that I'm not lying to you? Are you skipping to the bottom of this article to read the justification? Are you just bored?
I love watching people's reactions to hearing this problem for the first time. Their reactions reveal so much about their mindset.
My favorite people are the ones who react with a genuine curiosity. They're open to the possibility that their instinct was wrong, and they're eager to learn why. It's no surprise that these people are great to be around and to work with. They value truth and learning above ego. I feel lucky that most of my colleagues and students over the years have fallen into this category.
My least favorite people are the ones who react defensively. Happily, these people are rare. But they do exist. A non-technical colleague once told me, as I tried to explain the answer to him, that I must not understand probability very well if I thought that switching doors changed the odds. We had worked together for over a year; I was the team's data scientist, and a credible one; and he knew about my Ph.D. and my role as an ML instructor. Now, it's entirely possible that my explanation was unclear and confusing. But for him to leap from his confusion straight to the assumption that I was somehow confused or incorrect was a telling leap indeed. No wonder this guy was tough to work with. He left no room for the possibility that he was incorrect, either in this problem or in our professional work.
The most interesting people, to me, are the ones who don't really care one way or the other. They believe the right answer, once it's given, but they aren't particularly invested in understanding why it's correct. This is fascinating to me because I can't relate to it. Not even a little bit. An unsolved puzzle will eat away at me until I understand the solution. This trait has upsides and downsides, of course. It's one of the reasons I'm great at my work, but it can also lead to distraction. Complementary perspectives usually make good collaborations, though, and it's fun to find people who are motivated by different things than I am.
I would never use the Monty Hall problem as an interview question. Questions like this are notoriously unhelpful for predicting job performance, and also create notoriously bad candidate experiences (even the New Yorker agrees). But I do wish I had a way of weeding out defensive or ego-driven candidates as efficiently as I could using the Monty Hall problem. It would save so much time!
Anyway, for those who care, let's get back to the actual Monty Hall problem. Why is it that switching doors doubles your chances of winning?
First, consider the strategy of sticking with your original door. Your odds of having chosen the right door off the bat are 1 in 3, so those are your odds of winning with this strategy. This is the case where intuition and probability align.
Now, consider the strategy of switching doors. How does this strategy play out when your original choice of door is correct (1/3 of the time), versus when it's not (2/3 of the time)?
If your original choice is correct, then the switching strategy always leads you to a goat. It doesn't matter what door Monty opens. You're switching away from the car, and you're guaranteed to lose.
What happens when your original choice is incorrect? Then one of the unchosen doors hides a goat, and the other hides a car. We know that Monty will never reveal the car. He will open the door hiding the goat. Thus, the door he doesn't open always contains the car. The switching strategy always leads you to the car. In the 2 out of 3 cases where your original choice was incorrect, switching doors guarantees that you'll win.
So there you have it. Under the switching strategy, you win when your original choice is a goat (2/3 of the time), and you lose when your original choice is the car (1/3 of the time). So overall, the switching strategy leads to a win 2/3 of the time, while staying with your original choice leads to a win 1/3 of the time.
That explanation reflects the way I think about the problem. It's the explanation that resonates most with me, but even I have to admit it's pretty dry. The intuitive key is that Monty is giving you information through his choice of door. In particular, when you've initially chosen a goat door, Monty has no choice about which door he opens, since he's not allowed to reveal the car. You've forced him to reveal the location of the car. His actions have given you new information, and by switching doors you can take advantage of that new information.
There are a few other explanations out there that use very different reasoning to reach the same conclusion. Others have written about them more clearly than I can. This site has a few nice explanations, and also a simulator that lets you play the game over and over, so you can see for yourself that your win rate settles around either 1/3 or 2/3, depending on the strategy you use.
The breadth of common reactions to this problem is also well-documented. Whatever your reaction, you're not alone. The Wikipedia page has a nice discussion around the original publishing of the problem in Marilyn Vos Savant's newspaper column back in the 1990's. The public reaction included a lot of people with strong statistical credentials doubling down on the intuitive but incorrect answer.
This Thanksgiving, if you're looking to fill a conversational lull with something non-political, give the Monty Hall problem a spin. Everyone knows that math is the key to any successful party! See whether any of your relatives come to the correct conclusion on their own. More interestingly, see how they react when they don't. See if anyone surprises you!