## Does Changing Your Anwer In The Exam Help?

*The Monty Hall Paradox*

*One of the 3 doors hides a car. The other two hide a goat each. In search of a new car, the player picks a door, say 1. The game host then opens one of the other doors, say 3, to reveal a goat and offers to let the player pick door 2 instead of door 1. Is there an advantage if the the player decides to switch? (Courtesy: Wikipedia)
*

Hola amigos! Yes, I’m back! It’s been eons and I’m sure many of you may have been wondering why I was MIA. Let’s just say it was academia as usual.

This post is unique as it’s probably the first where I’ve actually learned something from contributors and feedback. A very critical audience and pure awesome discussion. The main thrust was going to be an analysis of the question, “If you had to pick an answer in an MCQ randomly, does changing your answer alter the probabilities to success?” and it was my hope to use decision trees to attack the question. I first learned about decision trees and decision analysis in Dr. Harvey Motulsky’s great book, “Intuitive Biostatistics“. I do highly recommend his book. As I pondered over the question, I drew a decision tree that I extrapolated from his book. Thanks to initial feedback from BrownSandokan (my venerable computer scientist friend from yore :P) and Dr. Motulsky himself, who was so kind as to write back to just a random reader, it turned out that my diagram was wrong and so was the original analysis. The problem with the original tree (that I’m going to maintain for other readers to see and reflect on here) was that the tree in the book is specifically for a math (or rather logic) problem called the **Monty Hall Paradox**. You can read more about it here. As you can see, the Monty Hall Paradox is a special kind of unequal conditional probability problem, in which knowing something for sure, influences the probabilities of your guesstimates. It’s a very interesting problem, and has bewildered thousands of people, me included. When it was originally circulated in a popular magazine, “nearly 1000 PhDs” (cf. Wikipedia) wrote back to say that the solution put forth was wrong, prompting numerous psychoanalytical studies to understand human behavior. A decision tree for such a problem is conceptually different from a decision tree for our question and so my original analysis was incorrect.

So what the heck are decision trees anyway? They are basically conceptual tools that help you make the right decisions given a couple of known probabilities. You draw a line to represent a decision, and explicitly label it with a corresponding probability. To find the final probability for a number of decisions (or lines) in sequence, you multiply or add their individual probabilities. It takes skill and a critical mind to build a correct tree, as I learned. But once you have a tree in front of you, its easier to see the whole picture.

Let’s just ignore decision trees completely for the moment and think in the usual sense. How good an idea is it to change an answer on an MCQ exam such as the USMLE? The Kaplan lecture notes will tell you that your chances of being correct are better off if you don’t. Let’s analyze this. If every question has 1 correct option and 4 incorrect options (the total number of options being 5), then any single try on a random choice gives you a probability of 20% for the correct choice and 80% for the incorrect choice. The odds are higher that on any given attempt, you’ll get the answer wrong. If your choice was correct the first time, it still doesn’t change these basic odds. You are still likely to pick the incorrect choice 80% of the time. Borrowing from the concept of “regression towards the mean” (repeated measurements of something, yield values closer to said thing’s mean), we can apply the same reasoning to this problem. Since the outcomes in question are categorical (binomial to be exact), the measure of central tendency used is the Mode (defined as the most commonly or frequently occurring thing in a series). In a categorical series – cat, dog, dog, dog, cat – the mode is ‘dog’. Since the Mode in this case happens to be the category “incorrect”, if you pick a random answer and repeat this multiple times, you are more likely to pick an incorrect answer! See, it all make sense :) ! It’s not voodoo after all :D !

Coming back to decision analysis, just as there’s a way to prove the solution to the Monty Hall Paradox using decision trees, there’s also a way to prove our point on the MCQ problem using decision trees. While I study to polish my understanding of decision trees, building them for either of these problems will be a work in progress. And when I’ve figured it all out, I’ll put them up here. A decision tree for the Monty Hall Paradox can be accessed here.

To end this post, I’m going to complicate our main question a little bit and leave it out in the void. What if on your initial attempt you have no idea which of the answers is correct or incorrect but on your second attempt, your mind suddenly focuses on a structure flaw in one or more of the options? Assuming that an option with a structure flaw can’t be correct, wouldn’t this be akin to Monty showing the goat? One possible structure flaw, could be an option that doesn’t make grammatical sense when combined with the stem of the question. Does that mean you should switch? Leave your comments below!

Hope you’ve found this post interesting. Adios for now!

Copyright © Firas MR. All rights reserved.

*Readability grades for this post:*

*Flesch reading ease score: 72.4
Automated readability index: 7.8
Flesch-Kincaid grade level: 7.3
Coleman-Liau index: 8.5
Gunning fog index: 11.4
SMOG index: 10.7*

Intuitive Biostatistics, by Harvey Motulsky

*Powered by ScribeFire.*

Salaam

Um your analysis is incorrect. Your expected winning isn’t $800 if you switch, because you can still only choose one of the remaining four. If you could choose all four and win if one of the four was correct, then it would indeed be $800. So your chances if you switch are still $1000 * (4/5)(1/4) = $200 (chances of your current answer NOT being the correct one, multiplied by the chance of the new answer being correct).

The reason this is different from the Monty Hall problem is because you’re given new information after you choose. You’re shown the empty door and that changes around the probabilities. Suddenly the combined chances of two doors collapses into one door because one is shown to be empty.

Be careful with the math :)

BrownSandokanAugust 11, 2009 at 4:42 pm

W’salam,

Interesting :) . I’d actually thought about this. But I think it makes sense if you think of the two pathways as two independent branches. In fact this is how the problem is approached in the book. Your decision to pick an answer is not necessarily correlated or dependent upon your earlier choice. Every answer is an equal candidate of being the right option, as long as you can’t establish that for a fact – that is your outcomes are truly random.

And when you want to figure out the probabilities for each outcome – $0 or $1000, I think it makes sense to multiply each outcome with its own probability. To calculate the combined probability of the two mutually exclusive outcomes just find the sum.

Thus, if your calculation can be written as [(4/5 * 0) + (1/4 * 1000)], you end up with $250, and you’re still better off switching :) .

I agree, the Monty Hall problem is a little different. I’m going to make a mention of that in the post. Geez, I’m so awful at arithmetic! I might have to completely rewrite this if I’m wrong!

Firas MRAugust 11, 2009 at 5:15 pm

Nope .. absent any new information, you can’t get any more out of the system than an expected winning of $200.

I don’t see how you got $250. What are you multiplying? You have to multiply the 1/4 with 4/5 also because both events must happen.

BrownSandokanAugust 11, 2009 at 5:22 pm

@BrownSandokan

Yes, I understand where you’re coming from and your point makes sense. Absent any new information, how can you get anything more than $200? Valid question. Tell you what, I’m going to send you the excerpts on decision trees from his book and maybe you can help me learn something! And I’ll keep updating this post as I go along. Maybe I should turn this article in to a wiki :) .

Firas MRAugust 11, 2009 at 5:54 pm

Just as a quick refresher to readers, there are three different kinds of probabilities:

Independent:- Two events x and y are considered independent if the occurrence of x doesn’t preclude the occurrence of y and vice versa. Not only that, the occurrence of x doesn’t tell you anything about the occurrence of y. To find the combined probability of two independent events x AND y multiply like soExample: What is the combined probability that a patient has a foot infection and is bald?

Conditional: Two events x and y are said to be conditional if giving the probability of one tells you something about the other. Not only that, the occurrence of x doesn’t preclude the occurrence of y or vice versa. To find the combined probability of two conditional event x and y, where y is conditional on x:Example: What is the combined probability that a person has diabetes AND atherosclerosis? We all know atherosclerosis and diabetes are usually seen together and having one predisposes to the other.

Mutually exclusive: Two events x and y are mutually exclusive if the occurrence of x precludes the occurrence of y or vice versa. To find the combined probability of two mutually exclusive events x and y, find the sum like so:Example: What is the combined probability that patient has hyperacidity OR hypoacidity?

Firas MRAugust 11, 2009 at 6:13 pm

[...] I’d initially written about decision trees, it did not at all occur to me at the time how this stuff was so familiar to me [...]

Decision Tree Questions In Genetics And The USMLE « My Dominant HemisphereSeptember 3, 2009 at 4:07 pm

[...] we are aware of what it is to be aware of. This can often lead to trouble down the road. I’ve talked about numerous PhDs having failed at the Monty Hall Paradox before. But a recent talk I watched, touched [...]

Let’s Face It, We Are Numskulls At Math! « My Dominant HemisphereNovember 3, 2010 at 8:32 pm

[...] Monty Hall paradox, about which I’ve written before is another good example. You’re presented with new information midway in the game and you use [...]

Meeting Ghosts In The Chase For Reality « My Dominant HemisphereNovember 16, 2010 at 9:00 am