Medical test paradox

A patient takes a test for some disease and the test comes back positive, that is, the test says the patient has the disease. In reality, though, the probability that the patient really has the disease can be much smaller than the probability that they do not.

The story

Swine flu has appeared in London. A maths student goes to the doctor, who gives them a test to find out whether or not they have the flu. The test has a success rate of 99.95%. After a week the student learns that the test came back positive. They feel sad for a while – after all, the test is 99.95% accurate, so the chance that it made a mistake is only 0.05%. But then they start calculating and cheer up. They find that the probability that they really have swine flu is only about 17%. How is that possible?

Let’s go through it all. To keep things simple, imagine that our imaginary test works as follows. (In practice a test could hardly be 100% successful, but let’s assume it for simplicity.)

• If you have the disease, the test tells you that you are ill in 100% of cases.
• If you do not have the disease, the test is correct in 99.95% of cases. So even if you do not have the disease, in 0.05% of cases the test says that you do.
• If the test says that you do not have the disease, you are definitely not ill.
• If the test says that you have the disease, it may be true or it may be a mistake.

That is not everything we need to know, though. We also need to know how many people in the population are ill.

Lots of ill people

Now let’s see how the test performs in different settings. In this case the result is what you would expect. Imagine a population A in which 200 people out of 10,000 really have swine flu, that is, 2% of people are infected. We test one million people. That means 20,000 people have the flu and the others do not.

Since the test is always correct when a person has the disease, it marks 20,000 people as ill. That leaves 980,000 people who do not have the disease. For them the test has a success rate of 99.95%, that is, a failure rate of 0.05%, so it marks 980,000 · 0.0005 = 490 people as ill even though they are not.

In total, the test marks 20,490 people as ill. Of these, only 20,000 people are actually ill. So the probability that you really have the disease is 20,000 / 20,490 = 0.9760858956, which is approximately 97.6%.

Few ill people

Now let’s look at a population B in which only 1 person in 10,000 is ill, that is, 0.01% of the inhabitants have the disease. We carry out the same test on one million people.

Out of one million people, only 100 are ill – the test correctly marks these people as ill. That leaves 999,900 healthy people. Because of its error, the test marks 0.05% of them as ill. So the test marks 999,900 · 0.0005 ≈ 500 healthy people as ill. The rest are correctly marked as healthy.

In total, 100 + 500 = 600 people are marked as ill. Of these, only a hundred are actually ill, so the probability that you really have the disease is only 100/600 = 1/6, about 17%. Even though the test marked you as ill, you still have a much higher chance of not having the disease.

One ill person

The previous result – that if the test marks you as ill, you have only about a 17% chance of really being ill – may surprise some people. It is even easier to see if we take the example to the extreme.

Suppose a population of one million people in which only a single person is ill. The test we use has a 90% success rate for healthy people. What does that mean? Out of 999,999 healthy people, the test marks 10% as ill, that is, it marks roughly a hundred thousand people as ill. Yet only one person is actually ill. Even if the test marks you, you have only a 1/100000 chance of really having the disease, that is, approximately 0.001%.