Here is a story from US News about German’s COVID-19 death rate that demonstrates how one should hesitate to draw conclusions from the face of a statistic.
First, some basic math. The death rate for COVID-19 (indeed any rate) is based on a fraction. We all learned in grammar school that fractions have two parts: a numerator (the number on top), which is divided by the denominator (the number on bottom). The fraction for the death rate is [total deaths]/[total cases].
Now, it should be obvious that there are two ways for the rate to go down. (1) hold the line on total deaths in proportion to total cases OR (2) pump the number of total cases in proportion to deaths. It should also be obvious that only way (1) is a measure of the efficacy of a country’s health care efforts. Here is an example. Assume two countries with equal populations: Country, X and Country Y. Let’s assume that both countries have the exact same number of deaths at a given time, say 400. So the numerator for both cases is the same.
Now let’s assume that the total number of identified cases for County X is 20,000 and 40,000 for Country Y. So the death rate for County X is 2%. And the death rate for Country Y is 1%. It is obvious that Country Y’s healthcare system is twice as good as Country X’s. After all the death rate in Country X is double the death rate in Country Y.
Not so fast. Note the key word is “identified.” If we assume that identified cases track with actual cases, then the conclusion that Country Y’s healthcare system is better may be warranted. But what if Country Y has instituted a massive testing program with the goal of testing every single person in the country. And at the same time, County X tests only those presenting for treatment? If that were the case, the difference in rates would probably be attributable in part (almost certainly very large part) to the fact that Country Y has included a huge number of asymptomatic people in its total cases number and Country X has not. In other words, the difference in death rates would have little to do with the relative efficacy of each country’s healthcare system, and almost everything to do with the difference in testing.
Now let’s go to the story. The reporter notes that Germany’s death rate is low compared to other countries and concludes this “suggests that Germany is doing something right that the others aren’t.” To his credit the reporter demonstrates that he understands the basic math we just went through:
The most important factor contributing to the low death rate is that Germany appears to be that it is testing far more people than any other European country. Scientists agree that a large number – probably a big majority – of all coronavirus cases never make it into the official figures because they are not severe enough for hospital treatment. The more widely a country tests, the more of these milder cases it will find. Since the most severe cases are almost always tested, the number of coronavirus deaths will likely stay the same. The net effect is that more testing leads to a lower-looking death rate.
So what is the big deal about Germany? It is lowering its death rate by identifying a lot of asymptomatic cases. That says practically nothing about the efficacy of its healthcare system in lowering the rate by reducing deaths through superior treatment. Yet, the reporter goes on to praise Germany’s universal healthcare system, suggesting that system has something to do with the lower rate. Maybe it does. But the reporter does not know that and his musings are nothing but pure speculation. And he should know this, because he has already shown earlier in the story that Germany’s low death rate is certainly almost entirely an artifact of higher testing rates.
Yet another lesson from the “lies, damn lies and statistics” category.