The above question is extremely important during the current COVID-19 panic.
Tyler Cowen quotes from a Stat Reports article by Sharon Begley:
A widely followed model for projecting Covid-19 deaths in the U.S. is producing results that have been bouncing up and down like an unpredictable fever, and now epidemiologists are criticizing it as flawed and misleading for both the public and policy makers. In particular, they warn against relying on it as the basis for government decision-making, including on “re-opening America.”And then Cowen wisely asks:
“It’s not a model that most of us in the infectious disease epidemiology field think is well suited” to projecting Covid-19 deaths, epidemiologist Marc Lipsitch of the Harvard T.H. Chan School of Public Health told reporters this week, referring to projections by the Institute for Health Metrics and Evaluation at the University of Washington.
Others experts, including some colleagues of the model-makers, are even harsher. “That the IHME model keeps changing is evidence of its lack of reliability as a predictive tool,” said epidemiologist Ruth Etzioni of the Fred Hutchinson Cancer Center, home to several of the researchers who created the model, and who has served on a search committee for IHME. “That it is being used for policy decisions and its results interpreted wrongly is a travesty unfolding before our eyes.”
…The chief reason the IHME projections worry some experts, Etzioni said, is that “the fact that they overshot” — initially projecting up to 240,000 U.S. deaths, compared with fewer than 70,000 now — “will be used to suggest that the government response prevented an even greater catastrophe, when in fact the predictions were shaky in the first place.”
To be clear, I am (and always have been) fully aware that there are more nuanced epidemiological models “sitting on on the shelf,” just as is true for macroeconomics and many other areas. But I ask you, where are the numerous cases of leading epidemiologists screaming bloody murder to the press, or on their blogs, or in any other manner, that the most commonly used model for this all-important policy analysis is deeply wrong and in some regards close to a fraud? Yes I know you can point to a few tweets from the more serious people, but where has the profession as a whole been?Are epidemiologists that compromised by the establishment? They apparently know the truth.
Cowen again:
And to be clear, I have heard [the more nuanced] model cited and discussed in many (off the record) policy discussions.And some reporting by Reason in the pre-COVID-19 world of 2016:
When it comes to separating the wheat of good public health research from the chaff of studies that are mediocre or just plain bad, Albert Einstein College of Medicine epidemiologist Geoffrey Kabat is a national treasure. "Most research findings are false or exaggerated, and the more dramatic the result, the less likely it is to be true," he declares in his excellent new book Getting Risk Right...
"As we have seen," concludes Kabat, "the landscape in which health risks are studied and in which findings are disseminated is pervaded by false claims, oversold results, biases operating at the level of observational studies as well as psychological and cognitive biases, and professional and political agendas."
-RW
Statistical models are of no use when making personal health decisions. However, people make mistakes, so-called experts accept these models as being helpful when they are not and people rely on experts rather than their "own eyes" and experience. But all of this has happened many times before. So-called "nuanced" models and less regarded "observational" models that were more actively reported were also around in 2018, 2017, 2016...., etc. So what triggered the hysteria this time?
ReplyDelete