Astoundingly, even after my last post (Orthomyxo’s Hero May be the Worst Scientist of All Time), Orthomyxo continues to defend disgraced scientist Neil Ferguson and the debunked Imperial College Model (ICM) that touched off worldwide panic and the tragic waste of trillions of dollars.
For further reading on the debunked model, the American Institute of Economic Research has issued: Imperial College Model Applied to Sweden Yields Preposterous Results. The article details two of the main flaws with the ICM — the inexcusable secrecy surrounding its source code and the “preposterous” predictions the model makes in the case of Sweden. The entire article is must reading.
First — and I found this to be utterly astounding — is the total lack of transparency surrounding the actual model runs. The model was released on March 16 — a date that will live in infamy. But despite the worldwide panic caused by the release, to this day Imperial College has refused to release the version of the model they ran to produce their results. Instead, the article explains
they released a heavily reorganized and generic file that would permit others to run their own version of the COVID model. They do not appear to have released the actual version they ran in the March 16th paper that shaped the US and UK government policies, or the results that came from that model
Secondly, the version of the model that they did produce shows that its assumptions would have made a total hash of predictions about Sweden’s actual results.
Although ICL only released scenarios and associated forecasts for the United Kingdom and United States, its model is theoretically adaptable to any country by changing the inputs to reflect its population, demographics, and the date its specific policies took effect.
In early April around the peak of the academic community’s backlash against the Swedish government’s strategy, a group of researchers at Uppsala University attempted to do just that. They released an epidemiological model for Sweden that adapted the ICL COVID-19 model from Ferguson and his colleagues, and attempted to project the effects of Sweden’s unique response on both hospital capacity and total fatalities.
The results of that study are telling. Under Ferguson’s model, the Swedish government’s response would result in 40,000 deaths shortly after May 1, 2020 and continue to rise to almost 100,000 deaths by June.
The actual results: As of April 29th, Sweden’s death toll from COVID-19 stands at 2,462.
One wonders if the fact that the model predicted results that were demonstrably wrong by orders of magnitude will shake Orthomyxo’s faith in his hero Ferguson and his favorite doomsday COVID 19 model. Actually, that is not true. Of course it will not. Long ago Orthomyxo and those like him proved they have no interest in data or science. Their only interest is in fearmongering and inducing panic on a unimaginable scale. They are dangerous. We must do all in our power to resist, debunk and foil their tactics.
12 Replies to “Must Reading on the Secrecy and Flaws of the Imperial College Model”
Again, the more important point is that we didn’t need ANY models. Public health methods have been tried and optimized over the last 100 years. They work as well as any human effort can work. Sweden and Belarus were simply following REALITY as tried and tested.
Every other country was completely rejecting REALITY and adopting the military tactics of sieges, which have ALSO been tried and tested for thousands of years. Sieges are intended to kill a city or country efficiently. Public health is intended to help people use their own immunity efficiently.
Both procedures are scientifically tested, but both procedures have precisely opposite PURPOSES. The only question we need to ask is why everyone except Sweden and Belarus decided to kill the people instead of helping the people.
I agree that it’s poor to release the model without the source code. I wish the overly optimistic IHME model was more transparent too.
Its not true that the Swedish model is a run of the Imperial college model. It’s an independent implementation of an agent based model, with the same basic structure but independent parameterisation. At a quick glance, they seem to have a higher implied IFR and shorter time till death, though I’d have to look more closely to confirm.
I guess the readers will have to decide whether to believe:
1. Ortho, an anonymous, tendentious internet troll who has frequently been caught pulling “facts” out of his ass; or
2. The authors of the study: Jasmine M. Gardner, Ph.D, Post-doc in Structural Biology; Lander Willem, Ph.D, Post-doc in Infectious Disease Modelling; Wouter Van Der Wijngaart, Ph.D, Professor in Engineering; Shina Caroline Lynn Kamerline, Ph.D, Professor of Structural Biology Peter Kasson, Associate Professor of Cell and Molecular Biology; and Nele Brusslaers, MD, MsC, Ph.D, Associate Professor in Clinical Epidemiology of the Science for Life Laboratory of the internationally renowned Uppsala University.
Ortho, why don’t you give up? You are well and truly beaten. It is embarrassing to watch your antics. You really are at the legless, armless black knight stage: See here.
If the Imperial College model’s source code was not release until April 27th and the paper appeared on April 15th I think we can be fairly confident this is not from, as you claimed, a run of the source code the Imperial College released.
Please let me know which facts I’ve pulled out of my ass. I was accused of doing so re: Fergusson not running a model on Sweden, but I was right on that one. You got angry when I suggested 65% of the population may be infected by this virus, but I laid out my assumptions in that calculations and you’ve yet to tell me where I went wrong. On the other hand, you have frequently been wrong, most obviously about the total number of deaths that would occur (15 -fold wrong so far, with good reason to think we are still early in teh epidemic).
So I’ll ask you again to tell us approximately how many deaths you now think have occured in the USA by the start of August this year. You can give a range if you’d like, and mention any caveats/assumptions about how the epidemic is managed etc.
1. Ortho’s estimate of total US deaths (extrapolated from 80,000 in California alone): 667,000.
2. Initially, like most people, I believed COVID19 was a media-hyped panic (based on decades of experience with media-hyped panic). I thought there would be about 5,000 deaths. I was wrong.
3. On March 22, estimated fewer than 60,000. See here. I was wrong again.
4. But in August (when most models predict the worst will be over) we will see who is wrong but closest to right, approx. 60,000 or Ortho’s 670,000.
Or my prediction if 100,000+.
Anyone who wishes to can read my comment here, showing the estimate Barry refers to: https://uncommondescent.com/medicine/what-are-total-deaths-telling-us/#comment-700632
I was talking about California, Barry decided to forget his motto that geography matters and scale it up to the whole USA.
You’ll see I said this might occur in a scenario where suppression measures are lifted, and over the time it takes for an epedemic to burn out. Epidemics burn out when the number of susceptible hosts is reduced to the point it can’t spread. No model predicts that will happen by August. The only model I know that predicts a step decline is the IHME model, which assumes the lockdown stays in place indefinitely.
So I do think it’s plausible that, given relaxing of measures and assuming no treatment arises, on the order of 650,000 Americans will die. I do not think that will happen by August.
Ortho, if you don’t put a time limit on your prediction it is worthless. What, are you going to haunt these pages for 30 years and if at the end of those three decades the total COVID deaths have accumulated to 650,000 come back and say “See, I told ya”?
OK, let’s see. p8:
For those of you unfamiliar with the nuances of English, “based on” does not mean “the same as”.
FWIW, the GitHub repository with the code was opened on the 20th of March, just after the Imperial model came out, and before they released their code.
The Economist (hardly a liberal publication) has a page tracking excess deaths this year and discusses how that can be interpreted.
A month ago, most models and experts predicted Nebraska to be past its peak by now. With the biggest single day jump – 640 new COVID-19 cases on Friday – that’s not the case. The outbreaks at many Nebraska meatpacking plants are drawing national attention, including a clash between Gov. Pete Ricketts and MSNBC’s Rachel Maddow.
On April 1, according to our data, Nebraska had 218 cases. By May 8, that total was 7,831.
Throughout April, the numbers doubled about every seven days, however that has slowed slightly. Nebraska’s case count has increased by roughly 2,800 cases over the last week, or about 400 cases per day. Nebraska reached its 500th case on April 8, its 1,000th case on April 16, its 2,000th case on April 23, its 4,000th case on April 30, its 5,000th case on May 1, its 6,000th case on May 4, and its 7,000th case on May 7.
A model recently released by the University of Nebraska Medical Center predicts the state’s death toll likely ranging from around 200 to 350-plus, depending on when the peak is. If the peak is today, it predicts roughly 200 deaths and 16,500 cases. If the peak isn’t until May 20, it predicts 340 deaths and around 34,000 total cases.
The IHME Model, which has fluctuated throughout April and May, now predicts Nebraska to peak over the next few days. It was last updated May 4. It’s now projecting nearly 380 statewide deaths, with zero shortages in ICU beds or ventilators.
1 Make it clear in your coverage that models are only as good as the data used to build them, and that researchers currently lack high-quality data about this pandemic.
2 Explain to your audience that researchers also make assumptions when creating models.
3 Keep in mind that researchers use a variety of models to study infectious diseases. They are designed to answer different questions.
4 When reporting on a model that makes a numerical prediction — for example, the number of Americans who will die from COVID-19 during a period of time in the future — emphasize that the prediction is a ballpark estimate represented by a range of possible numbers.
5 Tell your audience what the study adds to what we know about that particular topic and which big questions remain.
6 Ask these seven questions when interviewing researchers about
What type of model was used and what are its strengths and weaknesses?
What assumptions went into creating the model?
What was this model designed to do?
Where did the data used for the model come from and how did using this specific data affect results?
What factors or data were intentionally left out of this study and why?
Does this study focus on a best-case or worst-case scenario?
What caveats must be included in an explanation of this study’s findings?
7 Give additional scrutiny to models created by researchers who have not demonstrated expertise in model building.
Be leery of epidemiology models from scientists who aren’t experts in epidemiology.
9 Use Twitter to find out what academics and others are saying about new research.
10 Learn more about epidemiological models. It will help you ask stronger questions and better explain coronavirus research in plain language.