Last month I mentioned that conservatives were using talking points from John Ioannidis to bolster their claims that COVID-19 is being over-diagnosed and just isn’t really that big of a deal. I didn’t go into it then but that is kind of a big deal. Here’s why.
First of all, it has forced me to finally learn once and for all how to pronounce John Ioannidis’s name. Long-time fans of this channel already know that thanks to growing up in an environment that wasn’t terribly intellectual but did provide me with a lot of books, I’m really good at reading things and not so good at pronouncing them. Ee-ya-nee-dees. Got it. Now I basically know Greek. Thank you John Ioannidis.
Second of all, it’s a big deal because Ioannidis is a big name, both amongst the smaller sphere of skeptics and in the larger world of the study of science and epidemiology. He is most well-known for his debunking — his biggest studies have been about how fucked up many scientific studies are. He was one of the loudest voices calling attention to how bad the replication crisis is throughout many different disciplines, including medicine. Researchers are rewarded for publishing novel findings, so too often they (consciously or unconsciously) massage their data to get something interesting. They aren’t rewarded for replicating, so shoddy research often gets published and taken as “truth” for far too long.
He’s also on record rightly criticizing sloppy meta-analyses. As a reminder, meta-analyses are studies that take a look at all the previous research on a particular subject and combine them into one larger dataset, to give you an idea of the current state of the research. For instance, you can say that one particular study shows that humans are responsible for heating up the planet, but it’s much more powerful and helpful to say that 97% of all climate research supports the fact that humans are responsible for heating up the planet. One study can be flawed, which is why redundancy and replication is so great for our body of scientific knowledge. But to make a meta-analysis, the researcher must pick and choose which studies to include. You could do a meta-analysis on whether or not the earth is flat and only include data from studies conducted by Tila Tequila, and come to the conclusion that the earth is, in fact, flat. That’s an egregious example but you can imagine how subtle this kind of cherrypicking can be. Ioannidis points out that scientists with “pre-determined agendas” can easily manipulate the data to show whatever they want, or what their sponsors want, as he pointed out to Retraction Watch, when he said “sponsors and other conflicted stakeholders want to exploit them to promote their products, beliefs, and agendas” He has described this misuse of meta-analyses as an “epidemic.”
All of this is true, and particularly ironic because researchers are now pointing out that Ioannidis himself has published a meta-analysis that they say manipulates data to downplay an actual literal epidemic to promote the agenda that we should reopen the US economy at the risk of spreading COVID-19 and killing more people. And his meta-analysis includes his own much-pilloried (and as of right now still not officially peer reviewed) antibody study claiming to show that coronavirus had infected 85 times more people than we currently know about in Silicon Valley, making it much less deadly than other studies suggest. Thanks to a whistle blower, the public has just learned that that study was funded in part by David Neeleman, the founder of JetBlue Airways who is outspoken in his belief that the virus isn’t that bad and we should let everyone leave their homes and start, oh, I don’t know, flying in jets to other places? It’s almost like that’s a conflicted sponsor with a pre-determined agenda paying for someone to promote their agenda. Almost!
Much like his initial study, “COVID-19 Antibody Seroprevalence in Santa Clara County, California,” the meta-analysis has been the subject of some lively criticism on social media. Honestly that’s the nice thing about these sort of papers being made publicly available — social media peer review tends to be much more thorough than the traditional type. And more entertaining!
For instance, Australian epidemiologist Gid MK pointed out some of the flaws of the meta-analysis over on Twitter. Ioannides fails to include cogent information on why he chose the studies he did, and similarly why he chose to exclude others. No government reports are included in the research, which is strange because that’s where the bulk of the testing data is coming from. For instance, the largest dataset to date comes from a Spanish report that estimated an infection/fatality rate of 1 to 1.3%. Conveniently, none of the studies chosen by Ioannaidis have an IFR greater than .28%.
Gid also points out that several of the studies included in the meta-analysis suffer from a thing called right-censoring. In common parlance, “right-censoring” refers to YouTube removing neo-Nazi and Alex Jones videos, but in scientific terms it means a faulty method of data analysis in which subjects leave a study too early or the study itself ends too early before the desired result could actually be seen. Gid has a good example: let’s say you want to see if smoking is correlated with lung cancer, so you gather 1,000 20-year old smokers and 1,000 20-year old non-smokers. You track them for 5 years. By the age of 25, 11 smokers have lung cancer and 10 non-smokers do, which is an insignificant difference. If the study ends there, you can say there’s no correlation. But if you check again in 25 years, you’ll see that in fact the number of smokers with lung cancer skyrocket past the nonsmokers. Boom! Correlation.
What’s this got to do with COVID-19? Well, if you test a random population for antibodies today and find that 100 people have them but only 1 person died, you might think that’s an IFR of 1%. But COVID-19 doesn’t kill people immediately — it can take two to three weeks. That means that your calculated IFR is going to be much lower than what it actually is. That’s right-censoring, baby! To figure out the actual IFR, you’d want to test 100 people today and see how many people are dead in a few weeks, or use statistical modeling to correct for the problem. Several of Ioannides’s studies didn’t do that, and he didn’t correct it for them. Instead, he took the lowest possible IFR from each study (for instance, just choosing “.37%” for a German study that found IFR of 0.37-.46%) and skewed them in the opposite direction, making them much lower than what the studies suggest and claiming that it’s appropriate because those studies didn’t test for all the various antibodies. So the German study that found an IFR of .37-.46% gets marked down as “.37” and then “adjusted” to “.28%.” Shady!
Like I said in my previous video about Ioannides’s initial seroprevalance study, it is useful to have more information, even if that data wasn’t necessarily handled in the correct fashion. And I don’t think Ioannides is an evil person who is trying to get more people killed. So I won’t call his studies trash — I’ll leave it to the experts to do that. And I won’t call him trash, but I will point out that he very clearly has a bias. Over on Wired, David H. Freedman points out that when he interviewed Ioannides back in 2004, the scientist said outright that his own bias is finding that researchers are doing good work. “If I did a study and the results showed that in fact there wasn’t really much bias in research, would I be willing to publish it?” he said then. “That would create a real psychological conflict for me.”
“Ioannidis was acknowledging that he’s invested in showing that other scientists tend to get it wrong, and that he might end up being skeptical of data suggesting they are, in fact, getting it right.”
And I have to say that even though I think Ioannidis is simply the victim of his own internal biases, and I don’t think he’s in the pocket of Big COVID-19 Conspiracy Theory, it’s extremely shady that he didn’t disclose an obvious conflict of interest, that his research was being funded by a multi-millionaire who is on record supporting Donald Trump’s plan to reopen the economy because the lives we will lose are worth less than his stock. I don’t think that David Neeleman passed Ioannidis a suitcase of money and told him to make up the data, but I can’t help but be skeptical when Ioannidis fails to disclose the connection and when called on it claims that he didn’t know Neeleman was behind the money. Neeleman himself told Buzzfeed that he gave the $5,000 to Stanford specifically for Ioannidis’s study, and added that he was in communication with the researchers throughout the study. He even wrote a post for Ben Shapiro’s website The Daily Wire in which he said he got to know all of them, including Ioannidis, personally.
I highly recommend you check out the Buzzfeed reporting, which has uncovered a lot of similar shady business surrounding Ioannidis. It’s so disappointing to see someone I really admired for their unbridled skepticism suddenly descend into anti-science neo-conservative/Libertarian buffoonery but honestly I have to say…the feeling is becoming pretty familiar and cozy. Like an old pair of pajama bottoms that you can’t wear around other people due to the smell and the holes in delicate places.
I’ll conclude by saying what I already said in that previous video: none of this matters to you, the general public. Whether they find that COVID-19 has an IFR of .01% or .1% or 1%, none of that has to do with your health right now. None of those figures will apply to you if you catch this virus. Your odds of survival depend on your health and your luck. So don’t leave it up to that. Stay away from people. If you have to go near people, wear a mask. Wash your hands. Take care of yourself.