This post contains a video, which you can also view here. To support more videos like this, head to patreon.com/rebecca!
Transcript:
Last month, I talked about an MIT preprint published on arXiv about AI. It’s a little early for an update on that, but there is some interesting news about another MIT preprint published on arXiv about AI, and the news around it really highlights why, whenever I discuss a preprint, I take pains to point out that it all might be bullshit unless and until it passes peer review and is published in a reputable journal. I do NOT think that mainstream media has any business covering preprints at all.
Back in December, the Wall Street Journal published a glowing article about supposedly groundbreaking research just posted on arXiv from Aidan Toner-Rodgers, a 26-year old doctoral student at MIT, one of the top economics schools in the country. The article is topped with a photo showing Toner-Rodgers flanked by two of the most influential living economists of our time and his advisors, David Autor and Daron Acemoglu (who recently won the “Nobel” Prize for economics).
“It’s fantastic,” said Acemoglu, about Toner-Rodgers’ paper.
“I was floored,” said Autor.
Toner-Rodgers’ paper was about using AI to accelerate scientific discovery. He decided to look at material science, because much of the work of discovering new materials is trial-and-error, something that an algorithm could presumably tackle. Here he is describing the research in an hourlong conversation with the Atlantic’s podcast.
The results he got were astounding: the researchers using the AI tool “discovered 44% more materials, their patent filings rose by 39% and there was a 17% increase in new product prototypes. Contrary to concerns that using AI for scientific research might lead to a “streetlight effect”—hitting on the most obvious solutions rather than the best ones—there were more novel compounds than what the scientists discovered before using AI.”
Wow, incredible! I take back every bad thing I’ve ever said about AI. I know you don’t believe that because I already told you in my introduction that this was, in fact, bullshit. I’ve talked about some past headline-grabbing academic frauds, like Dan Ariely (who got a prime time TV show) and Francesca Gino (who got fired). And usually the charlatans in question actually perform an experiment, but then fudge the numbers a little bit to make sure the results show what will get them the headlines and book deals. The funniest example is Ariely and Gino working together on a study that claimed to show that people are more honest when they sign an honesty pledge before doing a task where they could cheat to gain a benefit. The jokes just write themselves.
But this is not another case of a few fudged numbers. This is a case of a guy who just straight made up an entire study and convinced the world that it was real.
After Toner-Rodgers got so much great press, not just from the Wall Street Journal and Atlantic but also NPR and Freakonomics and basically any outlet that regurgitates economics research without looking at it too closely, a computer science expert was curious as to where, exactly, Toner-Rodgers found all this beautiful data. Supposedly, this lab that employs 1,000 researchers to study potential new materials every day (which would narrow down the lab to a handful, despite Toner-Rodgers’ insistence on keeping them anonymous) started their own clinical trial a year before Toner-Rodgers was even accepted into MIT, and then agreed to share the very sensitive data they gathered with this first-year doctoral student.
According to the Wall Street Journal in a follow-up article a few months ago, this computer science expert had experience in materials science, and was baffled that he hadn’t heard about any lab experiencing so much success from using this new tool, to the point where even their patents have substantially increased. So, where did the data come from, exactly? This simple question caused MIT to investigate the matter, and they quickly realized that Toner-Rodgers was full of shit. They published a press release that didn’t name Toner-Rodgers but did say that they have “no confidence in the provenance, reliability or validity of the data and…no confidence in the veracity of the research contained in the paper.” They state simply, “The author is no longer at MIT.”
It’s short and to the point and drama-free, which is why I am SO, SO glad that there still exists a lively and gossipy scientific community online to share the juicy details. Because the details are, in fact, completely bonkers.
On the same day as MIT’s press release, a materials scientist named Ben Shindel posted a newsletter describing some of the red flags that people really should have noticed before. For instance, as I alluded to earlier, there are no firms that fit the profile of the anonymous lab in the research, though he lists four as the closest possibilities: 3M, Dupont, Dow, and Corning. Put a pin in that. Shindel says he’ll be very embarrassed, though, if any real company was behind this data. He thinks the data MUST have been made up out of whole cloth, and he lists a lot of very good reasons for this. But I simply MUST skip ahead to his postscript, where he found a social media post pointing to a complaint lodged against the author in question, Toner-Rodgers, for domain name infringement, filed by….drumroll please…CORNING INCORPORATED. This document asserts that on January 15th of 2025, Aidan Toner-Rodgers registered corningresearch.com with Squarespace and did so in bad faith with the intent to deceive people into thinking it was the domain name of the actual material sciences firm. The court found in favor of Corning and turned the domain over to them.
Now, here’s where we can only make logical deductions from the facts at hand: Corning is one of only a few firms that even came close to fitting the anonymous lab in Toner-Rodgers’ paper; January is around the time that MIT was asking him where he got his data; he only had a “coming soon” parked on the site, so maybe he only needed the domain for something else, like, say, an email address? That’s the deduction Shindel made, writing “It’s possible he was using the domain name to send fake emails to himself, or to generate pdf files at plausible-sounding urls, to show his advisor.”
All of this brings me to a post written a few days later from Andrew Gelman, a statistics professor at Columbia who hilariously titles his piece “If only Arxiv required researchers to sign at the top rather than the bottom of the page, none of this would’ve happened.” Gelman calls out a lot of statistical red flags in Toner-Rodgers’ paper, which I won’t get into here but if that kind of thing interests you, I love that for you and please go to the transcript for this video to click through and read it all in full. I’ll just skip to a choice quote from Gelman:
“It’s kind of amazing for someone to have put so much effort into (allegedly) faking an entire study and then get sloppy at that last bit. Maybe he should’ve faked all the raw data so as to ensure internal consistency of his results. I kinda wonder where all these numbers came from. Maybe he used a chatbot to produce them? It would be kind of exhausting to construct them all from scratch.”
After reading a number of these criticisms, I have to say that it is my personal opinion that that’s exactly what Toner-Rodgers did. He wanted to get economics fame, fortune, and (let’s be honest about the most likely goal) a professorship with tenure at a prestigious school, and he did it by completely fabricating a “study” about how “AI” is so great for science. Why wouldn’t he get a large language model to just generate the data he needed? Why on earth would he, who obviously thinks AI is just great, NOT use AI to do the boring work of coming up with fake numbers?
Two more fun quotes from Gelman. First, he goes through what would have been necessary to actually get away with a fake paper, concluding that “the only way to produce a truly convincing fake experiment is . . . to do the experiment for real. But that would take a lot of work!” Which echos the point that scientists have made for years about the idea that Stanley Kubrick helped NASA fake the moon landing. In the 1960s, we had the technology to send men to the moon but it actually would have been way more difficult to fake it, a fact that is probably going to fade into nonexistence as “AI” allows any moron to fake anything if the people viewing it are also morons.
Finally, he speculates as to what’s next for Aidan Toner-Rodgers, suggesting that he can just get a job at Duke. As a long-time fan of Fark and their Duke Sucks tag, this amuses me greatly.
That said, my cynical side says he’s probably correct. The world is built for well-connected white boys. My guess is that he’ll switch to using his middle name and only one of the hyphenated last names and get a 6-figure job working for some tech company’s AI division.
