There’s a new preprint out about how tools like ChatGPT might be affecting human cognition. It comes courtesy of MIT Media Lab titled “Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task.”
As I mention every time I talk about a preprint, I’d like to remind you that you should always take results with a grain of salt because it has yet to go through peer review. In this relatively small study, researchers asked 54 subjects to write essays using either ChatGPT, Google search, or nothing at all. They gave them each EEGs while they worked and found that the group that used ChatGPT “consistently underperformed at neural, linguistic, and behavioral levels.”
That’s obviously not shocking: the entire point of ChatGPT is to save you some brain energy, right? But the actually interesting part is that the researchers had the groups go on to write more essays, and they found that with each new essay, the ChatGPT group got lazier and lazier, until their final essays were just copied and pasted from the results of a single prompt.
In one final section, the researchers asked the ChatGPT group to go back and rewrite one of their previous essays without using ChatGPT. The subjects hardly remembered their original work and showed lower brain activity, possibly because they weren’t accessing deep memory. The group that didn’t have any tools were also asked to rewrite an essay but they were allowed to use ChatGPT to do it. The researchers found that their brains still fired on all cylinders and produced good essays, giving hope that there is a decent use case for the tool.
The paper has been submitted to a journal, and so I was going to hold off on discussing it until it passed peer review, but the other day I happened to see a post on the “skeptic” subreddit linking to a newsletter from someone criticizing it–a computer programmer named Greg Fish. In that post, he mischaracterizes the study pretty badly, failing to mention the researchers’ finding that the ChatGPT group is the only one that actually got worse over the course of the study. It’s not just that they showed less brain activity, it’s that they got lazier and lazier and had no ability to remember what they had “written” later. That’s significant, because in school the point of having someone write an essay is often to get them to think deeply about a topic, maybe improve their research abilities, and come away with a better understanding of the topic and how to convey an idea.
Fish also badly maligns Arxiv, the site where the preprint was published, writing that “the whole thing took a strange turn” when “The lead investigator, Nataliya Kos’myna, decided to publish it directly to the arXiv pre-print repository and publicize her findings before a formal peer review process because she found them so startling and alarming, which is often a red flag in academia.
“I say that because academics usually tend to do this if their paper either seems likely to be rejected, or makes serious leaps to conclusions they’re not sure will be given a pass by the reviewers.”
That is simply not true. Yes, Kos’myna told the press that this was her first time publishing a preprint on arXiv, but the site hosts nearly 3 million papers and is a critical tool for researchers around the world. University of California, the Department of Energy, Oxford, Harvard, and MIT are just a few of the hundreds of institutions that regularly submit. There’s even a common phrase amongst physicists: “If it’s not on arXiv, it doesn’t exist.” It is absolutely in NO WAY a “red flag” to publish a preprint there or anywhere else, and it is absolutely NOT done because academics think their papers will be rejected by a peer-reviewed journal. Publishing a preprint is a great way to get immediate feedback from peers to refine the paper and make sure it’s been properly represented. And allowing anyone and everyone to access these preprints gives important data to researchers who otherwise might not be able to afford to get the data from peer-reviewed journals.
“It’s like the backbone for our field,” CERN’s Alex Kohls told Scientific American. “It’s not only a tool for physicists and computer scientists—it has had an impact on the overall scholarly communication process.” For instance, arXiv-inspired preprint servers in the life sciences, such as bioRxiv and medRxiv, have proved invaluable for speeding up biomedical research during the coronavirus pandemic.”
So please don’t take that newsletter seriously or share it with anyone who might end up with the exactly wrong idea of arXiv, preprints, or this particular study.
But I actually agree with the rest of that newsletter on a broader point, which I actually just discussed last month with a good friend of mine who asked me what I thought about “AI” chatbots making kids dumber. She was referring to a series of articles that had recently been published, starting with New York magazine’s piece by James D. Walsh titled “Everyone Is Cheating Their Way Through College: ChatGPT has unraveled the entire academic project.” That essay detailed the experiences of dozens of students various colleges who were using the chatbot to write all of their essays for them and essentially offload all of that silly “thinking” business.”
Walsh cited some ethicists, one of whom said that “Massive numbers of students are going to emerge from university with degrees, and into the workforce, who are essentially illiterate…Both in the literal sense and in the sense of being historically illiterate and having no knowledge of their own culture, much less anyone else’s.” Another agreed, stating, “We’re talking about an entire generation of learning perhaps significantly undermined here. It’s short-circuiting the learning process, and it’s happening fast.”
A week later, Jason Koebler at 404 Media reported that he had already filed dozens of public record requests with schools and state departments of education to better understand how they were dealing with this apparent ChatGPT problem, with his conclusion being that they, well, weren’t. If they took any action at all, it was to bring in consultants who were “pro-AI…to give presentations that largely encouraged teachers to use generative AI in their classrooms. Each of these presentations noted potential “challenges” with the technology but none of them anticipated anything as extreme as what is described in the New York magazine article or as troublesome as what I have heard anecdotally from my friends who are teachers, who say that some students rely almost entirely on ChatGPT to make it through school.”
Finally, Koebler followed that piece up with reports from teachers who contacted him to describe their experiences dealing with chatbots in the classroom, with his conclusion being that “Teachers Are Not OK.” Respondents detailed their frustration with needing to come up with new techniques for teaching and grading and catching “cheaters,” many of which don’t even know or feel like they’re cheating.
And thus my friend asked me if I thought that this was shaping up to be an existential threat of sorts, considering that I’ve been pretty critical in the past of how large language models like ChatGPT make the world a worse place in a lot of other ways, like sucking up a bunch of energy and spreading misinformation and, uh, creating cult leaders. As you do.
But I had yet to comment on whether or not it was bad for the kids, because at that time there was little to no hard data on how chatbots affect kids, and I’ve always been skeptical of claims that new tools and new forms of media will naturally lead to some kind of downfall of society. According to Plato, Socrates felt that the written word was destroying people’s minds, something that we only know about because, well, yeah, Plato wrote it down.
My friend pointed out that in a way, Socrates was correct. Writing things down allowed humans to offload a part of their thinking that was previously occupied with remembering things, and so humans overall got worse at remembering. But in that case, it was likely overall beneficial: we now have a better and more factual record of history AND we have more cognitive energy to perform higher level tasks.
The same can be said about cell phones. When I was a kid, I had at least a dozen phone numbers memorized: my parents’ line, my brothers’ line, my grandmom’s, and a few friends. Today, I have one number memorized: my parents’ line, because they’re boomers who never gave up their landline. I have to ask my own husband what his cell phone number is whenever I fill out emergency contact forms. My brain no longer recalls a dozen 10-digit numbers, but overall that’s perfectly fine because my cell phone does it for me. There IS a slight problem in that if I’m ever in an emergency without my phone, well…yeah I should probably commit my husband’s number to memory. But for the most part, who cares?
This debate over that push and pull of technology has only heightened in recent decades. In 2011, Nicholas Carr published “The Shallows: What the Internet Is Doing to Our Brains,” arguing that neurological research showed “online reading” was lowering our cognition and making it more difficult for people to engage in “deep” reading of literature. I mean, I think that’s what it was about–I never got around to reading it. I did try to read his earlier 4,000-word Atlantic article that inspired it but I ran out of time so I hit up the Wikipedia page, which is only 106 words but before I finished I got bored and noticed that there’s a song about it by a band called I Like Trains, which…I’m sorry, the song is 41 minutes long???? Forget it.
Okay, joke’s aside, the Wikipedia page for the Atlantic article is actually quite extensive and provides a good overview of Carr’s position as well as the corresponding pushback, with some arguing that Carr is being overly deterministic by suggesting that the internet is literally atrophying our brains as opposed to, if anything, tempting us with consuming information in a way that is easier but also shallower, meaning that each individual can ultimately decide for themselves if they’d like to continue putting in the effort to read deeply. To refer back to my earlier point, I don’t actually think I’ve literally lost the ability to remember a dozen phone numbers, I think I just don’t feel the need to. If I make a good decision and sit down and try I’m fairly certain I can retain my husband’s number for emergencies.
So yeah, as I’ve mentioned in many previous videos, there’s not a lot of evidence to suggest that humans are literally evolving to have smaller or less capable brains. We’re just choosing to use them in different ways, which can be good or it can be bad. We may look back and think “Oh, Socrates was being pretty silly with all that whining about the written word,” but that doesn’t mean that all technological advances are necessarily neutral or beneficial for us as a society. It’s fair to say that the internet, for instance, has had some unintended negative consequences, or at least has contributed to some unintended negative consequences. As I mentioned in my video about people using ChatGPT to experience a spiritual awakening, yeah, people have always had psychotic episodes; people have always used religion and spirituality to manipulate others; people have always used new tech to find “God.” But now instead of needing to go outside and actually find people to bilk, a burgeoning cult leader can just reach in his pocket and use social media. And instead of making up his own nonsensical doctrines, he can just ask ChatGPT to tell the masses that he’s the second coming of Pazuzu, or whatever. It’s easier now, and that’s not good.
In my discussion with my friend, that was her ultimate worry: that ChatGPT and its ilk aren’t necessarily comparable to the written word supplanting the oral tradition, because instead of replacing rote memorization, ChatGPT is replacing thinking and learning in general. It’s replacing critical analysis and the active finding and sorting of information, and that is, in fact, bad.
I thought she made a good point, but her point highlighted a concern that I’ve had for a very long time now: that while chatbots are doing that, are replacing critical thinking, they are only doing it because we as a society have devalued critical thinking for decades. And I, myself am a good example.
I was a “gifted” kid. Before my elementary school established a “gifted and talented” program, I had already started school a year early, read books assigned to the next class up, and would need to go to the next grade up to take certain math and science classes. I don’t say this to brag–kids all learn at different rates and I had the benefit of a stable family where I learned to read early and often. And what the entire experience of elementary school taught me was that things should come easily to me, and if I had to work at anything it meant that I was dumb and bad and a failure.
And that left me very cynical going forward in that I refused to really push myself. I knew what I was good at, like passing tests, and I used that to help me get good grades in advanced and AP courses despite never, and I mean NEVER doing the homework. And when it came time to figure out what I wanted to do in college, I decided that since I was a good writer I would go to school to become a…copywriter. Why not a novelist? Because I had read the autobiography of a copywriter in the 1960s who made tons of money doing easy work and he wrote a book in his spare time.
At the time, I thought it was a pragmatic decision: I would go to the best advertising school in the country, work around the clock to graduate early so I’d pay less in tuition and get into the workplace faster, using my degree to get a high paying job and live a good life.
In retrospect, “pragmatic” has turned to “cynical.” I didn’t actually go into college to learn anything. I went in to do what I knew I did best: pass tests and get the hell out.
But when I got to college, I actually became enamored with learning. I paid my tuition by working full time in a magic shop, where I was introduced to James Randi and through him the works of Carl Sagan and Martin Gardner, and I reflected that back by wandering off the syllabus and picking weird and interesting classes that had nothing to do with advertising, like Greek literature and the origins of homo sapiens and structures of the brain as it relates to psychology and the history of heresy and the writers of the Harlem Renaissance. And I remembered that before I got obsessed with always already pretending to be the smartest kid in class, I loved learning. And learning requires that you first be ignorant, that you ask questions, that you embarrass yourself sometimes, and that you actually put in the effort to solve a problem, and that, yeah, you even try to remember something because remembering something is the first step to being able to know how to even use a tool like Google to find the thing you want to learn next.
I got lucky. Because even back then, in the ‘90s and 2000s, the American education system had already morphed into a system that valued cynicism in a student who wanted to succeed. What mattered was scoring high on that test to get into that specific expensive school that would give me that piece of paper that would get me that job that would give me healthcare and a good salary so that I could live a good life.
Of course now we have a tool to exploit that further, and of course students are using it because their education isn’t about being educated any more. And of course people in the workforce are using it, because their jobs are meaningless. DOING the job is not beneficial to them. Having DONE the job and getting paid is the only good result.
Sorry, I know I got a little long winded and personal there. Honestly, back when I made that joke about the 42-minute song, The Shallows? Well, at this point in writing the script I was almost finished. It’s really good, actually!
Anyway, the point is that yeah, ChatGPT probably IS bad for people, because it is replacing learning and critical thought. But it’s exacerbating a problem that was already there: we, as a culture, no longer value learning for the sake of learning. If we could snap our fingers and disappear chatbots tomorrow, that glaring problem would still be there. The only way to truly solve the problem? That’s right: Fully Automated Gay Luxury Space Communism.
