Should Scary Movies Be Rated By the Compounds They Make Humans Emit (Seriously)?

Support more videos like this at patreon.com/rebecca!

Transcript:

It’s Halloween, the best time of year! To celebrate, I’ve been binging spooky movies, like A Quiet Place, The Thing, Get Out, and, well, Halloween. Three Halloweens, in fact. Here’s the thing, though — even though I love love love Halloween, I am in actuality a giant wimp. If you saw a horror movie and left thinking it wasn’t scary at all, you can be assured that I spent the entire runtime as a giant ball of stress.

All of which brings me to a fun new study with a bit of a wacky headline: Testing human volatile organic compounds as tools for age classification of films. Put another way, researchers are claiming that it’s possible to judge what rating a movie should get based on how much isoprene is in the air, since humans release more isoprene when they’re stressed out.

Skeptical? Yeah, you should be. But the results are still pretty interesting. The researchers hooked up a mass spectrometer to a movie theater ventilation unit, allowing them to tell how much isoprene people released while watching one of eleven different movies over the course of 135 screenings. They found that the isoprene levels correlated to various film ratings, so a movie that is rated R, for example, would make humans release more isoprene than a movie that’s rated PG. They used that information to create a model that would be able to categorize the movies accurately based on isoprene levels.

The researchers say that this will improve the movie rating system, since currently it’s “subjective,” while this is an “objective” measurement that you can’t argue with. That’s the point where I pretty much lost it, so let’s talk about what’s wrong with all this.

First of all, they didn’t go into this with the hypothesis that isoprene would correlate to movie ratings. They used the mass spectrometer to test the levels of more than 60 different compounds that humans release, and then they picked out the one that most closely showed what they were looking for. This is something I’ve talked about in previous videos — it’s p-hacking, and it should only be used in preliminary studies if at all. This is obviously a very preliminary study because of that — when you’re looking at more than 60 different compounds, yeah, you have a pretty good chance of finding one that does what you want just by random chance.

So a follow-up will need to be done in which researchers only look at isoprene to confirm that the first study wasn’t a statistical blip, and to make sure that the model they created actually, you know, works and continues to correlate with the currently established ratings system.

Another issue is that this is not, in fact, an “objective” measure. This is something that researchers in other fields rightfully call out in the social sciences — just because you quantified something doesn’t mean that it’s now objective. Humans made the subjective decision to look at stress levels as a way to determine what age should or should not experience a movie. That’s just as subjective as determining that stress may be fine but certain ages shouldn’t be subjected to specific scenes, like bloody violence or graphic sex. And if you’re going to use this to determine ratings, then your test audiences are going to subjectively experience stress in different levels. Isoprene is released in part due to people squirming in their seats — does that mean that a particularly cringey episode of The Office would be rated X? Again, that’s subjective.

It’s a pernicious myth that we can emotion-proof our lives through science. No matter what, with just about every field and every situation, you end up falling back on humans who are making subjective decisions. The secret, at least in this case, isn’t to come up with a better algorithm — it’s to actually study the potential negative effects of scenes of sex and violence on children of certain ages.

And then, you know, to actually stop idiot parents from dragging their 8-year olds to Deadpool 2. Seriously, I saw that happen and those kids are either going to be scarred for life or super fun at parties later. Probably both.