Top Five Signs That Study Is Shady
Note: Some parts of this may have actually originated from the scintillating mind behind BlagHag. We are hivemind, after all. That, or we discussed how to spot bad science journalism together at our workshop on blogging at Skepticon.
If you’re an American and don’t live on the West Coast, chances are that by this point, you are slowly digesting copious quantities of nutritionally-bankrupt food. As a good skeptic, you know that the sleepiness you’re feeling isn’t from tryptophan but is, instead, a result of your own decadence and poor life choices.
The shame you feel about overindulging pales in comparison to the shame you’d feel if you still believed that silly myth.
Except that you probably still believe some sort of silly myth.
We’ve all fallen for a spurious study, faulty fact, or convincingly clever concoction. I know that I have. For that matter, at this very moment, my mouth could very well be triumphantly quoting things that will turn out to be utter bunk — when it isn’t busy partaking in the holiday repast, of course. As someone with a vested interest in Not Being Wrong, however, I’ve started to notice patterns in what often turns out to be bullshit.
5. You couldn’t resist clicking on it.
This is really more taking issue with faulty reporting than with the studies themselves, but as more and more people write about science online (cough), more and more articles — and especially their SEO-enhancing titles — will overblow the results of studies. Looking at the actual study can clarify what actually is going on rather than what will lead to the most clicks.
4. It’s based on evidence that’s fairly limited in scope.
Like this list is. Oops?
Many studies select from a very particular population, i.e. the willing guinea pigs that are undergrads looking for some course credit and/or cash. College students are a pretty self-selected, if growing, segment of the population.
3. The results sound like something an “edgy” stand-up comedian would say.
In other words, if it magically confirms some sort of stereotype that a stereotypical dudebro would promote, it’s probably false (thereby confirming the stereotype of dudebros and “edgy” stand-up comedians?).
Are our furry overlords controlling us through more than just vocalizations? As it turns out, cats aren’t much of a source of brain-control parasites that turn women into crazy cat ladies.
And, once and for all, semen isn’t a boob-saving elixir.
2. It attributes a major phenomenon to something that is far less wide-ranging.
Lady slut pills aren’t destroying the environment and feminism isn’t shrinking penises, which is sad news for us here at the Femistasi.
1. It’s being reported on by someone who just plays a scientist on the Internet.
This guy’s an epidemiologist. This dude has Master’s Degree in Mathematical Behavioral Science. This person wrote a three-parter explicitly for laypeople so I’m assuming they aren’t a layperson.
For the record, every debunked study (and “study”) I’ve cited on this list is something that I bought into, at least for a second.
Happy debunking — or digesting, if that’s all you can do for now.
This is a good list, Heina. Thank you for the post! I’ve found #3 in particular is a huge warning sign. If a “scientific” finding conveniently enforces the status quo, odds are it’s fake, and such assertions tend to send my BS meter into overdrive.
Just out of curiosity: what’s this myth about tryptophan that’s been going around? Are the alt-med bunch spreading some woo about the dangers of this lovely protein?
Sadly, the myth is far more mundane than that. People think that there’s enough tryptophan in turkey to make them sleepy when, in reality, it’s usually how much they ate.
Yes, it *is* used as a sleep aid, but saying that your turkey dinner is responsible for doing it, not the calorie intake, is about as accurate as claiming that you can die from alkaloid poisoning, by eating tomatoes (after all, its just “weaker” nightshade). The cause of the myth is, like most “altie med” BS, a complete lack of comprehension as to how/why 800mg of something might have a bigger effect than 2mg of it. Though, of course, the “homeopathy” people would probably suggest diluting your turkey down to the point you can’t even taste it, in order to “enhance” its tryptophan effects. lol
Ah, homeopathy! Now I understand why we get so sick of turkey in the days and weeks after the feast – the less you eat, the stronger the effect!
Also, it would be a stimulant.
You must log in to post a comment.