Top Five Signs That Study Is Shady
Note: Some parts of this may have actually originated from the scintillating mind behind BlagHag. We are hivemind, after all. That, or we discussed how to spot bad science journalism together at our workshop on blogging at Skepticon.
If you’re an American and don’t live on the West Coast, chances are that by this point, you are slowly digesting copious quantities of nutritionally-bankrupt food. As a good skeptic, you know that the sleepiness you’re feeling isn’t from tryptophan but is, instead, a result of your own decadence and poor life choices.
The shame you feel about overindulging pales in comparison to the shame you’d feel if you still believed that silly myth.
Except that you probably still believe some sort of silly myth.
We’ve all fallen for a spurious study, faulty fact, or convincingly clever concoction. I know that I have. For that matter, at this very moment, my mouth could very well be triumphantly quoting things that will turn out to be utter bunk — when it isn’t busy partaking in the holiday repast, of course. As someone with a vested interest in Not Being Wrong, however, I’ve started to notice patterns in what often turns out to be bullshit.
This is really more taking issue with faulty reporting than with the studies themselves, but as more and more people write about science online (cough), more and more articles — and especially their SEO-enhancing titles — will overblow the results of studies. Looking at the actual study can clarify what actually is going on rather than what will lead to the most clicks.
4. It’s based on evidence that’s fairly limited in scope.
Like this list is. Oops?
Many studies select from a very particular population, i.e. the willing guinea pigs that are undergrads looking for some course credit and/or cash. College students are a pretty self-selected, if growing, segment of the population.
In other words, if it magically confirms some sort of stereotype that a stereotypical dudebro would promote, it’s probably false (thereby confirming the stereotype of dudebros and “edgy” stand-up comedians?).
Are our furry overlords controlling us through more than just vocalizations? As it turns out, cats aren’t much of a source of brain-control parasites that turn women into crazy cat ladies.
And, once and for all, semen isn’t a boob-saving elixir.
2. It attributes a major phenomenon to something that is far less wide-ranging.
1. It’s being reported on by someone who just plays a scientist on the Internet.
For the record, every debunked study (and “study”) I’ve cited on this list is something that I bought into, at least for a second.
Happy debunking — or digesting, if that’s all you can do for now.