Why Drug-sniffing Dogs Don’t Work: Racism and the Clever Hans Effect
Support more videos like this at patreon.com/rebecca!
Are police officers allowing their personal biases to influence how they treat the general public and possible criminal suspects? Yes, obviously. Have you not been paying attention?
What’s newsworthy at the moment, though, is the discovery that those personal biases are even screwing up canine units. Police often use drug-sniffing dogs in places like border crossings to identify and apprehend possible smugglers. Dogs are really good at this job: they have about 50x more olfactory receptors in their noses than humans, and the part of a dog’s brain that processes that information is about 40x the size of ours, proportionately speaking.
Dogs can use their noses to find drugs, people, and according to some preliminary research, even cancer cells. Here’s the thing, though: they need to be trained by a human to do that, and after training they have to work with a human to properly apply their training. And that’s where we run into the Clever Hans effect.
Clever Hans was a horse who got famous in the early 20th century for supposedly being able to solve math puzzles. You would ask him or write a problem on a chalk board, like “1+2”, and Clever Hans would stamp his hoof three times. Some of the questions were pretty complex, even, like “If the eighth day of the month comes on a Tuesday, what is the date of the following Friday?”
Clever Hans was usually right, but when researchers studied him more closely, they realized that he was only good at getting the answers right if the person asking the question knew the correct answer, and if Hans could see the questioner. In other words, Clever Hans really was clever, but he was much better at the art of reading a person than at mathematics. Even his owner probably didn’t realize that this was happening, at least in the beginning.
It’s a great example of the importance of double blinding your experiments — making sure that the person running the experiment doesn’t know the desired result, and that the subject of the experiment doesn’t, either. Because it turns out it’s really hard to remain perfectly unbiased, as we’re all constantly giving off subtle hints about what we’re thinking or what we want to happen, and these can unknowingly influence the experiment.
You may already be able to guess how Clever Hans applies to drug-sniffing dogs. The dogs are usually properly trained in how to detect a drug, but when they’re working in the field, they are with a human who they desperately want to please.
A federal appeals court has just ruled that the use of a drug-sniffing dog was legal, despite the findings that the dog in question, Lex, signalled that he had found drugs 93% of the time he was used. His actual success rate at finding the drugs was only 59%. So he was constantly subjecting people to invasive searches, but was barely better than a coin flip at actually finding drugs.
It turns out that Lex’s handler was giving him a reward every time he alerted, whether or not the alert led to a drug discovery. So he learned that alerting = treat.
It also turns out that Lex’s success rate is higher than what was found in a study of Chicago drug dogs, who managed to correctly ID drugs just 44% of the time, or worse than a coin flip. And if you narrowed it down to Latino drivers, the dogs were only accurate 27% of the time. But they’re still used, and the federal courts have given their approval for their continued use, despite the fact that there are no controls in place to prevent the Clever Hans effect from subjecting millions of innocent people to unnecessary search and seizure.
The only scientifically reasonable next step is to either stop using the dogs or better yet, stop using police officers, since an unbiased dog should be able to do their job about 50x better than they can, and we’d probably see a significant decrease in the number of innocent people gunned down without provocation. Also a significant increase in tummy rubs. Everyone wins.