On the 12th anniversary of 9/11, Dylan Matthews decides to find out whether we’ve learned much of anything about fighting terrorism. Luckily, a trio of researchers produced a broad review of the counterterrorism literature in 2009. Unluckily, they didn’t find much to review:
The first problem the review identifies is that barely any of the terrorism literature even tries to answer questions about effective counterterrorism. “Of the over 20,000 reports regarding terrorism that we located,” the authors write, “only about 1.5 percent of this massive
literature even remotely discussed the idea that an evaluation had been conducted of counter-terrorism strategies.”
They found 354 studies that did, however. Further culling left them 80 studies that could be reasonably said to evaluate the effectiveness of counterterrorism measures. Of these, only 21 of those 80 studies “appeared to at least attempt to connect an outcome or effect with a program through a minimally rigorous scientific test.” Of those 21, only 10 met the Campbell review’s methodological standards. Three of those were medical studies dealing with the effects of bioterrorism, leaving seven for the review to consider.
But wait! It’s even worse than that. Not only did they find only seven relevant studies—which is probably less than the number of studies of LOLcats in popular culture over the past decade1—but those seven studies were all basically negative. None of the counterterrorism strategies studied actually reduced terrorism.
In fairness, it’s possible there are classified studies we don’t know about. It’s also worth pointing out that supposedly rigorous academic studies aren’t the be-all and end-all of human knowledge. It’s perfectly reasonable for us to take actions based on our best intuitions about how our fellow human beings react to various carrots and sticks.
Nonetheless—and even granting that this is a difficult area to study—this is a pretty remarkable finding. You’d think that testing our intuitions about what works and what doesn’t would be of far greater interest that it is. I guess we’d all rather just blather and toss bombs around instead.
1After I wrote this, I got curious. Are there more studies of LOLcats than of counterterrorism strategies? That depends on your definition, but at the very least it’s a close call. A quick search of Google Scholar turned up an awful lot of citations for LOLcats. Among them were “Wants moar: Visual media’s use of text in LOLcats and silent film,” “I @m teh 1337 h@xx0r: A closer look at Internet Englishes and their sociolinguistic implications,” “I Can Haz an Internet Aesthetic?!? LOLCats and the Digital Marketplace,” and “I Can Has Cultural Influenz?: The Effects of Internet Memes on Popular Culture.” Among other things, this demonstrates that scholars of popular culture all apparently think they’re a lot cleverer than they really are.
