Al Grillo/ZUMApress
Last week, Glenn Beck was elated to uncover some data suggesting that the mean things people say about tea partiers—”maybe you’re a conspiracy theorist, or maybe you are somebody who’s crazy, or you’re uneducated, or you’re rude,” as the conservative talker summed it up on his radio show—are just false. Instead, explained Beck and his on-air cohorts, the new evidence suggests that members of the tea party have a better grasp on science, on average, than other Americans.
But in highlighting this, Beck and his gang managed to show that they don’t understand statistics.
Beck and his co-hosts were citing the research of Yale psychology professor Dan Kahan, who had been analyzing some data and found a slight positive relationship between scores on a science “comprehension” scale and identifying with the tea party. How slight? For data nerds, the correlation was .05. Correlations range between 0 and 1, where 1 is a perfect positive correlation, and from 0 to -1, where -1 is a perfect negative correlation. In social science (or any science) a correlation of .05 would be considered “weak” or perhaps even “trivial,” because it is pretty close to zero. (For a very approachable explanation of correlations, see here.)
At the same time, Kahan also found a similarly small relationship, but negative, between science comprehension and being a conservative Republican (-.05). And he found a somewhat larger, but still quite modest, negative relationship between religiosity and science comprehension (-.26). Keep all of this in mind.
Now, back to The Glenn Beck Program. “It turns out there is a strong correlation between scores and [sic] the science comprehension scale and identifying with the tea party,” declared Beck’s fellow talking head Steve Burguiere (A.K.A. “Stu”). But there isn’t. It’s a very weak correlation at best.
Stu appeared to be quoting Kahan, but the researcher had actually written something quite different:
It turns out that there is about as strong a correlation between scores on the science comprehension scale and identifying with the Tea Party as there is between scores on the science comprehension scale and Conservrepub [conservative Republicans]….
(Italics are mine.) What he’s saying is that there’s just as slight of a positive relationship between being a tea partier and comprehending science as there is a negative relationship between being a conservative Republican and comprehending science.
Kahan went on to add:
Again, the relationship is trivially small, and can’t possibly be contributing in any way to the ferocious conflicts over decision-relevant science that we are experiencing.
Beck and gang managed to miss this pretty important point.
What’s the actual takeaway from this study? Tea partiers are a teeny bit better at science comprehension than non-tea partiers—even as liberal Democrats are a teeny bit better at science comprehension than conservative Republicans.
In truth, it’s not surprising to find a small correlation between science comprehension and those who affiliate with the tea party. The tea party is, in significant part, a libertarian movement, and libertarians have been found to outscore both liberals and conservatives on the cognitive-reflection test, which is a measurement of how you fare on tricky math problems.
But whether tea partiers have basic science comprehension and whether they accept what science has to say are two different things. Tea party members score even lower than Republicans when it comes to accepting established scientific knowledge about evolution and climate change. Democrats and Independents score much better.
What’s more, we also know that on the right, more knowledge about science can lead to less scientific acceptance. Kahan’s research itself is a major source of this finding. He shows that as conservatives become more scientifically literate, they also become stronger deniers of climate science.
So, in sum: The Glenn Beck Program, in trying to show that tea partiers are good at science, muffed the statistics. More important still, it missed the point.
Beck himself then ended the segment by veering toward some conspiratorial thinking about academia. To quote from an online write-up of the segment on Beck’s website:
In theory, Kahan could have chosen to hide his results or manipulate the data to prove a different point, but, instead, he openly admitted that while he was surprised by his results, this is the results his research turned up.
“I think this is great. Dan Kahan will be looking for a job [soon],” Glenn said laughing. “I hope you have tenure, Dan, because Yale’s not going to keep you around, dude.”
Update: Kahan has blogged at length about how extensively his research has been misunderstood on the right, writing:
My observation that the size of the effect was “trivial,” and my statement that the “statistical” significance level was practically meaningless and as likely to disappear as reappear in any future survey (where one observes a “p-value” very close to 0.05, then one should expect half of the attempted replications to have a p-value above 0.05 and half below that) was conveniently ignored (indeed, writers tried to add force to the reported result by using meaningless terms like “solid” etc. to the describe it).
Also ignored, of course, was that liberals scored higher than conservatives on the same measure and in the same dataset.
Did these zealots feel the sting of 50,000 logic arrows burrowing into their chests moments after they got done beating on them? Doubt it.
