For the past couple of weeks, Bob Somerby has been reviewing Amanda Ripley’s new book, The Smartest Kids in the World. I haven’t linked to any of Somerby’s increasingly acerbic posts about Ripley because I haven’t read the book and can’t vouch for how fair they are. But one point he makes is simple enough: for her international comparisons, Ripley relies entirely on a single test, the PISA, on which American students do relatively poorly. She ignores others with longer pedigrees, like the TIMSS, on which Americans do fairly well.
Ripley apparently has some arguments about why PISA is a better test, and I can’t really offer an assessment of that—though, like Somerby, it’s hard not to suspect that part of the motivation is a desire to tell an alarming story about how poorly American kids are doing. However, it turns out that, in fact, Ripley doesn’t always rely solely on PISA. On at least one occasion, when she’s praising the improvement that Minnesota has made in math scores, she merely refers to a “major international test”:
Ripley never names the international tests to which she refers in this passage, not even in her endnotes, which run 35 pages….Here’s the rest of the story:
In each case, Ripley is referring to Minnesota’s performance on the Trends in International Mathematics and Science Study (the TIMSS). In 1995 and in 2007, Minnesota participated in the TIMSS as a stand-alone entity….Minnesota’s fourth graders did score quite well on that TIMSS math test in 2007. Minnesota’s eighth graders did a bit less well, but they outscored most foreign nations too.
Having said that, please note a key point:
In this passage, Ripley accepts Minnesota’s performance on the TIMSS as a marker of the state’s elite status in math. And yet, all through the rest of her book, she completely ignores the TIMSS.
That’s odd, all right. It’s almost as if Ripley has a story she wants to tell, and cherry picks whatever statistics help her tell it. For the record, TIMSS (despite its name) also tests reading these days, and it turns out that American kids in general—not just Minnesotans—did pretty well in the latest round of testing: 9th out of 56 in math, 10th out of 56 in science, and 6th out of 53 in reading. For some reason, though, you never hear about that. After all, everyone, both liberals and conservatives, has their own educational hobbyhorses, and it’s a lot easier to promote them if you tell an alarming story of educational decline. But the truth is different. If you look at all the evidence—TIMSS, PIRLS, PISA, NAEP, and other metrics—the story is rather more mixed and nuanced. America continues to do a poor job of educating its low-income kids and its black and Hispanic kids, something that’s especially inexcusable given the increasing evidence that these children are far behind their peers even before they get to kindergarten. On the other hand, American kids more broadly are (a) doing better over time and (b) doing fairly well compared to kids in other countries. Like it or not, that’s the story.
If you’re interested, the latest TIMSS results are below. I originally posted these in December, but it might be worth seeing them again.
UPDATE: Mike the Mad Biologist has some more technical critiques of PISA here. Among other things, it turns out there was a sampling error in the U.S. administration of PISA that overrepresented low-income children.
