Yet Another Look at How Our Kids Are Really Doing in School

Fight disinformation: Sign up for the free Mother Jones Daily newsletter and follow the news that matters.


So how are our kids doing? I mean, really doing? In particular, how are our black high-school kids doing at math?

A few days ago I showed the results for the Long-Term NAEP math test. This is a version of the NAEP that’s stayed fairly similar over the years so that it’s possible to see long-term trends. But Bob Somerby isn’t buying it. Why not look at the Main NAEP instead, since that’s the standard version of the NAEP that usually gets all the headlines?

There are two reasons. First, the Main NAEP starts in 1990, so if you want to see longer-term trends, it’s useless. More to the point, it’s not even that useful for medium-term trends because there was a major break in 2005: the test changed and the scale changed, from a 500-point scale to a 300-point scale. So what happened between 2000 and 2005? No one knows. There are no official comparisons.

Still, you can do this: look at the change from 1990-2000 and the change from 2005-2013. That should give you a reasonable idea of what’s happened over the past 25 years. When Somerby does this, he gets 6.11 + 5.24 = +11.34 points. That’s a pretty good gain. By contrast, when you look at the Long-Term NAEP scores over that same period, you get a drop of -1 points. That’s a huge difference. What’s going on?

Let’s take a crack at figuring this out. The long-term scores are easy: neither the test nor the scale have changed, so you just look at the numbers and multiply all of them by 3/5 to norm them to a 300-point scale. For the main test, we need to norm the 1990-2000 scores to a 300-point scale and then paste them together with the 2005-2013 scores. The chart on the right shows what you get.

On the long-term test, scores are still down by about 1 point. Nothing much has changed. But on the main test, scores are up by only 1 point instead of 11 points. What happened? Two things:

  • The 6-point increase from 1990-2000 becomes a 3.6-point increase when you renorm it to a 300-point scale.
  • There’s an unrecorded drop of 7.4 points between 2000 and 2005.

Altogether, this shaves about 10 points from the raw 11-point gain. If that’s accurate, it means there’s no mystery. One test is up by a point and the other is down by a point. Since these tests have a margin of error of about one point, that’s close enough to identical not to worry about.

Needless to say, this leaves us with some questions. Is it acceptable to casually renorm scores by simple multiplication? Is the drop between 2000 and 2005 real? Or is it because the test got harder? Why do scores on the main test bounce around considerably while scores on the long-term test stay pretty stable? There hardly seems to be any correlation between scores on the two tests at all.

Almost certainly, experts would be aghast at all this renorming and extrapolation. But I think it gets us closer to the truth. And one way or another, you have to account for that 2000-05 gap. If you ignore it, you’re ignoring what could be a substantial part of the story.

In any case, this is why I think you’re better off looking at the long-term test if you want to see long-term trends. That’s what it’s designed for, and you don’t have to monkey with the data. Either way, though, we end up with pretty much the same story: black test scores (and white scores and Hispanic scores) have been pretty stagnant since 1990 for high school seniors. This doesn’t mean the gains in earlier grades are nothing to celebrate. They are, and reporters should pay more attention to them. In the end, though, it doesn’t matter what the score is in the sixth inning if your bullpen consistently blows big leads. What we care about is how well educated our kids are when they leave school and enter the world. Until our high schools are able to build on the big gains they’re inheriting from middle schools, we’re not going to see any improvement on that score.

POSTSCRIPT: If you want to look at the raw data yourself, there are plenty of ways to do it. However, the following printed reports provide easy access to all of it:

For what it’s worth, two more notes. First, the main test is given to 12th graders. The long-term test is given to 17-year-olds, who are both 11th and 12th graders. Also: since 2000, the two tests have been given a year apart. Neither of these is likely to affect scores or trends in any material way.

WE'LL BE BLUNT

It is astonishingly hard keeping a newsroom afloat these days, and we need to raise $253,000 in online donations quickly, by October 7.

The short of it: Last year, we had to cut $1 million from our budget so we could have any chance of breaking even by the time our fiscal year ended in June. And despite a huge rally from so many of you leading up to the deadline, we still came up a bit short on the whole. We can’t let that happen again. We have no wiggle room to begin with, and now we have a hole to dig out of.

Readers also told us to just give it to you straight when we need to ask for your support, and seeing how matter-of-factly explaining our inner workings, our challenges and finances, can bring more of you in has been a real silver lining. So our online membership lead, Brian, lays it all out for you in his personal, insider account (that literally puts his skin in the game!) of how urgent things are right now.

The upshot: Being able to rally $253,000 in donations over these next few weeks is vitally important simply because it is the number that keeps us right on track, helping make sure we don't end up with a bigger gap than can be filled again, helping us avoid any significant (and knowable) cash-flow crunches for now. We used to be more nonchalant about coming up short this time of year, thinking we can make it by the time June rolls around. Not anymore.

Because the in-depth journalism on underreported beats and unique perspectives on the daily news you turn to Mother Jones for is only possible because readers fund us. Corporations and powerful people with deep pockets will never sustain the type of journalism we exist to do. The only investors who won’t let independent, investigative journalism down are the people who actually care about its future—you.

And we need readers to show up for us big time—again.

Getting just 10 percent of the people who care enough about our work to be reading this blurb to part with a few bucks would be utterly transformative for us, and that's very much what we need to keep charging hard in this financially uncertain, high-stakes year.

If you can right now, please support the journalism you get from Mother Jones with a donation at whatever amount works for you. And please do it now, before you move on to whatever you're about to do next and think maybe you'll get to it later, because every gift matters and we really need to see a strong response if we're going to raise the $253,000 we need in less than three weeks.

payment methods

WE'LL BE BLUNT

It is astonishingly hard keeping a newsroom afloat these days, and we need to raise $253,000 in online donations quickly, by October 7.

The short of it: Last year, we had to cut $1 million from our budget so we could have any chance of breaking even by the time our fiscal year ended in June. And despite a huge rally from so many of you leading up to the deadline, we still came up a bit short on the whole. We can’t let that happen again. We have no wiggle room to begin with, and now we have a hole to dig out of.

Readers also told us to just give it to you straight when we need to ask for your support, and seeing how matter-of-factly explaining our inner workings, our challenges and finances, can bring more of you in has been a real silver lining. So our online membership lead, Brian, lays it all out for you in his personal, insider account (that literally puts his skin in the game!) of how urgent things are right now.

The upshot: Being able to rally $253,000 in donations over these next few weeks is vitally important simply because it is the number that keeps us right on track, helping make sure we don't end up with a bigger gap than can be filled again, helping us avoid any significant (and knowable) cash-flow crunches for now. We used to be more nonchalant about coming up short this time of year, thinking we can make it by the time June rolls around. Not anymore.

Because the in-depth journalism on underreported beats and unique perspectives on the daily news you turn to Mother Jones for is only possible because readers fund us. Corporations and powerful people with deep pockets will never sustain the type of journalism we exist to do. The only investors who won’t let independent, investigative journalism down are the people who actually care about its future—you.

And we need readers to show up for us big time—again.

Getting just 10 percent of the people who care enough about our work to be reading this blurb to part with a few bucks would be utterly transformative for us, and that's very much what we need to keep charging hard in this financially uncertain, high-stakes year.

If you can right now, please support the journalism you get from Mother Jones with a donation at whatever amount works for you. And please do it now, before you move on to whatever you're about to do next and think maybe you'll get to it later, because every gift matters and we really need to see a strong response if we're going to raise the $253,000 we need in less than three weeks.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate