About That New Lead Study….

Fight disinformation: Sign up for the free Mother Jones Daily newsletter and follow the news that matters.


A new study was released recently about the effect of childhood lead poisoning on future academic performance. After reading it, I decided not to post about it, but since it’s getting some attention I should probably explain why. This will take a while, so be patient.

First things first: The basic idea here is uncontroversial. We’ve known for decades that childhood lead exposure reduces IQ, stunts academic development, and leads to lower test scores. But most of the original studies in this area were done a long time ago, when childhood lead levels were much higher than they are now. Blood lead levels are measured in micrograms per deciliter, and kids in the 70s and 80s frequently had levels as high as 20 or 30. Today that’s rare, so this paper focuses on something different: small changes in children who already had fairly low lead levels. For example, what would be the effect of a drop from 4 to 3?

To measure this, they rounded up records for nearly every third-grader in Rhode Island. These records included both blood lead levels in infancy and academic performance later in childhood, which is just what you need. The problem is that you can’t just compare those two things. It’s common knowledge that kids with high lead levels also tend to be poor, have less educated mothers, belong to minority groups, etc. Since all of these things are correlated with poor academic performance, you have to control for them somehow. It’s very difficult to do properly since you can never be entirely sure there isn’t something you haven’t overlooked.

So the authors looked at another variable unique to Rhode Island. Starting in 1997, Rhode Island required landlords to certify their rentals as lead-free. Kids who live in certified housing are likely to have lower lead levels, which means you can compare that to academic performance instead. Unfortunately, you run into the same problem: people who live in certified housing are unlikely to be a random subset. You have to control for different stuff, but you still have to run a lot of controls.

To address this, the authors used an instrumental variables approach. They constructed a remarkably complex variable that models “the probability that a child’s home was certified at the time of birth as a function of the number of certificates that had been issued in their census tract as of their year of birth, as well as family characteristics, and tract, year, and month of birth fixed effects.” After all that, though, they found only small effects:

The estimated effects of lead in these models are strongly statistically significant but relatively small: The column (4) estimates suggest that a one point increase in mean BLLs is estimated to reduce reading scores by .306, and math scores by .193.

So going from a lead level of 4 to 3 raises test scores by less than a third of a point on an 80-point scale. A 3-point reduction—which is fairly large these days—would raise test scores by about a point in reading and half a point in math.

But that’s not the end. There are two ways of measuring lead levels: venous (a standard blood draw) and finger pricks. Venous is more accurate, but finger pricks are more common. The venous measures show a stronger effect from lead exposure, so the authors constructed yet another instrumental variable to take this into account, and that produced a bigger estimate of lead on test scores: about half a point for reading and a third of a point for math.

But we’re not done yet. The authors then generate another instrumental variable, along with all the usual controls, and this produces an even bigger estimate: about one point for reading and 0.4 points for math. In both cases, however, the standard errors are quite large and the correlation coefficients are minuscule. In the case of math, the results are not statistically significant even at the 10 percent level.

This is the point at which I emphasize that I’m no expert in the design of studies like this. Controls are perfectly legit. Instrumental variables are perfectly legit—though you have to be careful not to get over-clever about them. Trying to correct for measurement problems is perfectly legit. And yet, when you put this all together it leaves a bad taste in my mouth. There are lots of controls. The main instrumental variable might be appropriate, but I couldn’t quite convince myself of that. It’s also a very complex instrument, which makes it hard to evaluate. The measurement stuff looks suspiciously like a post-hoc way of generating a bigger effect. It all feels very fragile. And even after all this, the statistical value of the results is weak.

I may be wrong about every aspect of this. It will take a real expert to go through the paper and make an informed judgment. In the meantime, though, I’d take it with a grain of salt. There’s no question that childhood lead exposure reduces academic performance, but for now I’d say I’m skeptical that the effect is as large at low levels as the authors suggest.

WE'LL BE BLUNT

It is astonishingly hard keeping a newsroom afloat these days, and we need to raise $253,000 in online donations quickly, by October 7.

The short of it: Last year, we had to cut $1 million from our budget so we could have any chance of breaking even by the time our fiscal year ended in June. And despite a huge rally from so many of you leading up to the deadline, we still came up a bit short on the whole. We can’t let that happen again. We have no wiggle room to begin with, and now we have a hole to dig out of.

Readers also told us to just give it to you straight when we need to ask for your support, and seeing how matter-of-factly explaining our inner workings, our challenges and finances, can bring more of you in has been a real silver lining. So our online membership lead, Brian, lays it all out for you in his personal, insider account (that literally puts his skin in the game!) of how urgent things are right now.

The upshot: Being able to rally $253,000 in donations over these next few weeks is vitally important simply because it is the number that keeps us right on track, helping make sure we don't end up with a bigger gap than can be filled again, helping us avoid any significant (and knowable) cash-flow crunches for now. We used to be more nonchalant about coming up short this time of year, thinking we can make it by the time June rolls around. Not anymore.

Because the in-depth journalism on underreported beats and unique perspectives on the daily news you turn to Mother Jones for is only possible because readers fund us. Corporations and powerful people with deep pockets will never sustain the type of journalism we exist to do. The only investors who won’t let independent, investigative journalism down are the people who actually care about its future—you.

And we need readers to show up for us big time—again.

Getting just 10 percent of the people who care enough about our work to be reading this blurb to part with a few bucks would be utterly transformative for us, and that's very much what we need to keep charging hard in this financially uncertain, high-stakes year.

If you can right now, please support the journalism you get from Mother Jones with a donation at whatever amount works for you. And please do it now, before you move on to whatever you're about to do next and think maybe you'll get to it later, because every gift matters and we really need to see a strong response if we're going to raise the $253,000 we need in less than three weeks.

payment methods

WE'LL BE BLUNT

It is astonishingly hard keeping a newsroom afloat these days, and we need to raise $253,000 in online donations quickly, by October 7.

The short of it: Last year, we had to cut $1 million from our budget so we could have any chance of breaking even by the time our fiscal year ended in June. And despite a huge rally from so many of you leading up to the deadline, we still came up a bit short on the whole. We can’t let that happen again. We have no wiggle room to begin with, and now we have a hole to dig out of.

Readers also told us to just give it to you straight when we need to ask for your support, and seeing how matter-of-factly explaining our inner workings, our challenges and finances, can bring more of you in has been a real silver lining. So our online membership lead, Brian, lays it all out for you in his personal, insider account (that literally puts his skin in the game!) of how urgent things are right now.

The upshot: Being able to rally $253,000 in donations over these next few weeks is vitally important simply because it is the number that keeps us right on track, helping make sure we don't end up with a bigger gap than can be filled again, helping us avoid any significant (and knowable) cash-flow crunches for now. We used to be more nonchalant about coming up short this time of year, thinking we can make it by the time June rolls around. Not anymore.

Because the in-depth journalism on underreported beats and unique perspectives on the daily news you turn to Mother Jones for is only possible because readers fund us. Corporations and powerful people with deep pockets will never sustain the type of journalism we exist to do. The only investors who won’t let independent, investigative journalism down are the people who actually care about its future—you.

And we need readers to show up for us big time—again.

Getting just 10 percent of the people who care enough about our work to be reading this blurb to part with a few bucks would be utterly transformative for us, and that's very much what we need to keep charging hard in this financially uncertain, high-stakes year.

If you can right now, please support the journalism you get from Mother Jones with a donation at whatever amount works for you. And please do it now, before you move on to whatever you're about to do next and think maybe you'll get to it later, because every gift matters and we really need to see a strong response if we're going to raise the $253,000 we need in less than three weeks.

payment methods

We Recommend

Latest

Sign up for our free newsletter

Subscribe to the Mother Jones Daily to have our top stories delivered directly to your inbox.

Get our award-winning magazine

Save big on a full year of investigations, ideas, and insights.

Subscribe

Support our journalism

Help Mother Jones' reporters dig deep with a tax-deductible donation.

Donate