Discarded syringes lay near near train tracks in Philadelphia.Matt Rourke/AP
First came the headlines in August that 64,000 Americans had died of overdoses in 2016—a 21 percent jump from the previous year. Then came the news this month that the number of heroin users more than doubled between 2002 and 2016. The two biggest national data sets on drug deaths and use, from the Centers for Disease Control and Prevention and the Substance Abuse and Mental Health Administration, respectively, show a quickly worsening epidemic—one that Heath and Human Services Secretary Tom Price called “heartbreaking.”
Yet the data that these figures are based on is murky at best. The CDC numbers are likely undercounting opioid deaths by thousands, and some estimate that the drug-use data falls short by hundreds of thousands.
Data tracking isn’t a particularly sexy thing to fund—and is often one of the first line items cut in tight budget years, says Keith Humphreys, a Stanford psychiatry professor who advised the Obama administration on drug policy. It may seem extravagant to keep up data systems rather than fund treatment or better law enforcement, he adds, “but that’s actually what they should do—it’s not extravagant because without good data you have no idea where to target your efforts [or whether] you’re making any difference.”
Indeed, overdose numbers determine the funding and programming that state and federal agencies devote to the epidemic. As the Tennessean recently noted, more than 50 federal programs rely on death data amassed by the Centers for Disease Control and Prevention to make policy decisions.
Here’s a primer on what’s wrong with the data we have—and what information we need—when it comes to opioid use and overdose.
Lots of overdoses deaths are “unspecified”
Theoretically, when a person overdoses, the drug that caused it shows up on a death certificate from the local medical examiner or coroner’s office. That information, over time, makes its way to the state and eventually to a national database maintained by the CDC, which publishes the definitive annual overdose numbers used each year by federal agencies, the media, politicians, and more.
But different localities have different procedures for how to record overdoses: Some examiners perform autopsies to see which drugs were in the body at the time of death; some simply list a death as an overdose from an “unspecified” drug. For about 1 in 7 overdose deaths, records don’t specify the type of drug that caused it. This may seem like a trivial technicality, but when Christopher Ruhm, a public policy and economics professor at the University of Virginia, used computer modeling based on things like age, sex, race, and location to predict the proportion of those so-called “unspecified” overdoses that involved opioids, the overall number of opioid overdoses nationally jumped by more than 20 percent—from 28,647 in 2014 to 35,624.
Some states did a better job than others at noting the specifics: In Vermont, virtually all overdoses listed specific drugs. But in Pennsylvania, the opioid overdose rate more than doubled when Ruhm applied his corrections, leading the state’s heroin overdose rate to jump from 32nd- to the 7th-highest in the country.
“If you’re Pennsylvania and you’re applying for the billion-dollar fund that the Cures Act created, you would be in a much better position if you actually knew the extent of your problem,” says Humphreys, referring to last year’s legislation that allotted money for addiction prevention and treatment. “I suspect they got less money than they should have.”
Below are the five states that Ruhm found most severely undercount opioid overdoses:
Local overdose protocols vary dramatically
We know from local studies that the opioid epidemic today is driven by fentanyl, a synthetic opioid 100 times more powerful than morphine, along with fentanyl analogs, or slight tweaks on the original fentanyl molecule. But testing bodies for specific analogs is expensive—it’s cheaper and faster to simply test for the existence of opioids in general. Because lots of medical examiner’s offices don’t test for fentanyl analogs, the national CDC numbers tell us very little about which opioids killed people. That makes a big difference: Recently published data from southern Ohio, where fentanyl analogs were tested, found that in the first two months of this year, 90 percent of overdoses involved fentanyl, and about half involved analogs like acetyl fentanyl and norfentanyl—drugs that we hear about far less often than, say, heroin, which was only present in 6 percent of overdoses.
The lack of standardization—in whether or not to test bodies for specific drugs, and what specific drugs to test for—means that an overdose in one place may be classified differently from an overdose somewhere else. Anita Wadhwani of the Tennessean asked medical examiners in the state about their procedures and found dramatically different answers:
Bradley County Medical Examiner Jeffrey Miller has typically written “polypharmacy overdose” for deaths in which multiple drugs are involved, said Wittmaier, who noted that “lately he’s gotten better at listing specific drugs.” Portis lists all identified drugs on the death certificate. Joe Fite, medical examiner for Giles County, does not. Instead writes “prescription drug toxicity” and notes all specific drugs on a separate form sent to the department of health that does not get included in death data. Medical examiners also vary in noting the manner of death for suspected drug overdoses. In Bradley County, the medical examiner typically categorizes overdose deaths as “accidents” unless there is clear evidence of suicide. In Benton County, they are suicides if lethal drug doses are present. In Hamblen County, Thompson takes the unusual step of classifying the majority of overdose deaths as “natural,” because, he said, “death is a natural progression of the disease of addiction.”
The overdose data is old
The process of getting the death data that we do have—doing the autopsies, amassing local numbers to send to the state, which gathers state numbers to send to the CDC, which then processes national numbers—can take more than a year. Until a month ago, the most recent national numbers were from 2015.
The time lag in death data is problematic because molecular variations of opioids—particularly fentanyl analogs—change every few weeks. Thomas Gilson, the medical examiner of Cuyahoga County in Cleveland, Ohio, is constantly updating its website with detailed local overdose data. Over the past year, he explained, fentanyl has become far more prevalent and is making its way into the cocaine supply, so the demographics of those dying has changed dramatically. “If you can’t get that number nationally, you’re relying then on offices like this to provide data. But is my data applicable to New England? Is it applicable to the Deep South? Who generates that data? Nobody, essentially.”
We have no idea how many people use heroin and fentanyl
The death numbers, as faulty as they may be, are the primary way that local and federal agencies are gauging the scope of the problem. But Dan Ciccarone, an epidemiologist at the University of California-San Francisco, argues that other metrics—like drug use rates and drug supply data—may provide a more complete picture. “We’re following this old paradigm of ‘Counting the bodies as they lie” and trying to determine how bad a problem is based on that,” says Ciccarone. “What is the number of active heroin users in the United States right now? Don’t you think we need to know that number?”
For years, the go-to data on drug use has been from the National Survey on Drug Use and Health, an annual survey run by the federal government. Each year, about 70,000 Americans take the hourlong survey on computers at homes, homeless shelters, military bases, and elsewhere. Like the CDC’s death data, NSDUH is the foundation of programming and funding decisions made by local and federal agencies.
But experts worry that the NSDUH is missing a large portion of regular hard drug users, since it doesn’t survey jails, prisons, hospitals or homeless people on the streets. “If you want to understand what’s happening, you have to get information from your heavy users,” says Beau Kilmer, co-director of the Drug Policy Research Center at RAND, noting that an estimated 80 percent of heroin purchased in the United States is by daily or near-daily users. “Erratic lifestyles don’t lend themselves well to being home at a certain time to take a survey on a computer,” he adds. “There’s more stigma associated with cocaine, meth, and heroin than alcohol and marijuana”
Until a few years ago, there were two other data sets that surveyed populations more likely to be heavy users. The Arrestee Drug Abuse and Monitoring program, known as ADAM, surveyed participants in jails in more than 35 urban counties throughout the country, asking them what drugs they took and how much they bought them for. The answers, which had no affect on the arrestees’ cases, were confirmed with urine tests. But ADAM, which cost $10 million per year, was cut from the Department of Justice’s budget in the mid-2000s. The Office of National Drug Control Policy revived a smaller version of it, but that, too, was cut in 2013. The Drug Abuse Warning Network, a surveillance program that tracked drug-related emergency room visits, was cut from the Substance Abuse and Mental Health Administration’s budget in 2011.
When, in 2014, the White House hired Kilmer and his colleagues to put together a more accurate picture of drug use, they used available data from NSDUH, ADAM, and other data sets to estimate that in 2010, there were roughly 1 million daily or near-daily heroin users in the United States—compared with the 60,000 estimated by NSDUH.
As Kilmer wrote in 2014, “Our nation’s richest source of information about cocaine, heroin and methamphetamine abuse has been terminated. How can we make sensible decisions about treatment funding without knowing how many people are suffering from dependence on drugs and whether the number is rising or falling?”
Money is tight
All of this fuzziness in the data stems in part from lack of local resources. Medical examiner’s offices have been slammed by the epidemic. A White House report from December estimated that the caseload requiring autopsies had increased by 10 percent in a single year. In Connecticut, the autopsy caseload quadrupled in a year. In Ohio, some medical examiners’ offices are struggling to find storage space for all the bodies. Doing rigorous testing of bodies is expensive: The Tennessean found that autopsies in the state cost between $1,600 and $2,000. As one medical examiner said, “If we autopsied every overdose, the county would go broke.”
Meanwhile, many medical examiners’ offices still rely on paper records. “During a war, they know exactly how many people are killed and where, and they’re monitoring the battle field all the time—they have it computer-integrated,” says Humphreys. “This is almost a 19th-century companion to that: We don’t know whats going on, the data’s unreliable, we don’t have any kind of coordinated system.”
HHS Secretary Tom Price seems to understand the need for better data collection: “One piece of HHS’s five-point strategy for combating the opioid crisis is improving our understanding of the epidemic through better public health data,” he said earlier this month. Yet President Donald Trump’s budgets have proposed cutting funding from agencies that monitor overdose and drug use, including the Substance Abuse and Mental Health Administration and the Office of National Drug Control Policy. “He did appoint an Opioid Commission with smart people on it,” says Humphreys. “He has yet to listen to any of them.”
