Advertisement

FDA and NIH let clinical trial sponsors keep results secret and break the law

Science investigation of ClinicalTrials.gov reveals that federal promises to enforce trial transparency have fizzled

Conceptual illustration. People are walking around in lab coats and suits. Two are in line to put files in a file cabinet marked required. Two officials are sitting nearby, seemingly not paying attention
DAVIDE BONAZZI/SALZMAN ART

For 20 years, the U.S. government has urged companies, universities, and other institutions that conduct clinical trials to record their results in a federal database, so doctors and patients can see whether new treatments are safe and effective. Few trial sponsors have consistently done so, even after a 2007 law made posting mandatory for many trials registered in the database. In 2017, the National Institutes of Health (NIH) and the Food and Drug Administration (FDA) tried again, enacting a long-awaited "final rule" to clarify the law's expectations and penalties for failing to disclose trial results. The rule took full effect 2 years ago, on 18 January 2018, giving trial sponsors ample time to comply. But a Science investigation shows that many still ignore the requirement, while federal officials do little or nothing to enforce the law.

Science examined more than 4700 trials whose results should have been posted on the NIH website ClinicalTrials.gov under the 2017 rule. Reporting rates by most large pharmaceutical companies and some universities have improved sharply, but performance by many other trial sponsors—including, ironically, NIH itself—was lackluster. Those sponsors, typically either the institution conducting a trial or its funder, must deposit results and other data within 1 year of completing a trial. But of 184 sponsor organizations with at least five trials due as of 25 September 2019, 30 companies, universities, or medical centers never met a single deadline. As of that date, those habitual violators had failed to report any results for 67% of their trials and averaged 268 days late for those and all trials that missed their deadlines. They included such eminent institutions as the Harvard University–affiliated Boston Children's Hospital, the University of Minnesota, and Baylor College of Medicine—all among the top 50 recipients of NIH grants in 2019.

The violations cover trials in virtually all fields of medicine, and the missing or late results offer potentially vital information for the most desperate patients. For example, in one long-overdue trial, researchers compared the efficacy of different chemotherapy regimens in 200 patients with advanced lymphoma; another—nearly 2 years late—tests immunotherapy against conventional chemotherapy in about 600 people with late-stage lung cancer.

Other leading NIH grantees did only slightly better in Science's analysis based on data collected from the TrialsTracker website of the University of Oxford, which automatically mines information from ClinicalTrials.gov. The University of Texas MD Anderson Cancer Center and the Mayo Clinic both failed to report results on time, or at all, in about two-thirds of their trials. Yale University failed to do so in 84% of its trials. NIH's own institutes also had a bad record. They are directly responsible for reporting results when they sponsor studies done by agency staff or some grantees, and the top four NIH institute sponsors, taken together, reported results late or not at all in more than six of every 10 trials Science looked at.

Contacted for comment, none of the institutions disputed the findings of this investigation. In all 4768 trials Science checked, sponsors violated the reporting law more than 55% of the time. And in hundreds of cases where the sponsors got credit for reporting trial results, they have yet to be publicly posted because of quality lapses flagged by ClinicalTrials.gov staff (see sidebar).

Although the 2017 rule, and officials' statements at the time, promised aggressive enforcement and stiff penalties, neither NIH nor FDA has cracked down. FDA now says it won't brandish its big stick—penalties of up to $12,103 a day for failing to report a trial's results—until after the agency issues further "guidance" on how it will exercise that power. It has not set a date. NIH said at a 2016 briefing on the final rule that it would cut off grants to those who ignore the trial reporting requirements, as authorized in the 2007 law, but so far has not done so.

Advertisement

Missed deadlines

Among more than 4700 clinical trials examined by Science, less than 45% had their results reported early or on time to ClinicalTrials.gov.
undefined: NaN 150631.6% Not reported undefined: NaN 113223.7% undefined: NaN 213044.7% Reported ontime or early Reported late
(GRAPHIC) N. DESAI/SCIENCE; (DATA) CLINICALTRIALS.GOV, VIA TRIALSTRACKER

Many scientists who conduct clinical trials, and their sponsors or funders, have downplayed concerns about late or missing results in ClinicalTrials.gov. Researchers, doctors, and patients can instead learn about trial outcomes from peer-reviewed publications, they say. But thousands of trials are never published, particularly when they find treatments ineffective, history has shown. ClinicalTrials.gov also uses a common format, allowing relatively easy comparisons of results across trials that journal articles rarely make possible. Doctors, researchers, and potential trial participants rely on the site, to judge from its 215 million monthly page views.

Deborah Zarin, a physician at Brigham and Women's Hospital and Harvard who headed ClinicalTrials.gov between 2005 and 2018, says the Science findings show failures of the research culture, FDA, and NIH. "If this was a priority for the leadership of NIH, then they could ensure that high-quality, timely reporting happened all of the time," says Zarin, an NIH-paid research consultant for the database. "You can set up processes so trial reporting is an expectation. You can't pass ‘go' and collect $200 until this is done."

Zarin, who works in a program to advance clinical research, adds that the problem persists because "reporting to ClinicalTrials.gov is frequently seen by sponsors, funders, and trialists as an annoying administrative and perhaps legal burden, not a scientific imperative. Human nature being what it is, people follow the requirements when forced to do so."

NIH and FDA officials do not seem inclined to apply that pressure. Lyric Jorgenson, NIH deputy director for science policy, says her agency has been "trying to change the culture of how clinical trial results are reported and disseminated; not so much on the ‘aha, we caught you,' as much as getting people to understand the value, and making it as easy as possible to share and disseminate results." To that end, she says, ClinicalTrials.gov staff have educated researchers about the website and improved its usability.

As for FDA, Patrick McNeilly, an official at the agency who handles trial enforcement matters, recently told an industry conference session on ClinicalTrials.gov that "FDA has limited resources, and we encourage voluntary compliance." He said the agency also reviews reporting of information on ClinicalTrials.gov as part of inspections of trial sites, or when it receives complaints.

McNeilly declined an interview request, but at the conference he discounted violations of ClinicalTrials.gov reporting requirements found by journalists and watchdog groups. "We're not going to blanketly accept an entire list of trials that people say are noncompliant," he said. Such determinations require "nonpublic information" submitted to the agency by trial sponsors. In response to Science's findings, a spokesperson said an absence of posted results on ClinicalTrials.gov did not mean a trial sponsor has broken the 2007 law.

Yet that law and the 2017 final rule detail only a few exemptions that would allow trial sponsors to withhold results on the basis of nonpublic information. The very few registered trials that qualify for those exemptions are not flagged as violators by TrialsTracker or in Science's analysis.

Congress approved the creation of ClinicalTrials.gov in 1997, after allegations that patients were harmed because companies withheld evidence showing their medicines were ineffective or hazardous. A widely cited case involved the GlaxoSmithKline antidepressant Paxil (paroxetine). According to legal filings and a report in The BMJ, the firm held secret data showing that in clinical trials the drug was ineffective and caused suicidal thoughts in teenagers, yet encouraged doctors to prescribe it for young people.

Registration was only required initially for trials of treatments for serious or life-threatening diseases. But the 2007 law, the Food and Drug Administration Amendments Act, required sponsors to register a much broader range of trials within 21 days of enrolling the first patient, and to post summary results, adverse events, and other data to ClinicalTrials.gov within 1 year of collecting the last patient data. Although many trials, such as industry-sponsored early-stage evaluations of drug safety, are exempt from reporting, about 326,000 have been registered, and results have been posted for more than 40,000.

Yet until 2015, even the most active investigators at clinical research institutions treated the law more as a suggestion—not surprising given that the government enforced no penalties and did not publicly identify violators. A report on the news website STAT by this author and Talia Bronshtein first drew significant attention to specific trial sponsors—companies, government agencies, universities, and individuals—that routinely ignored reporting requirements. It sparked immediate improvement, according to NIH. (Those same authors documented some of that improvement in a 2018 STAT article.)

At a 2016 press briefing, NIH and FDA rolled out the final rule, aimed at boosting even greater compliance with the 2007 law. It took effect in January 2017, with first deadlines for results, and ostensibly enforcement, 1 year later. Then–FDA Commissioner Robert Califf said it would thereafter "be pretty hard to hide that you are doing a clinical trial or hide the result." FDA, he vowed, was finally prepared, if necessary, to enforce the daily $10,000 penalty for noncompliance allowed under the law. (Adjusted for inflation, that figure recently rose above $12,000.)

"I don't think anybody wants to be on the wall of shame," NIH Director Francis Collins said at the press event, promising that NIH would publicly flag reporting violations on ClinicalTrials.gov itself.

"We are serious about this," Collins said, threatening for the first time to enforce provisions of the 2007 law that allow NIH to rescind funding to grantees who violate the statute. "It's hard to herd cats, but you can … take their food away," he said. "This is about maintaining the trust that we have with participants in clinical trials. … If we fail to live up to that expectation, then that is an ethical failure."

Three years later, TrialsTracker conservatively estimates that FDA could have collected more than $6 billion in ClinicalTrials.gov penalties so far. The agency has yet to demand a single dollar. And despite more than 2600 trials for which results are overdue or were filed late, NIH has yet to withhold a single grant as a result or post a single violation notice on ClinicalTrials.gov. No "wall of shame" exists.

"Public-facing websites run by the government should be accurate. That's not asking much," Senator Chuck Grassley (R–IA), who advocated for the 2007 law, wrote in an email after reviewing a summary of the Science findings. "It's a question of basic management and agency competence. The government has a duty to police its work product, especially because the public trusts .gov websites will be accurate and reliable."

To physician Ben Goldacre, who directs the Oxford program behind TrialsTracker, "The lack of urgency is really troubling."

Reporting problems

Science analyzed ClinicalTrials.gov records of all clinical trials with results legally required to be reported between 18 January 2018 and 25 September 2019. The chart, covering trial sponsors with 15 or more results due in that window or reported early, shows that some results deposited were not posted due to quality lapses. It also highlights that pharma's record has been markedly better than that of academia and the federal government.
A graphic depicting academic/nonprofit, federal government, and industry clinical trial sponsors; their total trials; whether results were reported on time or early, late, not at all, or whether there were quality problems; and average days late
(GRAPHIC) N. DESAI/SCIENCE; (DATA) CLINICALTRIALS.GOV, VIA TRIALSTRACKER

In a recent article in The BMJ, Goldacre and colleagues highlighted a long-running MD Anderson trial as an example of what's at stake when clinical research results go AWOL. Started in 1999, the trial tested a specialized hormone therapy in patients who had surgery for prostate cancer and faced a high risk of recurrence. The treatment might reduce that risk significantly, but other trials suggested it had serious side effects, including reduced bone density, sexual dysfunction, and greater risk of diabetes and cardiovascular events. When Science collected the ClinicalTrials.gov data, the MD Anderson trial's results were overdue by 1 year and 8 months without explanation, and no journal appears to have published them. Doctors and prostate cancer patients weighing the most appropriate treatment have been left in the dark.

Stephen Hahn, who served as chief medical officer at MD Anderson until last month, when he became the new FDA commissioner, was unavailable for comment. An MD Anderson spokesperson said the center "believes in transparency," and is making "every effort to comply" with trial reporting rules, but did not respond to a question on the missing cancer trial data.

Mayo, Yale, the University of Minnesota, Baylor, and Boston Children's, which have similarly poor reporting records, all said via email that they, too, were committed to fulfilling ClinicalTrials.gov requirements. Mayo said it deployed dedicated staff to assist researchers. Yale noted it had registered hundreds of trials exempt from reporting requirements and added a layer of review to help ensure compliance. The University of Minnesota recently created a monitoring system and has made rapid progress on reporting, according to a spokesperson. Baylor said it planned to centralize trial monitoring, as "a top priority." Boston Children's said it was "committed to achieving 100% compliance." It submitted data for one long-overdue study after being contacted by Science, but four others remain in apparent violation as of the end of 2019.

The slow, apparently forgiving regulatory approach favored by NIH and FDA will never force such organizations into full compliance, some advocates of clinical trial transparency say. "In an era when every restaurant is obliged to publish on its front door the hygiene rating in their kitchen, we're seriously not saying whether a trial that costs millions of dollars has broken the law and its obligation to … patient participants by failing to report its results?" Goldacre asks. "That seems like extraordinary special treatment for clinical trialists."

Select institutions have made serious efforts to comply. Twenty big pharma companies met all reporting requirements under the 2017 rule and some major academic centers improved sharply compared with data collected in 2017 (as detailed in the second STAT investigation). Memorial Sloan Kettering Cancer Center, Duke University, and Johns Hopkins University—poor performers in 2017—complied with the law in nearly all their registered trials covered by the new rule. Johns Hopkins added staff to track and assist on reporting and to identify "problem records." And it enlisted university executives to crack down on recalcitrant investigators, according to Anthony Keyes, a clinical research manager there.

But such good performance shouldn't be an exception, Harvard's Zarin says. "Further public accountability of the trialists, but also our government organizations, has to happen. One possibility is that FDA and NIH will be shamed into enforcing the law. Another possibility is that sponsors will be shamed into doing a better job. A third possibility is that ClinicalTrials.gov will never fully achieve its vital aspirations."

This story was supported by the Science Fund for Investigative Reporting.

*Update, 17 January, 6:30 p.m.: Ben Goldacre, head of TrialsTracker, and colleagues have published a similar ClinicalTrials.gov analysis with comparable conclusions today in The Lancet.

Related story

Gaming the system

By Charles Piller

ClinicalTrials.gov relies on the organizations that fund or run clinical trials to register studies and report accurate results on time. Many miss the legal reporting deadlines (see main story). But even those that comply often do so with substandard effort. They face no penalty: Sponsors are credited with fulfilling their legal obligations on the date they file the faulty data.

Deborah Zarin, who headed the National Institutes of Health data site from 2005 to 2018, recently reported in JAMA Internal Medicine that typical lapses in reported results included showing data on more participants than were enrolled and providing inconsistent units of measurement—rendering the results incoherent. ClinicalTrials.gov analysts reject such results until they pass basic quality-control standards—a process than can take many months. Science has found that one of every seven trials credited with reporting results shows no posted data as a result of quality-control reviews. Results for many other trials were posted only after multiple rejections.

Similar quality issues often delay an earlier step: registering a trial at the outset. According to Zarin, ClinicalTrials.gov has historically rejected half of all initial registrations because of a basic flaw: unclear primary outcome measures—what a trial seeks to examine. Often, rejections stem from "sloppiness or lack of rigor," she says. But she has heard of companies that want to appear to comply with ClinicalTrials.gov's legal mandate but deliberately file shoddy trial registrations—likely to be rejected and therefore delay public posting—to temporarily protect information they regard as proprietary.

Jennifer Miller, an ethicist at Yale University who created the Good Pharma Scorecard to rank drug companies on ethics and transparency, says some trial sponsors go even further. In a recent study in The BMJ, Miller and colleagues examined 19 drugs approved by the U.S. Food and Drug Administration (FDA) in 2015, all from large companies. In only 10 cases were all the trials provided to FDA for its decisions registered on ClinicalTrials.gov. When studies aren't registered, a failure to report results is impossible to detect. "It's easy to game the system," Miller says. "Just don't register your trial."

"Minimal compliance with the law should be a given," she adds. But trial sponsors should do more than just go through the motions of registering their studies and posting results, Miller says. Evidence-based medicine demands complete and careful reporting: "Ethics is not minimal compliance with the law."


Support nonprofit science journalism

Help News from Science publish trustworthy, high-impact stories about research and the people who shape it. Please make a tax-deductible gift today.

Donate
1800
1801
1802
1803
1804