Former FDA Staffer Says Missing Data Across Antithrombotic Trials Cast Doubt on Drug Effects

Published rates of “lost to follow-up” across a host of oral antithrombotic drug trials are far lower than the gaps recently found in an FDA review, a new analysis suggests. In many cases, the rates published in the FDA document exceed the absolute difference found for the primary end point in those trials, calling into question the true drug effect, according to the study authors.

The comparison between FDA and trial data, published online today as a research letter in JAMA: Internal Medicine, should compel regulatory agencies to consider incomplete follow-up every bit as seriously as they do P values as “critical estimates of the reliability of trial results,” write the authors, who were led by Thomas Marciniak, MD, a former Medical Team Leader in the FDA’s Center for Drug Evaluation and Research.

Speaking with TCTMD, the letter’s senior author, Victor Serebruany, MD, of Johns Hopkins University (Baltimore, MD), put it succinctly. “There is a huge discrepancy between what is published in the medical literature for all these glorious papers and the reality,” he said. “The main thing is that the [number of] patients lost in translation is way higher than all the benefit reported in these trials.”

Marciniak, also speaking with TCTMD, was even more blunt: “Quite truthfully I think this is atrocious. We are slaves to the P value in terms of whether we think a trial is successful or not, but P values are only accurate if the data are accurate and complete. If you have sloppy data, I don’t care what the P value is, I have great uncertainty about that trial.”

Absent Patients

Just prior to his retirement from the FDA in December 2014, Marciniak took the step of publishing an independent addendum to the new drug application for edoxaban addressing cancer risks across 23 antithrombotic drug trials. Internally, he also attached this review to all of the already approved drugs covered in his analysis. In it, he compared the earliest last follow-up date as defined by each study, with the last contact date for each patient at which endpoints (beyond just vital status) were ascertained. In Marciniak’s analysis—and in the current research letter—patient follow-up was deemed “incomplete” if this last contact date occurred prior to the earliest last follow-up date defined in the study.

For the current analysis, Marciniak and colleagues pulled numbers on completed follow-up from 21 of the 23 trials considered in Marciniak’s 2014 cancer risk review and compared those with published follow-up numbers in the scientific literature (2 trials from Marciniak’s original review were excluded for not having published follow-up). Drugs included new oral anticoagulant and antiplatelet drugs, as well as clopidogrel in some older trials, for a total of 270,089 patients and a median follow-up duration of 20 months.

Taking numbers from the scientific literature, the mean published rate of loss to follow-up was just 0.4%, rising as high as 2% in the SPS3 trial and falling as low as .005% in ENGAGE.

By contrast, the mean incomplete follow-up rate for the 21 trials in the original Marciniak analysis was 12% and ranged from 2% to 23%, with no correlation between the published and FDA-calculated rates and no correlation with drug discontinuation rates across studies.

“Whereas the published rate of loss to follow-up is usually less than the endpoint rate difference, the FDA-calculated rates of loss to follow-up were substantially greater than the latter differences,” the authors write.

Of note, patient loss-to-follow-up is increasingly on the radar of the experts charged with reviewing drugs for approval. In today’s research letter, authors note that the FDA advisory committee tasked with reviewing the ATLAS trial data, supporting an ACS indication for rivaroxaban, ultimately recommended against the expanded indication. This was in large part due to the panel’s acceptance of a review by Marciniak showing that the loss to follow-up rate in the published paper was 0.3%, while the FDA-calculated rate was 20%. The FDA never granted the ACS indication.

‘The Shakespearean Drama’

“There are 2 numbers which all trials report,” Serebruany observed. “One number ... is drug discontinuation rates, usually present in all papers. And then there is what I would call the Shakespearean drama with regard to what is named ‘loss to follow-up.’” This, he argued, is a catch-all term—for patients who don’t show up for appointments, don’t return calls, whose relatives persuade them to drop out, or can’t be reached for various reasons—but the range of explanations is not necessarily well captured across studies.

“There is a game of words here, because there are many, many more patients who really do not have final outcome data with regards to the trial [than is captured under this umbrella] and the better name for this is ‘unwillingness to complete,’” he stressed.

Serebruany’s theory is that patients deemed lost to follow-up can provide a convenient pool of patient endpoints for sponsors needing to nudge their results in one direction or another—they can be drug discontinuations, or they can more simply be missing. “It’s like adjudication,” he said. “If you don’t need it, you don’t use it, but if you need to apply some creative adjudication then you have this pool.”

Serebruany stopped short of saying that trial sponsors were intentionally rigging results in the antithrombotic drug trials studied, saying he has no proof. “I would need very hard evidence to prove that intention. . . . What we have here is, we can see how [numbers] can be manipulated, but not that they were definitely manipulated,” he noted.

The current research letter does not include data on whether patients lost to follow-up are more common in the investigational arm than the comparator group for the trials, but Serebruany says they have done some of these analyses and the numbers clearly show that more patients are “lost” in the investigational group. He says they hope to publish these findings at a future date.

The Quest for Full Data

Robert Harrington, MD, of Stanford University, CA, who served as a senior investigator for many of the drug trials cited in Marciniak et al’s analysis as well as the past director of the Duke Clinical Research Institute (DCRI), which was the coordinating center for a range of the studies, commented on the research letter for TCTMD.

“FDA as well as trial investigators have grappled with this issue,” he said. “It’s critically important that as we do large trials that are looking at incremental improvements, that we get as much of the data on as many of the patients as is humanly possible. Over the years, as we’ve moved to varying definitions of patient privacy in clinical research, it has become increasingly difficult to get all of the data required in these trials and that’s despite the fact that an enormous amount of effort is spent on this.”

Harrington, however, took issue with any insinuation that researchers or sponsors could be leveraging this situation as a way to manipulate trial results. “If [the authors of the research letter] are trying to create a conspiracy theory out of this, saying that people are deciding which patients are or are not going to be followed, then I would say that’s absolutely ridiculous,” he emphasized.

Harrington said he stands by all of the studies overseen by the DCRI and points out that all of the data were collected and adjudicated in a blinded fashion. “So yes, this is an important topic [and] it’s been incredibly difficult to collect this information, but no, it’s not a global conspiracy to hide these results. ... If Serebruany and Marciniak want to find out how hard this is, they should try and run a trial,” he commented.

Much Ado About ‘Missingness’

During Marciniak’s years at the agency he earned a reputation as an outspoken critic of numerous investigational drugs considered for approval and for encouraging black box warnings by the FDA, most famously the diabetes drug rosiglitazone as well as angiotensin receptor blockers, which he long maintained are carcinogenic. In recent years, Marciniak has repeatedly raised an alarm about what he saw as a mounting trend towards “missingness” in the final datasets of pivotal trials conducted to support new drug approvals in the United States.

Of note, however, a published memorandum from the CDER makes clear that Marciniak’s 2014 cancer risk review contains a number of “unusual features,” chief among them that it was appended to a range of drugs that had already been approved. Others include the fact that Marciniak was never assigned to conduct the review; the review spans multiple pharmacological classes rather than a specific application; the review was conducted independently, without collaboration or consultation with other specialists including statisticians or pharmacologists; and no senior managers were involved or informed about Marciniak’s review before it was finalized and filed.

Asked about this, Marciniak told TCTMD that his superiors could hardly have been surprised, since he had conducted his own review for “most other” drugs heading through the regulatory process, whether he was asked to do so or not. Moreover, sometime after the FDA’s review of edoxaban, to which Marciniak’s antithrombotic drug was attached, was made public in the early 2015, all preclinical reviews “mysteriously disappeared” from the FDA”s website. When Marciniak’s analysis reappeared a few months later, it was appended to the new drug application for ivabridine. He says he was told they were removed in order to undergo some redacting, then republished.

Contacted by TCTMD, the FDA’s press office denied requests for interviews regarding the JAMA research letter, saying that the agency does not comment on specific studies, even one which, in this case, specifically calls into question the transparency of the FDA’s internal deliberations.

Sources:
1. Marciniak T, Cherepanov V, Golukhova E, et al. Drug discontinuation and follow-up rates in oral antithrombotic trials. JAMA Internal Med. 2016:Epub ahead of print.

2. Marciniak TA. Antiplatelet and anticoagulant drugs: cancer risks. Appended to: Indication statement proposed for the ivabradine label. Application number: 206143Orig1s000 Medical Review(s). Center for Drug Evaluation and Research. December 2014;62-63, 72-73, 76, 81-82. www.accessdata.fda.gov/drugsatfda_docs/nda/2015/206143Orig1s000MedR.pdf. Accessed: January 11, 2016.

Related Stories:

• Rivaroxaban Added to Dual Antiplatelet Therapy Cuts Stent Thrombosis in ACS Patients
• Meta-analysis Looks at Novel Anticoagulants Plus Dual Therapy in ACS Patients