Data Submitted to FDA in Support of Cardiovascular Devices Often Differ From Those Published in Peer-Reviewed Papers

Premarket approval documents for novel cardiovascular devices often differ substantially from the published results of supporting trials according to a study released online June 10, 2015, ahead of print in the BMJ. Among devices approved in the last decade, discrepancies were noted in trial characteristics, primary endpoints, and safety and efficacy results reported in medical journals following approval. 

“Clinicians might not be aware of the FDA device summaries and so might not critically examine these data,” say Rita F. Redberg, MD, MSc, of the University of California, San Francisco Medical Center (San Francisco, CA), and colleagues. “Thus, for many high-risk devices, clinical trial evidence might never be made readily available to the medical community or might be made available only after a long delay.”

The researchers compared safety and efficacy results of clinical trials reported in the FDA premarket approval documents of 106 cardiovascular devices approved between January 2000 and December 2010 with their respective published results.

Of 177 supporting studies (mean 1.7 per device), only 49% had been published in a peer-reviewed journal. These 86 published studies pertained to a total of 60 devices. The mean time from FDA approval to publication was 6.5 months (range −4.8 to7.5 years).

Of the published studies, 66 were considered pivotal by virtue of being the only 1 included in the summary, a multicenter RCT, or explicitly noted in the FDA summary as being pivotal. Of these, the average time to publication was 7.9 months, and 33% were published before FDA approval. Of those that specified a funding source, all were industry funded. Five of the pivotal studies were found to have been presented as pooled data and not as individual studies when published.  

Among the 86 published studies, 26% had a discrepancy between the number of enrolled participants as it was presented in the FDA summary and that given when it was published. The mean difference in the stated total number was 12.8 patients (a 4% difference), with a maximum difference of 181 patients.

Some demographic information also differed between FDA summaries and published studies. In 11%, the average age was off by more than 1 year, and in 16%, the breakdown by sex differed by more than 1% in absolute terms.

Of 152 possible primary endpoints identified in the FDA summaries, a review of the subsequent published studies found them to be labeled as secondary in 2%, unlabeled in 28%, and not found at all in 10%. When primary endpoints were compared between the summaries and the published data, 45% were identical, while 23% were similar, 11% were different, and 20% could not be compared.

Furthermore, in 1 particular instance in which the researchers noted disparity in the significance of the reported outcome between the summary and the published report—involving the Talent thoracic stent (Medtronic)—it “substantially changed the interpretation of the result such that the device seemed more favorable in the summary than in the publication.”

Mandatory Registration of Trial Data

Dr. Redberg, editor of JAMA Internal Medicine, and colleagues argue that the study, which identifies selective reporting at both the study and outcome level, “underscores a broader problem in the way clinical trial evidence is presented.”

In the case of the discrepancy in numbers of enrolled subjects, the suggestion is “that the composition of the patient population had changed substantially… introducing a possible source of bias in reporting,” they say.

Taken as a whole, the authors assert that available evidence points to the need for “mandatory registration on a public clinical trials platform.” Clinicaltrials.gov is an example of attempts to achieve this, although “recent data show that published trials often have discrepant findings between [that site] and publications,” they say.

Adherence to Ethical Principles

In an editorial accompanying the study, Sidney M. Wolfe, MD, of Public Citizen’s Health Research Group (Washington, DC), notes that the lack of publicly available data from clinical trials violates an important ethical principle of the Declaration of Helsinki, which was developed to regulate research involving human experimentation. Specifically, it states: “Negative and inconclusive as well as positive results must be published or otherwise made publicly available.”

But Dr. Wolfe points out that attempts “to increase the public availability of clinical trial data to prevent the serious public health consequences of overstating benefits and understating risks” has led to industry opposition in Europe and the United States.

In 2012, for example, the European Medicines Agency (EMA) sought “proactive publication of clinical trial data, once the marketing-authorization process has ended,” according to Dr. Wolfe. But after 2 pharmaceutical companies sued the EMA to prevent disclosure, the agency “watered down its original plans,” he says. Likewise, he adds that the Pharmaceutical Research and Manufacturers of America urged the US government to influence the European Union against the data disclosure policy.

But both the study authors and Dr. Wolfe argue that accessibility of the data is needed to maintain integrity of clinical trial research and patient safety. Additionally, clinical practice guidelines might be affected in that they are not “based on complete and accurate information about drugs or devices,” Dr. Redberg and colleagues say.

1. Chang L, Dhruva SS, Chu J, et al. Selective reporting in trials of high risk cardiovascular devices: cross sectional comparison between premarket approval summaries and published reports. BMJ. 2015;Epub ahead of print. 
2. Wolfe SM. Selective clinical trial reporting: betraying trial participants, harming patients: reporting biases found in trials of cardiovascular devices [editorial]. BMJ. 2015;Epub ahead of print. 

Disclosures:

• Dr. Redberg reports serving as on the FDA Circulatory System Devices Panel. 
• Dr. Wolfe reports being the founder and advisor of a consumer organization that campaigns for safer medicines and medical devices.

Related Stories:

• Data Supporting FDA Approval for Cardiovascular Devices Called into Question