FDA Clearance of Modified Devices Relies Heavily on Weak Studies, Surrogate Endpoints

The evidence used to support significant changes in the design or performance of high-risk medical devices rarely comes from high-quality clinical studies, according to the results of a new analysis.

In the last 10 years, the approval of modifications to many medical devices, such as coronary stents and implantable cardioverter defibrillators (ICDs), has been based largely on studies that are nonrandomized or unblinded, with many of the trials using surrogate endpoints like percent diameter stenosis, report investigators.

“This means lower-quality data often supported changes to high-risk devices that are modifications to previously approved devices,” Sarah Zheng, MD (University of California, San Francisco), and colleagues reported online August 15, 2017, in the Journal of the American Medical Association.

High-risk medical devices, defined by the US Food and Drug Administration (FDA) as those that “support or sustain human life, are of substantial importance in preventing impairment of human health, or which present a potential, unreasonable risk of illness or injury,” are assessed via the premarket approval (PMA) pathway.

These devices can undergo changes in design over time and manufacturers must submit supplementary applications for any changes that affect safety and effectiveness.

“When we first started looking at device approval, I was surprised to learn that even for the high-risk devices, there wasn’t a lot of high-quality data supporting their safety and effectiveness before they get to market,” senior investigator Rita Redberg, MD (University of California, San Francisco), told TCTMD. “Over the years, there have been some very high-profile recalls, such as ICD leads, and other kinds of devices. Looking at those, we noticed that they had gone through a supplementary approval.”

For example, the researchers note there have been more than 5,800 supplements for 77 original PMAs for cardiac implantable electronic devices. “We’re talking about a lot of modifications,” said Redberg.

Look Back at Past Decade     

In their analysis, the researchers specifically looked at data supporting 78 panel-track supplements approved between 2006 and 2015. Panel-track supplements are one of six different premarket approval (PMA) pathways and are used when a manufacturer wants to make a “significant change in design or performance of the device” or wants to gain a new indication. The panel-track supplementary pathway requires clinical data to support the change.

We’re talking about a lot of modifications. Rita Redberg

Of the 78 panel-track supplements, 62 were submitted to gain a labelling change/new indication while 14 were submitted based on changes to the device’s design, components, or specifications. In total, 53% of the supplements were for cardiovascular devices, 19% were ophthalmic products, and 12% were devices used in general and plastic surgery. The mean number of studies supporting each panel-track supplement was 1.1, and the mean time between submission and FDA approval was 355 days.

In total, 83 studies supported the panel-track supplement submissions. Regarding the quality of these data, though, researchers found significant flaws. Just 45% were randomized clinical studies, and only 30% were blinded. Of the 41 cardiovascular studies, only 39% were randomized clinical trials and just 15% were single or double blinded.

Regarding the types of endpoints used in the 83 studies, there were an average of 1.8 primary endpoints per study, with the type of analysis explicitly stated for 57% of the 150 primary endpoints identified. Of these, 26% were superiority analyses, 14% were noninferiority analyses, and 58% were comparisons against an objective performance criterion. Of the primary endpoints, 30% were composite endpoints and 81% were surrogates.      

“In order to know if something is due to a change in the device, the best data is going to come from a randomized controlled trial,” said Redberg. “Blinding is a particularly big issue for devices because there is such a powerful placebo effect when you do a procedure, which is how most devices get implanted. Without a blinded trial, you don’t know if any of the effects you’re looking at are due to the placebo effect or the device.”

Redberg highlighted the case of renal denervation in the hypertension field. Until a randomized controlled trial with a sham procedure was conducted, experts believed renal denervation would result in large reductions in systolic blood pressure. The enthusiasm for renal denervation evaporated, though, when SYMPLICITY HTN-3 was published showing that the intervention was no better than a sham procedure at reducing systolic blood pressure in patients with resistant hypertension.

Buzzword Is Innovation

Regarding the “ethics” of doing a sham-controlled trial with medical devices, Redberg turned the argument around, questioning whether it was ethical to implant a device into a patient without gold-standard evidence that the product works as advertised. For the vast majority of medical devices evaluated via the panel-track pathway, Redberg said a randomized controlled trial with adequate blinding should be required.

“Unless we’re talking about some lifesaving device and there’s no other treatment, I think we really need to consider that we’re putting these devices into patients permanently,” she said. “We really should have good data on safety and effectiveness.”

A spokesperson for AdvaMed, the global trade organization representing medical device companies, told TCTMD via email that there are ethical and practical reasons why a randomized controlled trial, or a double-blinded study, might not be appropriate or necessary for PMA supplements. They added the FDA has the authority to demand any new data it deems necessary before allowing a device, or modified device, onto the market and these decisions are based on the agency’s experience with similar products, some of which have been commercially available for years.

“Device innovation is an iterative process and the information included in a PMA supplement does not stand alone, but is built on the comprehensive data that FDA reviewed favorably to approve the original PMA application,” according to the AdvaMed spokesperson. 

Redberg said the current buzzword in medical device circles is “innovation,” adding that the primary concern appears to be getting these products onto the market as quickly as possible. The argument behind the urgency is that patient lives are at stake, yet “most of these devices aren’t lifesaving devices,” said Redberg. “We have the time to show that they’re effective before they’re actually put onto the market.”

Former Commissioner Weighs in on Delicate Balancing Act

In a second article published in JAMA, Huseyin Naci, PhD (London School of Economics and Political Science, England), and colleagues evaluated the preapproval and confirmatory clinical trials of 22 drugs granted accelerated approval, the vast majority of which were indicated for cancer treatment. While there were 30 preapproval studies supporting the 24 indications, 14 indications were based on studies that included just a single-intervention arm.

Additionally, only 19 of 39 confirmatory studies were completed a minimum of 3 years after approval while postapproval requirements were completed—and showed efficacy—in just 10 of the 24 indications. For eight drug indications, the confirmatory studies were not completed more than 5 years after FDA approval via the accelerated pathway.

In an editorial, Robert Califf, MD (Duke University School of Medicine, Durham, NC), said the two studies raise concerns that US regulation is too lax in not requiring traditional randomized controlled trials for supplemental design modifications for devices and for the postmarket evaluation of drugs approved through the accelerated approval pathway. Yet on the other hand, he noted, the accelerated pathway is appreciated by patients and their families, and it reflects the views of patient advocates, academics, and regulators when other options aren’t available or an inadequate for life-threatening disease.

A “sweeping overhaul” is coming to the FDA, Califf adds, noting that as electronic records, information storage, and analytic methods continue to improve, “there are no technical limitations that would prevent tracking a marketed drug or device throughout its life cycle.” He notes that the National Evaluation System for Health Technology is currently being developed, the goal being the synthesis of data from different sources, including registries, electronic records, and billing claims. Interconnected networks will also make it possible to conduct high-quality, low-cost trials and continuously monitor device performance.