Twitter, Bias, and Trial Tweaks: ISCHEMIA Invites Debate on Study Designs and Endpoint Combos

SAN DIEGO, CA—Does the use of a primary composite endpoint help or harm when it comes to interpreting modern-day clinical trials? Do softer clinical endpoints, such as recurrent angina, restenosis, or the need for coronary revascularization, help with understanding the effectiveness of a drug or device? Or do they only contribute background noise?  

Last week at TCT 2018, several experts tried to sort through the vagaries of clinical trial design and data interpretation by taking a closer look in the context of the $100 million ISCHEMIA trial, still a few years away from reporting results, in addition to a number of prior trials that have faced their share of discussion.

From an evidence-based medicine perspective, Sanjay Kaul, MD (Cedars-Sinai Medical Center, Los Angeles, CA), argued that use of a composite endpoint helps to estimate the net clinical benefit of therapy and overcomes the limitations of an insufficiently powered trial, but it also poses major challenges. If clinical trialists are forced to use a composite endpoint, the individual components should lie on the same pathophysiologic spectrum, should occur with similar frequency, and be equally meaningful, he said.

“The success [of a clinical trial] should not be concluded if it is driven by a less meaningful component or if there is evidence of a therapeutic disadvantage on a more meaningful component,” said Kaul. In terms of preferred endpoints, Kaul said he values Q-wave MI, disabling stroke, and cardiovascular mortality but conceded that “if I were just to design trials with these endpoints, it would require 30,000 or 40,000 patients.” 

ISCHEMIA’s Many Faces

The pitfalls of composite endpoints have been in the spotlight of late because of the ISCHEMIA trial, a study intended to determine the best treatment course in patients with stable coronary heart disease with moderate-to-severe ischemia on stress testing. How best to help this specific group of patients has been called one of the last big unknowns in cardiovascular medicine. ISCHEMIA is testing whether patients identified as having ischemia on one or more tests, then randomized to an invasive strategy of cardiac catheterization followed by coronary revascularization (PCI or CABG) plus optimal medical therapy, fare better than similarly identified patients managed using a conservative treatment strategy of optimal medical therapy alone.

Earlier this year, the ISCHEMIA trialists drew criticism for changing the primary endpoint from cardiovascular death and MI to an expanded five-component MACE endpoint. As reported by TCTMD, the original trial protocol included a provision that allowed an independent advisory panel to look at the possibility of a prespecified primary endpoint change from CVD death and MI to the expanded five-component MACE in the event it became necessary to preserve study power.

For Kaul, hospitalization for heart failure and unstable angina—two of the five outcomes added to the ISCHEMIA composite—are “softer endpoints” that are clinically relevant, albeit less so than the other three: nonfatal MI, resuscitated cardiac arrest, and cardiovascular death. However, these two endpoints are also prone to bias, Kaul noted. In addition, hospitalization for heart failure is an endpoint that occurs frequently but is not known to be clinically affected by revascularization in patients with stable coronary artery disease.

“When you have increased frequency of an endpoint with less potential for a treatment benefit, all you’re doing is adding noise and your bias will be towards the null, which is very difficult for a superiority trial,” said Kaul.

The ISCHEMIA investigators are hardly alone in their use of a multifaceted composite MACE endpoint, though. As Kaul showed during his talk, a 2017 study looked at the primary endpoint of 140 phase III cardiology trials published in  Circulation, JAMA, the Lancet, and the New England Journal of Medicine. Of these, 22 trials included a dual composite endpoint, 63 trials included a triple composite endpoint, 31 trials used a four-component composite endpoint, and 15 trials had a five-component primary endpoint.

High Stakes, Many Tweets

Defending the switch to the five-component MACE endpoint at TCT, ISCHEMIA coprimary investigator David Maron, MD (Stanford University School of Medicine, CA), said that any time a primary endpoint is changed, questions should be raised. ISCHEMIA is a high-profile trial with a large public investment. “The stakes are high and that’s why there is a lot of fuss,” said Maron.

He stressed, though, that the endpoint change—which was a “reversion” back to the original five-component endpoint—was based on an independent evaluation of the accrued endpoint events. Nothing nefarious, he said, was at play. Maron added they developed “stringent criteria” to define hospitalization for unstable angina and heart failure and that all five endpoints are to be adjudicated by an independent, blinded clinical events committee.

Nonetheless, the endpoint change “unleashed a great amount of controversy and discussion,” said Maron, highlighting the provocative editorial by the ORBITA investigators that accused the ISCHEMIA group of “moving the goalposts into unblinded territory.” The endpoint change also launched a storm on social media, with some stating that ISCHEMIA would no longer be a “definitive” clinical trial, particularly if the benefit of PCI was driven by hospitalizations.

“In the Twitter era, things have changed and we all need to get used to it,” said Maron. “The trials are being reviewed before they are published. There are bits of information floating around that may or may not be accurate. Twitter doesn’t require peer review.”

While the ISCHEMIA investigators have previously admitted to being stung by the criticisms, Maron worries that Twitter’s rapid dissemination of (mis)information can adversely affect participant recruitment and retention and undermine a clinical trial. “Let’s say you’re a patient who is considering going into a trial and you’re aware of the buzz that’s going on in the Twittersphere. Maybe you have second thoughts about participating? If you’re a physician, maybe you have second thoughts about allowing your patient to participate. Twitter needs to be responsible. It can affect the integrity of a trial.”

However, several panelists argued that the quality of medical information could be enhanced if physicians participated in these social media discussions and that censoring the online discussion of clinical trials is not the right approach, nor is it possible. For his part, Stuart Pocock, PhD (London School of Hygiene and Tropical Medicine, England), said Twitter is not unlike democracy. “Most people are ignorant about most things, but if you give them an audience and you give them a vote, you see what on earth happens,” he said. “But the alternative is a dictatorship of the experts, which is another equally dangerous thing.”

Composite Endpoints Popular Out of Necessity?

E. Magnus Ohman, MBBS (Duke University School of Medicine, Durham, NC), who moderated the session, also questioned the use of composite endpoints, noting that clinical trials from the 70s, 80s and early 90s only studied all-cause mortality.

Trials conducted in the past, however, had the luxury of testing therapies when cardiovascular event rates were higher and there were no other effective interventions, responded Kaul. “It was easier to design pragmatic trials answering clinically relevant questions,” he said. “In the last three or four decades, all the low-hanging fruit has been plucked. In order to keep innovation going, we’re looking at incremental treatment effects over and beyond what we’ve observed with other interventions impacting mortality.”   

But there are other ways that trial design can run into problems, he warned. In modern-era clinical research testing pharmacologic and device interventions, trialists have sometimes opted for multipronged endpoints that combine efficacy and safety, which can lead to misinterpretation of the results. For example, he cautioned, when a net adverse clinical events (NACE) endpoint is driven by greater differences in safety compared with efficacy, a “relatively ineffective but safer drug can be made to appear as good or even better than effective drugs.”

Several primary PCI trials, including HORIZONS-AMI and EUROMAX, showed that bivalirudin reduced the primary NACE endpoint as well as major bleeding. In HEAT-PPCI, however, which did not use a “net benefit” endpoint, there was no treatment effect of bivalirudin on the primary major adverse cardiovascular and cerebrovascular event (MACCE) endpoint. HEAT-PPCI, despite being a single-center trial, ultimately prompted a reevaluation of bivalirudin’s role in the setting of acute MI.

Kaul had similar concerns about the 2009 PROTECT-AF trial, which tested the Watchman device in left atrial appendage closure for stroke. This time around, he noted, the primary composite endpoint was driven by reduced bleeding but “not an improved thromboembolic profile.”  

The Role of Sham-Controlled Trials

No session on clinical trial design, particularly one at an interventional cardiology meeting, would be complete without a discussion of the feasibility of sham-controlled trials for medical devices. Last year, ORBITA shook things up when investigators reported no significant improvement in total exercise time or angina symptoms among patients with stable coronary artery disease treated with PCI compared with those treated with a sham procedure. 

Bernard Gersh, MB, ChB, DPhil (Mayo Clinic, Rochester, MN), who argued that sham-controlled trials should be the gold standard for testing new devices, noted that data and safety monitoring boards are critical in such trials to ensure that those exposed to the sham procedure are at as little risk as possible. Pocock, for his part, said that sham-controlled trials a “good idea in the right context.” For example, the sham-controlled SYMPLICITY HTN-3 study was particularly helpful in cutting through the overly positive claims about renal denervation for the treatment of high blood pressure.

Panelist David Cohen, MD (Saint Luke’s Mid America Heart Institute, Kansas City, MO), who was involved in the sham-controlled study of percutaneous transmyocardial laser revascularization (TMR) for the treatment of refractory angina, agreed there is a role for sham trials. With TMR, for example, the unblinded studies suggested significant benefit of TMR for reducing symptoms. 

“This was clearly a procedure with substantial ability to harm,” said Cohen. “When a population that is likely to be affected by the treatment is very large, or the potential for harm from the procedure is substantial, then the stakes are even higher. We owe more to our patients and more to society to do the sham-controlled trials to make sure we don’t persist in harming patients for a long period of time by doing something erroneous.”

Andrew Sharp, MBChB, MD (The Royal Devon and Exeter Hospital, England), who has enrolled patients into six sham-controlled device studies, including SYMPLICITY and ORBITA, said patients had very little issue with entering into such a trial “as long they understood they would end up getting the treatment at some point and they understood the reason behind [the study].”