Site Investigators Missed Clinical Events in GLOBAL LEADERS

In an international pragmatic study testing a strategy of shortened dual antiplatelet therapy (DAPT) for patients who receive a drug-eluting stent, nearly one in five clinical outcome events were missed by site investigators, according to an analysis from the GLOBAL LEADERS Adjudication Substudy (GLASSY).

Overall, the diagnostic accuracy of clinical events reported by site investigators was modest, while the concordance was generally weak between investigator-reported MIs, strokes, bleeding events, and stent thromboses and those captured by the formal clinical event committee (CEC). 

“We did see that relying exclusively on investigator-reported events is very suboptimal, as events were reported by investigators which were not confirmed by the central process and, also vice versa, that the central process detected a considerable number of events which were not reported originally by the investigators,” said senior author Marco Valgimigli, MD, PhD (Cardiocentro Ticino, Lugano, Switzerland).   

Their findings, with Sergio Leonardi , MD (University of Pavia and Fondazione IRCCS Policlinico S. Matteo, Pavia, Italy), as lead author, were published February 4, 2021, in Circulation: Cardiovascular Quality and Outcomes.

Based on these results, the researchers say they believe a systematic process of adjudication is feasible within a pragmatic trial and should be routinely implemented to standardize and independently assess the effects of a new treatment. To TCTMD, Valgimigli suggested a hybrid approach for randomized trials, one that relies on a central process for scrutinizing the case report forms (and on asking the investigator to provide relevant documentation) rather than relying on costly and resource-intensive site visits and collection of source documents.

“Investigators themselves will be more careful in reporting or not reporting an event if they know that there will be central scrutiny,” he said.  

Mohammed Meah, MBBS (University of Edinburgh, Scotland), who wasn’t involved in the study, pointed out that it’s never been proven that CEC-adjudicated events are the “correct” clinical outcomes. For that reason, he takes slight issue with the latest analysis in that it treats CEC-adjudicated events in GLASSY as the gold standard. Clinical medicine is subjective, he said, and it can be argued that trial investigators—those physicians on the front lines taking care of the patient—would be in the best position to make decisions about events.

In clinical practice, “you don’t have a coterie of four cardiologists and a radiologist looking at every patient who’s been discharged” to determine if the investigator-reported outcome is correct, said Meah.

GLASSY: Trial in a Trial

GLOBAL LEADERS, which was conducted at 130 sites in 18 countries, tested two DAPT strategies in patients undergoing PCI with a biolimus A9-eluting stent (BioMatrix, Biosensors International): 75-100 mg aspirin daily plus 90 mg ticagrelor twice daily for 1 month followed by ticagrelor monotherapy for 23 months versus DAPT with either 75 mg clopidogrel (for stable CAD) or 90 mg ticagrelor (for ACS) for 12 months followed by aspirin alone for 12 months. As reported by TCTMD, no difference between the two antithrombotic strategies was seen for the primary endpoint of all-cause mortality or nonfatal Q-wave MI, and there was no difference in BARC 3 or 5 major bleeding.

The pragmatic trial was unique in that while outcomes were investigator reported, study leadership included a formal CEC at the top 20 enrolling sites. In 2019, researchers published data based on the CEC-adjudicated events, the GLASSY analysis, and showed that ticagrelor monotherapy reduced the composite endpoint of all-cause death, MI, stroke, and urgent TVR compared with conventional DAPT.

“The need for pragmatic trials comes from the high costs of running good quality research in today’s practice,” said Valgimigli. Revenues from pharmaceutical/device companies are decreasing and nobody can sustain the current research costs with the exception of doing the minimum of getting products passed regulatory hurdles. Moreover, sophisticated regulatory trials are designed to show the best possible treatment effect but do not reflect clinical practice. Physicians need ongoing studies to understand how a drug works in the real world or when it might not work best.

“Pragmatic trials allow for continuous research in a sustainable format so that clinicians can afford it without being tied to companies,” said Valgimigli.

In this newest analysis, the researchers wanted to assess the accuracy of clinical endpoint ascertainment—MI, bleeding, stroke, and stent thrombosis—by site investigators. They used the CEC-adjudicated events as their standard and assessed the sensitivity, specificity, positive and negative predictive value, and diagnostic accuracy of investigator-reported events. To do so, they assessed study “triggers,” which was any outcome evaluated by the CEC. This included all investigator-reported events as well as potential outcomes picked up by case report form-based algorithms that were not reported by the investigators. 

Overall, 2,632 triggers were identified and 98.3% of those identified had enough evidence to be adjudicated by the CEC. In total, 87.9% of triggers reported by the investigators were adjudicated as outcome events while 87.2% of triggers not reported by the investigators, but captured in the case reports, were adjudicated formally. Among clinical stent thromboses reported by investigators, just 75.2% were confirmed as events by the CEC. The CEC confirmation rate was higher for MI, bleeding, and stroke. For triggers not captured by investigators but picked up in case reports, MI was adjudicated by the CEC in just 65.1% of cases.

Specificity and negative predictive values were low to very low for all investigator-reported clinical event types and this led to very limited diagnostic accuracy, according to investigators. When using CEC adjudication as the standard, the diagnostic accuracy of investigator-reported events ranged from 59% for stent thrombosis to 77% for bleeding. Diagnostic accuracy was 70% for both MI and stroke, respectively. Concordance between investigator-reported and CEC-adjudicated was moderate for bleeding events, weak for MI and stroke, and minimal for stent thrombosis.

As to why investigators miss events or report “triggers” that turn out not to be events, Valgimigli said they might not be familiar with specific event definition or they may not simply have the time to focus on the event of interest. There is a risk of bias, too, in open-label studies. Sometimes study nurses might be forced to make the call on whether a certain outcome truly constitutes a clinical event, he added.

Train and Incentivize Site Investigators

In an editorial, Sanket Dhruva, MD (University of California, San Francisco), said the “generally weak concordance with CEC findings” suggests CEC adjudication can play an important role for improving the accuracy of randomized trial results over investigator-reported endpoints. The “imprecision of effect estimates introduced through investigator reporting could obscure the true effect of the intervention,” he writes. “Trials may inaccurately meet noninferiority criteria or inaccurately fail to meet superiority criteria.”   

Dhruva states there are strategies to help bring greater confidence to pragmatic randomized trial designs. Physicians in these studies should receive additional, ongoing training and support in outcome reporting, or trials might include a midstream evaluation of events by CECs, such as after a prespecified number of investigator-reported events. Incentivizing physicians for their time and effort should also be a priority, he writes, because doctors in busy clinical practices can be overwhelmed by participation in a pragmatic trial.

“Without support to better align research with clinical care, the notoriously nonstop pressures of healthcare delivery will continue precluding meaningful clinician engagement,” writes Dhruva.

Meah, who along with senior author David Newby, MD, PhD (University of Edinburgh), and others published an editorial last year in the Lancet on clinical endpoint adjudication, said that when trials are designed, and later criticized, it’s quite easy to come down on those that don’t adjudicate clinical events. In some types of studies, though, it simply isn’t warranted.

For example, he highlighted SCOT-HEART, a trial that aimed to establish the benefit of CT angiography in routine clinical practice. In that trial, the primary endpoint of coronary heart disease mortality or nonfatal MI was collected through health records data, which was a better way of gathering events because the trial was intended to assess the benefit of CT on National Health Service resources. While there might be some disagreement between CEC-adjudicated and investigator-reported or hospital-recorded events, effect size estimates tend to remain relatively unchanged. 

“While I completely understand there’ll be some patients who were miscoded or weren’t coded correctly, we just don’t think that margin of error is large enough to detract away from the conclusions that can be drawn from hospital episode statistics, particularly since they’re getting more and more accurate as time goes on,” Meah told TCTMD. While the results may be a little less accurate and the confidence intervals wider, the effect estimates of investigator-reported trials will probably be about the same, he said.   

The bottom line, Meah told TCTMD, is that trials using CEC-adjudicated or investigator-reported outcomes each have strengths and weaknesses. In regulatory drug trials or studies testing new technologies, such as devices, where investigators or patients are unblinded, endpoint adjudication with a CEC is an appropriate option, said Meah. Standardized endpoint adjudication is also helpful in international trials where information is recorded differently across borders. He noted that when drugs are tested in randomized controlled trials, these results can be at odds with real-world clinical practice and this discrepancy might be the result of strict standardization of clinical outcomes with adjudication.