AI Tool Doesn’t Increase Appropriate Referrals to Angiography, but Might Help Some

Using artificial intelligence (AI) to help decide whether to send patients undergoing stress echocardiography to invasive angiography was not noninferior to standard practice in the randomized PROTEUS trial, but researchers say there is still hope that this technology can have a benefit in low-volume centers.

Ross Upton, PhD (University of Oxford, England), who presented the findings at the European Society of Cardiology Congress 2024 earlier this week, said the trial likely failed due to a low angiography referral rate—only 8% compared with 15% in EVAREST—as well as effects of the COVID-19 pandemic which lowered elective procedure rates.

“Despite the low number of angiography referrals, the results indicate AI augmentation may have utility in low-volume stress echocardiography centers,” Upton said during his presentation. “Clinicians who perform fewer stress echocardiograms a year may have less opportunity to maintain level of expertise.”

Lior Jankelson, MD, PhD (NYU Grossman School of Medicine, New York, NY), who commented on the results for TCTMD, agreed that this is a “reasonable explanation” and the technology “didn't necessarily get the full chance to shine.” Still, the study shows where AI can be transformative in medicine.

”That’s from the perspective of being able to show statistically significant benefit in centers that are low volume,” said Jankelson. “What we really are saying, I think from the start, is that the aim of AI is not to replace physicians, not to replace healthcare providers, but just to make them better.”

Noninferiority Only in Low-Volume Centers

For PROTEUS, Upton and colleagues screened 4,907 patients (mean age 63 years; 45% female) who underwent stress echocardiography at 20 UK centers between 2021 and 2023. The researchers randomized patients to standard decision-making (n = 1,175) or evaluation with an AI-based tool that analyzes LV motion between rest and stress and indicates the likelihood of severe CAD (n = 1,166; EchoGo Pro; Ultromics Ltd; Oxford, England).

About three-quarters of patients underwent stress echo with dobutamine, and contrast was also used in about three-quarters. Fewer than 10% of patients had resting wall motion abnormalities.

After follow-up of 6 months, the intervention arm did not meet noninferiority criteria for the primary endpoint of evidence of severe coronary disease in patients referred for angiography or evidence of an event (AUROC difference -0.09; 95% CI -0.22-0.39).

Eighty-five patients were referred for angiography, with positive predictive values of 75.0% and 69.3% and negative predictive values of 98.0% and 98.2%, respectively, in the control and intervention arms.

Upton pointed out that there were “a very low amount of positives in the trial and a very high amount of negatives.” 

The differences in sensitivity between the control and intervention groups were 9.05% (95% CI -21.8%-39.59%) and -0.58% (95% CI -2.13%-0.94%), respectively.

In subgroup analyses, there were trends toward a benefit for the intervention, but the confidence intervals were too wide to meet noninferiority criteria. For low-volume sites performing 155 to 375 stress echoes annually, however, AI guidance was noninferior to standard practice given that the lower bound of the 95% confidence interval did not surpass -0.05 (AUROC difference 0.25; 95% CI -0.02-0.62).

“Despite the low number of angio referrals, the results still indicated that AI augmentation may have utility in low-volume stress echo centers,” Upton concluded. “Clinicians who perform stress echoes in these centers, we hypothesize, may have less opportunity to maintain the level of expertise versus their higher-volume peers.”

More generally, he added, the results, which are hypothesis-generating, “should inform further trials and may identify areas where AI diagnostics are useful in the patient care pathway.”

Still Hope

Discussing the study after Upton’s presentation, Victoria Delgado, MD, PhD (Hospital University Germans Trias i Pujol, Badalona, Spain), acknowledged the difficulty of executing a trial like PROTEUS not only given the pandemic conditions but also the low referral to angiography overall.

“But I think that the trial adds a further step in the knowledge,” she said. “I think that confirmation of the results in a large population with higher referrals to invasive coronary angiography or CT coronary angiography will help us, and for sure following these patients in much longer follow-up and providing prognostic data will enhance the results of this first step.”

During the panel discussion, Linda Gillam, MD (Morristown Medical Center, NJ), told Upton she shares in “your frustration and disappointment that, unbeknownst to you, changes in referral patterns for angiography and, of course, the pandemic undoubtedly influenced your ability to hit your primary endpoint.”

She further questioned the potential for this technology going forward, positing that repeating the study in a region with a higher rate of angiography referrals might help.

“Pointing this where we can find higher referral rates may see some benefit,” Upton said. “I think a longer follow-up might be warranted. . . . We can't know for sure those ones that didn't receive an angio and didn't have an event in the 6-month time frame did not have severe coronary artery disease. So following them up for a year, I think, could have helped improve the referral rate for angio.  And then finding another method to potentially treat those patients would be very important.”

Moving forward, this type of technology is very promising and might help increase equity in care, Delgado said. “I think that here is perhaps where we need to see how the technique that is valuable to diagnose coronary artery disease can help other labs to gain access to many more patients and to eliminate even the biases.”

Jankelson, additionally, called the findings “optimistic,” especially because a large proportion of stress echoes performed are not in high-volume centers, but rather lower-volume ones that could see true benefit with the addition of AI.

“This is probably worth repeating in a scenario where the clinical benefit could be more easily demonstrated, meaning in a subpopulation that has a high proportion of referral for angio and maybe with the adoption of the new referral statistics post the ISCHEMIA trial,” he said. “If it's designed under these newer circumstances, newer assumptions or hypotheses, there's a very good chance for that type of study to be positive.”