Improving Echo Appropriateness: Audit and Feedback Intervention Works

Letting physicians know how they’re doing in terms of the appropriateness of the transthoracic echocardiograms (TTEs) they’re ordering appears to improve their decision-making, a randomized trial shows.

An educational intervention that included monthly feedback reports lowered the rate of TTEs deemed “rarely appropriate” according to appropriate use criteria (AUC) from 12.4% to 9.5% among US and Canadian cardiologists and primary care physicians, R. Sacha Bhatia, MD (Women’s College Hospital, Toronto, Canada), report in a study published in the August 29, 2017, issue of the Journal of the American College of Cardiology.

“This is probably one of the largest studies that shows that just giving data to doctors about their performance without any financial incentives attached to them actually does still have an impact on their performance,” Bhatia told TCTMD.

Commenting for TCTMD, David Wiener, MD (Jefferson Heart Institute, Philadelphia, PA), said he was impressed by the effect of the intervention, noting that the rate of rarely appropriate TTEs in the control group was in line with prior studies. “To knock off a quarter of that is quite an achievement in my estimation,” he said.

Although efforts like this one theoretically would be worthwhile to use more widely, there are some issues to consider, added Wiener, a member of the board of directors for the American Society of Echocardiography (ASE). Audit and feedback interventions are labor intensive, making implementation challenging, and it remains unclear whether the beneficial effects wear off after they end, he said.

Bhatia agreed that these types of programs take time and staffing, pointing to the need to train clinical research coordinators on coding the appropriateness of TTEs, which involves going through charts and referring to AUC documents.

“What would obviously be really great is if that process could be automated, using free-text analysis and [artificial intelligence (AI)] algorithms to be able to determine appropriateness based on the chart or based on the electronic medical record,” he said. “If that could be done and the process could be automated, I think that would be really beneficial to the sustainability of a program and significantly reduce the cost of capturing the data in particular.”

Echo WISELY

There has been a growing focus on ensuring more judicious use of medical testing in recent years. Cardiac tests have come under particular scrutiny, and professional societies including the American College of Cardiology (ACC) and others have released several AUC documents to address the issue. In 2011, ACC, along with ASE and other societies, published AUC for echocardiography.

Last year, Bhatia’s group published results from a single-center, randomized trial evaluating an educational intervention for reducing the ordering of “rarely appropriate” TTEs by attending cardiologists that showed a significant benefit.

Such an intervention received a more rigorous vetting in the current trial, called Echo WISELY. The trial was conducted at eight centers—seven in Ontario, Canada, and one in Massachusetts—among cardiologists and primary care physicians providing care in ambulatory settings. The analysis included 179 physicians and 14,697 TTEs ordered between December 2014 and April 2016. Nearly all of the echoes (99.4%) were classifiable according to the 2011 AUC.

Physicians randomized to the educational intervention received an email containing a 20-minute video lecture describing the AUC for echocardiography and detailing common clinical scenarios, instructions for accessing the ASE’s decision-support mobile app and a pdf of the AUC document, and monthly email reports containing summaries of their individual TTE ordering behavior and the appropriateness of the tests.

Volume of TTE ordering did not differ between the intervention and control groups, but audit and feedback reduced the proportion of rarely appropriate TTEs ordered overall and among physicians who, on average, ordered at least 1 TTE per month (8.6% vs 11.1%; OR 0.76; 95% CI 0.57-0.99). Results were similar in the subgroup of cardiologists alone, and did not differ between US and Canadian physicians.

“Although the overall impact of the intervention appeared modest, the potential for significant reductions in low-value cardiac testing, when applied across a large population, is substantial,” Bhatia et al write. “Future research efforts should be directed to large, community-based practices to assess the broad impact and sustainability of the intervention.”

How to Make Sure Echoes Are Appropriate

In an accompanying editorial, Randolph Martin, MD (Emory University, Atlanta, GA), a past president of ASE, says that “although the investigators are to be complimented for extending their very important work in the field of AUC, there are some significant limitations to their findings, but also lessons learned that will be of benefit moving forward.”

He points out that the investigators’ prior research shows that the effect of an educational intervention like this one wears off after it’s stopped and that a small number of physicians in the intervention group accessed a website each month to check their results, as opposed to looking at the email only. In addition, physicians in Echo WISELY ordered a small number of TTEs on average.

To ensure that echoes are ordered under appropriate circumstances, several options can be considered, Martin indicates.

First, he acknowledges the importance of educational efforts by ACC, ASE, and others before highlighting the potential of audit and feedback systems to help make physicians aware of AUC. However, such efforts need to be sustained, he says.

Secondly, any interventions should take aim at the top three to five “rarely appropriate” types of TTEs ordered, he says. “These educational activities should be aimed at those physicians, nurse practitioners, physician assistants, or extenders, and even house staff, who order the highest volume of echoes in both inpatient and outpatient settings (point of care),” he argues.

And finally, Martin underscores the potential of integrating AI and machine learning into electronic medical records. “If companies such as Amazon can know what your online ordering history and preferences are,” he asks, “why can’t similar AI machine-learning algorithms be applied to the ordering of echocardiograms by individual physicians or healthcare providers?”

That’s an idea that got some strong support from Wiener. “I think Dr. Martin’s right on the money in saying that this begs for automation by AI or machine learning to provide physicians with regular reminders,” he told TCTMD.