MI Definitions in SCOT-HEART Prompt Outcry—Then Silence—on Twitter

Sparring over SCOT-HEART resumed this week following a series of anonymous tweets pointing out that the MI definition used in the trial included two ICD codes not typically included in this endpoint.

“The devil is in the details as usual,” an account named Segmentation Fault tweeted. “When I drill down to read the supplements of #SCOTHEART, it turns out that the main outcome of nonfatal MI, which drove the primary endpoint of the trial, includes I24.9 and I25.6. Which are not MI events at all.”

That tweet sparked a maelstrom, drawing out a wide range of cardiologists and academics, both those who have roundly rejected the study findings and others coming to their defense. As previously reported by TCTMD, SCOT-HEART showed a 41% reduction in the endpoint of coronary heart disease death or nonfatal MI at 5 years when coronary CT angiography was used instead of standard care, predominantly stress testing, to guide the management of patients presenting with chest pain. Those findings, presented last year at the European Society of Cardiology (ESC) Congress 2018 and published simultaneously in the New England Journal of Medicine, prompted a debate that shows no signs of settling down.

This week, critics of the trial were quick to jump on the new (tweeted) revelations, heralding them as an explanation for CT’s win or as evidence of some deeper problems with the trial.

“More shenanigans,” Venkatesh Murthy, MD (University of Michigan, Ann Arbor), tweeted. “When is an MI not an MI? Sometimes in SCOT-HEART, apparently.” In later tweets, Murthy suggested that the SCOT-HEART manuscript’s description of methods or reporting of results, and by extension its peer review, were “sloppy,” and called out the ESC guideline-writing committee for failing to notice the inclusion of these ICD codes. “What does that say about us?” he tweeted.

That question drew out David Cohen, MD (University of Missouri-Kansas City), who replied: “It says that no single reviewer (or set of reviewers) can possibly cover everything, and assuming otherwise is naive. So, we need to still rely on the integrity of the investigators. And we need to really scrutinize results that are inexplicable (as has been done here).”

Speaking with TCTMD, Cohen said he, too, was surprised by how quickly the discussion grew “fevered” on Twitter, but also marveled at the platform’s ability to get quick answers.

Indeed, within 24 hours, SCOT-HEART coinvestigator Marc Dweck, MD, PhD (University of Edinburgh, Scotland), had provided the numbers, by Twitter: just two cases were coded as I24.9 (acute ischemic heart disease, unspecified) in the standard-care arm and none in the CT arm. No cases were coded as I25.6 (silent myocardial ischemia) in either arm.

Almost immediately, the squawking and finger-pointing on the social media platform settled down—for now.

Trial by Tweet

Speaking with TCTMD, Dweck expressed surprise at how swift and acrimonious the Twitter discussion had become.

This experience, Dweck said, “does raise some questions about this process of ‘trial by Twitter’ after you publish a randomized controlled trial. There are some advantages to it: in this instance, [Tweeters] actually raised what I think is a very legitimate question and ultimately Twitter got to the bottom of it quite quickly. Had this gone through the traditional peer-review process, that might have taken months, if not longer. So that’s an advantage of Twitter. But what is not an advantage is that someone raises a concern and then people just go off on it, basically, and that concern is amplified and seen to be a major issue. There were tweets going around questioning the integrity of the authors of the study, tweets going around questioning the integrity of the ESC guideline committee, and it really got way out of hand very, very quickly on the basis of, ultimately, a fairly legitimate question.”

The “crazy comments” he saw this week, Dweck said, stem from the “challenging messages” of SCOT-HEART. “It challenges a lot of our preconceptions about how we should be managing patients with chest pain, and it challenges the paradigms that we have,” he observed.

Asked by TCTMD whether Dweck’s response helped quell his concerns, Murthy insisted that event definition and patient inclusion in SCOT-HEART need more scrutiny. “Ultimately, the way events were identified is a serious limitation,” he said in an email. “We really don't know much of anything about these MIs.” He also expressed concern about the types of symptoms used to admit patients into the trial in the first place.

“My feeling is we have a large number of neutral trials and one exception: [SCOT-HEART],” Murthy said. “That exception has a control that doesn't reflect US practice, relatively high event rates (possibly because unstable angina was included), relatively high disease prevalence, and suboptimal event ascertainment.”

SCOT-HEART investigators have a new paper in press, due out next month, that they hope will satisfy some of the critics and answer questions that have been raised on social media, via stand-alone editorials, and through letters to the editor. Moreover, said Dweck, SCOT-HEART investigators have shared their data “widely with study groups all around the world. People have access, there's nothing to hide, and if people ask us a reasonable question we will provide an answer.”

For his part, SCOT-HEART principal investigator David Newby, MD, PhD (University of Edinburgh), told TCTMD he finds it “fascinating” that people continue to find fault with SCOT-HEART. “No trial is perfect and other trials (CT and functional testing trials) have important deficiencies—I would argue they have even greater problems—and these are rarely discussed as much as SCOT-HEART,” he said in an email. “However, SCOT-HEART has given us the clearest answer to the question, hence why it is under fire!”

He continued: “People have often said to me that there is a question of invested interests in functional testing, because many clinicians have put a lot of energy into developing excellent high-quality functional imaging services. Of course, reimbursement is also high for tests like nuclear perfusion imaging, too. So, the results of SCOT-HEART are going to be difficult for some healthcare providers to accept. Functional testing has also been the bedrock of assessing patients with stable chest pain for 40 to 50 years. So, when a test comes along that is clearly better than previous approaches, people struggle with this and are resistant to change. We have seen this throughout history.”

There are many suggestions as to how SCOT-HEART might have been done differently to assuage the concerns now being raised, chief among them interdependent event adjudication, both of MI type and of whether deaths were indeed coronary in nature. Alternatively, Dweck offered, all patients could have undergone CT imaging, with results suppressed in the standard-care arm—such a strategy would have enabled investigators to drill down into the question of whether intensification of preventive medication had reached the right patients in both arms. But blinding to CT, Dweck pointed out, would raise profound ethical questions, particularly if the test had picked up incidental findings, such as cancers that were then suppressed during the course of the trial.

Cohen raised the prospect of a trial with coronary death as the only, unambiguous endpoint. Such studies were done, he pointed out, to establish Medicare reimbursement for screening colonoscopies. No such trial has been done for an imaging test in cardiology, however, including for the screening tests that are currently the standard of care.

Asked whether, in retrospect, he would have done anything differently in SCOT-HEART, Newby was succinct. “No, I would not have done anything differently,” he said. “I think the trial design is robust and the results compelling and clear. Indeed, we are soon to launch SCOT-HEART 2 using the same approach.”

SCOT-HEART 2, which Dweck confirmed has been funded, is looking at a primary prevention population. The recent brouhaha on Twitter, he noted, “has been quite useful from our point of view as well in thinking about this in more detail and about the codes that we included.”