Biomarkers, New Data, and Revised Methods Boost Accuracy of the ACC/AHA Risk Equations

Since the American College of Cardiology (ACC) and the American Heart Association (AHA) introduced a tool for calculating atherosclerotic cardiovascular disease risk—also called the Pooled Cohort Equations (PCEs)—as part of a suite of prevention guidelines published in 2013, there have been questions raised about its accuracy in certain groups. Now, two new studies have proposed ways to make the PCEs better.

In the first, researchers led by Anum Saeed, MD (Baylor College of Medicine, Houston, TX), showed that the addition of three easily measured biomarkers—N-terminal pro-B-type natriuretic peptide (NT-proBNP), high-sensitivity cardiac troponin T (hs-cTnT), and high-sensitivity C-reactive protein (hs-CRP)—improves discrimination of risk of overall CVD events, including heart failure, in older adults.

And in the second study, Steve Yadlowsky, MS (Stanford University, CA), and colleagues found that using newer data and revised methodology boosts the accuracy of the equations, particularly when it came to estimating risk in black patients.

Having a firmer grasp of an individual patient’s cardiovascular risk has important implications as medicine moves toward more personalized care with an emphasis on shared decision-making, Christie Ballantyne, MD (Baylor College of Medicine), senior author of the first study, told TCTMD.

“The goal is that you would be able to give someone a more accurate estimation of what their risk is for the different types of events and also what therapies are most likely to be useful,” he said, noting that future assessment tools will involve traditional risk factors, imaging, biomarkers, and potentially genetic information. “That’s what we’re moving towards, and we’ve actually made a huge amount of progress since I started in cardiology in terms of what we do for risk factor modification. The next 20 years is going to be, I think, accelerated. We’ll be much better at this.”

Addressing Gaps in Older Adults

Ballantyne said that the PCEs have some limitations when it comes to estimating CVD risk in older adults. Specifically, the equations are not meant for people older than 79, they do not incorporate risk of heart failure hospitalizations, and they look at risk over the next 10 years, a time span that might not be relevant for making decisions in an older population.

In their study, published in the June 5, 2018, issue of the Journal of the American College of Cardiology, he, Saeed, and colleagues looked into whether the addition of biomarkers readily available in the clinic—NT-proBNP, hs-cTnT, and hs-CRP—could improve how the PCEs perform for estimating risk of global CVD, including coronary heart disease, stroke, and heart failure events, over a shorter time period in older adults. They used data from the most recent visit in the Atherosclerosis Risk in Communities (ARIC) study, during which participants were, on average, 75 years old. The analysis included 4,760 participants who were followed for a median of 3.6 years.

In this cohort, Ballantyne said, the PCE alone “wasn’t terribly effective in regards to assessing risk in this older population,” with C-statistics ranging from 0.597 to 0.647.

Each biomarker individually improved performance, but the model with the highest accuracy incorporated all three. Compared with the original PCEs, that model increased the area under the receiver operating characteristic curve by 0.103, continuous net reclassification index by 0.484, and integrated discrimination index by 0.075.

Even a simpler “lab model” that included only age, race, sex, and the three biomarkers had better discrimination than the original PCEs.

Ballantyne said that biomarkers may be especially useful for risk estimation in older individuals.

“When you’re 55 years old, hypertension as a risk factor is very important, but when you’re 75 years old and everybody’s got hypertension, what’s much more important is not whether you have hypertension, it’s whether you have some end-organ damage from hypertension,” which can be picked up with biomarkers, he explained.

Though the approach explored in this study “may be beneficial,” Ballantyne stressed that future studies should validate the utility of the biomarker models, which could be easily integrated into clinical practice, and address issues around appropriate cutoffs for defining high risk when heart failure events are included and how to manage patients identified as high risk.

“Can this be implemented fairly rapidly? Yes. But are we in the position right now to have organized the information to make it clear to physicians and patients how to use the data? This is a step along that process. We’ve got a ways to go,” Ballantyne said.

In an accompanying editorial, Jennifer Robinson, MD (University of Iowa, Iowa City), and Adrian Hernandez, MD (Duke University, Durham, NC), agree that validation of these findings in other populations is essential.

In addition, “Future research should explore whether risk prediction equations that include biomarkers will predict other cardiovascular outcomes such as atrial fibrillation or asymptomatic valvular heart disease,” they say. “Most importantly, further evidence will be needed regarding whether intensifying therapy based on improved risk prediction prevents or improves these outcomes. Ultimately, any treatment strategy will need to be evaluated for cost-effectiveness from a societal perspective.”

Revising the PCEs

Speaking with TCTMD, Sanjay Basu, MD, PhD (Stanford University), senior author of the study led by Yadlowsky, said even though the biomarker study showed improved discrimination, it didn’t address the more important issue of calibration, or how well estimated rates correspond to observed rates.

“If you add a variable, you are almost always going to increase your discrimination,” Basu said. “That’s why so many biomarkers have come and gone over the years. It’s not because they’re too expensive. It’s just because they don’t actually help with calibration; they just help with discrimination, and that’s kind of a statistical overfitting problem.”

In their study, published online June 4, 2018, ahead of print in Annals of Internal Medicine, Yadlowsky, Basu, and colleagues tackle the calibration question by assessing how two changes—inclusion of more recent data and use of updated derivation techniques—affected performance of the PCEs.

Their cohort included 26,689 adults ages 40 to 79 who were free from cardiovascular disease at baseline and who were participating in one of six studies: ARIC, the Cardiovascular Health Study (CHS), the Coronary Artery Risk Development in Young Adults (CARDIA) study, the Framingham Heart Study (FHS) offspring cohort, the Jackson Heart Study (JHS), and the Multi-Ethnic Study of Atherosclerosis (MESA). There was some overlap with the cohorts used to derive the original PCEs, but this new grouping removed the original FHS cohort and added two newer studies (JHS and MESA).

The researchers found that either using newer data or updating derivation techniques individually did not have a major impact on the ability of the PCEs to predict risk of atherosclerotic cardiovascular disease. Putting both together, however, resulted in “substantially improved calibration” and better discrimination compared with the original equations, they say.

Of note, the updated model narrowed differences in how risk was predicted in black versus white people.

The researchers estimated that about 11.8 million US adults who would be labeled as high risk—a 10-year risk of 7.5% or greater—by the original equations would be relabeled as lower-risk by the revised models.

“Clinically, our results suggest that the revised PCEs will reduce overestimation of risk in general and may prevent adverse events, healthcare costs, and inflated expectations of absolute risk and corresponding absolute therapeutic benefit,” the authors write.

Like Ballantyne, Basu said that the findings need to be validated by other research groups before they can be implemented clinically, noting that his group has put all of the statistical code used in the study online for others to access.

What can be taken away from the study right now, however, is that risk scores will need to be continually updated over time to keep them performing well, Basu said. “Every few years we should probably plan in advance to update these equations because of dramatic changes in environment, physical activity, and nutrition, all these other factors that aren’t included that make up the background risk of people for heart disease and stroke.”

Andrew DeFilippis, MD, and Patrick Trainor, MS, MA (University of Louisville, KY), also highlight the need for ongoing work to keep risk scores up to date.

“Risk assessment based on cardiovascular risk factors is a major medical advancement, but continued effort is needed to produce accurate risk assessment tools for specific patient populations,” they write in an accompanying editorial.

“Risk prediction is an evolving science and will require continual updating through the study of contemporary data from various sources, including consortia of traditional cohorts and ‘big data’ from electronic medical records,” they continue. “Yadlowsky and colleagues show us that contemporary cohorts and statistical methods beyond the purview of classical epidemiology are important for accomplishing this goal.”