Cardiac Surgery Risk Prediction with EuroSCORE Improved with Updated, Dynamic Modeling

Predicting mortality after cardiac surgery could be improved using different methods of “dynamic modeling” to account for time-dependent changes in risk factors, healthcare improvements, and changes in the patient population, suggest the results of a new analysis.    

In a study that examined different methods for updating the EuroSCORE, which is commonly used in Europe to predict the risk of death following cardiac surgery, researchers report that the progressive overestimation of mortality with the risk-prediction model, also known as calibration drift, can be corrected using several different methods for dynamic modeling.

“The original EuroSCORE is quite an old score and the problem we’ve seen from the beginning, since it was published, is that the performance in terms of calibration and discrimination, especially the calibration, is drifting, which means it's not spot-on even if you have big groups of patients,” lead investigator Sabrina Siregar, MD (Leiden University Medical Center, the Netherlands), told TCTMD. “It seems to deteriorate over time so the problem is only getting worse as the score gets older.”

The EuroSCORE has been shown to overestimate mortality after cardiac surgery, according to one systematic review, while another European study showed the tool progressively overestimated mortality risk between 2001 and 2011. The underlying reason for the calibration drift include changes in the patient risk profile and improved clinical outcomes, according to the researchers. In their paper, published online March 1, 2016 in Circulation: Cardiovascular Quality and Outcomes, they explain that as the population gets older, the indication for surgery expands , and surgical care improves, “the relation between prognostic factors and outcomes may gradually change too.”

“Altogether, the whole situation changes, which means the relationship, or the strength of some of the risk factors, also changes [with] time,” said Siregar. “The result is that the predictive performance of the EuroSCORE gets less over time.” 

To address the deterioration of the EuroSCORE, the researchers tested six dynamic modeling methods to update the risk-prediction calculator using data from one or three consecutive years from the Netherlands Association for Cardiothoracic dataset. These methods included a basic recalibration, re-estimating the individual prognostic risk factor effects, and updating the model using Bayesian learning (prior background information from the EuroSCORE was combined with new patient information to obtain updated estimates of the prognostic risk factor effects), among others.   

Overall, all six methods improved the calibration of the risk model compared with the original EuroSCORE. Discrimination, as measured by the area-under-the-curve, was similar in all methods. In their analysis, the largest driver for calibration drift in the Dutch cohort was the overall decrease in mortality following cardiac surgery and not relative changes in the strength of prognostic risk factors. 

The statistical analysis is more than an esoteric exercise to improve the risk-prediction model. As Siregar told TCTMD, the EuroSCORE, as well as the updated EuroSCORE II, are used to select patients eligible for surgery or other procedures. “The problem is if the calibration is wrong, or is drifting, then you can't really depend on the score to discriminate which patients are better off with surgery or with another intervention. A score if it's used for patient selection should obviously have good predictive performance, or adequate predictive performance. If the calibration is drifting, then the predictive performance is, of course, inadequate.”

Siregar said there are multiple methods for dynamic modeling, but it all depends on the size of the dataset. With very large datasets, such as the 15,000 patients per year undergoing CABG, aortic-valve replacement (AVR), combined CABG and AVR surgery, and other cardiac surgery included in the Dutch database, researchers can perform extensive remodeling, effectively allowing them to re-estimate each of the individual prognostic risk factors that comprise the EuroSCORE.

“Overall, I think all models are good, and all dynamic modeling methods are adequate,” she said. “It just depends on how much data you have. If you have a small dataset, keep it simple.”

The researchers note that some statewide and national registries frequently update their risk-prediction models. The New York State Cardiac Surgery Reporting System revises their risk-prediction model each year, while the Netherlands and the Society for Cardiothoracic Surgery in Britain and Ireland calibrate the EuroSCORE to their databases periodically.

---

Source: 
Siregar S, Nieboer D, Vergouwe Y, et al. Improved prediction by dynamic modeling. Circ Cardiovasc Qual Outcomes. 2016:Epub ahead of print.

Related Stories:

• Poor Mobility Strongly Predicts Mortality in TAVR Patients
• Most Late Cardiac Mortality After TAVR Due to Heart Failure, Sudden Cardiac Death
• Risk Score Predicts Long-Term Mortality After PCI