Major TAVI Studies Have ‘Methodological Issues,’ INTEGRITTY Group Contends

Some of the landmark clinical trials comparing transcatheter aortic valve replacement versus surgery in aortic stenosis may not be completely reliable, say the authors of a new systematic review and meta-analysis.

Among the trials, they say, there are “serious methodological imbalances” that may increase the risk of bias, including differences in the loss to follow-up, rates of additional procedures, and deviations from the randomly assigned treatment.

Lead investigator Fabio Barili, MD, PhD (Harvard T.H. Chan School of Public Health, Boston, MA/S. Croce Hospital, Cuneo, Italy), said that while there may be “simple explanations” for the imbalances, they must be factored in when assessing the strength of evidence comparing TAVI versus SAVR across the risk spectrum.

“We agree that randomized, controlled trials are the best evidence we have, but they’re not gospel,” he told TCTMD. “We have to look at them as scientists and our duty is to create a discussion, to simply put something on the table. If there is no controversy, there is no discussion and there is no progress.”

The paper, which was published this week in JAMA Network Open, is the work of the INTEGRITTY group, a multidisciplinary team of academics with the common objective of critical appraisal of evidence to support better and optimized patient management. 

Barili, a cardiac surgeon, stressed that he strongly believes in TAVI, and said the paper is not meant to cast aspersions on the procedure or trialists. Instead, it is meant to highlight some of the concerns they have when analyzing the major TAVI-vs-SAVR studies.

“We’re focusing on methodological issues simply because starting with the first trials published in high-risk patients, we noticed that there were some issues, some concerns about the selection of patients, patients lost to follow-up, and so on,” he said. “Going forward into the intermediate- and low-risk patients, we noticed that these potential biases were not addressed.”

Based on their assessment of the data, Barili said the “major message is that there is a systematic imbalance in the trials—there’s a difference between the TAVI and SAVR study arms.”

Tsuyoshi Kaneko, MD (Washington University School of Medicine in St. Louis, MO), a cardiac surgeon who was involved in the intermediate- and low-risk PARTNER studies, said the new analysis shows how challenging it is to conduct clinical research. “No matter how good a job you do in a randomized, controlled study, there’s no perfect trial,” he told TCTMD. “This highlights that point. They dug in very deeply about follow-up, among other things, and I think the points they raised are valid.” 

If there is no controversy, there is no discussion and there is no progress. Fabio Barili

Given some of the limitations of even the best research, Kaneko said the new paper is a reminder for clinicians to “dig in when we interpret data from a randomized, controlled study.”

Kaneko pointed out that the PARTNER and CoreValve studies were the first randomized, controlled trials published in the field of valve interventions. And while the studies may have some methodological issues that could lead to bias, he doesn’t believe these issues rise to the level where they challenge the overall validity of the study findings.

David Cohen, MD (St. Francis Hospital and Heart Center, Roslyn, NY), who has also been involved in the major PARTNER and CoreValve studies, agreed that the critiques are on point from a theoretical perspective. In the real world where research is executed, however, it’s not possible to conduct a flawless randomized trial.

“We’ve all been very proud of these trials as they’ve tackled an important question in a reasonably rigorous way, but they’re not perfect,” he said. “These are the practicalities of doing clinical research on human beings.”

Surgery Patients Getting More Things Done

The review is based on eight clinical trials: PARTNER 1A, 2A, and 3; US CoreValve Pivotal High-Risk Trial, SURTAVI, and Evolut Low-Risk Trial; NOTION; and UK TAVI. The purpose, according to the INTEGRITTY researchers, was to determine if randomization protects the TAVI-vs-SAVR studies from biases other than nonrandom allocation. To do so, they quantitatively assessed deviation from the assigned treatment, loss to follow-up, and the receipt of other treatments aside from the aortic valve intervention.

“The most evident point here is with the associated interventions,” said Barili. “Starting with the high-risk patients, we noticed there was a big difference between the TAVI and surgical groups with the rate of associated procedures. In a randomization process, you’d expect the patients to be the same. If patients in one group had a higher probability to undergo coronary artery bypass or mitral repair than the other group, then there is the risk of performance bias because we’ve treated the patients differently.”

No matter how good a job you do in a randomized, controlled study, there’s no perfect trial. Tsuyoshi Kaneko

Overall, the pooled proportion of patients who underwent additional procedures was 10.4%, but this rate differed between the TAVI and surgical groups (4.6% vs 16.5%, respectively). This imbalance was statistically significant. Looking specifically at myocardial revascularization, 4.5% of the TAVI-treated patients underwent the additional procedure versus 10.8% in the SAVR arms, an imbalance that was also statistically significant.

The European Society of Cardiology/European Association for Cardio-Thoracic Surgery (ESC/EACTS) valvular heart disease guidelines state that the higher rate of coronary revascularization in the surgically treated patients would ultimately be protective, but Barili disagrees with that assumption. An adjunctive procedure increases perioperative risk, and while it may lower the future risk of clinical events, that’s not yet proven.

“At the moment we can’t say what’s happening long-term because we don’t have data,” he said.

‘Reasonable Concern’

Cardiac surgeon Michael J. Reardon, MD (Houston Methodist DeBakey Heart & Vascular Center, TX), who led SURTAVI and was involved in the CoreValve clinical trials program, said the issue of concomitant procedures is a reasonable concern, but he views it as a strength of the research, one that should skew the benefit towards TAVI. For example, a surgical patient with chronic total right coronary artery occlusion would also receive a bypass graft but the same patient allocated to TAVI would not under CTO PCI.

“I understand that if you have an AVR/coronary [surgery] that’s an added risk in the STS [risk calculator], but we as surgeons have argued for years that if you need your coronaries done, and you don’t do them, that in itself is a risk,” said Reardon.

A landmark analysis from PARTNER 2A did suggest there was a lower risk of death or disabling stroke between 2 and 5 years among those treated with surgery, and this benefit has been hypothesized to be because more surgical patients underwent coronary revascularization.

These are the practicalities of doing clinical research on human beings. David Cohen

Kaneko said the treatment protocols for SAVR allowed operators flexibility when it came to adjunctive procedures. During screening, researchers attempted to screen out those with moderate or severe mitral regurgitation, as well as those with tricuspid regurgitation, “but if we saw it in the operating room, we were allowed to treat it because it was the right thing to do,” said Kaneko. “Surgeons are always told you should fix everything when you’re in the operating room. There’s not going to be a second chance.”

While there is an argument that the trials should have been an apples-to-apples comparison of isolated valve replacement, the counter argument is that the full benefits of surgery wouldn’t be achieved if operators didn’t take care of other problematic valves or coronary lesions, said Kaneko.    

Like others, Cohen doesn’t think the criticism about additional procedures is fair. The trials weren’t designed as strict comparisons of a transcatheter valve versus a surgical valve, but instead as one strategy with TAVI against the surgical approach. In that way, the studies were constructed to maximize their applicability to clinical practice.

“That allowed us to have a broader, more real-world population with coronary disease, with atrial fibrillation,” said Cohen.

Loss to Follow-up

Despite studies extending follow-up and the data they generate, that leads to the group’s second issue with the major trials: many are losing too many patients. For example, the pooled proportion of loss to follow-up was 4.8%, a figure that increased with time (1.4% at 1 year vs 8.9% at 5 years). In the analysis, though, loss to follow-up was significantly less in the TAVI-treated patients than in the surgical arms. In some of the trials, loss to follow-up was as high as 20% in the SAVR groups at 5 years.

The 2021 ESC/EACTS clinical guidelines have now adopted an age-based approach when it comes to treating severe symptomatic aortic stenosis, an approach that aims to highlight the patient’s life expectancy. In the field, there is a lot of discussion about valve durability and many of the clinical trials are extending follow-up to assess long-term outcomes. The problem, said Barili, is that the TAVI studies were designed to be relative short in duration, and even if follow-up is extended, the sample size may no longer be large enough to make any meaningful conclusions.

We as surgeons have argued for years that if you need your coronaries done, and you don’t do them, that in itself is a risk. Michael J. Reardon

“In higher-risk patients, TAVI is the treatment of choice simply because the durability is not going to be a factor,” he said. “Obviously, if you move down to a patient with 15 years of life left, to pretend that 2 years follow-up can guarantee the durability of your device is not possible. We need to wait to see what’s happening, keeping in mind that many of these trials are 1 or 2 years long and may be underpowered to detect differences at 10 years.”

Kaneko said the differential losses to follow-up are difficult to understand, adding there isn’t really a great explanation for why it’s occurring.

In terms of the issues raised, Cohen said the loss to follow-up is the most important concern with the major TAVI studies. “It is a problem,” he told TCTMD. “The rates are higher than you’d ideally like to see.”

As an investigator, Cohen believes that patients who underwent surgery may have felt less invested in the trial because they didn’t receive the experimental treatment. “At some point in follow-up, they may have felt, ‘I got the standard treatment and I’m doing OK so I’m going to dropout at this point.’ That’s a legitimate issue and unfortunately there isn’t much you can do to fix it,” he suggested.

As for the impact on the long-term trial results, Cohen said it’s difficult to know. “It’s particularly challenging because most of the trials were conducted in the United States, where we don’t have access to good national death registries like in other countries,” he said.

To counter the dropouts, one way to conduct trials is to allow patients to withdraw in different ways, said Cohen. For example, they might opt for a complete withdrawal, but they might also withdraw so that they’re just not being contacted. In that case, they would consent to have their medical records used to track outcomes. Those types of approaches should be undertaken to minimize loss to follow-up, he said.

To TCTMD, Reardon believe the proportions of patients followed long-term is “quite good and well within what would be expected for [trials] of these durations.”

However, as a counter to some of the concerns about loss of follow-up, Reardon pointed to research from the EXTEND study, which attempted to adjudicate clinical outcomes from the US CoreValve Pivotal High-Risk study and SURTAVI using administrative data from the Centers for Medicare & Medicaid Services. In that study, ascertainment of the studies’ primary endpoints using the claims data reproduced the magnitude and direction of the treatment effect, he said.

Dropping Out Before Randomization

Finally, the third issue raised by the INTEGRITTY researchers is that there is a higher rate of deviation from the assigned treatment in the trials. After randomization, more patients assigned to TAVI followed through with the treatment, while more patients assigned to SAVR backed out.

Speaking with TCTMD, the researchers involved in the clinical trials said the deviation from assigned treatment would be expected from studies comparing surgery against a minimally invasive procedure. Because TAVI indications came in waves, starting with the highest-risk patients, those currently ineligible for the procedure may have hoped to get lucky in a randomized trial.

“The way these clinical trials were sold, patients were told they would get the opportunity to be treated with a transcatheter valve,” said Kaneko. “Inadvertently, some patients who had expected to undergo TAVR were randomized to SAVR and wanted to withdraw. We had multiple patients like that. On the other side, patients randomized to TAVR, are they going to withdraw? The likelihood is much, much lower.”

Reardon pointed out that the only trial that didn’t have high rates of deviation from the assigned treatment was NOTION, which was conducted before TAVI valves were widely available.

“If you got assigned to surgery and you didn’t want to have surgery, you didn’t have any other options,” he said in reference to those earliest days. “As we started running through these trials in US, typically once high-risk TAVI was approved, if you were an intermediate-risk patient assigned to surgery in a trial, you just had to find a cardiologist and surgeon who would deem you high risk so you could get a TAVR.” 

No Easy Solutions

Like some of the TAVI investigators, the INTEGRITTY researchers say there is no real solution to the concerns they raised. Moreover, it’s impossible to predict how the identified imbalances even affected the overall treatment effect.

To TCTMD, Barili said the ideal clinical trial would involve isolated aortic-valve treatment with long-term follow-up and complete patient capture, but that’s highly unlikely at this juncture.

“At the moment, we simply need to critically read the literature,” Barili told TCTMD.