Lancet COVID-19 Hydroxychloroquine Study Faces ‘Data Integrity’ Questions

A group of more than 140 scientists, researchers, and statisticians have written an open letter to the Lancet and the authors of a recently published observational study showing that the use of chloroquine, and its newer derivative hydroxychloroquine, increased the risk of mortality and the occurrence of cardiac arrhythmias when used in the treatment of hospitalized patients with COVID-19.

The letter writers, who include James Watson, PhD (Mahidol Oxford Tropical Medicine Research Unit, Thailand), a statistician who first raised concerns about the integrity of the results in a review on Twitter, call for the release of patient data and for the findings to be validated by the World Health Organization (WHO), or at least one other independent institution.

Contacted by TCTMD, one of the Lancet study investigators said that they, too, will conduct an independent review of the data they used for their analysis, which was provided by a private company.

Published May 22, 2020, and reported on by TCTMD along with a wide range of other media outlets, including the New York Times, the Wall Street Journal, and the Washington Post, the study in question focused on 96,032 patients hospitalized with COVID-19 between December 20, 2019, and April 14, 2020, at 671 hospitals on six continents. On multivariable adjustment, use of chloroquine and hydroxychloroquine with or without a macrolide antibiotic was associated with a 33% to 45% higher risk of in-hospital mortality. The risks of ventricular arrythmia were increased by several orders of magnitude.

The observational study has had a “considerable impact on public health practice and research,” according to the letter writing group, which also noted that the WHO paused the recruitment of patients into the hydroxychloroquine arm of the SOLIDARITY trial the day after the study was published. Other regulators around the globe have also pressed pause on their studies of hydroxychloroquine in COVID-19.

“The subsequent media headlines have caused considerable concern to participants and patients enrolled in randomized controlled trials seeking to characterize the potential benefits and risks of these drugs in the treatment and prevention of COVID-19 infections,” according to the letter. “There is uniform agreement that well-conducted randomized controlled trials are needed to inform policies and practices.”

In an email to TCTMD, lead study investigator Mandeep Mehra, MD (Brigham and Women’s Hospital, Boston, MA), said his group used data from Surgisphere, a private company whose president and chief executive officer Sapan Desai was one of the study authors, because of the absence of large, publicly available data sets on hydroxychloroquine or chloroquine and the lack of evidence regarding the safety and benefits of those treatments for hospitalized COVID-19 patients. Like the letter writers, he stressed the importance of conducting randomized controlled trials before any conclusions can be reached.  

“However, results are not anticipated from such [randomized controlled] trials until the summer and, given the urgency of the situation, leveraging the available data set was an intermediary step,” said Mehra.

Some Specific Concerns

The letter raises several concerns about the observational analysis, among them inadequate adjustment for known and measured confounders and the absence of an ethics review for the study. The authors question some of the data from Australia, noting there were too many COVID-19 cases and too many deaths, numbers that are inconsistent with government reports. They also challenge some of the findings from Africa, question the dosing of hydroxychloroquine, and point to unusually small reported variances in baseline variables, interventions, and outcomes between the continents.   

Mehra said he and his co-author had been made aware of a discrepancy in the data reported by the Australian hospitals, which was the result of a hospital in Asia being lumped together with the Australian data set. The updated findings have been submitted to the Lancet and a correction published. Mehra said the academic investigators are also planning on initiating an independent review of the data.

In their letter, Watson et al call on the company to provide details on data provenance, which, “at the very minimum, means sharing the aggregated patient data at the hospital level.” 

On their website, Surgisphere published a statement saying that given their research contracts with hospitals, they are unable to share data with other researchers. They state the company is certified as adhering to “very strict data security and data integrity” standards and that mandatory audits occur at least four times per year. “Everything from data acquisition to data reporting is independently reviewed by an external third-party auditor,” the company writes. They point out the results of their analysis line up with other observational studies reported previously, and that the US Food and Drug Administration has already issued warnings about the risks of hydroxychloroquine or chloroquine.