ICMJE Gets an Earful on Its Data-Sharing Proposal for Clinical Trials

 

“I could not disagree with this document more. . . . If such a policy were put in force to require de-identified datasets be submitted, I simply would never publish in any of the journals affiliated with this.”   

“This is a good idea in theory, but needs more thought.”

This proposal is “too narrowly focused on journals’ parochial interests over and above the public’s right to derive maximum benefit from medical research.”

“Uncontrolled public access to individual patient data is unethical so needs to be restricted to research purposes only.”

“ICMJE’s proposal is a significant step forward in improving the transparency of clinical trials for consumers and the academic medical community.”

“I believe this is a well-meaning but hopelessly naive proposal which actually undercuts the fundamental fabric of patient confidentiality and protections. . . . I strongly oppose this proposal.”

The open public comment period closed last week on the International Committee of Medical Journal Editors’ (ICMJE’s) proposal for sharing clinical trial data as a prerequisite for publication in an ICMJE journal with a whopping 319 comments. The 66 pages of responses—should you have an idle 2 hours to troll through them—reads in part like a Who’s Who in Medicine and, very occasionally, like the sardonic eye-rolling of comments sections that newspapers everywhere are gradually opting to close down.

I spoke with Christine Laine, MD, editor-in-chief of Annals of Internal Medicine, one of the member journals of the ICMJE and the secretary site for the organization, which, she pointed out, is volunteer-led, with no board and no designated leadership. Laine said she’d looked at “just a few” of the posted comments so far and that it will likely take many months before ICMJE members can review them all.

I myself spent some time wading through them last week. There are plenty of thoughtful responses, and some blanket agreement for the proposal. Most, however, mix cautious support with questions, caveats, and concerns. It’s also a highly international chorus, with respondents hailing from Australia, Canada, India, Greece, Iran, Japan, Italy, Germany, the United States, and beyond.

Who’s Who

Flurries of responses from the same institution hint that certain hospitals put out a reminder to their staff to get their comments in before the deadline (and, at least in one case, look to have supplied a ready-made letter to staff). There’s also input from a wide range of international medical organizations, trade associations, medical publishers, drug and device companies, and medical librarians, although the bulk of comments are from clinical trialists and researchers. A minority of clinical trial participants also had their say.

Cardiologist Manesh Patel, MD (Duke Clinical Research Institute, Durham, NC), noted in his comments that he and others, including C. Michael Gibson, MD (Beth Israel Deaconess Medical Center, Boston, MA), have formed a group to consider the issues: the Academic Research Organization Consortium for Continuing Evaluation of Scientific Studies - Cardiovascular (ACCESS CV). The group is preparing a formal response, to be published in a peer-reviewed publication, Patel noted.

“We agree with transparency and data sharing. However, we have concerns if there is not an orderly manner in which this is done,” he wrote. “Specifically, our two main concerns revolve around the potential for inaccurate and erroneous results from unstructured nonspecified analysis and analysis without hypothesis. We worry about the multiplicity of testing and a variety of potential errors that we highlight in our response. We are also concerned about patient privacy and methods to get access to the data.”

Harry Greenberg, MD, senior associate dean of research at Stanford University (Stanford, CA), also listed his concerns.

“The devil is clearly in the details, which makes me worry quite a lot about this proposal that seems utterly well-intentioned and aimed at a big problem. There are several critical details, three come immediately to mind: 1. How to finance and certify de-identification? 2. How to deal with the inevitable misuse of the data by parasites: cherry-picking, data dredging, and deliberate misconstruing, and how to separate misuse from the intended, useful, corrective, and creative uses? 3. How to prevent someone nasty and clever from re-identifying patients and embarrassing everyone, followed by the inevitable pillory for the hapless investigator who was a little less than perfect at de-identification? This last one seems inevitable to me. 4. Is there any data to indicate that benefits (discussed in the proposal) of implementing this proposal will outweigh the potential issues created?”

Rory Collins, FMedSci, of the University of Oxford’s Nuffield Department of Population Health in England, had similar concerns about the evidence underpinning the ICMJE proposal. “We do not support this proposal as it might result in less rather than more transparency,” he wrote. “We therefore encourage the Editors to seek advice from people who have experience both of generating the data and of using other people’s data before proposing a course of action that is evidence-free.”

‘Data Parasites’

Several comments gave a nod to a point made in a controversial New England Journal of Medicine editorial that followed the ICMJE announcement, namely that “data parasites” would capitalize on the hard work of others.

Cardiologist Robert Sheldon, MD, PhD (Libin Cardiovascular Institute of Alberta, Calgary, Canada), wrote in to say that he agrees with the general proposal, but notes that rules for data sharing developed through the Canadian Arrhythmia Network, with which he is an investigator, have revealed some pitfalls. “While preparing this we were cognizant of large centers in Canada that are indeed data parasites: never enroll, always happy to have the data. . . .  I note that the people who prefer immediate data dumping lack an MD, and therefore are clueless about the effort involved in conducting clinical trials,” he says.

Professor Anushka Patel, MBBS, PhD (George Institute for Global Health, Sydney, Australia), said the institute recognizes the potential substantial community benefits that sharing of de-identified clinical trial data offers but is concerned that some risks have not been adequately acknowledged. “While the need for a prespecified hypothesis and analysis plan that serves a genuine scientific and/or public safety need should be a fundamental requirement for any new analyses, safeguards against misuse of data that serve specific interest groups must also be put in place,” she commented. “In particular, the original investigators should have the right to refuse to share the data with any individual, group, or entity that has a demonstrable major conflict of interest.”

A number of cardiologists, including Salim Yusuf and Stuart Connolly, were signatories on a public letter submitted by the Population Health Research Institute at McMaster University in Hamilton, Canada, outlining their concerns over the proposal. “We believe that the current proposed plan of forced data sharing will not result in a net benefit to patients, investigators, sponsors, or society,” they wrote.

Meanwhile, investigators from the Yale Open Data Access (YODA) Project, including Harlan Krumholz, MD, and Joseph Ross, MD, commented that they support the ICMJE’s ‘general approach.” They point out, however, that certain points remain to be clarified, including, “as a safeguard,” the need for authors of secondary analyses to explain how their proposed study differs from that already conducted by the trial investigators.

As well, they suggest, the ICMJE should have a plan in place for penalizing investigators who do not share their data within the proposed 6 months. “We believe this proposal could be strengthened by explicitly defining penalty(ies), such as exclusion of all associated trial authors from publishing articles in ICMJE member journals for a 3-year period.”

Time and Money

One of the most disputed components of the proposal appears to be specification that trial investigators make their data available within 6 months. The bulk of comments—even those supportive of the proposal—stipulated that 6 months was not enough time for investigators to reasonably pull together all of their analyses and substudies. A minority disagreed, saying 6 months “is too much,” with several comments calling for data sharing to happen concurrently with publication.

Several comments queried the significant work required to provide data at a level that could be shared, and how it would be paid for. “I do not think that funding agencies provide anywhere near the funding that will be required to create such a data set with the necessary documentation so someone outside the study could reliably use the information,” Sharon-Lise Normand, PhD (Harvard Medical School, Boston, MA), commented. “Data are valuable: patients contributed data, researchers recruited participants, data safety committees watched over, and study investigators analyzed the data. Creating a usable data set having reliable and accurate documentation is a critical part of the study that needs sufficient funding if this effort is taken seriously.”

Endocrinologist Michael Jensen, MD (Mayo Clinic, Rochester, MN), was even more cynical: “Great—another burden for investigators who already spend too much time doing paperwork. . . . I see this as adding another unfunded mandate that will take more time away from investigators and/or their staff to do their jobs.”

Even those supportive of the plan still had questions about how to control the provision and dissemination of patient information.

Jacques Demotes, MD, PhD (European Clinical Research Infrastructures Network), wrote, “We strongly agree with this initiative. The question is not if but how to responsibly share data with appropriate control.” 

Patients and Participants

A number of responses, both from trialists and from patients, raised the issue of confidentiality, and the possibility that the specter of data sharing would deter patients from enrolling in trials.

One commenter, who self-identified as a researcher, clinical trialist, and clinical trial participant at the University of Michigan wrote that providing individual patient-level data would violate institutional review board approvals for conducting clinical studies. Moreover, she said, “It may be possible for patients to identify themselves from a few demographic descriptors and the fact that they participated in a specific trial at the institution. These are serious issues!”

Another commenter wrote: “The fact that patients’ data will be placed in the public domain implies that participants will need to be told at the time of study enrollment that this will ultimately happen when the study results are published; the informed consent form being the obvious document for disclosing this fact. Could this influence a patient’s decision to enter the study? Would concerns about confidentiality prevent the patient (or disease-free subject in some types of trials) from accepting to participate?”

One of the few anonymous posts, from a commenter identifying him/herself as a patient in a clinical trial, said, “I am terrified by the ICMJE proposal. The informed consent that I signed does not contemplate patient-level data sharing, and I worry that the sponsor of my trial may publish in an ICMJE journal and be forced to disclose my data, which I would have never permitted. I consented to participation in the study for the purposes of helping the sponsor evaluate the safety and efficacy of an investigational drug and gain approval, not for the purposes of sharing my data (even de-identified) with the public. . . . . Even the fact that you require people to supply an email address to provide comments indicates that you have no respect for patient privacy.”

Next Steps

According to Laine, the Annals editor, the ICMJE member journals meet just once a year and won’t meet again until late fall 2016. While the ICMJE has published policies before, they’ve never actually issued a draft policy, and thrown that open for public comment. “We’re going to try and agree about some of the changes that might be made, and potentially could publish a revised policy before [next fall], but given the large number of comments, it may be that the group can’t reach consensus until we have another in-person meeting,” she said.