Smartphone App Hears Speech Changes Indicating Fluid Buildup in HF

A smartphone app designed to detect changes in speech may one day provide an early warning to heart failure (HF) patients that they’ve begun to develop pulmonary edema, researchers say.

Their automated approach to speech analysis was able to tease out differences in nearly all cases.

“I think the desire to develop this approach really came from a recognized unmet need in heart failure management, and that is really being able to assess—particularly remotely—when patients are well compensated and dry versus when they’re retaining fluid and becoming congested, because we know that the latter situation often results in worsening signs and symptoms and the need for hospitalization,” investigator William T. Abraham, MD (The Ohio State University, Columbus), told TCTMD.

“So our goal here is to develop a tool to keep patients well and to keep them out of the hospital,” he said.

For the current proof-of-concept study, published in JACC: Heart Failure, they worked in the opposite direction: using the technology in patients already admitted with acute decompensated heart failure (ADHF) to differentiate between their voice patterns before and after treatment.

Our goal here is to develop a tool to keep patients well and to keep them out of the hospital. William T. Abraham

Led by Offer Amir, MD (Hadassah Medical Center, Jerusalem, and Bar-Ilan University, Zfat, Israel), the researchers asked 40 adult patients admitted with ADHF at Rambam Medical Center to record five short sentences through the HearO app (version 0.85) on an iPhone 6 Plus smartphone (Apple), both at the time of their hospital admission (wet) and at discharge (dry). Recordings were done in the patient’s native language of Hebrew, Arabic, or Russian. The app recorded and digitized speech then uploaded the files to the server for storage and analysis.

They collected 1,484 recordings. By looking at five distinct speech measures, the app could successfully identify wet versus dry cases 94% of the time, and it showed distinct differences for all of the speech measures in 87.5% of cases.

A Proactive Approach to HF

Could patients and their families also detect these differences on their own, without technology as a tool? Abraham said that’s unlikely, based on prior studies of implantable hemodynamic monitors like the CardioMEMS HF System (Abbott).

“It turns out that it is difficult for patients to judge changes in their clinical status early on, and often they don’t appreciate or don’t feel changes in symptoms until it’s too late: until they become acutely symptomatic and end up in the emergency department or admitted to the hospital,” he observed.

The idea, said Abraham, is to use the speech analysis app along with other clinical data to inform decision-making.

“What we would be looking for are trending data,” he explained, adding, “Some of these patients may have a day where they take in a little extra salt and fluid, and perhaps they correct that on their own over the next day or two. But when we see things trending in the wrong direction, it would provide an opportunity for a number of responses.

“One might be reaching out to the patient via a telehealth visit,” Abraham continued. “Another might be seeing them in the clinic. In either case, perhaps it would prompt a clinician to consider making a change in therapy, if the totality of evidence along with this system indicated that the patient was at increased risk for hospitalization.”

Next Steps

Neal G. Ravindra, PhD (Yale University, New Haven, CT), and David P. Kao, MD (University of Colorado, Aurora), in an accompanying editorial, say that “active speech analysis . . . is an important advance toward expanding the tools available to assess patients with HF.”

They, too, call for a holistic approach that integrates information from the app with other clinical factors. Yet there may be challenges related to getting patients to consistently engage with the technology, as well as issues with privacy.

Beyond these concerns, the editorialists caution against bias, which historically has hindered artificial intelligence’s potential.

“In addition to standard requirements for efficacy, any patient-monitoring technology should ensure demographic parity, equalized odds, equal opportunity, treatment equality, and fairness,” they stress. “The HearO app highlights that limited access to new smartphones (eg, the latest iPhone or Android devices) may affect quality of care in the future, because voice-recording quality could be stratified according to socioeconomic status. As more fields of medicine collect data with these devices, the HF community should think carefully about how such disparities might adversely affect our patients.”

Abraham said next steps for the app’s development include conducting larger studies done elsewhere in the world, including the United States, and in outpatients who are “starting out dry, . . . so that we can demonstrate the overall performance of this system as a day-to-day remote monitoring tool.”

In this research, it’s important to capture the diversity of the population, he added. “We want to make sure that dialects that might differ based on where you live and what your background might be perform equally well, and if there are differences we want to be able to account for those.”

Abraham said that based on his long years of working with diagnostic and monitoring devices he’s not worried about patients’ engagement with technology: “We live in a world where even elderly heart failure patients mostly have access to smartphones. They engage in social media. They’re very active users of technology now and very well informed. What I’ve found is these types of technologies can often better engage patients in their own self-care.”