mHealth Apps: Trust the Studies, Not the Stars

Screenshot courtesy Johns Hopkins University.

In 2016, the Instant Blood Pressure app developed by AuraLife was a big success story. It sold for $4.99, claiming the ability to estimate blood pressure using a smartphone camera. It was purchased over 140,000 times and was one of the most popular health applications available for the iPhone.

But in 2016, Johns Hopkins university researchers published a study in JAMA Internal Medicine that showed the app’s readings were highly inaccurate.

This week, some of those same researchers published a second study on Instant blood Pressure in the Journal of the American Medical Informatics Association. The new study focuses on user reviews of the app in light of its noted inaccuracy. Alongside—and likely fueling—the app’s popularity was a very high satisfaction score: 4.8 stars out of 5 after hundreds of reviews in the Apple App Store. Nearly 60% of reviews gave Instant Blood Pressure 5 stars.

>>>READ: mHealth Powered by Potential but Dogged by Dubious Studies

Through thematic analysis of 261 reviews, the team found that 42% of them extolled the program’s accuracy—compared to only 10% who mentioned inaccuracy.

Like most health apps, Instant Blood Pressure included a disclaimer saying it was developed for “recreation” and not intended for medical insight, but that did not deter many users from expressing that they did use it for medical reasons. Almost 10% of surveyed reviews mentioned using Instant Blood Pressure to manage legitimate medical conditions and their treatments, including 11 people claiming to be high blood pressure patients, 1 person claiming to suffer kidney disease, and 1 claiming a recent heart transplant.

“The data showed that disclaimers aren’t a complete solution. Consumers will continue to use these devices to manage their health care, which could be dangerous if they are substituting the app for medical care with a professional,” study co-author Seth Martin, MD, of the Johns Hopkins University School of Medicine, said.

There were even 6 reviewers who said they were medical professionals (1 physician, 4 nurses, 1 unspecified) who gave the app an average rating of 4.2 stars. And users also asserted, at times, that healthcare professionals had approved of their use of Instant Blood Pressure: 11 reviews made that claim, while only 2 said that a healthcare professional had disapproved (both of those reviews came with a 1-star rating).

“People tend to trust user reviews when shopping online and use them to decide which products to purchase, but that doesn’t cut it for medical apps,” said lead author Timothy Plante, MD, formerly of Johns Hopkins and now at the University of Vermont.

Since disclaimers discouraging medical use have proven ineffective, the onus shifts to healthcare providers and consumers to be diligent before engaging with any mhealth app. Physicians must be careful of what applications they endorse or that their patients are using, while consumers must beware of the content in anonymous online reviews—particularly from those people claiming to be healthcare professionals.

The authors believe their work shines a light on the dearth of regulatory scrutiny that mhealth apps receive. “A five-star rating doesn’t replace clinical validation studies and FDA review,” Plante said. And with the number of smartphone owners and available health apps rapidly increasing, the problem isn’t going away anytime soon.

Soon after the critical 2016 study was published, Instant Blood Pressure disappeared from the App Store and AuraLife found itself in dispute with the US Federal Trade Commission. It eventually settled the case for $600,000—likely more than it took home from app sales after taxes and Apple’s cut.

“AuraLife views the settlement and the understanding it established…as a win for the company,” it said in its statement at the time. The company apparently ran out of money not long after and ceased making settlement payments, but its founder and lead investor moved on to start another health app company.

Their new venture, Bodymatter Inc., also launched a blood pressure app called NextHeart. It was free and didn’t claim to estimate health metrics: Instead, it relied on user-inputted medication information and measurements from legitimate arm cuffs to encourage management and prescription adherence. “Make your doctor happy,” its site said. That site is still active, though NextHeart no longer appears in the App Store.

Related Coverage:

First Clinical Trial of mHealth Adherence App as a Standalone Yields Little Improvement

Will the mHealth Market Break $28B in 2018?

No, Apple Watch Can't Detect Irregular Heart Rhythms with 97% Accuracy (Yet)