No, Apple Watch Can't Detect Irregular Heart Rhythms with 97% Accuracy (Yet)

In a recent study, researchers from the University of California San Francisco tested out whether heart rate data collected by Apple Watch, crunched through a deep learning network, could accurately identify atrial fibrillation (AF) in people already known to have irregular heartbeats. That’s another medical feather in the device’s crowded cap.

As some publications noted and others (mostly in the tech sphere) essentially ignored, however, the study had some huge caveats. It doesn’t exactly hide them: Its “Limitations” section points to 2 that are very important.

First, the authors wrote that, “These data focused on individuals with a known history of AF. Therefore, we did not demonstrate an ability to identify a new diagnosis of the disease.” There is, of course, terrific value in the ability to monitor an existing condition, and the study goes out of its way to specify that it is “proof of concept” that the device, paired with a good algorithm, can recognize the condition with regularity.

The Apple Watch/deep learning combination did perform well in a small subset of patients with defined AF who were stationary when measurements were taken (the same group also had a coordinator present to assist in use of the device). The system didn’t do nearly as well operating passively in the real world. What will it take to refine these detection mechanisms, and if they can be improved, what are the next steps? Will the Apple Watch just alert a patient and urge them to seek medical attention? Will it directly contact their provider through some channel in Apple’s evolving healthcare capabilities?

The other big limitation has to do with generalizability. Even without the first asterisk, there’s a question of who this technology can help. Because all of the participants in the remote cohort already owned a smartwatch, the authors postulate that the results might not “generalize to less tech-savvy individuals.”

It isn’t just the “less tech-savvy” excluded: There’s also barriers to device ownership. The team wrote that smartwatches may become more mainstream over time, the same way smartphones and internet use have, but that's to be seen. Only about a tenth of US adults own such a device, and that’s skewed to younger generations. Smartwatches have yet to establish the same vitality in day-to-day life that smartphones and the internet itself have, but they add new capabilities daily and that could well change.

And as shown in iPhone-based research, Apple device ownership trends towards the white and wealthy. As any population health researcher will quickly tell you, poorer people are far more vulnerable to chronic health conditions. The questions that emerge from that limitation are: Do the populations being observed in existing studies contribute bias to the detection algorithms being developed? And if all the other questions are answered and the technology unlocks life-saving potential, will there be a push to make the devices more available to those that would otherwise lack the “savvy” (or money) to buy them?

Cardiogram, the app-maker whose namesake application was used in the study, will certainly continue to research how its algorithms can be refined, and Apple has numerous healthcare projects out there, including a massive heart health study that’s looking to recruit half-a-million patients (but again, they must already own an iPhone and Apple Watch). The device's capabilities will continue to grow more impressive. But does it “detect abnormal heart rhythms with a 97% accuracy rate” now?

Not without an asterisk.

Related Coverage:

Could an iPhone App Dramatically Reduce Heart Attack Readmissions?

Reports of a ResearchKit Revolution May be Premature

A New Ethical Wrinkle for Medical Algorithms