The Numbers Behind the First FDA-Approved Autonomous AI Diagnostic System

The first artificial intelligence (AI) diagnostic system to gain clearance from the U.S. Food and Drug Administration beat out all predetermined benchmarks, achieving “high diagnostic accuracy” for patients with certain forms of diabetic retinopathy, according to clinical trial findings.

IDx, the developer of the system, IDx-DR, published its results this week in the peer-reviewed journal Nature Digital Medicine. The paper provides an inside look into a technology that could transform how the industry diagnoses diabetic retinopathy, a condition that can cause blindness, bringing the process from the specialist’s office to primary care — without the need for a clinician to interpret the results.

>> READ: First-of-Its-Kind AI Tool for Diabetic Retinopathy Detection Approved by FDA

“This is formerly uncharted territory in healthcare, making it especially critical that we ensure the highest level of safety before introducing autonomous AI into patient care,” Michael D. Abràmoff, M.D., Ph.D., IDx’s founder and president and the study’s principal investigator, said in a statement.

In April, the FDA cleared IDx-DR, which analyzes images of the eye, for detection of “more than mild” diabetic retinopathy in adults with diabetes. This week’s published findings buttressed that groundbreaking decision.

The AI system achieved 87 percent sensitivity, 90 percent specificity and a 96 percent imageability rate, according to the findings. All of these figures surpassed pre-specified superiority endpoints. (Under the same tested standard for sensitivity, for instance, ophthalmologists perform at an average that is high as 73 percent and low as 33 percent.)

IDx’s clinical trial spanned 900 participants with diabetes at 10 primary care centers in the United States.

IDx officials stressed the need to “develop an exceptionally rigorous study” with review by independent physician-scientists. Now, Abràmoff said, physicians and patients are free to vet the system’s safety and efficacy, a critical element to adoption of this sort of novel technology.

“Not all trials are as thorough as this one, in that the IDx-DR system’s accuracy was checked against the leading reference standard for assessing diabetic retinopathy,” John C. Parker, M.D., an endocrinologist who served as principal investigator at a trial site, said in a statement.

So, what’s the big deal? The AI system does not require a clinician to interpret the results. IDx said this could bring “specialty-level diagnostics” to primary care settings — and even retail clinics. That new arrangement could further early detection of diabetic retinopathy, which may help prevent blindness. The study authors wrote that IDx-DR could “increase access and lower cost.”

Of note, however, was the AI system’s lower sensitivity score (87 percent) compared to that achieved in the lab data set (97 percent). IDx attributed this disconnect to the use of stereo widefield photography and larger image sizes; the use of primary care staffers rather than ophthalmic photographers to capture the images; or the reduction of selection and spectrum bias.

IDx-DR also achieved better specificity in patients who are 65 years and older. Study authors pointed to “the prevalence of highly reflective internal limiting membrane in younger people,” which can be confused for a sign of diabetic macular edema.

The first implementation of IDx-DR went live in June at University of Iowa Health Care—Iowa River Landing.

Get the best insights in healthcare analytics directly to your inbox.

Related

Employing Eye Experts, Google Sharpens Its Disease-Detecting AI

Another Start-Up Advances AI's Assault on Diabetes

Expedited FDA Review for Novel AI That Detects Diabetic Retinopathy