Without Context, a Neural Network Diagnosed Melanoma Better Than 58 Dermatologists

Clinicians make better diagnoses with more context. But in a new study of an artificial intelligence (AI)-based dermatology diagnostic system, the algorithms outperformed humans even without important demographic and contextual information about the images presented to them.

“When I came across recent reports on deep-learning algorithms that outperform human experts in specific tasks, I immediately knew that we had to explore these artificial intelligence algorithms for diagnosing melanoma,” Holger Haenssle of Germany’s University of Heidelberg said. Haenssle, a professor of dermatology, co-authored a new study that documents the development and performance of a new convolutional neural network (CNN) meant to parse between benign moles and malignant melanomas.

>>>READ: UK PM Says Cancer-Detecting AI Could Save 22,000 Lives

The network was built on Google’s Inception v4 CNN architecture and trained with over 100,000 dermoscopic images. The researchers then developed a pair of test sets (which were not involved in the system’s training) and collected 58 dermatologists from 17 different countries. More than half of the clinicians had more than 5 years of experience.

From just dermoscopic images, the clinicians were asked first to diagnose either a benign or malignant condition and asked to assign a care plan (level 1). A month later, they were shown the same set of images matched to information like age and sex of the patient and the location of the lesion on their body, and again asked for assessment (level 2)

The dermatologists did well in both cases—they identified melanomas 86.6% and 88.9% of the time in levels 1 and 2 and picked out non-malignant cases 71.3% and 75.7% of the time.

But no matter how much they improved when given context, the CNN outperformed them, detecting 95% of melanomas. Haenssle said that the findings show that “deep learning convolutional neural networks are capable of out-performing dermatologists, including extensively trained experts, in the task of detecting melanomas.”

The team says that it’s prepping the CNN for subsequent research to gauge its potential for real-life application. The new study, published in the Annals of Oncology, was accompanied by a commentary that ponders that very question.

Echoing sentiments previously expressed to Healthcare Analytics News™ by various radiologists, the commentary authors do not believe AI will lead to the replacement or “de-skilling” of the workforce tasked with evaluating and assessing patients. They argue that system’s like the new dermatology CNN can serve as a decision support tool (emphasis on “support”) that can provide a second opinion to a trained physician.

They also argue that it’s unknown how the AI will perform when confronted with atypical melanomas—which “often lack pigment and may have dotted and linear irregular vessels.” Since those can be difficult to create images for, AI systems can’t easily be trained to detect them.

“This is the catch; for challenging lesions where machine-assisted diagnosis would be most useful, the reliability is lowest,” they wrote.

Related Coverage:

The Nanorobots That Could Choke Out a Tumor

Why Experts Are Developing AI, Medical Imaging Standards

AI Identifies Patients at Highest Risk of Cholera Infection