Machine Learning, EHR Data Help Predict Risk of Patient Violence

^{Photo/Thumb have been modified. U.S. Air Force} photo ^{by Airman 1st Class Rito Smith.}

Inpatient violence risk assessments can be performed automatically using available clinical notes without sacrificing predictive validity, according to the findings of a study published in JAMA Network Open.

Researchers used machine learning to analyze clinical notes in the electronic health records (EHRs) of two psychiatric institutions to predict inpatient violence. Investigators measured each site’s area under the curve to determine predictive validity. The first site had an area under the curve of 0.797, while the second registered at 0.764, meaning it is possible to use routinely registered clinical notes for automatic violence risk assessment.

The model performed with a specificity between 0.935 and 0.947 and a sensitivity between 0.334 and 0.336.

“Inpatient violence remains a significant problem despite existing risk assessment methods,” the study authors wrote. “The lack of robustness and the high degree of effort needed to use current methods might be mitigated by using routinely registered clinical notes.”

The research team used the first site, the psychiatry department of the academic medical center in Utrecht, Netherlands, for internal method validation. The data set consisted of 3,201 admissions of 2,211 unique patients. The second site, a general psychiatric hospital that delivers secondary care in Rotterdam, Netherlands, was used for external method validation. This data set consisted of 3,277 admissions of 1,937 unique patients.

Researchers extracted clinical notes written by psychiatrists and nurses from patients’ EHRs. The research team included notes written in the four weeks before admission up to the first 24 hours of admission. They excluded admissions with fewer than 100 words registered after 24 hours.

Reports of violent incidents helped determine the outcome for each admission. Staff members involved in a given incident at either site filled in structured information, a textual description of the incident and the severity of the incident as measured by the Staff Observation Aggression Scale-Revised.

Violent incidents included all threatening and violent behavior of verbal or physical nature directed at another person. This excluded self-harm and inappropriate behavior like substance abuse or vandalism.

Researchers examined predictive power hidden in the notes by extracting the 1,000 most frequent terms. A chi-squared test helped assess the strength of the term’s association with the outcome. The research team selected the top 10% of predictors based on their chi-squared test scores in 1,000 repeated samples.

Terms such as “aggressive,” “angry,” “verbal,” “threatening” and “irritated” can be directly associated with violence. Terms like “reacts,” “walks” and “speaks” describe behavioral cues that can be indirectly associated with violence.

Researchers then used a machine-learning approach to perform a violence risk assessment. Algorithms can detect patterns in historical data, and prediction can help the course of treatment based on those patterns.

The approach transformed clinical notes into a numerical representation and then fed the representations into a classification model.

Researchers trained the model using the internal set of clinical notes.

The risk of violent outcomes for patients with predicted high risk compared to low risk was 5.121 in the first site and 6.297 in the second.

“In the near future, we envision that further advancements towards a data-driven psychiatric practice will be made and that EHR data will become an even more valuable asset in supporting important decisions in the clinical practice,” the authors wrote.

Get the best insights in digital health directly to your inbox.

Related

Patients Are Not Taking Advantage of EHR Portals

AI Model Analyzes EHR Data, Identifies Child Leukemia Patients for Studies

Health Systems Want to Improve Patient Experience, Don’t Have Necessary Resources