Utilization of Machine Learning-Based Computer Vision and Voice Analysis to Derive Digital Biomarkers of Cognitive Functioning in Trauma Survivors.
Researchers
Journal
Modalities
Models
Abstract
Alterations in multiple domains of cognition have been observed in individuals who have experienced a traumatic stressor. These domains may provide important insights in identifying underlying neurobiological dysfunction driving an individual’s clinical response to trauma. However, such assessments are burdensome, costly, and time-consuming. To overcome barriers, efforts have emerged to measure multiple domains of cognitive functioning through the application of machine learning (ML) models to passive data sources.
We utilized automated computer vision and voice analysis methods to extract facial, movement, and speech characteristics from semi-structured clinical interviews in 81 trauma survivors who additionally completed a cognitive assessment battery. A ML-based regression framework was used to identify variance in visual and auditory measures that relate to multiple cognitive domains.
Models derived from visual and auditory measures collectively accounted for a large variance in multiple domains of cognitive functioning, including motor coordination (R2 = 0.52), processing speed (R2 = 0.42), emotional bias (R2 = 0.52), sustained attention (R2 = 0.51), controlled attention (R2 = 0.44), cognitive flexibility (R2 = 0.43), cognitive inhibition (R2 = 0.64), and executive functioning (R2 = 0.63), consistent with the high test-retest reliability of traditional cognitive assessments. Face, voice, speech content, and movement have all significantly contributed to explaining the variance in predicting functioning in all cognitive domains.
The results demonstrate the feasibility of automated measurement of reliable proxies of cognitive functioning through low-burden passive patient evaluations. This makes it easier to monitor cognitive functions and to intervene earlier and at a lower threshold without requiring a time-consuming neurocognitive assessment by, for instance, a licensed psychologist with specialized training in neuropsychology.
Copyright © 2020 by S. Karger AG, Basel.