UNN scientists train AI to detect cognitive stress by voice features
A machine learning model developed by researchers from the Cyberpsychology Laboratory at the UNN Faculty of Social Sciences was used to analyse the acoustic features of speechandto assess the level of cognitive stress experienced by the experiment participants. This software module can be applied in creating systems for monitoring the condition of operators, dispatchers, drivers, security personnel, and other professionals operating under high mental workload.
"Initially, the participants talked about themselves in a relaxed setting, then they read a complex scientific text aloud and summarised it. We recorded the shift from a calm state to physiological activation by tracking changes in heart rate metrics, which aligned with alterations in vocal characteristics. Our study demonstrates that voice reflects changes in an individual's functional state under cognitive stress," explained Valeria Demareva, head of the Cyberpsychology Laboratory at Lobachevsky University.
The scientists note that integrating the software module into modern speech analytics systems requires additional training for the AI to identify the level of cognitive stress during specific professional tasks. Once customised, the model developed by the Nizhny Novgorod researchers will be able to evaluate employees' readiness to work under high intellectual pressure and assist in personnel selection or training.
"Using the context provided by the customer, our tool will detect the presence or absence of cognitive stress through voice analysis in situations where individuals must simultaneously retain information, make quick decisions, and maintain focus. This will optimise work processes by redistributing task volume and employee specialisation,"noted Valeria Demareva.
In the future, the module will become part of a software package for monitoring an individual's functional state, currently under development at the UNN Cyberpsychology Laboratory.
"The results of this research underscore the university's strong scientific and technological potential, the research teams' focus on addressing pressing social issues, and the high level of interdisciplinary collaboration. Implementing this software module into systems for monitoring professionals' health conditions contributes to enhancing the safety and efficiency of intellectual labour," noted Oleg Trofimov, Rector of Lobachevsky University.
The study was supported by the Russian Science Foundation, and the findings were published in The European Physical Journal Special Topics.
