Evaluating Voice-Assistant Commands for Dementia Detection


Xiaohui Liang, John A. Batsis, Youxiang Zhu, Tiffany M. Driesse, Robert M. Roth, David Kotz, and Brian MacWhinney. Evaluating Voice-Assistant Commands for Dementia Detection. Elsevier Computer Speech and Language. September 2021. doi:10.1016/j.csl.2021.101297. ©Copyright Elsevier. Accepted for publication; Special Issue on Speech Based Evaluation of Neurological Diseases.


Early detection of cognitive decline involved in Alzheimer’s Disease and Related Dementias (ADRD) in older adults living alone is essential for developing, planning, and initiating inter- ventions and support systems to improve users’ everyday function and quality of life. In this paper, we explore the voice commands using a Voice-Assistant System (VAS), i.e., Amazon Alexa, from 40 older adults who were either Healthy Control (HC) participants or Mild Cog- nitive Impairment (MCI) participants, age 65 or older. We evaluated the data collected from voice commands, cognitive assessments, and interviews and surveys using a structured protocol. We extracted 163 unique command-relevant features from each participant’s use of the VAS. We then built machine-learning models including 1-layer/2-layer neural networks, support vec- tor machines, decision tree, and random forest, for classification and comparison with standard cognitive assessment scores, e.g., Montreal Cognitive Assessment (MoCA). Our classification models using fusion features achieved an accuracy of 68%, and our regression model resulted in a Root-Mean-Square Error (RMSE) score of 3.53. Our Decision Tree (DT) and Random For- est (RF) models using selected features achieved higher classification accuracy 80-90%. Finally, we analyzed the contribution of each feature set to the model output, thus revealing the com- mands and features most useful in inferring the participants’ cognitive status. We found that features of overall performance, features of music-related commands, features of call-related commands, and features from Automatic Speech Recognition (ASR) were the top-four feature sets most impactful on inference accuracy. The results from this controlled study demonstrate the promise of future home-based cognitive assessments using Voice-Assistant Systems.

Citable with [BibTeX]

Keywords: [mhealth] [sensors]

Available from the publisher: [DOI]

Available from the author: [bib]
No pdf available at this time; check later.

[Kotz research]