CogVox project

Summary

The CogVox project is led by Xiaohui Liang (UMass Boston) and John Batsis (UNC). It aims to leverage conversations with Amazon's Alexa voice-based assistant to predict early onset of dementia in older adults. Such systems raise potential privacy concerns, which is how I became involved.

People

John Batsis, Tiffany Driesse, Karen Fortuna, David Kotz, Xiaohui Liang, David Lynch, Brian MacWhinney, Robert Roth, Hillary Spangler, Youxiang Zhu.

Funding and acknowledgements

This project was funded by the US National Institute on Aging award R01AG067416-03; by the North Carolina Translational and Clinical Sciences Institute, under award numbers UL1TR00248 and ACTP1R1001; by the US National Science Foundation (SaTC Frontiers program) under award number 1955805; and by the US National Institute on Drug Abuse (National Institutes of Health) under award number P30DA029926.

The views and conclusions contained on this site and in its documents are those of the authors and should not be interpreted as necessarily representing the official position or policies, either expressed or implied, of the sponsor(s). Any mention of specific companies or products does not imply any endorsement by the authors or by the sponsor(s).

Papers (tagged 'cogvox')

[Also available in BibTeX]

Papers are listed in reverse-chronological order; click an entry to pop up the abstract. For full information and pdf, please click Details link. Follow updates with RSS.

2022:
Spangler, Hillary B., Driesse, Tiffany M., Lynch, David H., Liang, Xiaohui, Roth, Robert M., Kotz, David, Fortuna, Karen, and Batsis, John A. Privacy Concerns of Older Adults Using Voice Assistant Systems. Journal of the American Geriatrics Society. August 26, 2022. [Details]

Voice assistant systems (VAS) are software platforms that complete various tasks using voice commands. It is necessary to understand the juxtaposition of younger and older adults' VAS privacy concerns as younger adults may have different concerns impacting VAS acceptance. Therefore, we examined the differences in VAS related privacy concerns across the lifespan.

Xiaohui Liang, John A. Batsis, Youxiang Zhu, Tiffany M. Driesse, Robert M. Roth, David Kotz, and Brian MacWhinney. Evaluating Voice-Assistant Commands for Dementia Detection. Computer Speech and Language. March 2022. Special Issue on Speech Based Evaluation of Neurological Diseases. [Details]

Early detection of cognitive decline involved in Alzheimer’s Disease and Related Dementias (ADRD) in older adults living alone is essential for developing, planning, and initiating interventions and support systems to improve users’ everyday function and quality of life. In this paper, we explore the voice commands using a Voice-Assistant System (VAS), i.e., Amazon Alexa, from 40 older adults who were either Healthy Control (HC) participants or Mild Cognitive Impairment (MCI) participants, age 65 or older. We evaluated the data collected from voice commands, cognitive assessments, and interviews and surveys using a structured protocol. We extracted 163 unique command-relevant features from each participant’s use of the VAS. We then built machine-learning models including 1-layer/2-layer neural networks, support vector machines, decision tree, and random forest, for classification and comparison with standard cognitive assessment scores, e.g., Montreal Cognitive Assessment (MoCA). Our classification models using fusion features achieved an accuracy of 68%, and our regression model resulted in a Root-Mean-Square Error (RMSE) score of 3.53. Our Decision Tree (DT) and Random Forest (RF) models using selected features achieved higher classification accuracy 80%–90%. Finally, we analyzed the contribution of each feature set to the model output, thus revealing the commands and features most useful in inferring the participants’ cognitive status. We found that features of overall performance, features of music-related commands, features of call-related commands, and features from Automatic Speech Recognition (ASR) were the top-four feature sets most impactful on inference accuracy. The results from this controlled study demonstrate the promise of future home-based cognitive assessments using Voice-Assistant Systems.

[Kotz research]