David Kotz papers for project 'auracle'

Analog Gated Recurrent Unit Neural Network for Detecting Chewing Events

Thu, 01 Dec 2022 00:00:00

Kofi Odame, Maria Nyamukuru, Mohsen Shahghasemi, Shengjie Bi, and David Kotz. Analog Gated Recurrent Unit Neural Network for Detecting Chewing Events. IEEE Transactions on Biomedical Circuits and Systems, volume 16, number 6, pages 1106–1115. IEEE, December 2022. doi:10.1109/TBCAS.2022.3218889.

Abstract:

We present a novel gated recurrent neural network to detect when a person is chewing on food. We implemented the neural network as a custom analog integrated circuit in a 0.18 μm CMOS technology. The neural network was trained on 6.4 hours of data collected from a contact microphone that was mounted on volunteers’ mastoid bones. When tested on 1.6 hours of previously-unseen data, the analog neural network identified chewing events at a 24-second time resolution. It achieved a recall of 91% and an F1-score of 94% while consuming 1.1 μW of power. A system for detecting whole eating episodes— like meals and snacks— that is based on the novel analog neural network consumes an estimated 18.8 μW of power.

Eating detection with a head-mounted video camera

Wed, 01 Jun 2022 00:00:00

Shengjie Bi and David Kotz. Eating detection with a head-mounted video camera. Proceedings of the IEEE International Conference on Healthcare Informatics (ICHI), pages 60–66. IEEE, June 2022. doi:10.1109/ICHI54592.2022.00021.

Abstract:

In this paper, we present a computer-vision based approach to detect eating. Specifically, our goal is to develop a wearable system that is effective and robust enough to automatically detect when people eat, and for how long. We collected video from a cap-mounted camera on 10 participants for about 55 hours in free-living conditions. We evaluated performance of eating detection with four different Convolutional Neural Network (CNN) models. The best model achieved accuracy 90.9% and F1 score 78.7% for eating detection with a 1-minute resolution. We also discuss the resources needed to deploy a 3D CNN model in wearable or mobile platforms, in terms of computation, memory, and power. We believe this paper is the first work to experiment with video-based (rather than image-based) eating detection in free-living scenarios.

Eating detection with a head-mounted video camera

Wed, 01 Dec 2021 00:00:00

Shengjie Bi and David Kotz. Eating detection with a head-mounted video camera. Technical Report number TR2021-1002, Dartmouth Computer Science, December 2021.

Abstract:

Detection of health-related behaviours using head-mounted devices

Sat, 01 May 2021 00:00:00

Shengjie Bi. Detection of health-related behaviours using head-mounted devices. PhD thesis, Dartmouth Computer Science, Hanover, NH, May 2021. PhD Dissertation.

Abstract:

The detection of health-related behaviors is the basis of many mobile-sensing applications for healthcare and can trigger other inquiries or interventions. Wearable sensors have been widely used for mobile sensing due to their ever-decreasing cost, ease of deployment, and ability to provide continuous monitoring. In this dissertation, we develop a generalizable approach to sensing eating-related behavior.

First, we developed Auracle, a wearable earpiece that can automatically detect eating episodes. Using an off-the-shelf contact microphone placed behind the ear, Auracle captures the sound of a person chewing as it passes through the head. This audio data is then processed by a custom circuit board. We collected data with 14 participants for 32 hours in free-living conditions and achieved accuracy exceeding 92.8% and F1 score exceeding77.5% for eating detection with 1-minute resolution.

Second, we adapted Auracle for measuring children’s eating behavior, and improved the accuracy and robustness of the eating-activity detection algorithms. We used this improved prototype in a laboratory study with a sample of 10 children for 60 total sessions and collected 22.3 hours of data in both meal and snack scenarios. Overall, we achieved 95.5% accuracy and 95.7% F1 score for eating detection with 1-minute resolution.

Third, we developed a computer-vision approach for eating detection in free-living scenarios. Using a miniature head-mounted camera, we collected data with 10 participants for about 55 hours. The camera was fixed under the brim of a cap, pointing to the mouth of the wearer and continuously recording video (but not audio) throughout their normal daily activity. We evaluated performance for eating detection using four different Convolutional Neural Network (CNN) models. The best model achieved 90.9% accuracy and 78.7%F1 score for eating detection with 1-minute resolution. Finally, we validated the feasibility of deploying the 3D CNN model in wearable or mobile platforms when considering computation, memory, and power constraints.

Measuring children’s eating behavior with a wearable device

Tue, 01 Dec 2020 00:00:00

Shengjie Bi, Yiyang Lu, Nicole Tobias, Ella Ryan, Travis Masterson, Sougata Sen, Ryan Halter, Jacob Sorber, Diane Gilbert-Diamond, and David Kotz. Measuring children’s eating behavior with a wearable device. Proceedings of the IEEE International Conference on Healthcare Informatics (ICHI). IEEE, December 2020. doi:10.1109/ICHI48887.2020.9374304.

Abstract:

Poor eating habits in children and teenagers can lead to obesity, eating disorders, or life-threatening health problems. Although researchers have studied children’s eating behavior for decades, the research community has had limited technology to support the observation and measurement of fine-grained details of a child’s eating behavior. In this paper, we present the feasibility of adapting the Auracle, an existing research-grade earpiece designed to automatically and unobtrusively recognize eating behavior in adults, for measuring children’s eating behavior. We identified and addressed several challenges pertaining to monitoring eating behavior in children, paying particular attention to device fit and comfort. We also improved the accuracy and robustness of the eating-activity detection algorithms. We used this improved prototype in a lab study with a sample of 10 children for 60 total sessions and collected 22.3 hours of data in both meal and snack scenarios. Overall, we achieved an accuracy exceeding 85.0% and an F1 score exceeding 84.2% for eating detection with a 3-second resolution, and a 95.5% accuracy and a 95.7% F1 score for eating detection with a 1-minute resolution.

Auracle: Detecting Eating Episodes with an Ear-Mounted Sensor

Sat, 01 Sep 2018 00:00:00

Shengjie Bi, Tao Wang, Nicole Tobias, Josephine Nordrum, Shang Wang, George Halvorsen, Sougata Sen, Ronald Peterson, Kofi Odame, Kelly Caine, Ryan Halter, Jacob Sorber, and David Kotz. Auracle: Detecting Eating Episodes with an Ear-Mounted Sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (Ubicomp), volume 2, number 3, article 92, 27 pages. ACM, September 2018. doi:10.1145/3264902.

Abstract:

In this paper, we propose Auracle, a wearable earpiece that can automatically recognize eating behavior. More specifically, in free-living conditions, we can recognize when and for how long a person is eating. Using an off-the-shelf contact microphone placed behind the ear, Auracle captures the sound of a person chewing as it passes through the bone and tissue of the head. This audio data is then processed by a custom analog/digital circuit board. To ensure reliable (yet comfortable) contact between microphone and skin, all hardware components are incorporated into a 3D-printed behind-the-head framework. We collected field data with 14 participants for 32 hours in free-living conditions and additional eating data with 10 participants for 2 hours in a laboratory setting. We achieved accuracy exceeding 92.8% and F1 score exceeding 77.5% for eating detection. Moreover, Auracle successfully detected 20-24 eating episodes (depending on the metrics) out of 26 in free-living conditions. We demonstrate that our custom device could sense, process, and classify audio data in real time. Additionally, we estimate Auracle can last 28.1 hours with a 110 mAh battery while communicating its observations of eating behavior to a smartphone over Bluetooth.

Poster: Auracle --- A Wearable Device for Detecting and Monitoring Eating Behavior

Thu, 01 Jun 2017 00:00:00

Shengjie Bi, Ellen Davenport, Jun Gong, Ronald Peterson, Joseph Skinner, Kevin Storer, Tao Wang, Kelly Caine, Ryan Halter, David Kotz, Kofi Odame, Jacob Sorber, and Xing-Dong Yang. Poster: Auracle --- A Wearable Device for Detecting and Monitoring Eating Behavior. Proceedings of the ACM International Conference on Mobile Systems, Applications, and Services (MobiSys), page 176. ACM, June 2017. doi:10.1145/3081333.3089320.

Abstract:

The Auracle aims to be a wearable earpiece that detects eating behavior, to be fielded by health-science researchers in their efforts to study eating behavior and ultimately to develop interventions useful to individuals striving to address chronic disease related to eating.

Toward a Wearable Sensor for Eating Detection

Thu, 01 Jun 2017 00:00:00

Shengjie Bi, Tao Wang, Ellen Davenport, Ronald Peterson, Ryan Halter, Jacob Sorber, and David Kotz. Toward a Wearable Sensor for Eating Detection. Proceedings of the ACM Workshop on Wearable Systems and Applications (WearSys), pages 17–22. ACM, June 2017. doi:10.1145/3089351.3089355.

Abstract:

Researchers strive to understand eating behavior as a means to develop diets and interventions that can help people achieve and maintain a healthy weight, recover from eating disorders, or manage their diet and nutrition for personal wellness. A major challenge for eating-behavior research is to understand when, where, what, and how people eat. In this paper, we evaluate sensors and algorithms designed to detect eating activities, more specifically, when people eat. We compare two popular methods for eating recognition (based on acoustic and electromyography (EMG) sensors) individually and combined. We built a data-acquisition system using two off-the-shelf sensors and conducted a study with 20 participants. Our preliminary results show that the system we implemented can detect eating with an accuracy exceeding 90.9% while the crunchiness level of food varies. We are developing a wearable system that can capture, process, and classify sensor data to detect eating in real-time.