Papers with keyword 'sensors'

That is, papers related to Sensors

[Also available in BibTeX] [See also: all keywords]

These papers relate to the design or evaluation of devices that can sense the physical world.

Papers are listed in reverse-chronological order; click an entry to pop up the abstract. For full information and pdf, please click Details link. Follow updates with RSS.

2025:
Michael V. Heinz, George D. Price, Avijit Singh, Sukanya Bhattacharya, Ching-Hua Chen, Asma Asyyed, Monique B. Does, Saeed Hassanpour, Emily Hichborn, David Kotz, Chantal A. Lambert-Harris, Zhiguo Li, Bethany McLeman, Varun Mishra, Catherine Stanger, Geetha Subramaniam, Weiyi Wu, Cynthia I. Campbell, Lisa A Marsch, and Nicholas C. Jacobson. A longitudinal observational study with ecological momentary assessment and deep learning to predict non-prescribed opioid use, treatment retention, and medication nonadherence among persons receiving medication treatment for opioid use disorder. Journal of Substance Use and Addiction Treatment (JSAT). March 2025. [Details]

Background: Despite effective treatments for opioid use disorder (OUD), relapse and treatment drop-out diminish their efficacy, increasing the risks of adverse outcomes, including death. Predicting important outcomes, including non-prescribed opioid use (NPOU) and treatment discontinuation among persons receiving medications for OUD (MOUD) can provide a proactive approach to these challenges. Our study uses ecological momentary assessment (EMA) and deep learning to predict momentary NPOU, medication nonadherence, and treatment retention in MOUD patients.

Methods: Study participants included adults receiving MOUD at a large outpatient treatment program. We predicted NPOU (EMA-based), medication nonadherence (Electronic Health Record [EHR]- and EMA-based), and treatment retention (EHR-based) using context-sensitive EMAs (e.g., stress, pain, social setting). We used recurrent deep learning models with 7-day sliding windows to predict the next-day outcomes, using Area Under the ROC Curve (AUC) for assessment. We employed SHapley additive ExPlanations (SHAP) to understand feature latency and importance.

Results: Participants comprised 62 adults with 14,322 observations. Model performance varied across EMA subtypes and outcomes with AUCs spanning 0.59-0.97. Recent substance use was the best performing predictor for EMA-based NPOU (AUC=0.97) and medication nonadherence (AUC=0.68); life-contextual factors performed best for EHR-based medication nonadherence (AUC=0.89) and retention (AUC=0.80). SHAP revealed varying latencies between predictors and outcomes.

Conclusions: Findings support the effectiveness of EMA and deep learning for forecasting actionable outcomes in persons receiving MOUD. These insights will enable the development of personalized dynamic risk profiles and just-in-time adaptive interventions (JITAIs) to mitigate high-risk OUD outcomes.


2024:
Timothy J. Pierson, Cesar Arguello, Beatrice Perez, Wondimu Zegeye, Kevin Kornegay, Carl Gunter, and David Kotz. We need a “building inspector for IoT” when smart homes are sold. IEEE Security & Privacy. Nov-Dec. 2024. [Details]

Internet of Things (IoT) devices left behind when a home is sold create security and privacy concerns for both prior and new residents. We envision a specialized “building inspector for IoT” to help securely facilitate transfer of the home.

Beatrice Perez, Timothy Pierson, Gregory Mazzaro, and David Kotz. Harmonic Radar Scanner for Electronics. Patent Application 18/749,826, published as US2024/0426974, December 26, 2024. Priority date 6/21/23; filed 6/21/24; published 12/26/24. [Details]

A harmonic radar system for detecting an electronic device includes a signal generator for generating one or more transmit radio frequency (RF) signals, a transmitting antenna for sending the transmit RF signals into an environment, a receiving antenna for receiving signals reflected or re-radiated by the electronic device in the environment in response to the transmit RF signals, and a spectrum analyzer for identifying a harmonic frequency of the transmit RF signals in the filtered signals.

Chixiang Wang, Weijia He, Timothy Pierson, and David Kotz. Moat: Adaptive Inside/Outside Detection System for Smart Homes. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT). September 2024. [Details]

Smart-home technology is now pervasive, demanding increased attention to the security of the devices and the privacy of the home's residents. To assist residents in making security and privacy decisions - e.g., whether to allow a new device to connect to the network, or whether to be alarmed when an unknown device is discovered - it helps to know whether the device is inside the home, or outside.

In this paper we present MOAT, a system that leverages Wi-Fi sniffers to analyze the physical properties of a device's wireless transmissions to infer whether that device is located inside or outside of a home. MOAT can adaptively self-update to accommodate changes in the home indoor environment to ensure robust long-term performance. Notably, MOAT does not require prior knowledge of the home's layout or cooperation from target devices, and is easy to install and configure.

We evaluated MOAT in four different homes with 21 diverse commercial smart devices and achieved an overall balanced accuracy rate of up to 95.6%. Our novel periodic adaptation technique allowed our approach to maintain high accuracy even after rearranging furniture in the home. MOAT is a practical and efficient first step for monitoring and managing devices in a smart home.


Weijia He, Nathan Reitinger, Atheer Almogbil, Yi-Shyuan Chiang, Timothy J. Pierson, and David Kotz. Contextualizing Interpersonal Data Sharing in Smart Homes. Proceedings of the Privacy Enhancing Technologies Symposium (PETS). July 2024. [Details]

A key feature of smart home devices is monitoring the environment and recording data. These devices provide security via motion-detection video alerts, cost-savings via thermostat usage history, and peace of mind via functions like auto-locking doors or water leak detectors. At the same time, the sharing of this information in interpersonal relationships---though necessary---is currently accomplished on an all-or-nothing basis. This can easily lead to oversharing in a multi-user environment. Although prior work has studied people's perceptions of information sharing with vendors or ISPs, the sharing of household data among users who interact personally is less well understood. Interpersonal situations make data sharing much more context-based and, thus, more complicated. In this paper, we use themes from the theory of contextual integrity in an online survey (n=1,992) to study how people perceive data sharing with others in smart homes and inform future designs and research. Our results show that data recipients in a smart home can be reduced to three major groups, and data types matter more than device types. We also found that the types of access control desired by users can vary from scenario to scenario. Depending on whom they are sharing data with and about what data, participants expressed varying levels of comfort when presented with different types of access control (e.g., explicit approval versus time-limited access). Taken together, this provides strong evidence that a more dynamic access control system is needed, and we can design it in a more usable way.

Ravindra Mangar, Timothy J. Pierson, and David Kotz. A framework for evaluating the security and privacy of smart-home devices, and its application to common platforms. IEEE Pervasive Computing. July 2024. [Details]

In this article, we outline the challenges associated with the widespread adoption of smart devices in homes. These challenges are primarily driven by scale and device heterogeneity: a home may soon include dozens or hundreds of devices, across many device types, and may include multiple residents and other stakeholders. We develop a framework for reasoning about these challenges based on the deployment, operation, and decommissioning life cycle stages of smart devices within a smart home. We evaluate the challenges in each stage using the well-known CIA triad—Confidentiality, Integrity, and Availability. In addition, we highlight open research questions at each stage. Further, we evaluate solutions from Apple and Google using our framework and find notable shortcomings in these products. Finally, we sketch some preliminary thoughts on a solution for the smart home of the near future.

Cesar Arguello, Beatrice Perez, Timothy J. Pierson, and David Kotz. Detecting Battery Cells with Harmonic Radar. Proceedings of the ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec). May 2024. [Details]

Harmonic radar systems have been shown to be an effective method for detecting the presence of electronic devices, even if the devices are powered off. Prior work has focused on detecting specific non-linear electrical components (such as transistors and diodes) that are present in any electronic device. In this paper we show that harmonic radar is also capable of detecting the presence of batteries. We tested a proof-of-concept system on Alkaline, NiMH, Li-ion, and Li-metal batteries. With the exception of Li-metal coin cells, the prototype harmonic radar detected the presence of batteries in our experiments with 100% accuracy.

Mounib Khanafer, Logan Kostick, Chixiang Wang, Wondimu Zegeye, Weijia He, Berkay Kaplan, Nurzaman Ahmed, Kevin Kornegay, David Kotz, and Timothy Pierson. Device Discovery in the Smart Home Environment. Proceedings of the IEEE/ACM Workshop on the Internet of Safe Things (SafeThings). May 2024. [Details]

With the availability of Internet of Things (IoT) devices offering varied services, smart home environments have seen widespread adoption in the last two decades. Protecting privacy in these environments becomes an important problem because IoT devices may collect information about the home’s occupants without their knowledge or consent. Furthermore, a large number of devices in the home, each collecting small amounts of data, may, in aggregate, reveal non-obvious attributes about the home occupants. A first step towards addressing privacy is discovering what devices are present in the home. In this paper, we formally define device discovery in smart homes and identify the features that constitute discovery in that environment. Then, we propose an evaluative rubric that rates smart home technology initiatives on their device discovery capabilities and use it to evaluate four commonly deployed technologies. We find none cover all device discovery aspects. We conclude by proposing a combined technology solution that provides comprehensive device discovery tailored to smart homes.

2023:
Beatrice Perez, Cesar Arguello, Timothy J. Pierson, Gregory Mazzaro, and David Kotz. Evaluating the practical range of harmonic radar to detect smart electronics. Proceedings of the IEEE Military Communications Conference (MILCOM). October 2023. [Details]

Prior research has found that harmonic radar systems are able to detect the presence of electronic devices, even if the devices are powered off. These systems could be a powerful tool to help mitigate privacy invasions. For example, in a rental property devices such as cameras or microphones may be surreptitiously placed by a landlord to monitor renters without their knowledge or consent. A mobile harmonic radar system may be able to quickly scan the property and locate all electronic devices. The effective range of these systems for detecting consumer-grade electronics, however, has not been quantified. We address that shortcoming in this paper and evaluate a prototype harmonic radar system. We find the system, a variation of what has been proposed in the literature, is able to reliably detect some devices at a range of about two meters. We discuss the effect of hardware on the range of detection and propose an algorithm for automated detection.

Beatrice Perez, Timothy J. Pierson, Gregory Mazzaro, and David Kotz. Identification and Classification of Electronic Devices Using Harmonic Radar. Proceedings of the Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT). June 2023. [Details]

Smart home electronic devices invisibly collect, process, and exchange information with each other and with remote services, often without a home occupants' knowledge or consent. These devices may be mobile or fixed and may have wireless or wired network connections. Detecting and identifying all devices present in a home is a necessary first step to control the flow of data, but there exists no universal mechanism to detect and identify all electronic devices in a space. In this paper we present ICED (Identification and Classification of Electronic Devices), a system that can (i) identify devices from a known set of devices, and (ii) detect the presence of previously unseen devices. ICED, based on harmonic radar technology, collects measurements at the first harmonic of the radar's transmit frequency. We find that the harmonic response contains enough information to infer the type of device. It works when the device has no wireless network interface, is powered off, or attempts to evade detection. We evaluate performance on a collection of 17 devices and find that by transmitting a range of frequencies we correctly identify known devices with 97.6% accuracy and identify previously unseen devices as ‘unknown’ with 69.0% balanced accuracy.

Shrirang Mare, David Kotz, and Ronald Peterson. Effortless authentication for desktop computers using wrist wearable tokens. U.S. Patent 11,574,039, February 7, 2023. Priority date 2018-07-20; International application Filed 2019-07-19; National stage Filed 2021-01-20; Issued 2023-02-07. [Details]

A system and method for authenticating users of a digital device includes an authentication device attached to an authorized user. The authentication device includes one or more motion sensors and acts as a user identity token. To authenticate with a digital device, the user performs one or more interactions with the digital device using the hand associated with the authentication device. The digital device correlates the inputs received due to the interactions with the user's hand and/or wrist movement, as measured by the authentication device. Access to the digital device is allowed if the inputs and movements are correlated.

2022:
Kofi Odame, Maria Nyamukuru, Mohsen Shahghasemi, Shengjie Bi, and David Kotz. Analog Gated Recurrent Neural Network for Detecting Chewing Events. IEEE Transactions on Biomedical Circuits and Systems. December 2022. [Details]

We present a novel gated recurrent neural network to detect when a person is chewing on food. We implemented the neural network as a custom analog integrated circuit in a 0.18 μm CMOS technology. The neural network was trained on 6.4 hours of data collected from a contact microphone that was mounted on volunteers’ mastoid bones. When tested on 1.6 hours of previously-unseen data, the analog neural network identified chewing events at a 24-second time resolution. It achieved a recall of 91% and an F1-score of 94% while consuming 1.1 μW of power. A system for detecting whole eating episodes— like meals and snacks— that is based on the novel analog neural network consumes an estimated 18.8 μW of power.

Spangler, Hillary B., Driesse, Tiffany M., Lynch, David H., Liang, Xiaohui, Roth, Robert M., Kotz, David, Fortuna, Karen, and Batsis, John A. Privacy Concerns of Older Adults Using Voice Assistant Systems. Journal of the American Geriatrics Society. August 26, 2022. [Details]

Voice assistant systems (VAS) are software platforms that complete various tasks using voice commands. It is necessary to understand the juxtaposition of younger and older adults' VAS privacy concerns as younger adults may have different concerns impacting VAS acceptance. Therefore, we examined the differences in VAS related privacy concerns across the lifespan.

George Boateng, Curtis L. Petersen, David Kotz, Karen L. Fortuna, Rebecca Masutani, and John A. Batsis. A Smartwatch Step-Counting App for Older Adults: Development and Evaluation Study. JMIR Aging. August 10, 2022. [Details]

Background: Older adults who engage in physical activity can reduce their risk of mobility impairment and disability. Short amounts of walking can improve quality of life, physical function, and cardiovascular health. Various programs have been implemented to encourage older adults to engage in physical activity, but sustaining their motivation continues to be a challenge. Ubiquitous devices, such as mobile phones and smartwatches, coupled with machine-learning algorithms, can potentially encourage older adults to be more physically active. Current algorithms that are deployed in consumer devices (eg, Fitbit) are proprietary, often are not tailored to the movements of older adults, and have been shown to be inaccurate in clinical settings. Step-counting algorithms have been developed for smartwatches, but only using data from younger adults and, often, were only validated in controlled laboratory settings.

Objective: We sought to develop and validate a smartwatch step-counting app for older adults and evaluate the algorithm in free-living settings over a long period of time.

Methods: We developed and evaluated a step-counting app for older adults on an open-source wrist-worn device (Amulet). The app includes algorithms to infer the level of physical activity and to count steps. We validated the step-counting algorithm in the lab (counting steps from a video recording, n=20) and in free-living conditions—one 2-day field study (n=6) and two 12-week field studies (using the Fitbit as ground truth, n=16). During app system development, we evaluated 4 walking patterns: normal, fast, up and down a staircase, and intermittent speed. For the field studies, we evaluated 5 different cut-off values for the algorithm, using correlation and error rate as the evaluation metrics.

Results: The step-counting algorithm performed well. In the lab study, for normal walking (R2=0.5), there was a stronger correlation between the Amulet steps and the video-validated steps; for all activities, the Amulet’s count was on average 3.2 (2.1%) steps lower (SD 25.9) than the video-validated count. For the 2-day field study, the best parameter settings led to an association between Amulet and Fitbit (R2=0.989) and 3.1% (SD 25.1) steps lower than Fitbit, respectively. For the 12-week field study, the best parameter setting led to an R2 value of 0.669.

Conclusions: Our findings demonstrate the importance of an iterative process in algorithm development before field-based deployment. This work highlights various challenges and insights involved in developing and validating monitoring systems in real-world settings. Nonetheless, our step-counting app for older adults had good performance relative to the ground truth (a commercial Fitbit step counter). Our app could potentially be used to help improve physical activity among older adults.


Shengjie Bi and David Kotz. Eating detection with a head-mounted video camera. Proceedings of the IEEE International Conference on Healthcare Informatics. June 2022. [Details]

In this paper, we present a computer-vision based approach to detect eating. Specifically, our goal is to develop a wearable system that is effective and robust enough to automatically detect when people eat, and for how long. We collected video from a cap-mounted camera on 10 participants for about 55 hours in free-living conditions. We evaluated performance of eating detection with four different Convolutional Neural Network (CNN) models. The best model achieved accuracy 90.9% and F1 score 78.7% for eating detection with a 1-minute resolution. We also discuss the resources needed to deploy a 3D CNN model in wearable or mobile platforms, in terms of computation, memory, and power. We believe this paper is the first work to experiment with video-based (rather than image-based) eating detection in free-living scenarios.

Taylor Hardin. Information Provenance for Mobile Health Data. PhD thesis, May 2022. [Details]

Mobile health (mHealth) apps and devices are increasingly popular for health research, clinical treatment and personal wellness, as they offer the ability to continuously monitor aspects of individuals' health as they go about their everyday activities. Many believe that combining the data produced by these mHealth apps and devices may give healthcare-related service providers and researchers a more holistic view of an individual's health, increase the quality of service, and reduce operating costs. For such mHealth data to be considered useful though, data consumers need to be assured that the authenticity and the integrity of the data has remained intact — especially for data that may have been created through a series of aggregations and transformations on many input data sets. In other words, information provenance should be one of the main focuses for any system that wishes to facilitate the sharing of sensitive mHealth data. Creating such a trusted and secure data sharing ecosystem for mHealth apps and devices is difficult, however, as they are implemented with different technologies and managed by different organizations. Furthermore, many mHealth devices use ultra-low-power micro-controllers, which lack the kinds of sophisticated Memory Management Units (MMUs) required to sufficiently isolate sensitive application code and data.

In this thesis, we present an end-to-end solution for providing information provenance for mHealth data, which begins by securing mHealth data at its source: the mHealth device. To this end, we devise a memory-isolation method that combines compiler-inserted code and Memory Protection Unit (MPU) hardware to protect application code and data on ultra-low-power micro-controllers. Then we address the security of mHealth data outside of the source (e.g., data that has been uploaded to smartphone or remote-server) with our health-data system, Amanuensis, which uses Blockchain and Trusted Execution Environment (TEE) technologies to provide confidential, yet verifiable, data storage and computation for mHealth data. Finally, we look at identity privacy and data freshness issues introduced by the use of blockchain and TEEs. Namely, we present a privacy-preserving solution for blockchain transactions, and a freshness solution for data access-control lists retrieved from the blockchain.


Namya Malik. SPLICEcube Architecture: An Extensible Wi-Fi Monitoring Architecture for Smart-Home Networks. Master's thesis, May 2022. [Details]

The vision of smart homes is rapidly becoming a reality, as the Internet of Things and other smart devices are deployed widely. Although smart devices offer convenience, they also create a significant management problem for home residents. With a large number and variety of devices in the home, residents may find it difficult to monitor, or even locate, devices. A central controller that brings all the home’s smart devices under secure management and a unified interface would help homeowners and residents track and manage their devices.

We envision a solution called the SPLICEcube whose goal is to detect smart devices, locate them in three dimensions within the home, securely monitor their network traffic, and keep an inventory of devices and important device information throughout the device’s lifecycle. The SPLICEcube system consists of the following components: 1) a main cube, which is a centralized hub that incorporates and expands on the functionality of the home router, 2) a database that holds network data, and 3) a set of support cubelets that can be used to extend the range of the network and assist in gathering network data.

To deliver this vision of identifying, securing, and managing smart devices, we introduce an architecture that facilitates intelligent research applications (such as network anomaly detection, intrusion detection, device localization, and device firmware updates) to be integrated into the SPLICEcube. In this thesis, we design a general-purpose Wi-Fi architecture that underpins the SPLICEcube. The architecture specifically showcases the functionality of the cubelets (Wi-Fi frame detection, Wi-Fi frame parsing, and transmission to cube), the functionality of the cube (routing, reception from cubelets, information storage, data disposal, and research application integration), and the functionality of the database (network data storage). We build and evaluate a prototype implementation to demonstrate our approach is scalable to accommodate new devices and extensible to support different applications. Specifically, we demonstrate a successful proof-of-concept use of the SPLICEcube architecture by integrating a security research application: an "Inside-Outside detection" system that classifies an observed Wi-Fi device as being inside or outside the home.


Gregory Mazzaro, Kyle Gallagher, Kelly Sherbondy, Alex Bouvy, Beatrice Perez, Timothy Pierson, and David Kotz. Harmonic response vs. target orientation: a preliminary study of the effect of polarization on nonlinear junction detection. Proceedings of the SPIE Radar Sensor Technology XXVI. May 27, 2022. [Details]

When an electromagnetically-nonlinear radar target is illuminated by a high-power stepped-frequency probe, a sequence of harmonics is unintentionally emitted by that target. Detection of the target is accomplished by receiving stimulated emissions somewhere in the sequence, while ranging is accomplished by processing amplitude and phase recorded at multiple harmonics across the sequence. The strength of the harmonics reflected from an electronic target depends greatly upon the orientation of that target (or equivalently, the orientation of the radar antennas). Data collected on handheld wireless devices reveals the harmonic angular-dependence of commercially-available electronics. Data collected on nonlinearly-terminated printed circuit boards implies the origin of this dependency. The results of this work suggest that electronic targets may be classified and ultimately identified by their unique harmonic-response-vs.-angle patterns.

Lisa A. Marsch, Ching-Hua Chen, Sara R. Adams, Asma Asyyed, Monique B. Does, Saeed Hassanpour, Emily Hichborn, Melanie Jackson-Morris, Nicholas C. Jacobson, Heather K. Jones, David Kotz, Chantal A. Lambert-Harris, Zhiguo Li, Bethany McLeman, Varun Mishra, Catherine Stanger, Geetha Subramaniam, Weiyi Wu, and Cynthia I. Campbell. The Feasibility and Utility of Harnessing Digital Health to Understand Clinical Trajectories in Medication Treatment for Opioid Use Disorder: D-TECT Study Design and Methodological Considerations. Frontiers in Psychiatry. April 29, 2022. Section: Addictive Disorders. [Details]

Introduction: Across the U.S., the prevalence of opioid use disorder (OUD) and the rates of opioid overdoses have risen precipitously in recent years. Several effective medications for OUD (MOUD) exist and have been shown to be life-saving. A large volume of research has identified a confluence of factors that predict attrition and continued substance use during substance use disorder treatment. However, much of this literature has examined a small set of potential moderators or mediators of outcomes in MOUD treatment and may lead to over-simplified accounts of treatment non-adherence. Digital health methodologies offer great promise for capturing intensive, longitudinal ecologically-valid data from individuals in MOUD treatment to extend our understanding of factors that impact treatment engagement and outcomes.

Methods: This paper describes the protocol (including the study design and methodological considerations) from a novel study supported by the National Drug Abuse Treatment Clinical Trials Network at the National Institute on Drug Abuse (NIDA). This study (D-TECT) primarily seeks to evaluate the feasibility of collecting ecological momentary assessment (EMA), smartphone and smartwatch sensor data, and social media data among patients in outpatient MOUD treatment. It secondarily seeks to examine the utility of EMA, digital sensing, and social media data (separately and compared to one another) in predicting MOUD treatment retention, opioid use events, and medication adherence [as captured in electronic health records (EHR) and EMA data]. To our knowledge, this is the first project to include all three sources of digitally derived data (EMA, digital sensing, and social media) in understanding the clinical trajectories of patients in MOUD treatment. These multiple data streams will allow us to understand the relative and combined utility of collecting digital data from these diverse data sources. The inclusion of EHR data allows us to focus on the utility of digital health data in predicting objectively measured clinical outcomes.

Discussion: Results may be useful in elucidating novel relations between digital data sources and OUD treatment outcomes. It may also inform approaches to enhancing outcomes measurement in clinical trials by allowing for the assessment of dynamic interactions between individuals' daily lives and their MOUD treatment response.

Clinical Trial Registration: Identifier: NCT04535583.


Xiaohui Liang, John A. Batsis, Youxiang Zhu, Tiffany M. Driesse, Robert M. Roth, David Kotz, and Brian MacWhinney. Evaluating Voice-Assistant Commands for Dementia Detection. Computer Speech and Language. March 2022. Special Issue on Speech Based Evaluation of Neurological Diseases. [Details]

Early detection of cognitive decline involved in Alzheimer’s Disease and Related Dementias (ADRD) in older adults living alone is essential for developing, planning, and initiating interventions and support systems to improve users’ everyday function and quality of life. In this paper, we explore the voice commands using a Voice-Assistant System (VAS), i.e., Amazon Alexa, from 40 older adults who were either Healthy Control (HC) participants or Mild Cognitive Impairment (MCI) participants, age 65 or older. We evaluated the data collected from voice commands, cognitive assessments, and interviews and surveys using a structured protocol. We extracted 163 unique command-relevant features from each participant’s use of the VAS. We then built machine-learning models including 1-layer/2-layer neural networks, support vector machines, decision tree, and random forest, for classification and comparison with standard cognitive assessment scores, e.g., Montreal Cognitive Assessment (MoCA). Our classification models using fusion features achieved an accuracy of 68%, and our regression model resulted in a Root-Mean-Square Error (RMSE) score of 3.53. Our Decision Tree (DT) and Random Forest (RF) models using selected features achieved higher classification accuracy 80%–90%. Finally, we analyzed the contribution of each feature set to the model output, thus revealing the commands and features most useful in inferring the participants’ cognitive status. We found that features of overall performance, features of music-related commands, features of call-related commands, and features from Automatic Speech Recognition (ASR) were the top-four feature sets most impactful on inference accuracy. The results from this controlled study demonstrate the promise of future home-based cognitive assessments using Voice-Assistant Systems.

Beatrice Perez, Gregory Mazzaro, Timothy J. Pierson, and David Kotz. Detecting the Presence of Electronic Devices in Smart Homes Using Harmonic Radar. Remote Sensing. January 2022. Special issue on Nonlinear Junction Detection and Harmonic Radar. [Details]

Data about users is collected constantly by phones, cameras, Internet websites, and others. The advent of so-called ‘Smart Things' now enable ever-more sensitive data to be collected inside that most private of spaces: the home. The first step in helping users regain control of their information (inside their home) is to alert them to the presence of potentially unwanted electronics. In this paper, we present a system that could help homeowners (or home dwellers) find electronic devices in their living space. Specifically, we demonstrate the use of harmonic radars (sometimes called nonlinear junction detectors), which have also been used in applications ranging from explosives detection to insect tracking. We adapt this radar technology to detect consumer electronics in a home setting and show that we can indeed accurately detect the presence of even ‘simple’ electronic devices like a smart lightbulb. We evaluate the performance of our radar in both wired and over-the-air transmission scenarios.

2021:
Shengjie Bi and David Kotz. Eating detection with a head-mounted video camera. Technical Report, December 2021. [Details]

In this paper, we present a computer-vision based approach to detect eating. Specifically, our goal is to develop a wearable system that is effective and robust enough to automatically detect when people eat, and for how long. We collected video from a cap-mounted camera on 10 participants for about 55 hours in free-living conditions. We evaluated performance of eating detection with four different Convolutional Neural Network (CNN) models. The best model achieved accuracy 90.9% and F1 score 78.7% for eating detection with a 1-minute resolution. We also discuss the resources needed to deploy a 3D CNN model in wearable or mobile platforms, in terms of computation, memory, and power. We believe this paper is the first work to experiment with video-based (rather than image-based) eating detection in free-living scenarios.

Sougata Sen and David Kotz. VibeRing: Using vibrations from a smart ring as an out-of-band channel for sharing secret keys. Journal of Pervasive and Mobile Computing. December 2021. [Details]

Many Internet of Things (IoT) devices are capable of sensing their environment, communicating with other devices, and actuating on their environment. Some of these IoT devices, herein known as “smartThings”, collect meaningful information from raw data when they are in use and in physical contact with their user (e.g., a blood-glucose monitor); the smartThing’s wireless connectivity allows it to transfer that data to its user’s trusted device, such as a smartphone. However, an adversary could impersonate the user and bootstrap a communication channel with the smartThing while the smartThing is being used by an oblivious legitimate user.

To address this problem, in this paper, we investigate the use of vibration, generated by a smartRing, as an out-of-band communication channel to unobtrusively share a secret with a smartThing. This exchanged secret can be used to bootstrap a secure wireless channel over which the smartphone (or another trusted device) and the smartThing can communicate. We present the design, implementation, and evaluation of this system, which we call VibeRing. We describe the hardware and software details of the smartThing and smartRing. Through a user study we demonstrate that it is possible to share a secret with various objects quickly, accurately and securely as compared to several existing techniques. Overall, we successfully exchange a secret between a smartRing and various smartThings, at least 85.9% of the time. We show that VibeRing can perform this exchange at 12.5 bits/second at a bit error rate of less than 2.5%. We also show that VibeRing is robust to the smartThing’s constituent material as well as the holding style. Finally, we demonstrate that a nearby adversary cannot decode or modify the message exchanged between the trusted devices.


Varun Mishra. Towards Effective Delivery of Digital Interventions for Mental and Behavioral Health. PhD thesis, September 2021. [Details]

The pervasiveness of sensor-rich mobile, wearable, and IoT devices has enabled researchers to passively sense various user traits and characteristics, which in turn have the potential to detect and predict different mental and behavioral health outcomes. Upon detecting or anticipating a negative outcome, the same devices can be used to deliver in-the-moment in- terventions and support to help users. One important factor that determines the effectiveness of digital health interventions is delivering them at the right time: (1) when a person needs support, i.e., at or before the onset of a negative outcome, or a psychological or contextual state that might lead to that outcome (state-of-vulnerability); and (2) when a person is able and willing to receive, process, and use the support provided (state-of-receptivity). In this dissertation, we present our work on determining when to deliver interventions by exploring and detecting both vulnerability and receptivity.

In the first part of the thesis, we discuss our work on accurate sensing and detection of different states-of-vulnerability. We start by discussing our work on advancing the field of physiological stress sensing. We took the first step towards testing the reproducibility and validity of our methods and machine-learning models for stress detection. To this end, we analyzed data from 90 participants from four independent controlled studies, using two different types of sensors, with different study protocols and research goals. We evaluated new methods to improve the performance of stress-detection models and found that our methods led to a consistent increase in performance across all studies, irrespective of the device type, sensor type, or the type of stressor. Our thorough exploration of reproducibility in a controlled environment provides a critical foundation for deeper study of such methods, and is a prerequisite for tackling reproducibility in free-living conditions.

Next, we present our work on detecting at-risk indicators for patients undergoing Opioid Use Disorder (OUD) treatment. We conducted a 12-week study with 59 patients undergoing an OUD treatment and collected sensor data, like location, physical activity, sleep, and heart rate, from smartphones and wearables. We used the data collected to formulate low- level contextual features and high-level behavioral features and explored the feasibility of detecting self-reported stress, craving, and mood of the participants. Our results show that adaptive, personalized models can detect different at-risk behaviors with the area under the receiver operating characteristic (AUROC) values of up to 0.85.

In the second part of this dissertation, we discuss our contributions in the domain of state-of-receptivity for digital health interventions. We start by conducting a study with 189 participants in Switzerland to explore participant receptivity towards actual physical activity behavior change interventions and report novel and significant results, e.g., being more receptive to interventions leads to higher goal completion likelihood. We further built machine-learning models to predict state-of-receptivity and deployed those models in a real-world study with participants in the United States to evaluate their effectiveness. Our results show that participants were more receptive to interventions delivered at moments detected as ‘receptive’ by our models.

In addition to receptivity in daily living conditions, we explored how participants interact with affective health interventions while driving. We analyzed longitudinal data from 10 participants driving in their day-to-day lives for two months. In this exploratory work, we found that several high-level trip factors (traffic flow, trip length, and vehicle occupancy) and in-the-moment factors (road type, average speed, and braking behavior) showed significant associations with the participant’s decision to start or cancel an intervention. Based on our analysis, we provide solid recommendations on delivering interventions to maximize responsiveness and effectiveness and minimize the burden on the drivers.

Overall, this dissertation makes significant contributions to the respective sub-fields by addressing fundamental challenges, advancing the current state-of-the-art, and contribut- ing new knowledge, thereby laying a solid foundation for designing, implementing, and delivering future digital health interventions.


Paul Gralla. An inside vs. outside classification system for Wi-Fi IoT devices. June 2021. Undergraduate Thesis. [Details]

We are entering an era in which Smart Devices are increasingly integrated into our daily lives. Everyday objects are gaining computational power to interact with their environments and communicate with each other and the world via the Internet. While the integration of such devices offers many potential benefits to their users, it also gives rise to a unique set of challenges. One of those challenges is to detect whether a device belongs to one’s own ecosystem, or to a neighbor – or represents an unexpected adversary. An important part of determining whether a device is friend or adversary is to detect whether a device’s location is within the physical boundaries of one’s space (e.g. office, classroom, home). In this thesis we propose a system that is able to decide with 82% accuracy whether the location of an IoT device is inside or outside of a defined space based on a small number of transmitted Wi- Fi frames. The classification is achieved by leveraging a machine-learning classifier trained and tested on RSSI data of Wi-Fi transmissions recorded by three or more observers. In an initialization phase the classifier is trained by the user on Wi-Fi transmissions of a variety of locations, inside (and outside). The system can be built with off-the-shelf Wi-Fi observing devices that do not require any special hardware modifications. With the exception of the training period, the system can accurately classify the indoor/outdoor state of target devices without any cooperation from the user or from the target devices.

Shengjie Bi. Detection of health-related behaviours using head-mounted devices. PhD thesis, May 2021. PhD Dissertation. [Details]

The detection of health-related behaviors is the basis of many mobile-sensing applications for healthcare and can trigger other inquiries or interventions. Wearable sensors have been widely used for mobile sensing due to their ever-decreasing cost, ease of deployment, and ability to provide continuous monitoring. In this dissertation, we develop a generalizable approach to sensing eating-related behavior.

First, we developed Auracle, a wearable earpiece that can automatically detect eating episodes. Using an off-the-shelf contact microphone placed behind the ear, Auracle captures the sound of a person chewing as it passes through the head. This audio data is then processed by a custom circuit board. We collected data with 14 participants for 32 hours in free-living conditions and achieved accuracy exceeding 92.8% and F1 score exceeding77.5% for eating detection with 1-minute resolution.

Second, we adapted Auracle for measuring children’s eating behavior, and improved the accuracy and robustness of the eating-activity detection algorithms. We used this improved prototype in a laboratory study with a sample of 10 children for 60 total sessions and collected 22.3 hours of data in both meal and snack scenarios. Overall, we achieved 95.5% accuracy and 95.7% F1 score for eating detection with 1-minute resolution.

Third, we developed a computer-vision approach for eating detection in free-living scenarios. Using a miniature head-mounted camera, we collected data with 10 participants for about 55 hours. The camera was fixed under the brim of a cap, pointing to the mouth of the wearer and continuously recording video (but not audio) throughout their normal daily activity. We evaluated performance for eating detection using four different Convolutional Neural Network (CNN) models. The best model achieved 90.9% accuracy and 78.7%F1 score for eating detection with 1-minute resolution. Finally, we validated the feasibility of deploying the 3D CNN model in wearable or mobile platforms when considering computation, memory, and power constraints.


Shengjie Bi, Tao Wang, Nicole Tobias, Josephine Nordrum, Robert Halvorsen, Ron Peterson, Kelly Caine, Xing-Dong Yang, Kofi Odame, Ryan Halter, Jacob Sorber, and David Kotz. System for detecting eating with sensor mounted by the ear. U.S. Patent Application PCT/US2019/044317; Worldwide Patent Application WO2020028481A9, February 1, 2021. Priority date 2018-07-31; Filed 2019-07-31; Amended 2021-02-01; Rejected 2025-07-02. Abandoned. [Details]

A wearable device for detecting eating episodes uses a contact microphone to provide audio signals through an analog front end to an analog-to-digital converter to digitize the audio and provide digitized audio to a processor; and a processor configured with firmware in a memory to extract features from the digitized audio. A classifier determines eating episodes from the extracted features. In embodiments, messages describing the detected eating episodes are transmitted to a cell phone, insulin pump, or camera configured to record video of the wearer's mouth.

John A. Batsis, Curtis L. Petersen, Matthew M. Clark, Summer B. Cook, Francisco Lopez-Jimenez, Rima I. Al-Nimr, Dawna Pidgeon, David Kotz, Todd A. Mackenzie, and Steven J. Bartels. A Weight-Loss Intervention Augmented by a Wearable Device in Rural Older Adults with Obesity: A Feasibility Study. Journals of Gerontology - Series A: Biological Sciences and Medical Sciences. January 2021. First published 8 May 2020. [Details]

Background: Older persons with obesity aged 65+ residing in rural areas have reduced access to weight management programs due to geographic isolation. The ability to integrate technology into health promotion interventions shows a potential to reach this underserved population.

Methods: A 12-week pilot in 28 older rural adults with obesity (body mass index [BMI] ≥ 30 kg/m2) was conducted at a community aging center. The intervention consisted of individualized, weekly dietitian visits focusing on behavior therapy and caloric restriction with twice weekly physical therapist-led group strengthening training classes in a community-based aging center. All participants were provided a Fitbit Flex 2. An aerobic activity prescription outside the strength training classes was provided.

Results: Mean age was 72.9 ± 5.3 years (82% female). Baseline BMI was 37.1 kg/m2, and waist circumference was 120.0 ± 33.0 cm. Mean weight loss (pre/post) was 4.6 ± 3.2 kg (4.9 ± 3.4%; p < .001). Of the 40 eligible participants, 33 (75%) enrolled, and the completion rate was high (84.8%). Objective measures of physical function improved at follow-up: 6-minute walk test improved: 35.7 ± 41.2 m (p < .001); gait speed improved: 0.10 ± 0.24 m/s (p = .04); and five-times sit-to-stand improved by 2.1 seconds (p < .001). Subjective measures of late-life function improved (5.2 ± 7.1 points, p = .003), as did Patient-Reported Outcome Measurement Information Systems mental and physical health scores (5.0 ± 5.7 and 4.4 ± 5.0, both p < .001). Participants wore their Fitbit 93.9% of all intervention days, and were overall satisfied with the trial (4.5/5.0, 1–5 low–high) and with Fitbit (4.0/5.0).

Conclusions: A multicomponent obesity intervention incorporating a wearable device is feasible and acceptable to older adults with obesity, and potentially holds promise in enhancing health.


2020:
Shengjie Bi, Yiyang Lu, Nicole Tobias, Ella Ryan, Travis Masterson, Sougata Sen, Ryan Halter, Jacob Sorber, Diane Gilbert-Diamond, and David Kotz. Measuring children’s eating behavior with a wearable device. Proceedings of the IEEE International Conference on Healthcare Informatics (ICHI). December 2020. [Details]

Poor eating habits in children and teenagers can lead to obesity, eating disorders, or life-threatening health problems. Although researchers have studied children’s eating behavior for decades, the research community has had limited technology to support the observation and measurement of fine-grained details of a child’s eating behavior. In this paper, we present the feasibility of adapting the Auracle, an existing research-grade earpiece designed to automatically and unobtrusively recognize eating behavior in adults, for measuring children’s eating behavior. We identified and addressed several challenges pertaining to monitoring eating behavior in children, paying particular attention to device fit and comfort. We also improved the accuracy and robustness of the eating-activity detection algorithms. We used this improved prototype in a lab study with a sample of 10 children for 60 total sessions and collected 22.3 hours of data in both meal and snack scenarios. Overall, we achieved an accuracy exceeding 85.0% and an F1 score exceeding 84.2% for eating detection with a 3-second resolution, and a 95.5% accuracy and a 95.7% F1 score for eating detection with a 1-minute resolution.

Sougata Sen and David Kotz. VibeRing: Using vibrations from a smart ring as an out-of-band channel for sharing secret keys. Proceedings of the International Conference on the Internet of Things (IoT). October 2020. [Details]

With the rapid growth in the number of IoT devices that have wireless communication capabilities, and sensitive information collection capabilities, it is becoming increasingly necessary to ensure that these devices communicate securely with only authorized devices. A major requirement of this secure communication is to ensure that both the devices share a secret, which can be used for secure pairing and encrypted communication. Manually imparting this secret to these devices becomes an unnecessary overhead, especially when the device interaction is transient. In this paper, we empirically investigate the possibility of using an out-of-band communication channel -- vibration, generated by a custom smart ring, to share a secret with a smart IoT device. This exchanged secret can be used to bootstrap a secure wireless channel over which the devices can communicate. We believe that in future IoT devices can use such a technique to seamlessly connect with authorized devices with minimal user interaction overhead. In this paper, we specifically investigate (a) the feasibility of using vibration generated by a custom wearable for communication, (b) the effect of various parameters on this communication channel, and (c) the possibility of information manipulation by an adversary or information leakage to an adversary. For this investigation, we conducted a controlled study as well as a user study with 12 participants. In the controlled study, we could successfully share messages through vibrations with a bit error rate of less than 2.5%. Additionally, through the user study we demonstrate that it is possible to share messages with various types of objects accurately, quickly and securely as compared to several existing techniques. Overall, we find that in the best case we can exchange 85.9% messages successfully with a smart device.

Curtis Lee Petersen, Ryan Halter, David Kotz, Lorie Loeb, Summer Cook, Dawna Pidgeon, Brock C. Christensen, and John A. Batsis. Using Natural Language Processing and Sentiment Analysis to Augment Traditional User-Centered Design: Development and Usability Study. JMIR mHealth and uHealth. August 2020. [Details]

Background: Sarcopenia, defined as the age-associated loss of muscle mass and strength, can be effectively mitigated through resistance-based physical activity. With compliance at approximately 40% for home-based exercise prescriptions, implementing a remote sensing system would help patients and clinicians to better understand treatment progress and increase compliance. The inclusion of end users in the development of mobile apps for remote-sensing systems can ensure that they are both user friendly and facilitate compliance. With advancements in natural language processing (NLP), there is potential for these methods to be used with data collected through the user-centered design process.

Objective: This study aims to develop a mobile app for a novel device through a user-centered design process with both older adults and clinicians while exploring whether data collected through this process can be used in NLP and sentiment analysis.

Methods: Through a user-centered design process, we conducted semistructured interviews during the development of a geriatric-friendly Bluetooth-connected resistance exercise band app. We interviewed patients and clinicians at weeks 0, 5, and 10 of the app development. Each semistructured interview consisted of heuristic evaluations, cognitive walkthroughs, and observations. We used the Bing sentiment library for a sentiment analysis of interview transcripts and then applied NLP-based latent Dirichlet allocation (LDA) topic modeling to identify differences and similarities in patient and clinician participant interviews. Sentiment was defined as the sum of positive and negative words (each word with a +1 or −1 value). To assess utility, we used quantitative assessment questionnaires—System Usability Scale (SUS) and Usefulness, Satisfaction, and Ease of use (USE). Finally, we used multivariate linear models—adjusting for age, sex, subject group (clinician vs patient), and development—to explore the association between sentiment analysis and SUS and USE outcomes.

Results: The mean age of the 22 participants was 68 (SD 14) years, and 17 (77%) were female. The overall mean SUS and USE scores were 66.4 (SD 13.6) and 41.3 (SD 15.2), respectively. Both patients and clinicians provided valuable insights into the needs of older adults when designing and building an app. The mean positive-negative sentiment per sentence was 0.19 (SD 0.21) and 0.47 (SD 0.21) for patient and clinician interviews, respectively. We found a positive association with positive sentiment in an interview and SUS score (ß=1.38; 95% CI 0.37 to 2.39; P=.01). There was no significant association between sentiment and the USE score. The LDA analysis found no overlap between patients and clinicians in the 8 identified topics.

Conclusions: Involving patients and clinicians allowed us to design and build an app that is user friendly for older adults while supporting compliance. This is the first analysis using NLP and usability questionnaires in the quantification of user-centered design of technology for older adults.


Xiaohui Liang, Ronald Peterson, and David Kotz. Securely Connecting Wearables to Ambient Displays with User Intent. IEEE Transactions on Dependable and Secure Computing. July 2020. [Details]

Wearables are often small and have limited user interfaces, hence they often wirelessly interface with a personal smartphone or a personal computer to relay information from the wearable for display. In this paper, we envision a new method LightTouch by which a wearable can establish a secure connection to an ambient display, such as a television or computer monitor, based on the user's intention to connect to the display. Such connections must be secure to prevent impersonation attacks, must work with unmodified display hardware, and must be easy to establish. LightTouch uses standard RF methods for communicating the data to display, securely bootstrapped with a key shared via a brightness channel between the low cost, low power, ambient light sensor of a wearable and the screen of the display. A screen touch gesture is adopted by users to ensure the modulation of screen brightness can be accurately and securely captured by the ambient light sensor. We further propose novel on-screen localization and correlation algorithms to improve security and reliability. Through experiments we demonstrate that LightTouch is compatible with current display and wearable designs, easy-to-use (5-6 seconds), reliable for connecting displays (98 percent success connection ratio), and secure against impersonation attacks.

Xiaohui Liang, Tianlong Yun, Ron Peterson, and David Kotz. Secure System For Coupling Wearable Devices To Computerized Devices with Displays. U.S. Patent 10,581,606, March 3, 2020. Priority date 2014-08-18, Filed 2015-08-18; Issued 2020-03-03. [Details]

A system has a first electronic device with optical sensor, digital radio transceiver, and processor with firmware; this device is typically portable or wearable. The system also has a computerized device with a display, a second digital radio transceiver, and a second processor with firmware. The first and computerized devices are configured to set up a digital radio link when in radio range. The second processor uses a spot on the display to optically transmit a digital message including a secret such as an encryption key or subkey and/or an authentication code adapted for authenticating an encrypting the radio link. The first device receives the digital message via its optical sensor, and uses the digital message to validate and establish encryption on the radio link. In embodiments, the system determines a location of the first device on the display and positions the transmission spot at the determined location.

John Batsis, Stephen Bartels, Rachel Dokko, Alexandra Zagaria, John Naslund, Elizabeth Carpenter-Song, and David Kotz. Opportunities to Improve a Mobile Obesity Wellness Intervention for Rural Older Adults with Obesity. Journal of Community Health. February 2020. [Details]

Older adults with obesity are at a high risk of decline, particularly in rural areas. Our study objective was to gain insights into how a potential Mobile Health Obesity Wellness Intervention (MOWI) in rural older adults with obesity, consisting of nutrition and exercise sessions, could be helpful to improve physical function. A qualitative methods study was conducted in a rural community, community-based aging center. Four community leaders, 7 clinicians and 29 patient participants underwent focus groups and semi-structured interviews. All participants had a favorable view of MOWI and saw its potential to improve health and create accountability. Participants noted that MOWI could overcome geographic barriers and provided feedback about components that could improve implementation. There was expressed enthusiasm over its potential to improve health. The use of technology in older adults with obesity in rural areas has considerable promise. There is potential that this intervention could potentially extend to distant areas in rural America that can surmount accessibility barriers. If successful, this intervention could potentially alter healthcare delivery by enhancing health promotion in a remote, geographically constrained communities. MOWI has the potential to reach older adults with obesity using novel methods in geographically isolated regions.

2019:
George Boateng, Vivian Genaro Motti, Varun Mishra, John A. Batsis, Josiah Hester, and David Kotz. Experience: Design, Development and Evaluation of a Wearable Device for mHealth Applications. Proceedings of the International Conference on Mobile Computing and Networking (MobiCom). October 2019. [Details]

Wrist-worn devices hold great potential as a platform for mobile health (mHealth) applications because they comprise a familiar, convenient form factor and can embed sensors in proximity to the human body. Despite this potential, however, they are severely limited in battery life, storage, bandwidth, computing power, and screen size. In this paper, we describe the experience of the research and development team designing, implementing and evaluating Amulet -- an open-hardware, open-software wrist-worn computing device -- and its experience using Amulet to deploy mHealth apps in the field. In the past five years the team conducted 11 studies in the lab and in the field, involving 204 participants and collecting over 77,780 hours of sensor data. We describe the technical issues the team encountered and the lessons they learned, and conclude with a set of recommendations. We anticipate the experience described herein will be useful for the development of other research-oriented computing platforms. It should also be useful for researchers interested in developing and deploying mHealth applications, whether with the Amulet system or with other wearable platforms.

Sougata Sen, Varun Mishra, and David Kotz. Using vibrations from a SmartRing as an out-of-band channel for sharing secret keys. Adjunct Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp). September 2019. [Details]

With the rapid growth in the number of Internet of Things (IoT) devices with wireless communication capabilities, and sensitive information collection capabilities, it is becoming increasingly necessary to ensure that these devices communicate securely with only authorized devices. A major requirement of this secure communication is to ensure that both the devices share a secret, which can be used for secure pairing and encrypted communication. Manually imparting this secret to these devices becomes an unnecessary overhead, especially when the device interaction is transient. In this work, we empirically investigate the possibility of using an out-of-band communication channel -- vibration, generated by a custom smartRing -- to share a secret with a compatible IoT device. Through a user study with 12 participants we show that in the best case we can exchange 85.9% messages successfully. Our technique demonstrates the possibility of sharing messages accurately, quickly and securely as compared to several existing techniques.

John A. Batsis, Alexandra B. Zagaria, Ryan J. Halter, George G. Boateng, Patrick Proctor, Stephen J. Bartels, and David Kotz. Use of Amulet in behavioral change for geriatric obesity management. Journal of Digital Health. June 2019. [Details]

Background: Obesity in older adults is a significant public health concern. Weight-loss interventions are known to improve physical function but risk the development of sarcopenia. Mobile health devices have the potential to augment existing interventions and, if designed accordingly, could improve one’s physical activity and strength in routine physical activity interventions. Methods and results: We present Amulet, a mobile health device that has the capability of engaging patients in physical activity. The purpose of this article is to discuss the development of applications that are tailored to older adults with obesity, with the intention to engage and improve their health. Conclusions: Using a team-science approach, Amulet has the potential, as an open-source mobile health device, to tailor activity interventions to older adults.

John A. Batsis, George G. Boateng, Lillian M. Seo, Curtis L. Petersen, Karen L. Fortuna, Emily V. Wechsler, Ronald J. Peterson, Summer B. Cook, Dawna Pidgeon, Rachel S. Dokko, Ryan J. Halter, and David F. Kotz. Development and Usability Assessment of a Connected Resistance Exercise Band Application for Strength-Monitoring. World Academy of Science, Engineering and Technology. June 2019. Presented at the International Conference on Body Area Networks (ICBAN). [Details]

Resistance exercise bands are a core component of any physical activity strengthening program. Strength training can mitigate the development of sarcopenia, the loss of muscle mass or strength and function with aging. Yet, the adherence of such behavioral exercise strategies in a home-based setting is fraught with issues of monitoring and compliance. Our group developed a Bluetooth-enabled resistance exercise band capable of transmitting data to an open-source platform. In this work, we developed an application to capture this information in real-time and conducted three usability studies in two mixed-aged groups of participants (n=6 each) and a group of older adults with obesity participating in a weight-loss intervention (n=20). The system was favorable, acceptable and provided iterative information that could assist in future deployment on ubiquitous platforms. Our formative work provides the foundation to deliver home-based monitoring interventions in a high-risk, older adult population.

John A. Batsis, John A. Naslund, Alexandra B. Zagaria, David Kotz, Rachel Dokko, Stephen J. Bartels, and Elizabeth Carpenter-Song. Technology for Behavioral Change in Rural Older Adults with Obesity. Journal of Nutrition in Gerontology and Geriatrics. April 2019. [Details]

Background: Mobile health (mHealth) technologies comprise a multidisciplinary treatment strategy providing potential solutions for overcoming challenges of successfully delivering health promotion interventions in rural areas. We evaluated the potential of using technology in a high-risk population.

Methods: We conducted a convergent, parallel mixed-methods study using semi-structured interviews, focus groups, and self-reported questionnaires, using purposive sampling of 29 older adults, 4 community leaders and 7 clinicians in a rural setting. We developed codes informed by thematic analysis and assessed the quantitative data using descriptive statistics.

Results: All groups expressed that mHealth could improve health behaviors. Older adults were optimistic that mHealth could track health. Participants believed they could improve patient insight into health, motivating change and assuring accountability. Barriers to using technology were described, including infrastructure.

Conclusions: Older rural adults with obesity expressed excitement about the use of mHealth technologies to improve their health, yet barriers to implementation exist.


Emily Greene, Patrick Proctor, and David Kotz. Secure Sharing of mHealth Data Streams through Cryptographically-Enforced Access Control. Journal of Smart Health. April 2019. [Details]

Owners of mobile-health apps and devices often want to share their mHealth data with others, such as physicians, therapists, coaches, and caregivers. For privacy reasons, however, they typically want to share a limited subset of their information with each recipient according to their preferences. In this paper, we introduce ShareHealth, a scalable, usable, and practical system that allows mHealth-data owners to specify access-control policies and to cryptographically enforce those policies so that only parties with the proper corresponding permissions are able to decrypt data. The design and prototype implementation of this system make three contributions: (1) they apply cryptographically-enforced access-control measures to stream-based (specifically mHealth) data, (2) they recognize the temporal nature of mHealth data streams and support revocation of access to part or all of a data stream, and (3) they depart from the vendor- and device-specific silos of mHealth data by implementing a secure end-to-end system that can be applied to data collected from a variety of mHealth apps and devices.

Shrirang Mare, Reza Rawassizadeh, Ronald Peterson, and David Kotz. Continuous Smartphone Authentication using Wristbands. Proceedings of the Workshop on Usable Security (USEC). February 2019. [Details]

Many users find current smartphone authentication methods (PINs, swipe patterns) to be burdensome, leading them to weaken or disable the authentication. Although some phones support methods to ease the burden (such as fingerprint readers), these methods require active participation by the user and do not verify the user’s identity after the phone is unlocked. We propose CSAW, a continuous smartphone authentication method that leverages wristbands to verify that the phone is in the hands of its owner. In CSAW, users wear a wristband (a smartwatch or a fitness band) with built-in motion sensors, and by comparing the wristband’s motion with the phone’s motion, CSAW continuously produces a score indicating its confidence that the person holding (and using) the phone is the person wearing the wristband. This score provides the foundation for a wide range of authentication decisions (e.g., unlocking phone, deauthentication, or limiting phone access). Through two user studies (N=27,11) we evaluated CSAW’s accuracy, usability, and security. Our experimental evaluation demonstrates that CSAW was able to conduct initial authentication with over 99% accuracy and continuous authentication with over 96.5% accuracy.

David Kotz. Amulet: an open-source wrist-worn platform for mHealth research and education. Proceedings of the Workshop on Networked Healthcare Technology (NetHealth). January 2019. [Details]

The advent of mobile and wearable computing technology has opened up tremendous opportunities for health and wellness applications. It is increasingly possible for individuals to wear devices that can sense their physiology or health-related behaviors, collecting valuable data in support of diagnosis, treatment, public health, or other applications. From a researcher’s point of view, the commercial availability of these “mHealth” devices has made it feasible to conduct scientific studies of health conditions and to explore health-related interventions. It remains difficult, however, to conduct systems work or other experimental research involving the hardware, software, security, and networking aspects of mobile and wearable technology. In this paper we describe the Amulet platform, an open-hardware, open-software wrist-worn computing device designed specifically for mHealth applications. Our position is that the Amulet is an inexpensive platform for research and education, and we encourage the mHealth community to explore its potential.

2018:
John A. Batsis, Alexandra Zagaria, David F. Kotz, Stephen J. Bartels, George G. Boateng, Patrick O. Proctor, Ryan J. Halter, and Elizabeth A. Carpenter-Song. Usability evaluation for the Amulet wearable device in rural older adults with obesity. Gerontechnology. October 2018. [Details]

Mobile health (mHealth) interventions hold the promise of augmenting existing health promotion interventions. Older adults present unique challenges in advancing new models of health promotion using technology including sensory limitations and less experience with mHealth, underscoring the need for specialized usability testing. We use an open-source mHealth device as a case example for its integration in a newly designed health services intervention. We performed a convergent, parallel mixed-methods study including semi-structured interviews, focus groups, and questionnaires, using purposive sampling of 29 older adults, 4 community leaders, and 7 clinicians in a rural setting. We transcribed the data, developed codes informed by thematic analysis using inductive and deductive methods, and assessed the quantitative data using descriptive statistics. Our results suggest the importance of end-users in user-centered design of mHealth devices and that aesthetics are critically important. The prototype could potentially be feasibly integrated within health behavior interventions. Centralized dashboards were desired by all participants and ecological momentary assessment could be an important part of monitoring. Concerns of mHealth, including the prototype device, include the device’s accuracy, its intrusiveness in daily life and privacy. Formative evaluations are critically important prior to deploying large-scale interventions.

Varun Mishra, Gunnar Pope, Sarah Lord, Stephanie Lewia, Byron Lowens, Kelly Caine, Sougata Sen, Ryan Halter, and David Kotz. The Case for a Commodity Hardware Solution for Stress Detection. Proceedings of the Workshop on Mental Health: Sensing & Intervention. October 2018. [Details]

Timely detection of an individual's stress level has the potential to expedite and improve stress management, thereby reducing the risk of adverse health consequences that may arise due to unawareness or mismanagement of stress. Recent advances in wearable sensing have resulted in multiple approaches to detect and monitor stress with varying levels of accuracy. The most accurate methods, however, rely on clinical grade sensors strapped to the user. These sensors measure physiological signals of a person and are often bulky, custom-made, expensive, and/or in limited supply, hence limiting their large-scale adoption by researchers and the general public. In this paper, we explore the viability of commercially available off-the-shelf sensors for stress monitoring. The idea is to be able to use cheap, non-clinical sensors to capture physiological signals, and make inferences about the wearer's stress level based on that data. In this paper, we describe a system involving a popular off-the-shelf heart-rate monitor, the Polar H7; we evaluated our system in a lab setting with three well-validated stress-inducing stimuli with 26 participants. Our analysis shows that using the off-the-shelf sensor alone, we were able to detect stressful events with an F1 score of 0.81, on par with clinical-grade sensors.

Shengjie Bi, Tao Wang, Nicole Tobias, Josephine Nordrum, Shang Wang, George Halvorsen, Sougata Sen, Ronald Peterson, Kofi Odame, Kelly Caine, Ryan Halter, Jacob Sorber, and David Kotz. Auracle: Detecting Eating Episodes with an Ear-Mounted Sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (Ubicomp). September 2018. [Details]

In this paper, we propose Auracle, a wearable earpiece that can automatically recognize eating behavior. More specifically, in free-living conditions, we can recognize when and for how long a person is eating. Using an off-the-shelf contact microphone placed behind the ear, Auracle captures the sound of a person chewing as it passes through the bone and tissue of the head. This audio data is then processed by a custom analog/digital circuit board. To ensure reliable (yet comfortable) contact between microphone and skin, all hardware components are incorporated into a 3D-printed behind-the-head framework. We collected field data with 14 participants for 32 hours in free-living conditions and additional eating data with 10 participants for 2 hours in a laboratory setting. We achieved accuracy exceeding 92.8% and F1 score exceeding 77.5% for eating detection. Moreover, Auracle successfully detected 20-24 eating episodes (depending on the metrics) out of 26 in free-living conditions. We demonstrate that our custom device could sense, process, and classify audio data in real time. Additionally, we estimate Auracle can last 28.1 hours with a 110 mAh battery while communicating its observations of eating behavior to a smartphone over Bluetooth.

Shrirang Mare, Reza Rawassizadeh, Ronald Peterson, and David Kotz. SAW: Wristband-based authentication for desktop computers. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (Ubicomp). September 2018. [Details]

Token-based proximity authentication methods that authenticate users based on physical proximity are effortless, but lack explicit user intentionality, which may result in accidental logins. For example, a user may get logged in when she is near a computer or just passing by, even if she does not intend to use that computer. Lack of user intentionality in proximity-based methods makes them less suitable for multi-user shared computer environments, despite their desired usability benefits over passwords.

We present an authentication method for desktops called Seamless Authentication using Wristbands (SAW), which addresses the lack of intentionality limitation of proximity-based methods. SAW uses a low-effort user input step for explicitly conveying user intentionality, while keeping the overall usability of the method better than password-based methods. In SAW, a user wears a wristband that acts as the user’s identity token, and to authenticate to a desktop, the user provides a low-effort input by tapping a key on the keyboard multiple times or wiggling the mouse with the wristband hand. This input to the desktop conveys that someone wishes to log in to the desktop, and SAW verifies the user who wishes to log in by confirming the user’s proximity and correlating the received keyboard or mouse inputs with the user’s wrist movement, as measured by the wristband. In our feasibility user study (n=17), SAW proved quick to authenticate (within two seconds), with a low false-negative rate of 2.5% and worst-case false-positive rate of 1.8%. In our user perception study (n=16), a majority of the participants rated it as more usable than passwords.


Curtis L. Petersen, Emily V. Wechsler, Ryan J. Halter, George G. Boateng, Patrick O. Proctor, David F. Kotz, Summer B. Cook, and John A. Batsis. Detection and Monitoring of Repetitions Using an mHealth-Enabled Resistance Band. Proceedings of the IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE). September 2018. [Details]

Sarcopenia is defined as an age-related loss of muscle mass and strength which impairs physical function leading to disability and frailty. Resistance exercises are effective treatments for sarcopenia and are critical in mitigating weight-loss induced sarcopenia in older adults attempting to lose weight. Yet, adherence to home-based regimens, which is a cornerstone to lifestyle therapies, is poor and cannot be ascertained by clinicians as no objective methods exist to determine patient compliance outside of a supervised setting. Our group developed a Bluetooth connected resistance band that tests the ability to detect exercise repetitions. We recruited 6 patients aged 65 years and older and recorded 4 specific, physical therapist-led exercises. Three blinded reviewers examined the findings and we also applied a peak finding algorithm to the data. There were 16.6 repetitions per exercise across reviewers, with an intraclass correlation of 0.912 (95%CI: 0.853–0.953, p<0.001) between reviewers and the algorithm. Using this novel resistance band, we feasibly detected repetition of exercises in older adults.

Timothy J. Pierson. Secure Short-range Communications. PhD thesis, June 2018. Available as Dartmouth Computer Science Technical Report TR2018-845. [Details]

Analysts predict billions of everyday objects will soon become “smart” after designers add wireless communication capabilities. Collectively known as the Internet of Things (IoT), these newly communication-enabled devices are envisioned to collect and share data among themselves, with new devices entering and exiting a particular environment frequently. People and the devices they wear or carry may soon encounter dozens, possibly hundreds, of devices each day. Many of these devices will be encountered for the first time. Additionally, some of the information the devices share may have privacy or security implications. Furthermore, many of these devices will have limited or non-existent user interfaces, making manual configuration cumbersome. This situation suggests that devices that have never met, nor shared a secret, but that are in the same physical area, must have a way to securely communicate that requires minimal manual intervention. In this dissertation we present novel approaches to solve these short-range communication issues. Our techniques are simple to use, secure, and consistent with user intent. We first present a technique called Wanda that uses radio strength as a communication channel to securely impart information onto nearby devices. We focus on using Wanda to introduce new devices into an environment, but Wanda could be used to impart any type of information onto wireless devices, regardless of device type or manufacturer. Next we describe SNAP, a method for a single-antenna wireless device to determine when it is in close physical proximity to another wireless device. Because radio waves are invisible, a user may believe transmissions are coming from a nearby device when in fact the transmissions are coming from a distant adversary attempting to trick the user into accepting a malicious payload. Our approach significantly raises the bar for an adversary attempting such a trick. Finally, we present a solution called JamFi that exploits MIMO antennas and the Inverse-Square Law to securely transfer data between nearby devices while denying more distant adversaries the ability to recover the data. We find JamFi is able to facilitate reliable and secure communication between two devices in close physical proximity, even though they have never met nor shared a key.

Andrés D. Molina-Markham, Shrirang Mare, Ronald Peterson, Jr., and David Kotz. Continuous seamless mobile device authentication using a separate electronic wearable apparatus. U.S. Patent 9,961,547, May 1, 2018. Priority date 2016-09-30, Filed 2016-09-30; Issued 2018-05-01. [Details]

A technique performs a security operation. The technique includes receiving first activity data from a mobile device, the first activity data identifying activity by a user that is currently using the mobile device. The technique further includes receiving second activity data from an electronic wearable apparatus, the second activity data identifying physical activity by a wearer that is currently wearing the electronic wearable apparatus. The technique further includes, based on the first activity data received from the mobile device and the second activity data received from the electronic wearable apparatus, performing an assessment operation that provides an assessment result indicating whether the user that is currently using the mobile device and the wearer that is currently wearing the electronic wearable apparatus are the same person. With such a technique, authentication may be continuous but without burdening the user to repeatedly re-enter a password.

Varun Mishra, Byron Lowens, Sarah Lord, Kelly Caine, and David Kotz. Investigating Contextual Cues as Indicators for EMA Delivery. Technical Report, April 2018. [Details]

In this work, we attempt to determine whether the contextual information of a participant can be used to predict whether the participant will respond to a particular Ecological Momentary Assessment (EMA) prompt. We use a publicly available dataset for our work, and find that by using basic contextual features about the participant’s activity, conversation status, audio, and location, we can predict whether an EMA prompt triggered at a particular time will be answered with a precision of 0.647, which is significantly higher than a baseline precision of 0.410. Using this knowledge, the researchers conducting field studies can efficiently schedule EMA prompts and achieve higher response rates.

David Kotz, Ryan Halter, Cory Cornelius, Jacob Sorber, Minho Shin, Ronald Peterson, Shrirang Mare, Aarathi Prasad, Joseph Skinner, and Andrés Molina-Markham. Wearable computing device for secure control of physiological sensors and medical devices, with secure storage of medical records, and bioimpedance biometric. U.S. Patent 9,936,877; International Patent Application WO2013096954A1, April 10, 2018. This patent adds claims to its predecessor; Priority date 2011-12-23; Filed 2017-02-07; Issued 2018-04-10. [Details]

A wearable master electronic device (Amulet) has a processor with memory, the processor coupled to a body-area network (BAN) radio and uplink radio. The device has firmware for BAN communications with wearable nodes to receive data, and in an embodiment, send configuration data. The device has firmware for using the uplink radio to download apps and configurations, and upload data to a server. An embodiment has accelerometers in Amulet and wearable node, and firmware for using accelerometer readings to determine if node and Amulet are worn by the same subject. Other embodiments use pulse sensors or microphones in the Amulet and node to both identify a subject and verify the Amulet and node are worn by the same subject. Another embodiment uses a bioimpedance sensor to identify the subject. The wearable node may be an insulin pump, chemotherapy pump, TENS unit, cardiac monitor, or other device.

George Boateng, John A. Batsis, Patrick Proctor, Ryan Halter, and David Kotz. GeriActive: Wearable App for Monitoring and Encouraging Physical Activity among Older Adults. Proceedings of the IEEE Conference on Body Sensor Networks (BSN). March 2018. [Details]

The ability to monitor a person’s level of daily activity can inform self-management of physical activity and assist in augmenting behavioral interventions. For older adults, the importance of regular physical activity is critical to reduce the risk of long-term disability. In this work, we present GeriActive, an application on the Amulet wrist-worn device that monitors in real time older adults’ daily activity levels (low, moderate and vigorous), which we categorized using metabolic equivalents (METs). The app implements an activity-level detection model we developed using a linear Support Vector Machine (SVM). We trained our model using data from volunteer subjects (n=29) who performed common physical activities (sit, stand, lay down, walk and run) and obtained an accuracy of 94.3% with leave-one-subject-out (LOSO) cross-validation. We ran a week-long field study to evaluate the usability and battery life of the GeriActive system where 5 older adults wore the Amulet as it monitored their activity level. Their feedback showed that our system has the potential to be usable and useful. Our evaluation further revealed a battery life of at least 1 week. The results are promising, indicating that the app may be used for activity-level monitoring by individuals or researchers for health delivery interventions that could improve the health of older adults.

Rui Liu, Cory Cornelius, Reza Rawassizadeh, Ron Peterson, and David Kotz. Vocal Resonance: Using Internal Body Voice for Wearable Authentication. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (UbiComp). March 2018. [Details]

We observe the advent of body-area networks of pervasive wearable devices, whether for health monitoring, personal assistance, entertainment, or home automation. For many devices, it is critical to identify the wearer, allowing sensor data to be properly labeled or personalized behavior to be properly achieved. In this paper we propose the use of vocal resonance, that is, the sound of the person’s voice as it travels through the person’s body -- a method we anticipate would be suitable for devices worn on the head, neck, or chest. In this regard, we go well beyond the simple challenge of speaker recognition: we want to know who is wearing the device. We explore two machine-learning approaches that analyze voice samples from a small throat-mounted microphone and allow the device to determine whether (a) the speaker is indeed the expected person, and (b) the microphone-enabled device is physically on the speaker’s body. We collected data from 29 subjects, demonstrate the feasibility of a prototype, and show that our DNN method achieved balanced accuracy 0.914 for identification and 0.961 for verification by using an LSTM-based deep-learning model, while our efficient GMM method achieved balanced accuracy 0.875 for identification and 0.942 for verification.

Gunnar C. Pope, Varun Mishra, Stephanie Lewia, Byron Lowens, David Kotz, Sarah Lord, and Ryan Halter. An Ultra-Low Resource Wearable EDA Sensor Using Wavelet Compression. Proceedings of the IEEE Conference on Body Sensor Networks (BSN). March 2018. [Details]

This study presents an ultra-low resource platform for physiological sensing that uses on-chip wavelet compression to enable long-term recording of electrodermal activity (EDA) within a 64kB microcontroller. The design is implemented on a wearable platform and provides improvements in size and power compared to existing wearable technologies and was used in a lab setting to monitor EDA of 27 participants throughout a stress induction protocol. We demonstrate the device’s sensitivity to stress induction by providing descriptive statistics of 8 common EDA signal features for each stressor of the experiment. To the best of our knowledge, this is the first time a generic, 16-bit microcontroller (MCU) has been used to record real-time physiological signals on a wearable platform without the use of external memory chips or wireless transmission for extended periods of time. The compression techniques described can lead to reductions in size, power, and cost of wearable biosensors with little or no modifications to existing sensor hardware and could be valuable for applications interested in monitoring long-term physiological trends at lower data rates and memory requirements.

2017:
Shrirang Mare, Andrés Molina-Markham, Ronald Peterson, and David Kotz. System, Method and Authorization Device for Biometric Access Control to Digital Devices. U.S. Patent 9,832,206; International Patent Application WO2014153528A2, November 28, 2017. Priority date 2013-03-21; Filed 2014-03-21; Issued 2017-11-28. [Details]

A system and method for authenticating and continuously verifying authorized users of a digital device includes an authentication device attached to an arm or wrist of authorized users. The authentication device has an accelerometer, digital radio, a processor configured to provide identity information over the radio, and to transmit motion data. The motion data is received by the digital device and the identity transmitted is verified as an identity associated with an authorized user. Input at a touchscreen, touchpad, mouse, trackball, or keyboard of the digital device is detected, and correlated with the motion data. Access to the digital device is allowed if the detected input and the detected motion data correlate, and disallowed otherwise.

Varun Mishra, Byron Lowens, Sarah Lord, Kelly Caine, and David Kotz. Investigating Contextual Cues As Indicators for EMA Delivery. Proceedings of the International Workshop on Smart and Ambient Notification and Attention Management (UbiTtention). September 2017. [Details]

In this work, we attempt to determine whether the contextual information of a participant can be used to predict whether the participant will respond to a particular EMA trigger. We use a publicly available dataset for our work, and find that by using basic contextual features about the participant’s activity, conversation status, audio, and location, we can predict if an EMA triggered at a particular time will be answered with a precision of 0.647, which is significantly higher than a baseline precision of 0.41. Using this knowledge, the researchers conducting field studies can efficiently schedule EMAs and achieve higher response rates.

Emily Greene. ShareABEL: Secure Sharing of mHealth Data through Cryptographically-Enforced Access Control. Technical Report, July 2017. [Details]

Owners of mobile-health apps and devices often want to share their mHealth data with others, such as physicians, therapists, coaches, and caregivers. For privacy reasons, however, they typically want to share a limited subset of their information with each recipient according to their preferences. In this paper, we introduce ShareABEL, a scalable, usable, and practical system that allows mHealth-data owners to specify access-control policies and to cryptographically enforce those policies so that only parties with the proper corresponding permissions are able to decrypt data. The design (and prototype implementation) of this system makes three contributions: (1) it applies cryptographically-enforced access-control measures to wearable healthcare data, which pose different challenges than Electronic Medical Records (EMRs), (2) it recognizes the temporal nature of mHealth data streams and supports revocation of access to part or all of a data stream, and (3) it departs from the vendor- and device-specific silos of mHealth data by implementing a secure end-to-end system that can be applied to data collected from a variety of mHealth apps and devices.

Shengjie Bi, Ellen Davenport, Jun Gong, Ronald Peterson, Kevin Storer, Tao Wang, Kelly Caine, Ryan Halter, David Kotz, Kofi Odame, Jacob Sorber, and Xing-Dong Yang. Poster: Auracle --- A Wearable Device for Detecting and Monitoring Eating Behavior. Proceedings of the ACM International Conference on Mobile Systems, Applications, and Services (MobiSys). June 2017. [Details]

The Auracle aims to be a wearable earpiece that detects eating behavior, to be fielded by health-science researchers in their efforts to study eating behavior and ultimately to develop interventions useful to individuals striving to address chronic disease related to eating.

Shengjie Bi, Tao Wang, Ellen Davenport, Ronald Peterson, Ryan Halter, Jacob Sorber, and David Kotz. Toward a Wearable Sensor for Eating Detection. Proceedings of the ACM Workshop on Wearable Systems and Applications (WearSys). June 2017. [Details]

Researchers strive to understand eating behavior as a means to develop diets and interventions that can help people achieve and maintain a healthy weight, recover from eating disorders, or manage their diet and nutrition for personal wellness. A major challenge for eating-behavior research is to understand when, where, what, and how people eat. In this paper, we evaluate sensors and algorithms designed to detect eating activities, more specifically, when people eat. We compare two popular methods for eating recognition (based on acoustic and electromyography (EMG) sensors) individually and combined. We built a data-acquisition system using two off-the-shelf sensors and conducted a study with 20 participants. Our preliminary results show that the system we implemented can detect eating with an accuracy exceeding 90.9% while the crunchiness level of food varies. We are developing a wearable system that can capture, process, and classify sensor data to detect eating in real-time.

Rui Liu, Cory Cornelius, Reza Rawassizadeh, Ron Peterson, and David Kotz. Poster: Vocal Resonance as a Passive Biometric. Proceedings of the ACM International Conference on Mobile Systems, Applications, and Services (MobiSys). June 2017. [Details]

We present a novel, unobtrusive biometric measurement that can support user identification in wearable body-mounted devices: vocal resonance, that is, the sound of the person’s voice as it travels through the person’s body.

Rui Liu, Reza Rawassizadeh, and David Kotz. Toward Accurate and Efficient Feature Selection for Speaker Recognition on Wearables. Proceedings of the ACM Workshop on Wearable Systems and Applications (WearSys). June 2017. [Details]

Due to the user-interface limitations of wearable devices, voice-based interfaces are becoming more common; speaker recognition may then address the authentication requirements of wearable applications. Wearable devices have small form factor, limited energy budget and limited computational capacity. In this paper, we examine the challenge of computing speaker recognition on small wearable platforms, and specifically, reducing resource use (energy use, response time) by trimming the input through careful feature selections. For our experiments, we analyze four different feature-selection algorithms and three different feature sets for speaker identification and speaker verification. Our results show that Principal Component Analysis (PCA) with frequency-domain features had the highest accuracy, Pearson Correlation (PC) with time-domain features had the lowest energy use, and recursive feature elimination (RFE) with frequency-domain features had the least latency. Our results can guide developers to choose feature sets and configurations for speaker-authentication algorithms on wearable platforms.

Xiaohui Liang, Tianlong Yun, Ronald Peterson, and David Kotz. LightTouch: Securely Connecting Wearables to Ambient Displays with User Intent. Proceedings of the IEEE International Conference on Computer Communications (INFOCOM). May 2017. [Details]

Wearables are small and have limited user interfaces, so they often wirelessly interface with a personal smartphone/computer to relay information from the wearable for display or other interactions. In this paper, we envision a new method, LightTouch, by which a wearable can establish a secure connection to an ambient display, such as a television or a computer monitor, while ensuring the user’s intention to connect to the display. LightTouch uses standard RF methods (like Bluetooth) for communicating the data to display, securely bootstrapped via the visible-light communication (the brightness channel) from the display to the low-cost, low-power, ambient light sensor of a wearable. A screen ‘touch’ gesture is adopted by users to ensure that the modulation of screen brightness can be securely captured by the ambient light sensor with minimized noise. Wireless coordination with the processor driving the display establishes a shared secret based on the brightness channel information. We further propose novel on-screen localization and correlation algorithms to improve security and reliability. Through experiments and a preliminary user study we demonstrate that LightTouch is compatible with current display and wearable designs, is easy to use (about 6 seconds to connect), is reliable (up to 98% success connection ratio), and is secure against attacks.

George G. Boateng. ActivityAware: Wearable System for Real-Time Physical Activity Monitoring among the Elderly. Master's thesis, May 2017. Available as Dartmouth Computer Science Technical Report TR2017-824. [Details]

Physical activity helps reduce the risk of cardiovascular disease, hypertension and obesity. The ability to monitor a person’s daily activity level can inform self-management of physical activity and related interventions. For older adults with obesity, the importance of regular, physical activity is critical to reduce the risk of long-term disability. In this work, we present ActivityAware, an application on the Amulet wrist-worn device that monitors the daily activity levels (low, moderate and vigorous) of older adults in real-time. The app continuously collects acceleration data on the Amulet, classifies the current activity level, updates the day’s accumulated time spent at that activity level, displays the results on the screen and logs summary data for later analysis.

The app implements an activity-level detection model we developed using a Linear Support Vector Machine (SVM). We trained our model using data from a user study, where subjects performed common physical activities (sit, stand, lay down, walk and run). We obtained accuracies up to 99.2% and 98.5% with 10-fold cross validation and leave-one-subject-out (LOSO) cross-validation respectively. We ran a week-long field study to evaluate the utility, usability and battery life of the ActivityAware system where 5 older adults wore the Amulet as it monitored their activity level. The utility evaluation showed that the app was somewhat useful in achieving the daily physical activity goal. The usability feedback showed that the ActivityAware system has the potential to be used by people for monitoring their activity levels. Our energy-efficiency evaluation revealed a battery life of at least 1 week before needing to recharge. The results are promising, indicating that the app may be used for activity-level monitoring by individuals or researchers for epidemiological studies, and eventually for the development of interventions that could improve the health of older adults.


George Boateng, John A. Batsis, Ryan Halter, and David Kotz. ActivityAware: An App for Real-Time Daily Activity Level Monitoring on the Amulet Wrist-Worn Device. Proceedings of the IEEE PerCom Workshop on Pervasive Health Technologies (PerHealth). March 2017. [Details]

Physical activity helps reduce the risk of cardiovascular disease, hypertension and obesity. The ability to monitor a person’s daily activity level can inform self-management of physical activity and related interventions. For older adults with obesity, the importance of regular, physical activity is critical to reduce the risk of long-term disability. In this work, we present ActivityAware, an application on the Amulet wrist-worn device that measures daily activity levels (sedentary, moderate and vigorous) of individuals, continuously and in real-time. The app implements an activity-level detection model, continuously collects acceleration data on the Amulet, classifies the current activity level, updates the day’s accumulated time spent at that activity level, logs the data for later analysis, and displays the results on the screen. We developed an activity-level detection model using a Support Vector Machine (SVM). We trained our classifiers using data from a user study, where subjects performed the following physical activities: sit, stand, lay down, walk and run. With 10-fold cross validation and leave-one-subject-out (LOSO) cross validation, we obtained preliminary results that suggest accuracies up to 98%, for n=14 subjects. Testing the ActivityAware app revealed a projected battery life of up to 4 weeks before needing to recharge. The results are promising, indicating that the app may be used for activity-level monitoring, and eventually for the development of interventions that could improve the health of individuals.

David Kotz, Ryan Halter, Cory Cornelius, Jacob Sorber, Minho Shin, Ronald Peterson, Shrirang Mare, Aarathi Prasad, Joseph Skinner, and Andrés Molina-Markham. Wearable computing device for secure control of physiological sensors and medical devices, with secure storage of medical records, and bioimpedance biometric. U.S. Patent 9,595,187; International Patent Application WO2013096954A1, March 14, 2017. Priority date 2011-12-23; Filed 2012-12-24; Issued 2017-03-14. [Details]

A wearable master electronic device (Amulet) has a processor with memory, the processor coupled to a body-area network (BAN) radio and uplink radio. The device has firmware for BAN communications with wearable nodes to receive data, and in an embodiment, send configuration data. The device has firmware for using the uplink radio to download apps and configurations, and upload data to a server. An embodiment has accelerometers in Amulet and wearable node, and firmware for using accelerometer readings to determine if node and Amulet are worn by the same subject. Other embodiments use pulse sensors or microphones in the Amulet and node to both identify a subject and verify the Amulet and node are worn by the same subject. Another embodiment uses a bioimpedance sensor to identify the subject. The wearable node may be an insulin pump, chemotherapy pump, TENS unit, cardiac monitor, or other device.

George Boateng and David Kotz. StressAware: An App for Real-Time Stress Monitoring on the Amulet Wearable Platform. Proceedings of the IEEE MIT Undergraduate Research Technology Conference (URTC). January 2017. [Details]

Stress is the root cause of many diseases and unhealthy behaviors. Being able to monitor when and why a person is stressed could inform personal stress management as well as interventions when necessary. In this work, we present StressAware, an application on the Amulet wearable platform that classifies the stress level (low, medium, high) of individuals continuously and in real time using heart rate (HR) and heart-rate variability (HRV) data from a commercial heart-rate monitor. We developed our stress-detection model using a Support Vector Machine (SVM). We trained and tested our model using data from three sources and had the following preliminary results: PhysioNet, a public physiological database (94.5% accurate with 10-fold cross validation), a field study (100% accurate with 10-fold cross validation) and a lab study (64.3% accurate with leave-one-out cross-validation). Testing the StressAware app revealed a projected battery life of up to 12 days. Also, the usability feedback from subjects showed that the Amulet has a potential to be used by people for monitoring their stress levels. The results are promising, indicating that the app may be used for stress detection, and eventually for the development of stress-related intervention that could improve the health of individuals.

2016:
Josiah Hester, Travis Peters, Tianlong Yun, Ronald Peterson, Joseph Skinner, Bhargav Golla, Kevin Storer, Steven Hearndon, Sarah Lord, Ryan Halter, David Kotz, and Jacob Sorber. The Amulet Wearable Platform: Demo Abstract. Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys). November 2016. [Details]

In this demonstration we present the Amulet Platform; a hardware and software platform for developing energy- and resource-efficient applications on multi-application wearable devices. This platform, which includes the Amulet Firmware Toolchain, the Amulet Runtime, the ARP-View graphical tool, and open reference hardware, efficiently protects applications from each other without MMU support, allows developers to interactively explore how their implementation decisions impact battery life without the need for hardware modeling and additional software development, and represents a new approach to developing long-lived wearable applications. We envision the Amulet Platform enabling long-duration experiments on human subjects in a wide variety of studies.

Josiah Hester, Travis Peters, Tianlong Yun, Ronald Peterson, Joseph Skinner, Bhargav Golla, Kevin Storer, Steven Hearndon, Kevin Freeman, Sarah Lord, Ryan Halter, David Kotz, and Jacob Sorber. Amulet: An Energy-Efficient, Multi-Application Wearable Platform. Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys). November 2016. [Details]

Wearable technology enables a range of exciting new applications in health, commerce, and beyond. For many important applications, wearables must have battery life measured in weeks or months, not hours and days as in most current devices. Our vision of wearable platforms aims for long battery life but with the flexibility and security to support multiple applications. To achieve long battery life with a workload comprising apps from multiple developers, these platforms must have robust mechanisms for app isolation and developer tools for optimizing resource usage.

We introduce the Amulet Platform for constrained wearable devices, which includes an ultra-low-power hardware architecture and a companion software framework, including a highly efficient event-driven programming model, low-power operating system, and developer tools for profiling ultra-low-power applications at compile time. We present the design and evaluation of our prototype Amulet hardware and software, and show how the framework enables developers to write energy-efficient applications. Our prototype has battery lifetime lasting weeks or even months, depending on the application, and our interactive resource-profiling tool predicts battery lifetime within 6-10% of the measured lifetime.


George G. Boateng. StressAware: App for Continuously Measuring and Monitoring Stress Levels in Real Time on the Amulet Wearable Device. Technical Report, May 2016. [Details]

Stress is the root cause of many diseases. Being able to monitor when and why a person is stressed could inform personal stress management as well as interventions when necessary. In this thesis, I present StressAware, an application on the Amulet wearable platform to measure the stress levels of individuals continuously and in real time. The app implements a stress detection model, continuously streams heart rate data from a commercial heart-rate monitor such as a Zephyr and Polar H7, classifies the stress level of an individual, logs the stress level and then displays it as a graph on the screen. I developed a stress detection model using a Linear Support Vector Machine. I trained my classifiers using data from 3 sources: PhysioNet, a public database with various physiological data, a field study, where subjects went about their normal daily activities and a lab study in a controlled environment, where subjects were exposed to various stressors. I used 73 data segments of stress data obtained from PhysioNet, 120 data segments from the field study, and 14 data segments from the lab study. I extracted 14 heart rate and heart rate variability features. With 10-fold cross validation for Radial Basis Function (RBF) SVM, I obtained an accuracy of 94.5% for the PhysioNet dataset and 100% for the field study dataset. And for the lab study, I obtained an accuracy of 64.29% with leave-one-out cross-validation. Testing the StressAware app revealed a projected battery life of up to 12 days before needing to recharge. Also, the usability feedback from subjects showed that the Amulet and Zephyr have a potential to be used by people for monitoring their stress levels. The results are promising, indicating that the app may be used for stress detection, and eventually for the development of stress-related intervention that could improve the health of individuals.

Anna J. Knowles. Integrating Bluetooth Low Energy Peripherals with the Amulet. Technical Report, May 2016. [Details]

The Amulet is a health monitor, similar in size and shape to a smartwatch but specifically designed to have a longer battery life and handle data securely. It is equipped with a Bluetooth Low Energy (BLE) radio in order to receive data from BLE-enabled sensors and transmit data to smartphones, but the full implementation of BLE communication on the Amulet is still a work in progress. This thesis describes architectural changes that improve the Amulet’s ability to receive data from a variety of BLE-enabled sensors and make it easier for developers to integrate new BLE-enabled sensors with the Amulet by introducing support for connecting to multiple sensors at the same time, rewriting the radio code to be more generic, and exposing BLE functionality to the AmuletOS. We discuss the relevant parts of the AmuletOS and the BLE protocol as background, describe the current structure of BLE communications on the Amulet, and document the proposed changes to create a system for easily integrating new BLE-enabled sensors and handling connections to multiple sensors simultaneously.

Shrirang Mare. Seamless Authentication for Ubiquitous Devices. PhD thesis, May 2016. Available as Dartmouth Computer Science Technical Report TR2016-793. [Details]

User authentication is an integral part of our lives; we authenticate ourselves to personal computers and a variety of other things several times a day. Authentication is burdensome. When we wish to access to a computer or a resource, it is an additional task that we need to perform -- an interruption in our workflow. In this dissertation, we study people’s authentication behavior and attempt to make authentication to desktops and smartphones less burdensome for users.

First, we present the findings of a user study we conducted to understand people’s authentication behavior: things they authenticate to, how and when they authenticate, authentication errors they encounter and why, and their opinions about authentication. In our study, participants performed about 39 authentications per day on average; the majority of these authentications were to personal computers (desktop, laptop, smartphone, tablet) and with passwords, but the number of authentications to other things (e.g., car, door) was not insignificant. We saw a high failure rate for desktop and laptop authentication among our participants, affirming the need for a more usable authentication method. Overall, we found that authentication was a noticeable part of all our participants’ lives and burdensome for many participants, but they accepted it as cost of security, devising their own ways to cope with it.

Second, we propose a new approach to authentication, called bilateral authentication, that leverages wrist-wearable technology to enable seamless authentication for things that people use with their hands, while wearing a smart wristband. In bilateral authentication two entities (e.g., user’s wristband and the user’s phone) share their knowledge (e.g., about user’s interaction with the phone) to verify the user’s identity. Using this approach, we developed a seamless authentication method for desktops and smartphones. Our authentication method offers quick and effortless authentication, continuous user verification while the desktop (or smartphone) is in use, and automatic deauthentication after use. We evaluated our authentication method through four in-lab user studies, evaluating the method’s usability and security from the system and the user’s perspective. Based on the evaluation, our authentication method shows promise for reducing users’ authentication burden for desktops and smartphones.


Aarathi Prasad. Privacy-preserving controls for sharing mHealth data. PhD thesis, May 2016. Available as Dartmouth Computer Science Technical Report TR2016-794. [Details]

Mobile devices allow people to collect and share health and health-related information with recipients such as health providers, family and friends, employers and insurance companies, to obtain health, emotional or financial benefits. People may consider certain health information sensitive and prefer to disclose only what is necessary. In this dissertation, we present our findings about factors that affect people’s sharing behavior, describe scenarios in which people may wish to collect and share their personal health-related information with others, but may be hesitant to disclose the information if necessary controls are not available to protect their privacy, and propose frameworks to provide the desired privacy controls. We introduce the concept of close encounters that allow users to share data with other people who may have been in spatio-temporal proximity. We developed two smartphone-based systems that leverage stationary sensors and beacons to determine whether users are in spatio-temporal proximity. The first system, ENACT, allows patients diagnosed with a contagious airborne disease to alert others retrospectively about their possible exposure to airborne virus. The second system, SPICE, allows users to collect sensor information, retrospectively, from others with whom they shared a close encounter. We present design and implementation of the two systems, analyse their security and privacy guarantees, and evaluate the systems on various performance metrics. Finally, we evaluate how Bluetooth beacons and Wi-Fi access points can be used in support of these systems for close encounters, and present our experiences and findings from a deployment study on Dartmouth campus.

Bingyue Wang. Learning Device Usage in Context: A Continuous and Hierarchical Smartphone Authentication Scheme. Technical Report, March 2016. [Details]

Popular smartphone authentication schemes, such as PIN-based or biometrics-based authentication methods, require only an initial login at the start of a usage session to authorize the user to use all the apps on the phone during the entire session. Those schemes fail to provide continuous protection of the smartphone after the initial login. They also fail to meet the hierarchy of security requirements for different apps under different contexts. In this study, we propose a continuous and hierarchical authentication scheme. We believe that a user’s app-usage patterns depend on his location context. As such, our scheme relies on app-usage patterns in different location context to continuously establish the log probability density (LPD) of the authenticity of the current user. Based on different LPD thresholds corresponding to different security requirements, the current user either has a LPD higher than the threshold, which grants him continuous access to the phone or the app, or he has a LPD lower than the threshold, which locks him out of the phone or the app immediately. We test our scheme on 4,600 subjects from the Device Analyzer Dataset. We found that our scheme could correctly identify the authenticity of the majority of the subjects. However, app-usage patterns with or without location context yielded similar performances, indicating that user contexts did not contribute further information to establish user behavioral patterns. Based on our scheme, we propose a hypothetical Android app which would provide continuous and hierarchical authentication for the smartphone users.

2015:
Minho Shin, Cory Cornelius, Apu Kapadia, Nikos Triandopoulos, and David Kotz. Location Privacy for Mobile Crowd Sensing through Population Mapping. Sensors. June 2015. [Details]

Opportunistic sensing allows applications to “task” mobile devices to measure context in a target region. For example, one could leverage sensor-equipped vehicles to measure traffic or pollution levels on a particular street or users’ mobile phones to locate (Bluetooth-enabled) objects in their vicinity. In most proposed applications, context reports include the time and location of the event, putting the privacy of users at increased risk: even if identifying information has been removed from a report, the accompanying time and location can reveal sufficient information to de-anonymize the user whose device sent the report. We propose and evaluate a novel spatiotemporal blurring mechanism based on tessellation and clustering to protect users’ privacy against the system while reporting context. Our technique employs a notion of probabilistic k-anonymity; it allows users to perform local blurring of reports efficiently without an online anonymization server before the data are sent to the system. The proposed scheme can control the degree of certainty in location privacy and the quality of reports through a system parameter. We outline the architecture and security properties of our approach and evaluate our tessellation and clustering algorithm against real mobility traces.

2014:
Andrés Molina-Markham, Ronald Peterson, Joseph Skinner, Tianlong Yun, Bhargav Golla, Kevin Freeman, Travis Peters, Jacob Sorber, Ryan Halter, and David Kotz. Amulet: A secure architecture for mHealth applications for low-power wearable devices. Proceedings of the Workshop on Mobile Medical Applications-- Design and Development (WMMADD). November 2014. [Details]

Interest in using mobile technologies for health-related applications (mHealth) has increased. However, none of the available mobile platforms provide the essential properties that are needed by these applications. An mHealth platform must be (i) secure; (ii) provide high availability; and (iii) allow for the deployment of multiple third-party mHealth applications that share access to an individual’s devices and data. Smartphones may not be able to provide property (ii) because there are activities and situations in which an individual may not be able to carry them (e.g., while in a contact sport). A low-power wearable device can provide higher availability, remaining attached to the user during most activities. Furthermore, some mHealth applications require integrating multiple on-body or near-body devices, some owned by a single individual, but others shared with multiple individuals. In this paper, we propose a secure system architecture for a low-power bracelet that can run multiple applications and manage access to shared resources in a body-area mHealth network. The wearer can install a personalized mix of third-party applications to support the monitoring of multiple medical conditions or wellness goals, with strong security safeguards. Our preliminary implementation and evaluation supports the hypothesis that our approach allows for the implementation of a resource monitor on far less power than would be consumed by a mobile device running Linux or Android. Our preliminary experiments demonstrate that our secure architecture would enable applications to run for several weeks on a small wearable device without recharging.

Xiaohui Liang and David Kotz. Securely Connecting Wearable Health Devices to External Displays. Proceedings of the USENIX Summit on Health Information Technologies. August 2014. No paper -- workshop presentation only. [Details]

Wearable health technology is becoming a hot commodity as it has the potential to help both patients and clinicians continuously monitor vital signs and symptoms. One popular type of wearable devices are worn on human wrist and are equipped with sensors to passively perform sensing tasks. Their constrained user interface, however, is ineffective to display the sensory data for users. We envision connecting a wrist-worn device to a display device, such as a television, so the user is able to view the sensory data. Such connections must be secure to prevent the sensory data from being eavesdropped by other devices, must be made only when the user intends, and must be easy even when a new display is encountered (such as in a medical clinic, or a hotel room). In this presentation, we will discuss the secure wearable/display connection problem by revisiting existing methods and hardware designs of wrist-worn devices and display devices. We then present possible solutions that leverage the built-in hardware components of wrist-worn devices to implement, secure, intentional, easy connections to ambient display devices.

Rima Narayana Murthy. mCollector: Sensor-enabled health-data collection system for rural areas in the developing world. Master's thesis, August 2014. Available as Dartmouth Technical Report TR2015-788. [Details]

Health data collection poses unique challenges in rural areas of the developing world. mHealth systems that are used by health workers to collect data in remote rural regions should also record contextual information to increase confidence in the fidelity of the collected data.

We built a user-friendly, mobile health-data collection system using wireless medical sensors that interface with an Android application. The data-collection system was designed to support minimally trained, non-clinical health workers to gather data about blood pressure and body weight using off-the-shelf medical sensors. This system comprises a blood-pressure cuff, a weighing scale and a portable point-of-sales printer. With this system, we introduced a new method to record contextual information associated with a blood-pressure reading using a tablet’s touchscreen and accelerometer. This contextual information can be used to verify that a patient’s lower arm remained well-supported and stationary during her blood-pressure measurement. In a preliminary user study, we found that a binary support vector machine classifier could be used to distinguish lower-arm movements from stationary arms with 90% accuracy. Predetermined thresholds for the accelerometer readings suffice to determine whether the tablet, and therefore the arm that rested on it, remained supported. Together, these two methods can allow mHealth applications to guide untrained patients (or health workers) in measuring blood pressure correctly.

Usability is a particularly important design and deployment challenge in remote, rural areas, given the limited resources for technology training and support. We conducted a field study to assess our system’s usability in Kolar town, India, where we logged health worker interactions with the app’s interface using an existing usability toolkit. Researchers analyzed logs from this toolkit to evaluate the app’s user experience and quantify specific usability challenges in the app. We have recorded experiential notes from the field study in this document.


Cory Cornelius, Ronald Peterson, Joseph Skinner, Ryan Halter, and David Kotz. A wearable system that knows who wears it. Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys). June 2014. [Details]

Body-area networks of pervasive wearable devices are increasingly used for health monitoring, personal assistance, entertainment, and home automation. In an ideal world, a user would simply wear their desired set of devices with no configuration necessary: the devices would discover each other, recognize that they are on the same person, construct a secure communications channel, and recognize the user to which they are attached. In this paper we address a portion of this vision by offering a wearable system that unobtrusively recognizes the person wearing it. Because it can recognize the user, our system can properly label sensor data or personalize interactions.

Our recognition method uses bioimpedance, a measurement of how tissue responds when exposed to an electrical current. By collecting bioimpedance samples using a small wearable device we designed, our system can determine that (a)the wearer is indeed the expected person and (b) the device is physically on the wearer’s body. Our recognition method works with 98% balanced-accuracy under a cross-validation of a day’s worth of bioimpedance samples from a cohort of 8 volunteer subjects. We also demonstrate that our system continues to recognize a subset of these subjects even several months later. Finally, we measure the energy requirements of our system as implemented on a Nexus S smart phone and custom-designed module for the Shimmer sensing platform.


Andrés Molina-Markham, Ronald A. Peterson, Joseph Skinner, Ryan J. Halter, Jacob Sorber, and David Kotz. Poster: Enabling Computational Jewelry for mHealth Applications. Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys). June 2014. [Details]

We are developing wearable devices as the foundation for a consistently present and highly available body-area mHealth network. Our vision is that a small device, such as a bracelet or pendant, will provide the availability and reliability properties essential for successful body-area mHealth networks. We call this class of device computational jewelry, and expect it will be the next frontier of mobile systems. We prototyped our first piece of computational jewelry, which we call Amulet, to enable our previously proposed vision. It runs applications that may collect sensor data from built-in sensors or from other devices, analyze and log the data, queue information for later upload, and interact with the wearer. Independent developers can develop applications that can be vetted and installed on an Amulet.

Shrirang Mare, Andrés Molina-Markham, Cory Cornelius, Ronald Peterson, and David Kotz. ZEBRA: Zero-Effort Bilateral Recurring Authentication (Companion report). Technical Report, May 2014. This project has been renamed CSAW. [Details]

We describe and evaluate Zero-Effort Bilateral Recurring Authentication (ZEBRA) in our paper that appears in IEEE Symposium on Security and Privacy, May 2014. In this report we provide a more detailed comparative evaluation of ZEBRA against other related authentication schemes. The abstract of the paper follows. Common authentication methods based on passwords, tokens, or fingerprints perform one-time authentication and rely on users to log out from the computer terminal when they leave. Users often do not log out, however, which is a security risk. The most common solution, inactivity timeouts, inevitably fail security (too long a timeout) or usability (too short a timeout) goals. One solution is to authenticate users continuously while they are using the terminal and automatically log them out when they leave. Several solutions are based on user proximity, but these are not sufficient: they only confirm whether the user is nearby but not whether the user is actually using the terminal. Proposed solutions based on behavioral biometric authentication (e.g., keystroke dynamics) may not be reliable, as a recent study suggests. To address this problem we propose ZEBRA. In ZEBRA, a user wears a bracelet (with a built-in accelerometer, gyroscope, and radio) on her dominant wrist. When the user interacts with a computer terminal, the bracelet records the wrist movement, processes it, and sends it to the terminal. The terminal compares the wrist movement with the inputs it receives from the user (via keyboard and mouse), and confirms the continued presence of the user only if they correlate. Because the bracelet is on the same hand that provides inputs to the terminal, the accelerometer and gyroscope data and input events received by the terminal should correlate because their source is the same -- the user’s hand movement. In our experiments ZEBRA performed continuous authentication with 85% accuracy in verifying the correct user and identified all adversaries within 11 s. For a different threshold that trades security for usability, ZEBRA correctly verified 90% of users and identified all adversaries within 50 s.

Shrirang Mare, Andrés Molina-Markham, Cory Cornelius, Ronald Peterson, and David Kotz. ZEBRA: Zero-Effort Bilateral Recurring Authentication. Proceedings of the IEEE Symposium on Security & Privacy. May 2014. This project has been renamed CSAW. [Details]

Common authentication methods based on passwords, tokens, or fingerprints perform one-time authentication and rely on users to log out from the computer terminal when they leave. Users often do not log out, however, which is a security risk. The most common solution, inactivity timeouts, inevitably fail security (too long a timeout) or usability (too short a timeout) goals. One solution is to authenticate users continuously while they are using the terminal and automatically log them out when they leave. Several solutions are based on user proximity, but these are not sufficient: they only confirm whether the user is nearby but not whether the user is actually using the terminal. Proposed solutions based on behavioral biometric authentication (e.g., keystroke dynamics) may not be reliable, as a recent study suggests.

To address this problem we propose ZEBRA. In ZEBRA, a user wears a bracelet (with a built-in accelerometer, gyroscope, and radio) on her dominant wrist. When the user interacts with a computer terminal, the bracelet records the wrist movement, processes it, and sends it to the terminal. The terminal compares the wrist movement with the inputs it receives from the user (via keyboard and mouse), and confirms the continued presence of the user only if they correlate. Because the bracelet is on the same hand that provides inputs to the terminal, the accelerometer and gyroscope data and input events received by the terminal should correlate because their source is the same -- the user’s hand movement. In our experiments ZEBRA performed continuous authentication with 85% accuracy in verifying the correct user and identified all adversaries within 11 s. For a different threshold that trades security for usability, ZEBRA correctly verified 90% of users and identified all adversaries within 50 s.


Cory Cornelius, Zachary Marois, Jacob Sorber, Ron Peterson, Shrirang Mare, and David Kotz. Vocal resonance as a biometric for pervasive wearable devices. Technical Report, February 2014. [Details]

We anticipate the advent of body-area networks of pervasive wearable devices, whether for health monitoring, personal assistance, entertainment, or home automation. In our vision, the user can simply wear the desired set of devices, and they “just work”; no configuration is needed, and yet they discover each other, recognize that they are on the same body, configure a secure communications channel, and identify the user to which they are attached. This paper addresses a method to achieve the latter, that is, for a wearable device to identify the wearer, allowing sensor data to be properly labeled or personalized behavior to be properly achieved. We use vocal resonance, that is, the sound of the person’s voice as it travels through the person’s body. By collecting voice samples from a small wearable microphone, our method allows the device to determine whether (a) the speaker is indeed the expected person, and (b) the microphone device is physically on the speaker’s body. We collected data from 25 subjects, demonstrate the feasibility of a prototype, and show that our method works with 77% accuracy when a threshold is chosen a priori.

Rima Murthy and David Kotz. Assessing blood-pressure measurement in tablet-based mHealth apps. Proceedings of the Workshop on Networked Healthcare Technology (NetHealth). January 2014. [Details]

We propose a new method to record contextual information associated with a blood-pressure reading using a tablet’s touchscreen and accelerometer. This contextual information can be used to verify that a patient’s lower arm remained well-supported and stationary during her blood-pressure measurement. We found that a binary support vector machine classifier could be used to distinguish different types of lower-arm movements from stationary arms with 90% accuracy overall. Predetermined thresholds for the accelerometer readings suffice to determine whether the tablet, and therefore the arm that rested on it, remained supported. Together, these two methods can allow mHealth applications to guide untrained patients (or health workers) in measuring blood pressure correctly.

2013:
Cory T. Cornelius. Usable Security for Wireless Body-Area Networks. PhD thesis, September 2013. Available as Dartmouth Computer Science Technical Report TR2013-741. [Details]

We expect wireless body-area networks of pervasive wearable devices will enable in situ health monitoring, personal assistance, entertainment personalization, and home automation. As these devices become ubiquitous, we also expect them to interoperate. That is, instead of closed, end-to-end body-worn sensing systems, we envision standardized sensors that wirelessly communicate their data to a device many people already carry today, the smart phone. However, this ubiquity of wireless sensors combined with the characteristics they sense present many security and privacy problems.

In this thesis we describe solutions to two of these problems. First, we evaluate the use of bioimpedance for recognizing who is wearing these wireless sensors and show that bioimpedance is a feasible biometric. Second, we investigate the use of accelerometers for verifying whether two of these wireless sensors are on the same person and show that our method is successful as distinguishing between sensors on the same body and on different bodies. We stress that any solution to these problems must be usable, meaning the user should not have to do anything but attach the sensor to their body and have them just work.

These methods solve interesting problems in their own right, but it is the combination of these methods that shows their true power. Combined together they allow a network of wireless sensors to cooperate and determine whom they are sensing even though only one of the wireless sensors might be able to determine this fact. If all the wireless sensors know they are on the same body as each other and one of them knows which person it is on, then they can each exploit the transitive relationship to know that they must all be on that person’s body. We show how these methods can work together in a prototype system. This ability to operate unobtrusively, collecting in situ data and labeling it properly without interrupting the wearer’s activities of daily life, will be vital to the success of these wireless sensors.


2012:
Cory Cornelius and David Kotz. Recognizing whether sensors are on the same body. Journal of Pervasive and Mobile Computing. December 2012. [Details]

In an open mobile health (mHealth) sensing system, users will be able to seamlessly pair sensors with their cellphone and expect the system to just work. This ubiquity of sensors, however, creates the potential for users to accidentally wear sensors that are not paired with their own cellphone. Our method probabilistically detects this situation by finding correlations between embedded accelerometers in the cellphone and sensor. We evaluate our method over a dataset of seven individuals with sensors in various positions on their body and experimentally show that our method is capable of achieving an accuracy of 85%.

Cory Cornelius, Jacob Sorber, Ronald Peterson, Joe Skinner, Ryan Halter, and David Kotz. Who wears me? Bioimpedance as a passive biometric. Proceedings of the USENIX Workshop on Health Security and Privacy. August 2012. [Details]

Mobile and wearable systems for monitoring health are becoming common. If such an mHealth system knows the identity of its wearer, the system can properly label and store data collected by the system. Existing recognition schemes for such mobile applications and pervasive devices are not particularly usable -- they require active engagement with the person (e.g., the input of passwords), or they are too easy to fool (e.g., they depend on the presence of a device that is easily stolen or lost).

We present a wearable sensor to passively recognize people. Our sensor uses the unique electrical properties of a person’s body to recognize their identity. More specifically, the sensor uses bioimpedance -- a measure of how the body’s tissues oppose a tiny applied alternating current -- and learns how a person’s body uniquely responds to alternating current of different frequencies. In this paper we demonstrate the feasibility of our system by showing its effectiveness at accurately recognizing people in a household 90% of the time.


Cory Cornelius, Zachary Marois, Jacob Sorber, Ron Peterson, Shrirang Mare, and David Kotz. Passive Biometrics for Pervasive Wearable Devices (Poster paper). Proceedings of the Workshop on Mobile Computing Systems and Applications (HotMobile). February 2012. [Details]

Wearable devices -- like the FitBit, MOTOACTV, and Jawbone UP -- are increasingly becoming more pervasive whether for monitoring health and fitness, personal assistance, or home automation. While pervasive wearable devices have long been researched, we are now beginning to see the fruits of this research in the form of commercial offerings. Today, many of these commercial wearable devices are closed systems that do not interoperate with other devices a person might carry. We believe, however, these commercial offerings signal the coming of wireless body-area networks that will connect these pervasive wearable devices and leverage existing devices a user already owns (e.g., a smartphone). Such wireless body-area networks will allow devices to specialize and utilize the capabilities of other devices in the network. A sensor, for example, might harness the internet connectivity of a smartphone to store its data in the cloud. Utilized in this way, devices will become cheaper because they will only require the components necessary for their speciality, and they will also become more pervasive because they can easily be shared between users.

In order for such a vision to be successful, these devices will need to seamlessly interoperate with no interaction required of the user. As difficult as it is for users to manage their wireless area networks, it will be even more difficult for a user to manage their wireless body-area network in a truly pervasive world. As such, we believe these wearable devices should form a wireless body-area network that is passive in nature. This means that these pervasive wearable devices will require no configuration, yet they will be able form a wireless body-area network by (1) discovering their peers, (2) recognizing they are attached to the same body, (3) securing their communications, and (4) identifying to whom they are attached. While we are interested in all aspects of these passive wireless body-area networks, we focus on the last requirement: identifying who is wearing a device.


Jacob Sorber, Minho Shin, Ronald Peterson, Cory Cornelius, Shrirang Mare, Aarathi Prasad, Zachary Marois, Emma Smithayer, and David Kotz. An Amulet for trustworthy wearable mHealth. Proceedings of the Workshop on Mobile Computing Systems and Applications (HotMobile). February 2012. [Details]

Mobile technology has significant potential to help revolutionize personal wellness and the delivery of healthcare. Mobile phones, wearable sensors, and home-based tele-medicine devices can help caregivers and individuals themselves better monitor and manage their health. While the potential benefits of this “mHealth” technology include better health, more effective healthcare, and reduced cost, this technology also poses significant security and privacy challenges. In this paper we propose Amulet, an mHealth architecture that provides strong security and privacy guarantees while remaining easy to use, and outline the research and engineering challenges required to realize the Amulet vision.

Aarathi Prasad. Exposing Privacy Concerns in mHealth Data Sharing. Master's thesis, February 2012. Available as Technical Report TR2012-711. [Details]

Mobile health (mHealth) has become important in the field of healthcare information technology, as patients begin to use mobile devices to record their daily activities and vital signs. These devices can record personal health information even outside the hospital setting, while the patients are at home or at their workplace. However, the devices might record sensitive information that might not be relevant for medical purposes and in some cases may be misused. Patients need expressive privacy controls so that they can trade potential health benefits of the technology with the privacy risks. To provide such privacy controls, it is important to understand what patients feel are the benefits and risks associated with the technology and what controls they want over the information.

We conducted focus groups to understand the privacy concerns that patients have when they use mHealth devices. We conducted a user study to understand how willing patients are to share their personal health information that was collected using an mHealth device. To the best of our knowledge, ours is the first study that explores users’ privacy concerns by giving them the opportunity to actually share the information collected about them using mHealth devices. We found that patients tend to share more information with third parties than the public and prefer to keep certain information from their family and friends. Finally, based on these discoveries, we propose some guidelines to developing defaults for sharing settings in mHealth systems.


2011:
Cory Cornelius and David Kotz. Recognizing whether sensors are on the same body. Proceedings of the International Conference on Pervasive Computing (Pervasive). June 2011. [Details]

As personal health sensors become ubiquitous, we also expect them to become interoperable. That is, instead of closed, end-to-end personal health sensing systems, we envision standardized sensors wirelessly communicating their data to a device many people already carry today, the cellphone. In an open personal health sensing system, users will be able to seamlessly pair off-the-shelf sensors with their cellphone and expect the system to just work. However, this ubiquity of sensors creates the potential for users to accidentally wear sensors that are not necessarily paired with their own cellphone. A husband, for example, might mistakenly wear a heart-rate sensor that is actually paired with his wife’s cellphone. As long as the heart-rate sensor is within communication range, the wife’s cellphone will be receiving heart-rate data about her husband, data that is incorrectly entered into her own health record.

We provide a method to probabilistically detect this situation. Because accelerometers are relatively cheap and require little power, we imagine that the cellphone and each sensor will have a companion accelerometer embedded with the sensor itself. We extract standard features from these companion accelerometers, and use a pair-wise statistic -- coherence, a measurement of how well two signals are related in the frequency domain -- to determine how well features correlate for different locations on the body. We then use these feature coherences to train a classifier to recognize whether a pair of sensors -- or a sensor and a cellphone -- are on the same body. We evaluate our method over a dataset of several individuals walking around with sensors in various positions on their body and experimentally show that our method is capable of achieving an accuracies over 80%.


Jacob Sorber, Minho Shin, Ron Peterson, and David Kotz. Poster: Practical Trusted Computing for mHealth Sensing. Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys). June 2011. [Details]

Mobile sensing technologies present exciting opportunities for healthcare. Wireless sensors can automatically provide sensor data to care providers, dramatically improving their ability to diagnose, monitor, and manage a wide range of medical conditions. Using mobile phones to provide connectivity between sensors and providers is essential to keeping costs low and deployments simple. Unfortunately, software-based attacks against phones, which can have significant consequences for patients, are also on the rise.

This poster describes a simple, flexible, and novel approach to protecting both the confidentiality and integrity medical sensing and data processing on vulnerable mobile phones, using plug-in smart cards---even a phone compromised by malware. We describe our design, implementation, and initial experimental results using real smart cards and Android smartphones.


Minho Shin, Cory Cornelius, Dan Peebles, Apu Kapadia, David Kotz, and Nikos Triandopoulos. AnonySense: A System for Anonymous Opportunistic Sensing. Journal of Pervasive and Mobile Computing. February 2011. [Details]

We describe AnonySense, a privacy-aware system for realizing pervasive applications based on collaborative, opportunistic sensing by personal mobile devices. AnonySense allows applications to submit sensing tasks to be distributed across participating mobile devices, later receiving verified, yet anonymized, sensor data reports back from the field, thus providing the first secure implementation of this participatory sensing model. We describe our security goals, threat model, and the architecture and protocols of AnonySense. We also describe how AnonySense can support extended security features that can be useful for different applications. We evaluate the security and feasibility of AnonySense through security analysis and prototype implementation. We show the feasibility of our approach through two plausible applications: a Wi-Fi rogue access point detector and a lost-object finder.

2010:
Aarathi Prasad and David Kotz. Can I access your Data? Privacy Management in mHealth. Proceedings of the USENIX Workshop on Health Security (HealthSec). August 2010. Position paper. [Details]

Mobile health (mHealth) has become important in the field of healthcare information technology, as patients begin to use mobile medical sensors to record their daily activities and vital signs. Since their medical data is collected by their sensors, the patients may wish to control data collection and distribution, so as to protect their data and share it only when the need arises. It must be possible for patients to grant or deny access to the data on the storage unit (mobile phones or personal health records (PHR)). Thus, an efficient framework is required for managing patient consent electronically, i.e.to allow patients to express their desires about what data to collect, what to store, and how to share. We describe several challenges posed by privacy management in mobile health.

Dan Peebles, Cory Cornelius, Apu Kapadia, David Kotz, Minho Shin, and Nikos Triandopoulos. AnonyTL Specification. Technical Report, January 2010. [Details]

We provide a specification of AnonyTL, a domain-specific language that describes sensing tasks for mobile devices in a manner that facilitates automated reasoning about privacy.

2009:
Minho Shin, Patrick Tsang, David Kotz, and Cory Cornelius. DEAMON: Energy-efficient sensor monitoring. Proceedings of the IEEE Communications Society Conference on Sensor, Mesh, and Ad Hoc Communications and Networks (SECON). June 2009. [Details]

In people-centric opportunistic sensing, people offer their mobile nodes (such as smart phones) as platforms for collecting sensor data. A sensing application distributes sensing ‘tasks,’ which specify what sensor data to collect and under what conditions to report the data back to the application. To perform a task, mobile nodes may use on-board sensors, a body-area network of personal sensors, or sensors from neighboring nodes that volunteer to contribute their sensing resources. In all three cases, continuous sensor monitoring can drain a node’s battery.

We propose DEAMON (Distributed Energy-Aware MONitoring), an energy-efficient distributed algorithm for long-term sensor monitoring. Our approach assumes only that mobile nodes are tasked to report sensor data under conditions specified by a Boolean expression, and that a network of nearby sensor nodes contribute to monitoring subsets of the task’s sensors. Our algorithm to select sensor nodes and to monitor the sensing condition conserves energy of all nodes by limiting sensing and communication operations. We evaluate DEAMON with a stochastic analysis and with simulation results, and show that it should significantly reduce energy consumption.


Apu Kapadia, David Kotz, and Nikos Triandopoulos. Opportunistic Sensing: Security Challenges for the New Paradigm. Proceedings of the International Conference on COMmunication Systems and NETworkS (COMSNETS). January 2009. Invited paper. [Details]

We study the security challenges that arise in opportunistic people-centric sensing, a new sensing paradigm leveraging humans as part of the sensing infrastructure. Most prior sensor-network research has focused on collecting and processing environmental data using a static topology and an application-aware infrastructure, whereas opportunistic sensing involves collecting, storing, processing and fusing large volumes of data related to everyday human activities. This highly dynamic and mobile setting, where humans are the central focus, presents new challenges for information security, because data originates from sensors carried by people--- not tiny sensors thrown in the forest or attached to animals. In this paper we aim to instigate discussion of this critical issue, because opportunistic people-centric sensing will never succeed without adequate provisions for security and privacy. To that end, we outline several important challenges and suggest general solutions that hold promise in this new sensing paradigm.

2008:
Ming Li and David Kotz. Group-aware Stream Filtering for Bandwidth-efficient Data Dissemination. International Journal of Parallel, Emergent and Distributed Systems (IJPEDS). December 2008. Invited paper. [Details]

In this paper we are concerned with disseminating high-volume data streams to many simultaneous applications over a low-bandwidth wireless mesh network. For bandwidth efficiency, we propose a group-aware stream filtering approach, used in conjunction with multicasting, that exploits two overlooked, yet important, properties of these applications: 1) many applications can tolerate some degree of “slack” in their data quality requirements, and 2) there may exist multiple subsets of the source data satisfying the quality needs of an application. We can thus choose the “best alternative” subset for each application to maximize the data overlap within the group to best benefit from multicasting. An evaluation of our prototype implementation shows that group-aware data filtering can save bandwidth with low CPU overhead. We also analyze the key factors that affect its performance, based on testing with heterogeneous filtering requirements.

Cory Cornelius, Apu Kapadia, David Kotz, Dan Peebles, Minho Shin, and Nikos Triandopoulos. AnonySense: Privacy-Aware People-Centric Sensing. Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys). June 2008. [Details]

Personal mobile devices are increasingly equipped with the capability to sense the physical world (through cameras, microphones, and accelerometers, for example) and the network world (with Wi-Fi and Bluetooth interfaces). Such devices offer many new opportunities for cooperative sensing applications. For example, users’ mobile phones may contribute data to community-oriented information services, from city-wide pollution monitoring to enterprise-wide detection of unauthorized Wi-Fi access points. This people-centric mobile-sensing model introduces a new security challenge in the design of mobile systems: protecting the privacy of participants while allowing their devices to reliably contribute high-quality data to these large-scale applications.

We describe AnonySense, a privacy-aware architecture for realizing pervasive applications based on collaborative, opportunistic sensing by personal mobile devices. AnonySense allows applications to submit sensing tasks that will be distributed across anonymous participating mobile devices, later receiving verified, yet anonymized, sensor data reports back from the field, thus providing the first secure implementation of this participatory sensing model. We describe our trust model, and the security properties that drove the design of the AnonySense system. We evaluate our prototype implementation through experiments that indicate the feasibility of this approach, and through two applications: a Wi-Fi rogue access point detector and a lost-object finder.


Cory Cornelius, Apu Kapadia, David Kotz, Dan Peebles, Minho Shin, and Patrick Tsang. Poster Abstract: Reliable People-Centric Sensing with Unreliable Voluntary Carriers. Proceedings of the International Conference on Mobile Systems, Applications, and Services (MobiSys). June 2008. [Details]

As sensor technology becomes increasingly easy to integrate into personal devices such as mobile phones, clothing, and athletic equipment, there will be new applications involving opportunistic, people-centric sensing. These applications, which gather information about human activities and personal social context, raise many security and privacy challenges. In particular, data integrity is important for many applications, whether using traffic data for city planning or medical data for diagnosis. Although our AnonySense system (presented at MobiSys) addresses privacy in people-centric sensing, protecting data integrity in people-centric sensing still remains a challenge. Some mechanisms to protect privacy provide anonymity, and thus provide limited means for accountability; data integrity becomes even more difficult to protect.

We propose SenseRight, the first architecture for high-integrity people-centric sensing. The SenseRight approach, which extends and enhances AnonySense, assures integrity of both the sensor data (through use of tamper-resistant sensor devices) and the sensor context (through a time-constrained protocol), maintaining anonymity if desired.


Jeffrey Fielding. Linkability in Activity Inference Data Sets. Technical Report, June 2008. Available as Dartmouth Computer Science Technical Report TR2008-623. [Details]

Activity inference is an active area of ubiquitous computing research. By training machine learning algorithms on data from sensors worn by volunteers, researchers hope to develop software that can interact more naturally with the user by inferring what the user is doing. In this thesis, we use the same sensor data to infer which volunteer is carrying the sensors. Such inference could be useful -- for example, a mobile device might infer who is carrying it and adapt to that user's preferences. It also raises some privacy concerns, since an attacker could learn more about a user by linking together several sensor traces from the same user. We develop a model to differentiate users based on their sensor data, and examine its accuracy as well as the potential benefits and pitfalls.

Apu Kapadia, Nikos Triandopoulos, Cory Cornelius, Dan Peebles, and David Kotz. AnonySense: Opportunistic and Privacy-Preserving Context Collection. Proceedings of the International Conference on Pervasive Computing (Pervasive). May 2008. [Details]

Opportunistic sensing allows applications to “task” mobile devices to measure context in a target region. For example, one could leverage sensor-equipped vehicles to measure traffic or pollution levels on a particular street, or users’ mobile phones to locate (Bluetooth-enabled) objects in their neighborhood. In most proposed applications, context reports include the time and location of the event, putting the privacy of users at increased risk---even if a report has been anonymized, the accompanying time and location can reveal sufficient information to deanonymize the user whose device sent the report.

We propose AnonySense, a general-purpose architecture for leveraging users’ mobile devices for measuring context, while maintaining the privacy of the users. AnonySense features multiple layers of privacy protection---a framework for nodes to receive tasks anonymously, a novel blurring mechanism based on tessellation and clustering to protect users’ privacy against the system while reporting context, and k-anonymous report aggregation to improve the users’ privacy against applications receiving the context. We outline the architecture and security properties of AnonySense, and focus on evaluating our tessellation and clustering algorithm against real mobility traces.


Guanling Chen, Ming Li, and David Kotz. Data-centric middleware for context-aware pervasive computing. Pervasive and Mobile Computing. April 2008. [Details]

The complexity of developing and deploying context-aware pervasive-computing applications calls for distributed software infrastructures that assist applications to collect, aggregate, and disseminate contextual data. In this paper, we motivate a data-centric design for such an infrastructure to support context-aware applications. Our middleware system, Solar, treats contextual data sources as stream publishers. The core of Solar is a scalable and self-organizing peer-to-peer overlay to support data-driven services. We describe how different services can be systematically integrated on top of the Solar overlay and evaluate the resource discovery and data-dissemination services. We also discuss our experience and lessons learned when using Solar to support several implemented scenarios. We conclude that a data-centric infrastructure is necessary to facilitate both the development and deployment of context-aware pervasive-computing applications.

2007:
Apu Kapadia, Tristan Henderson, Jeffrey Fielding, and David Kotz. Virtual Walls: Protecting Digital Privacy in Pervasive Environments. Proceedings of the International Conference on Pervasive Computing (Pervasive). May 2007. [Details]

As pervasive environments become more commonplace, the privacy of users is placed at an increased risk. The numerous and diverse sensors in these environments can record contextual information about users, leading to users unwittingly leaving “digital footprints.” Users must therefore be allowed to control how their digital footprints are reported to third parties. While a significant amount of prior work has focused on location privacy, location is only one specific type of footprint, and we expect most users to be incapable of specifying fine-grained policies for a multitude of footprints. In this paper we present a policy language based on the metaphor of physical walls, and posit that users will find this to be an intuitive way to control access to their digital footprints. For example, users understand the physical privacy implications of conducting a meeting in a room enclosed by physical walls. By allowing users to deploy “virtual walls,” they can control the privacy of their digital footprints much in the same way they control their privacy in the physical world. We present a policy framework and model for virtual walls with three levels of transparency that correspond to intuitive levels of privacy. We also describe the results of a user study (N = 23) that indicates that our model is easy to understand and use.

Peter Johnson, Apu Kapadia, David Kotz, and Nikos Triandopoulos. People-Centric Urban Sensing: Security Challenges for the New Paradigm. Technical Report, February 2007. [Details]

We study the security challenges that arise in people-centric urban sensing, a new sensor-networking paradigm that leverages humans as part of the sensing infrastructure. Most prior work on sensor networks has focused on collecting and processing ephemeral data about the environment using a static topology and an application-aware infrastructure. People-centric urban sensing, however, involves collecting, storing, processing and fusing large volumes of data related to every-day human activities. Sensing is performed in a highly dynamic and mobile environment, and supports (among other things) pervasive computing applications that are focused on enhancing the user’s experience. In such a setting, where humans are the central focus, there are new challenges for information security; not only because of the complex and dynamic communication patterns, but also because the data originates from sensors that are carried by a person---not a tiny sensor thrown in the forest or mounted on the neck of an animal. In this paper we aim to instigate discussion about this critical issue---because people-centric sensing will never succeed without adequate provisions for security and privacy. To that end, we outline several important challenges and suggest general solutions that hold promise in this new paradigm of sensor networks.

2006:
Rajnish Kumar, Arnab Paul, Umakishore Ramachandran, and David Kotz. On improving wireless broadcast reliability of sensor networks using erasure codes. Proceedings of the International Conference on Mobile Ad-hoc and Sensor Networks (MSN). December 2006. [Details]

Efficient and reliable dissemination of information over a large area is a critical ability of a sensor network for various reasons such as software updates and transferring large data objects (e.g., surveillance images). Thus efficiency of wireless broadcast is an important aspect of sensor network deployment. In this paper, we study FBcast, a new broadcast protocol based on the principles of modern erasure codes. We show that our approach provides high reliability, often considered critical for disseminating codes. In addition FBcast offers limited data confidentiality. For a large network, where every node may not be reachable by the source, we extend FBcast with the idea of repeaters to improve reliable coverage. Simulation results on TOSSIM show that FBcast offers higher reliability with lower number of retransmissions than traditional broadcasts.

2005:
Guanling Chen and David Kotz. Policy-Driven Data Dissemination for Context-Aware Applications. Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom). March 2005. [Details]

Context-aware pervasive-computing applications require continuous monitoring of their physical and computational environment to make appropriate adaptation decisions in time. The data streams produced by sensors, however, may overflow the queues on the dissemination path. Traditional flow-control and congestion-control policies either drop data or force the sender to pause. When the data sender is sensing the physical environment, however, a pause is equivalent to dropping data. Instead of arbitrarily dropping data that may contain important events, we present a policy-driven data dissemination service named PACK, based on an overlay-based infrastructure for efficient multicast delivery. PACK enforces application-specified policies that define how to discard or summarize data flows wherever queues overflow on the data path, notably at the mobile hosts where applications often reside. A key contribution of our approach is to uniformly apply the data-stream “packing” abstraction to queue overflow caused by network congestion, slow receivers, and temporary disconnection. We present experimental results and a detailed application study of the PACK service.

2004:
Guanling Chen, Ming Li, and David Kotz. Design and implementation of a large-scale context fusion network. Proceedings of the International Conference on Mobile and Ubiquitous Systems: Networking and Services (Mobiquitous). August 2004. [Details]

In this paper we motivate a Context Fusion Network (CFN), an infrastructure model that allows context-aware applications to select distributed data sources and compose them with customized data-fusion operators into a directed acyclic information fusion graph. Such a graph represents how an application computes high-level understandings of its execution context from low-level sensory data. Multiple graphs by different applications inter-connect with each other to form a global graph. A key advantage of a CFN is re-usability, both at code-level and instance-level, facilitated by operator composition. We designed and implemented a distributed CFN system, Solar, which maps the logical operator graph representation onto a set of overlay hosts. In particular, Solar meets the challenges inherent to heterogeneous and volatile ubicomp environments. By abstracting most complexities into the infrastructure, we believe Solar facilitates both the development and deployment of context-aware applications. We present the operator composition model, basic services of the Solar overlay network, and programming support for the developers. We also discuss some applications built with Solar and the lessons we learned from our experience.

Guanling Chen. Solar: Building A Context Fusion Network for Pervasive Computing. PhD thesis, August 2004. Available as Dartmouth Computer Science Technical Report TR2004-514. [Details]

The complexity of developing context-aware pervasive-computing applications calls for distributed software infrastructures that assist applications to collect, aggregate, and disseminate contextual data. In this dissertation, we present a Context Fusion Network (CFN), called Solar, which is built with a scalable and self-organized service overlay. Solar is flexible and allows applications to select distributed data sources and compose them with customized data-fusion operators into a directed acyclic information flow graph. Such a graph represents how an application computes high-level understandings of its execution context from low-level sensory data. To manage application-specified operators on a set of overlay nodes called Planets, Solar provides several unique services such as application-level multicast with policy-driven data reduction to handle buffer overflow, context-sensitive resource discovery to handle environment dynamics, and proactive monitoring and recovery to handle common failures. Experimental results show that these services perform well on a typical DHT-based peer-to-peer routing substrate. In this dissertation, we also discuss experience, insights, and lessons learned from our quantitative analysis of the input sensors, a detailed case study of a Solar application, and development of other applications in different domains.

Guanling Chen and David Kotz. Application-Controlled Loss-Tolerant Data Dissemination. Technical Report, February 2004. [Details]

Reactive or proactive mobile applications require continuous monitoring of their physical and computational environment to make appropriate decisions in time. These applications need to monitor data streams produced by sensors and react to changes. When mobile sensors and applications are connected by low-bandwidth wireless networks, sensor data rates may overwhelm the capacity of network links or of the applications. In traditional networks and distributed systems, flow-control and congestion-control policies either drop data or force the sender to pause. When the data sender is sensing the physical environment, however, a pause is equivalent to dropping data. Arbitrary data drops are not necessarily acceptable to the reactive mobile applications receiving sensor data. Data distribution systems must support application-specific policies that selectively drop data objects when network or application buffers overflow.

In this paper we present a data-dissemination service, PACK, which allows applications to specify customized data-reduction policies. These policies define how to discard or summarize data flows wherever buffers overflow on the dissemination path, notably at the mobile hosts where applications often reside. The PACK service provides an overlay infrastructure to support mobile data sources and sinks, using application-specific data-reduction policies where necessary along the data path. We uniformly apply the data-stream “packing” abstraction to buffer overflow caused by network congestion, slow receivers, and the temporary disconnections caused by end-host mobility. We demonstrate the effectiveness of our approach with an application example and experimental measurements.


2003:
Guanling Chen and David Kotz. Context-Sensitive Resource Discovery. Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom). March 2003. [Details]

This paper presents the “Solar” system framework that allows resources to advertise context-sensitive names and for applications to make context-sensitive name queries. The heart of our framework is a small specification language that allows composition of “context-processing operators” to calculate the desired context. Resources use the framework to register and applications use the framework to lookup context-sensitive name descriptions. The back-end system executes these operators and constantly updates the context values, adjusting advertised names and informing applications about changes. We report experimental results from a prototype, using a modified version of the Intentional Naming System (INS) as the core directory service.

2002:
Guanling Chen and David Kotz. Context Aggregation and Dissemination in Ubiquitous Computing Systems. Proceedings of the IEEE Workshop on Mobile Computing Systems and Applications (WMCSA). June 2002. [Details]

Many “ubiquitous computing” applications need a constant flow of information about their environment to be able to adapt to their changing context. To support these “context-aware” applications we propose a graph-based abstraction for collecting, aggregating, and disseminating context information. The abstraction models context information as events, produced by sources and flowing through a directed acyclic graph of event-processing operators and delivered to subscribing applications. Applications describe their desired event stream as a tree of operators that aggregate low-level context information published by existing sources into the high-level context information needed by the application. The operator graph is thus the dynamic combination of all applications’ subscription trees.

In this paper, we motivate and describe our graph abstraction, and discuss a variety of critical design issues. We also sketch our Solar system, an implementation that represents one point in the design space for our graph abstraction.


Guanling Chen and David Kotz. Solar: An Open Platform for Context-Aware Mobile Applications. Proceedings of the International Conference on Pervasive Computing (Pervasive) (Short paper). June 2002. In an informal companion volume of short papers. [Details]

Emerging pervasive computing technologies transform the way we live and work by embedding computation in our surrounding environment. To avoid increasing complexity, and allow the user to concentrate on her tasks, applications in a pervasive computing environment must automatically adapt to their changing context, including the user state and the physical and computational environment in which they run. Solar is a middleware platform to help these “context-aware” applications aggregate desired context from heterogeneous sources and to locate environmental services depending on the current context. By moving most of the context computation into the infrastructure, Solar allows applications to run on thin mobile clients more effectively. By providing an open framework to enable dynamic injection of context processing modules, Solar shares these modules across many applications, reducing application development cost and network traffic. By distributing these modules across network nodes and reconfiguring the distribution at runtime, Solar achieves parallelism and online load balancing.

Guanling Chen and David Kotz. Context Aggregation and Dissemination in Ubiquitous Computing Systems. Technical Report, February 2002. [Details]

Many “ubiquitous computing” applications need a constant flow of information about their environment to be able to adapt to their changing context. To support these “context-aware” applications we propose a graph-based abstraction for collecting, aggregating, and disseminating context information. The abstraction models context information as events, produced by sources and flowing through a directed acyclic graph of event-processing operators and delivered to subscribing applications. Applications describe their desired event stream as a tree of operators that aggregate low-level context information published by existing sources into the high-level context information needed by the application. The operator graph is thus the dynamic combination of all applications’ subscription trees.

In this paper, we motivate and describe our graph abstraction, and discuss a variety of critical design issues. We also sketch our Solar system, an implementation that represents one point in the design space for our graph abstraction.


Guanling Chen and David Kotz. Solar: A pervasive-computing infrastructure for context-aware mobile applications. Technical Report, February 2002. [Details]

Emerging pervasive computing technologies transform the way we live and work by embedding computation in our surrounding environment. To avoid increasing complexity, and allow the user to concentrate on her tasks, applications must automatically adapt to their changing context, the physical and computational environment in which they run. To support these “context-aware” applications we propose a graph-based abstraction for collecting, aggregating, and disseminating context information. The abstraction models context information as events, which are produced by sources, flow through a directed acyclic graph of event-processing operators, and are delivered to subscribing applications. Applications describe their desired event stream as a tree of operators that aggregate low-level context information published by existing sources into the high-level context information needed by the application. The operator graph is thus the dynamic combination of all applications’ subscription trees. In this paper, we motivate our graph abstraction by discussing several applications under development, sketch the architecture of our system (“Solar”) that implements our abstraction, report some early experimental results from the prototype, and outline issues for future research.

2001:
Robert S. Gray, David Kotz, Ronald A. Peterson, Jr., Joyce Barton, Daria Chacón, Peter Gerken, Martin Hofmann, Jeffrey Bradshaw, Maggie Breedy, Renia Jeffers, and Niranjan Suri. Mobile-Agent versus Client/Server Performance: Scalability in an Information-Retrieval Task. Proceedings of the IEEE International Conference on Mobile Agents. December 2001. A corrected version of this paper is available on the Dartmouth web site. [Details]

Building applications with mobile agents often reduces the bandwidth required for the application, and improves performance. The cost is increased server workload. There are, however, few studies of the scalability of mobile-agent systems. We present scalability experiments that compare four mobile-agent platforms with a traditional client/server approach. The four mobile-agent platforms have similar behavior, but their absolute performance varies with underlying implementation choices. Our experiments demonstrate the complex interaction between environmental, application, and system parameters.

Guanling Chen and David Kotz. SOLAR: Towards a Flexible and Scalable Data-Fusion Infrastructure for Ubiquitous Computing. Proceedings of the UbiTools workshop at UbiComp 2001. October 2001. [Details]

As we embed more computers into our daily environment, ubiquitous computing promises to make them less noticeable and to avoid information overload. We see, however, few ubiquitous applications that are able to adapt to the dynamics of user, physical, and computational context. The challenge is to allow applications flexible access to these sources, and yet scale to thousands of devices and sensors. In this paper we introduce our proposed infrastructure, Solar. In Solar, information sources produce events. Applications may subscribe to interesting sources directly, or they may instantiate and subscribe to a tree of operators that filter, transform, merge and aggregate events. Applications use a subscription language to describe the tree, based on event streams registered in a context-sensitive naming hierarchy. Solar is flexible: modular operators can be composed to produce new event streams. Solar is scalable: it distributes operators across hosts called Planets, and it re-uses common subgraphs in the operator network.

Guanling Chen and David Kotz. Supporting Adaptive Ubiquitous Applications with the SOLAR System. Technical Report, May 2001. [Details]

As we embed more computers into our daily environment, ubiquitous computing promises to make them less noticeable and help to prevent information overload. We see, however, few ubiquitous applications that are able to adapt to the dynamics of user, physical, and computational context. We believe that there are two challenges causing this lack of ubiquitous applications: there is no flexible and scalable way to support information collection and dissemination in a ubiquitous and mobile environment, and there is no general approach to building adaptive applications given heterogeneous contextual information. We propose a system infrastructure, Solar, to meet these challenges. Solar uses a subscription-based operator graph abstraction and allows dynamic composition of stackable operators to manage ubiquitous information sources. After developing a set of diverse adaptive applications, we expect to identify fundamental techniques for context-aware adaptation. Our expectation is that Solar’s end-to-end support for information collection, dissemination, and utilization will make it easy to build adaptive applications for a ubiquitous mobile environment with many users and devices.


[Kotz research]