Related projects: [Amanuensis], [Amulet], [Auracle], [THaW], [TISH]
Related keywords: [mhealth], [sensors], [wearable]
In the SIMBA (Sensing and Intervention for Mental, Behavioral, and Affective health) project we explore opportunities to sense physiological states (such as stress) and the receptivity to behavioral interventions. For the complete set of work, see [mishra:thesis].
Recent advances in wearable sensing have resulted in multiple approaches to detect and monitor stress with varying levels of accuracy. The most accurate methods, however, rely on clinical-grade sensors to measure physiological signals; they are often bulky, custom made, and expensive, hence limiting their adoption by researchers and the general public. In our work, we explore the viability of commercially available off-the-shelf sensors for stress monitoring. Our analysis shows that using the off-the-shelf Polar H7 sensor, we were able to detect stressful events with an F1-score of up to 0.87 in the lab and 0.66 in the field, on par with clinical-grade sensors [mishra:jcommodity]. Then, a subsequent paper takes the first step towards testing reproducibility and validity of methods and machine-learning models for stress detection [mishra:stress-ml].
In another line of work, we explore receptivity to mHealth interventions, including Just-In-Time Adaptive Interventions (JITAI). We explored the factors affecting users' receptivity towards JITAIs by conducting a study with 189 participants, over a period of 6 weeks. We found that (1) several participant-specific characteristics (age, personality, and device type) show significant associations with the overall participant receptivity over the course of the study, and that (2) several contextual factors (day/time, phone battery, phone interaction, physical activity, and location), show significant associations with the participant receptivity, in-the-moment. We also built machine-learning models to detect receptivity, with up to a 77% increase in F1 score over a biased random classifier [kunzler:receptivity].
We then deployed the Ally app in a field study with 83 users. The app included two types of machine-learning model that used contextual information about a person to predict when a person is receptive: a static model that was built before the study started and remained constant for all participants and an adaptive model that continuously learned the receptivity of individual participants and updated itself as the study progressed. For comparison, we included a control model that sent intervention messages at random times. The app randomly selected a delivery model for each intervention message. We observed that the machine-learning models led up to a 40% improvement in receptivity as compared to the control model. Further, we evaluated the temporal dynamics of the different models and observed that receptivity to messages from the adaptive model increased over the course of the study. After an early version of this work appeared in arXiv [mishra:receptivity-tr], our full paper was published in UbiComp [mishra:receptivity] and won an IMWUT Distinguished Paper Award (DPA). Citation: "This research is a perfect example of how to improve mobile health interventions by exploring models that help to decide on the fly when the right moment for the intervention has arrived. It reports findings from an in-the-wild study with 83 participants over three weeks and thus constitutes a core Ubicomp research contributions." The work later appeared in ACM GetMobile as a summary highlight [mishra:receptivity-highlight].
We later conducted a study with 20 participants over 3 weeks, where participants received interventions about mental health, COVID-19, physical activity, and diet & nutrition. The interventions were delivered by the chatbot-based iOS app called Elena+, and via the MobileCoach platform. Our exploratory analysis found that significant differences in mean intrinsic motivation scores across topics were not associated with differences in mean receptivity metrics across topics. We also found that positive relationships exist between intrinsic motivation measures and receptivity for interventions about a topic. The student thesis [hong:receptivity] was later published as a workshop paper [mishra:wellcomp].
We also explored the receptivity of car drivers to in-car wellness interventions. Based on our analyses, we provide solid recommendations on how to deliver interventions to maximize responsiveness and effectiveness and minimize the burden on the drivers. [koch:car-receptivity].
To evaluate the effects of incentives, weekly planning, and daily self-monitoring prompts that were used as intervention components as part of the Ally app, we conducted a study with n = 274 insurees of a health insurance company in Switzerland. We found that daily cash incentives increased physical activity in the short term. Planning interventions and self-monitoring prompts require revision before they can be included in future versions of the app. Selection effects and engagement can be important challenges for physical-activity apps [kramer:step-goals].
Finally, we are exploring several of these ideas in the context of the D-TECT digital phenotyping project at CTBH. We are particularly interested in the utility of EMA and digital sensing in predicting OUD (opioid-use disorder) treatment retention and buprenorphine medication adherence. We published the research protocol for D-TECT [marsch:dtect-protocol], as well as a preliminary study of patient engagement [campbell:engagement]. More recently, we have published the first results from the study [heinz:ema].
Varun Mishra and David Kotz, with others: Asma Asyyed, Thomas Berger, Sukanya Bhattacharya, Kelly Caine, Cynthia I. Campbell, Ching-Hua Chen, Grace Chen, Monique B. Does, Elgar Fleisch, Ryan Halter, Tian Hao, Saeed Hassanpour, Michael V. Heinz, Emily Hichborn, Sarah Hong, Nicholas C. Jacobson, Florian Künzler, Kevin Koch, Tobias Kowatsch, Jan-Niklas Kramer, Chantal A. Lambert-Harris, Stephanie Lewia, Zhiguo Li, Shu Liu, Sarah Lord, Byron Lowens, Lisa A Marsch, Bethany McLeman, Gunnar Pope, Bastien Presset, George D. Price, Jeffrey Rogers, Urte Scholz, Sougata Sen, Avijit Singh, Shawna Smith, Catherine Stanger, Geetha Subramaniam, Felix Wortmann, and Weiyi Wu.
SIMBA was a project affiliated with Dartmouth's Center for Technology and Behavioral Health, which is supported by the US National Institute of Drug Abuse under award number NIH/NIDA P30DA029926.
Many SIMBA projects are affiliated with Dartmouth's Institute for Security, Technology, and Society (ISTS), and supported by the US National Science Foundation under award numbers CNS-1314281, CNS-1314342, CNS-1619970, and CNS-1619950.
Many SIMBA projects are a partnership with the Center for Digital Health Interventions at ETH Zürch. Some of those studies were funded by CSS Insurance, Switzerland.
The views and conclusions contained on this site and in its documents are those of the authors and should not be interpreted as necessarily representing the official position or policies, either expressed or implied, of the sponsor(s). Any mention of specific companies or products does not imply any endorsement by the authors or by the sponsor(s).
[Also available in BibTeX]
Papers are listed in reverse-chronological order;
click an entry to pop up the abstract.
For full information and pdf, please click Details link.
Follow updates with RSS.
Methods: Study participants included adults receiving MOUD at a large outpatient treatment program. We predicted NPOU (EMA-based), medication nonadherence (Electronic Health Record [EHR]- and EMA-based), and treatment retention (EHR-based) using context-sensitive EMAs (e.g., stress, pain, social setting). We used recurrent deep learning models with 7-day sliding windows to predict the next-day outcomes, using Area Under the ROC Curve (AUC) for assessment. We employed SHapley additive ExPlanations (SHAP) to understand feature latency and importance.
Results: Participants comprised 62 adults with 14,322 observations. Model performance varied across EMA subtypes and outcomes with AUCs spanning 0.59-0.97. Recent substance use was the best performing predictor for EMA-based NPOU (AUC=0.97) and medication nonadherence (AUC=0.68); life-contextual factors performed best for EHR-based medication nonadherence (AUC=0.89) and retention (AUC=0.80). SHAP revealed varying latencies between predictors and outcomes.
Conclusions: Findings support the effectiveness of EMA and deep learning for forecasting actionable outcomes in persons receiving MOUD. These insights will enable the development of personalized dynamic risk profiles and just-in-time adaptive interventions (JITAIs) to mitigate high-risk OUD outcomes.
Objective: The aim is to examine patient engagement with multiple digital phenotyping methods among patients receiving buprenorphine medication for OUD.
Methods: The study enrolled 65 patients receiving buprenorphine for OUD between June 2020 and January 2021 from 4 addiction medicine programs in an integrated health care delivery system in Northern California. Ecological momentary assessment (EMA), sensor data, and social media data were collected by smartphone, smartwatch, and social media platforms over a 12-week period. Primary engagement outcomes were meeting measures of minimum phone carry (≥8 hours per day) and watch wear (≥18 hours per day) criteria, EMA response rates, social media consent rate, and data sparsity. Descriptive analyses, bivariate, and trend tests were performed.
Results: The participants’ average age was 37 years, 47% of them were female, and 71% of them were White. On average, participants met phone carrying criteria on 94% of study days, met watch wearing criteria on 74% of days, and wore the watch to sleep on 77% of days. The mean EMA response rate was 70%, declining from 83% to 56% from week 1 to week 12. Among participants with social media accounts, 88% of them consented to providing data; of them, 55% of Facebook, 54% of Instagram, and 57% of Twitter participants provided data. The amount of social media data available varied widely across participants. No differences by age, sex, race, or ethnicity were observed for any outcomes.
Conclusions: To our knowledge, this is the first study to capture these 3 digital data sources in this clinical population. Our findings demonstrate that patients receiving buprenorphine treatment for OUD had generally high engagement with multiple digital phenotyping data sources, but this was more limited for the social media data.
International Registered Report Identifier (IRRID): RR2-10.3389/fpsyt.2022.871916
Methods: This paper describes the protocol (including the study design and methodological considerations) from a novel study supported by the National Drug Abuse Treatment Clinical Trials Network at the National Institute on Drug Abuse (NIDA). This study (D-TECT) primarily seeks to evaluate the feasibility of collecting ecological momentary assessment (EMA), smartphone and smartwatch sensor data, and social media data among patients in outpatient MOUD treatment. It secondarily seeks to examine the utility of EMA, digital sensing, and social media data (separately and compared to one another) in predicting MOUD treatment retention, opioid use events, and medication adherence [as captured in electronic health records (EHR) and EMA data]. To our knowledge, this is the first project to include all three sources of digitally derived data (EMA, digital sensing, and social media) in understanding the clinical trajectories of patients in MOUD treatment. These multiple data streams will allow us to understand the relative and combined utility of collecting digital data from these diverse data sources. The inclusion of EHR data allows us to focus on the utility of digital health data in predicting objectively measured clinical outcomes.
Discussion: Results may be useful in elucidating novel relations between digital data sources and OUD treatment outcomes. It may also inform approaches to enhancing outcomes measurement in clinical trials by allowing for the assessment of dynamic interactions between individuals' daily lives and their MOUD treatment response.
Clinical Trial Registration: Identifier: NCT04535583.
In the first part of the thesis, we discuss our work on accurate sensing and detection of different states-of-vulnerability. We start by discussing our work on advancing the field of physiological stress sensing. We took the first step towards testing the reproducibility and validity of our methods and machine-learning models for stress detection. To this end, we analyzed data from 90 participants from four independent controlled studies, using two different types of sensors, with different study protocols and research goals. We evaluated new methods to improve the performance of stress-detection models and found that our methods led to a consistent increase in performance across all studies, irrespective of the device type, sensor type, or the type of stressor. Our thorough exploration of reproducibility in a controlled environment provides a critical foundation for deeper study of such methods, and is a prerequisite for tackling reproducibility in free-living conditions.
Next, we present our work on detecting at-risk indicators for patients undergoing Opioid Use Disorder (OUD) treatment. We conducted a 12-week study with 59 patients undergoing an OUD treatment and collected sensor data, like location, physical activity, sleep, and heart rate, from smartphones and wearables. We used the data collected to formulate low- level contextual features and high-level behavioral features and explored the feasibility of detecting self-reported stress, craving, and mood of the participants. Our results show that adaptive, personalized models can detect different at-risk behaviors with the area under the receiver operating characteristic (AUROC) values of up to 0.85.
In the second part of this dissertation, we discuss our contributions in the domain of state-of-receptivity for digital health interventions. We start by conducting a study with 189 participants in Switzerland to explore participant receptivity towards actual physical activity behavior change interventions and report novel and significant results, e.g., being more receptive to interventions leads to higher goal completion likelihood. We further built machine-learning models to predict state-of-receptivity and deployed those models in a real-world study with participants in the United States to evaluate their effectiveness. Our results show that participants were more receptive to interventions delivered at moments detected as ‘receptive’ by our models.
In addition to receptivity in daily living conditions, we explored how participants interact with affective health interventions while driving. We analyzed longitudinal data from 10 participants driving in their day-to-day lives for two months. In this exploratory work, we found that several high-level trip factors (traffic flow, trip length, and vehicle occupancy) and in-the-moment factors (road type, average speed, and braking behavior) showed significant associations with the participant’s decision to start or cancel an intervention. Based on our analysis, we provide solid recommendations on delivering interventions to maximize responsiveness and effectiveness and minimize the burden on the drivers.
Overall, this dissertation makes significant contributions to the respective sub-fields by addressing fundamental challenges, advancing the current state-of-the-art, and contribut- ing new knowledge, thereby laying a solid foundation for designing, implementing, and delivering future digital health interventions.
We leveraged prior work regarding receptivity to JITAIs and deployed a chatbot-based digital coach - Ally - that provided physical-activity interventions and motivated participants to achieve their step goals. We extended the original Ally app to include two types of machine-learning model that used contextual information about a person to predict when a person is receptive: a static model that was built before the study started and remained constant for all participants and an adaptive model that continuously learned the receptivity of individual participants and updated itself as the study progressed. For comparison, we included a control model that sent intervention messages at random times. The app randomly selected a delivery model for each intervention message. We observed that the machine-learning models led up to a 40% improvement in receptivity as compared to the control model. Further, we evaluated the temporal dynamics of the different models and observed that receptivity to messages from the adaptive model increased over the course of the study.
Although past work on generic phone notifications has found evidence that users are more likely to respond to notifications with content they view as useful, there is no existing research on whether users' intrinsic motivation for the underlying topic of mHealth interventions affects their receptivity. In this work, we explore whether relationships exist between intrinsic motivation and receptivity across topics and within topics for mHealth interventions. To this end, we conducted a study with 20 participants over 3 weeks, where participants received interventions about mental health, COVID-19, physical activity, and diet & nutrition. The interventions were delivered by the chatbot-based iOS app called Elena+, and via the MobileCoach platform.
Our exploratory analysis found that significant differences in mean intrinsic motivation scores across topics were not associated with differences in mean receptivity metrics across topics. We also found that positive relationships exist between intrinsic motivation measures and receptivity for interventions about a topic.
To this end, we conducted a two-month longitudinal study with 10 participants, in which each participant was provided with a study car for their daily driving needs. We delivered two in-vehicle interventions - each aimed at improving affective well-being - and simultaneously recorded the participants' driving behavior. In our analysis, we found that several pre-trip characteristics (like trip length, traffic flow, and vehicle occupancy) and the pre-trip affective state of the participants had significant associations with whether the participants started an intervention or canceled a started intervention. Next, we found that several in-the-moment driving characteristics (like current road type, past average speed, and future brake behavior) showed significant associations with drivers' responsiveness to the intervention. Further, we identified several driving behaviors that "negated" the effectiveness of interventions and highlight the potential of using such "negative" driving characteristics to better inform intervention delivery. Finally, we compared trips with and without intervention and found that both interventions employed in our study did not have a negative effect on driving behavior. Based on our analyses, we provide solid recommendations on how to deliver interventions to maximize responsiveness and effectiveness and minimize the burden on the drivers.
This paper takes the first step towards testing reproducibility and validity of methods and machine-learning models for stress detection. To this end, we analyzed data from 90 participants, from four independent controlled studies, using two different types of sensors, with different study protocols and research goals. We started by evaluating the performance of models built using data from one study and tested on data from other studies. Next, we evaluated new methods to improve the performance of stress-detection models and found that our methods led to a consistent increase in performance across all studies, irrespective of the device type, sensor type, or the type of stressor. Finally, we developed and evaluated a clustering approach to determine the stressed/not-stressed classification when applying models on data from different studies, and found that our approach performed better than selecting a threshold based on training data. This paper's thorough exploration of reproducibility in a controlled environment provides a critical foundation for deeper study of such methods, and is a prerequisite for tackling reproducibility in free-living conditions.
Purpose: To evaluate the effects of incentives, weekly planning, and daily self-monitoring prompts that were used as intervention components as part of the Ally app.
Methods: We conducted an 8 week optimization trial with n = 274 insurees of a health insurance company in Switzerland. At baseline, participants were randomized to different incentive conditions (cash incentives vs. charity incentives vs. no incentives). Over the course of the study, participants were randomized weekly to different planning conditions (action planning vs. coping planning vs. no planning) and daily to receiving or not receiving a self-monitoring prompt. Primary outcome was the achievement of personalized daily step goals.
Results: Study participants were more active and healthier than the general Swiss population. Daily cash incentives increased step-goal achievement by 8.1%, 95% confidence interval (CI): [2.1, 14.1] and, only in the no-incentive control group, action planning increased step-goal achievement by 5.8%, 95% CI: [1.2, 10.4]. Charity incentives, self-monitoring prompts, and coping planning did not affect physical activity. Engagement with planning interventions and self-monitoring prompts was low and 30% of participants stopped using the app over the course of the study.
Conclusions: Daily cash incentives increased physical activity in the short term. Planning interventions and self-monitoring prompts require revision before they can be included in future versions of the app. Selection effects and engagement can be important challenges for physical-activity apps.
Clinical Trial Information: This study was registered on ClinicalTrials.gov, NCT03384550.
We define several metrics to gauge receptivity towards the interventions, and found that (1) several participant-specific characteristics (age, personality, and device type) show significant associations with the overall participant receptivity over the course of the study, and that (2) several contextual factors (day/time, phone battery, phone interaction, physical activity, and location), show significant associations with the participant receptivity, in-the-moment. Further, we explore the relationship between the effectiveness of the intervention and receptivity towards those interventions; based on our analyses, we speculate that being receptive to interventions helped participants achieve physical activity goals, which in turn motivated participants to be more receptive to future interventions. Finally, we build machine-learning models to detect receptivity, with up to a 77% increase in F1 score over a biased random classifier.
Objective: The primary objective of this study is to quantify main effects, interactions, and moderators of 3 intervention components of a smartphone-based intervention for physical activity. The secondary objective is the exploration of participants’ states of receptivity, that is, situations in which participants are more likely to react to intervention notifications through collection of smartphone sensor data.
Methods: In 2017, we developed the Assistant to Lift your Level of activitY (Ally), a chatbot-based mobile health intervention for increasing physical activity that utilizes incentives, planning, and self-monitoring prompts to help participants meet personalized step goals. We used a microrandomized trial design to meet the study objectives. Insurees of a large Swiss insurance company were invited to use the Ally app over a 12-day baseline and a 6-week intervention period. Upon enrollment, participants were randomly allocated to either a financial incentive, a charity incentive, or a no incentive condition. Over the course of the intervention period, participants were repeatedly randomized on a daily basis to either receive prompts that support self-monitoring or not and on a weekly basis to receive 1 of 2 planning interventions or no planning. Participants completed a Web-based questionnaire at baseline and postintervention follow-up.
Results: Data collection was completed in January 2018. In total, 274 insurees (mean age 41.73 years; 57.7% [158/274] female) enrolled in the study and installed the Ally app on their smartphones. Main reasons for declining participation were having an incompatible smartphone (37/191, 19.4%) and collection of sensor data (35/191, 18.3%). Step data are available for 227 (82.8%, 227/274) participants, and smartphone sensor data are available for 247 (90.1%, 247/274) participants.
Conclusions: This study describes the evidence-based development of a JITAI for increasing physical activity. If components prove to be efficacious, they will be included in a revised version of the app that offers scalable promotion of physical activity at low cost.
Trial Registration: ClinicalTrials.gov NCT03384550; https://clinicaltrials.gov/ct2/show/NCT03384550 (Archived by WebCite at http://www.webcitation.org/74IgCiK3d)
International Registered Report Identifier (IRRID): DERR1-10.2196/11540