IntelligentPooling: Practical Thompson Sampling for mHealth

by   Sabina Tomkins, et al.

In mobile health (mHealth) smart devices deliver behavioral treatments repeatedly over time to a user with the goal of helping the user adopt and maintain healthy behaviors. Reinforcement learning appears ideal for learning how to optimally make these sequential treatment decisions. However, significant challenges must be overcome before reinforcement learning can be effectively deployed in a mobile healthcare setting. In this work we are concerned with the following challenges: 1) individuals who are in the same context can exhibit differential response to treatments 2) only a limited amount of data is available for learning on any one individual, and 3) non-stationary responses to treatment. To address these challenges we generalize Thompson-Sampling bandit algorithms to develop IntelligentPooling. IntelligentPooling learns personalized treatment policies thus addressing challenge one. To address the second challenge, IntelligentPooling updates each user's degree of personalization while making use of available data on other users to speed up learning. Lastly, IntelligentPooling allows responsivity to vary as a function of a user's time since beginning treatment, thus addressing challenge three. We show that IntelligentPooling achieves an average of 26 lower regret than state-of-the-art. We demonstrate the promise of this approach and its ability to learn from even a small group of users in a live clinical trial.


page 1

page 2

page 3

page 4


Rapidly Personalizing Mobile Health Treatment Policies with Limited Data

In mobile health (mHealth), reinforcement learning algorithms that adapt...

The stratified micro-randomized trial design: sample size considerations for testing nested causal effects of time-varying treatments

Technological advancements in the field of mobile devices and wearable s...

Personalizing Intervention Probabilities By Pooling

In many mobile health interventions, treatments should only be delivered...

Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity

With the recent evolution of mobile health technologies, health scientis...

Evaluating Reinforcement Learning Algorithms in Observational Health Settings

Much attention has been devoted recently to the development of machine l...

AI-Augmented Behavior Analysis for Children with Developmental Disabilities: Building Towards Precision Treatment

Autism spectrum disorder is a developmental disorder characterized by si...

Fast Physical Activity Suggestions: Efficient Hyperparameter Learning in Mobile Health

Users can be supported to adopt healthy behaviors, such as regular physi...