Effective Warm Start for the Online Actor-Critic Reinforcement Learning based mHealth Intervention

by   Feiyun Zhu, et al.

Online reinforcement learning (RL) is increasingly popular for the personalized mobile health (mHealth) intervention. It is able to personalize the type and dose of interventions according to user's ongoing statuses and changing needs. However, at the beginning of online learning, there are usually too few samples to support the RL updating, which leads to poor performances. A delay in good performance of the online learning algorithms can be especially detrimental in the mHealth, where users tend to quickly disengage with the mHealth app. To address this problem, we propose a new online RL methodology that focuses on an effective warm start. The main idea is to make full use of the data accumulated and the decision rule achieved in a former study. As a result, we can greatly enrich the data size at the beginning of online learning in our method. Such case accelerates the online learning process for new users to achieve good performances not only at the beginning of online learning but also through the whole online learning process. Besides, we use the decision rules achieved in a previous study to initialize the parameter in our online RL model for new users. It provides a good initialization for the proposed online RL algorithm. Experiment results show that promising improvements have been achieved by our method compared with the state-of-the-art method.


page 1

page 2

page 3

page 4


Warm-Start Actor-Critic: From Approximation Error to Sub-optimality Gap

Warm-Start reinforcement learning (RL), aided by a prior policy obtained...

A Reduction from Reinforcement Learning to No-Regret Online Learning

We present a reduction from reinforcement learning (RL) to no-regret onl...

Personalization of Health Interventions using Cluster-Based Reinforcement Learning

Research has shown that personalization of health interventions can cont...

Did we personalize? Assessing personalization by an online reinforcement learning algorithm using resampling

There is a growing interest in using reinforcement learning (RL) to pers...

Timing Process Interventions with Causal Inference and Reinforcement Learning

The shift from the understanding and prediction of processes to their op...

Data-pooling Reinforcement Learning for Personalized Healthcare Intervention

Motivated by the emerging needs of personalized preventative interventio...

Online learning using multiple times weight updating

Online learning makes sequence of decisions with partial data arrival wh...

Please sign up or login with your details

Forgot password? Click here to reset