Robust Actor-Critic Contextual Bandit for Mobile Health (mHealth) Interventions

02/27/2018
by   Feiyun Zhu, et al.
0

We consider the actor-critic contextual bandit for the mobile health (mHealth) intervention. State-of-the-art decision-making algorithms generally ignore the outliers in the dataset. In this paper, we propose a novel robust contextual bandit method for the mHealth. It can achieve the conflicting goal of reducing the influence of outliers while seeking for a similar solution compared with the state-of-the-art contextual bandit methods on the datasets without outliers. Such performance relies on two technologies: (1) the capped-ℓ_2 norm; (2) a reliable method to set the thresholding hyper-parameter, which is inspired by one of the most fundamental techniques in the statistics. Although the model is non-convex and non-differentiable, we propose an effective reweighted algorithm and provide solid theoretical analyses. We prove that the proposed algorithm can find sufficiently decreasing points after each iteration and finally converges after a finite number of iterations. Extensive experiment results on two datasets demonstrate that our method can achieve almost identical results compared with state-of-the-art contextual bandit methods on the dataset without outliers, and significantly outperform those state-of-the-art methods on the badly noised dataset with outliers in a variety of parameter settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/17/2017

Robust Contextual Bandit via the Capped-ℓ_2 norm

This paper considers the actor-critic contextual bandit for the mobile h...
research
06/28/2017

An Actor-Critic Contextual Bandit Algorithm for Personalized Mobile Health Interventions

Increasing technological sophistication and widespread use of smartphone...
research
08/21/2022

Robust Tests in Online Decision-Making

Bandit algorithms are widely used in sequential decision problems to max...
research
06/18/2023

On the Global Convergence of Natural Actor-Critic with Two-layer Neural Network Parametrization

Actor-critic algorithms have shown remarkable success in solving state-o...
research
05/22/2022

Contextual Information-Directed Sampling

Information-directed sampling (IDS) has recently demonstrated its potent...
research
12/21/2020

Fast Physical Activity Suggestions: Efficient Hyperparameter Learning in Mobile Health

Users can be supported to adopt healthy behaviors, such as regular physi...

Please sign up or login with your details

Forgot password? Click here to reset