Active Screening for Recurrent Diseases: A Reinforcement Learning Approach

by   Han-Ching Ou, et al.

Active screening is a common approach in controlling the spread of recurring infectious diseases such as tuberculosis and influenza. In this approach, health workers periodically select a subset of population for screening. However, given the limited number of health workers, only a small subset of the population can be visited in any given time period. Given the recurrent nature of the disease and rapid spreading, the goal is to minimize the number of infections over a long time horizon. Active screening can be formalized as a sequential combinatorial optimization over the network of people and their connections. The main computational challenges in this formalization arise from i) the combinatorial nature of the problem, ii) the need of sequential planning and iii) the uncertainties in the infectiousness states of the population. Previous works on active screening fail to scale to large time horizon while fully considering the future effect of current interventions. In this paper, we propose a novel reinforcement learning (RL) approach based on Deep Q-Networks (DQN), with several innovative adaptations that are designed to address the above challenges. First, we use graph convolutional networks (GCNs) to represent the Q-function that exploit the node correlations of the underlying contact network. Second, to avoid solving a combinatorial optimization problem in each time period, we decompose the node set selection as a sub-sequence of decisions, and further design a two-level RL framework that solves the problem in a hierarchical way. Finally, to speed-up the slow convergence of RL which arises from reward sparseness, we incorporate ideas from curriculum learning into our hierarchical RL approach. We evaluate our RL algorithm on several real-world networks.


Exploratory Combinatorial Optimization with Reinforcement Learning

Many real-world problems can be reduced to combinatorial optimization on...

Combinatorial Optimization by Graph Pointer Networks and Hierarchical Reinforcement Learning

In this work, we introduce Graph Pointer Networks (GPNs) trained using r...

Reinforcement Learning to Solve NP-hard Problems: an Application to the CVRP

In this paper, we evaluate the use of Reinforcement Learning (RL) to sol...

Reinforcement Learning with Chromatic Networks

We present a new algorithm for finding compact neural networks encoding ...

Reversible Action Design for Combinatorial Optimization with Reinforcement Learning

Combinatorial optimization problem (COP) over graphs is a fundamental ch...

How to Stop Epidemics: Controlling Graph Dynamics with Reinforcement Learning and Graph Neural Networks

We consider the problem of monitoring and controlling a partially-observ...

Accelerating Psychometric Screening Tests With Bayesian Active Differential Selection

Classical methods for psychometric function estimation either require ex...