Reversible Action Design for Combinatorial Optimization with Reinforcement Learning

02/14/2021
by   Fan Yao, et al.
0

Combinatorial optimization problem (COP) over graphs is a fundamental challenge in optimization. Reinforcement learning (RL) has recently emerged as a new framework to tackle these problems and has demonstrated promising results. However, most RL solutions employ a greedy manner to construct the solution incrementally, thus inevitably pose unnecessary dependency on action sequences and need a lot of problem-specific designs. We propose a general RL framework that not only exhibits state-of-the-art empirical performance but also generalizes to a variety class of COPs. Specifically, we define state as a solution to a problem instance and action as a perturbation to this solution. We utilize graph neural networks (GNN) to extract latent representations for given problem instances for state-action encoding, and then apply deep Q-learning to obtain a policy that gradually refines the solution by flipping or swapping vertex labels. Experiments are conducted on Maximum k-Cut and Traveling Salesman Problem and performance improvement is achieved against a set of learning-based and heuristic baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/09/2019

Exploratory Combinatorial Optimization with Reinforcement Learning

Many real-world problems can be reduced to combinatorial optimization on...
03/07/2020

Reinforcement Learning for Combinatorial Optimization: A Survey

Combinatorial optimization (CO) is the workhorse of numerous important a...
05/27/2022

Learning to Solve Combinatorial Graph Partitioning Problems via Efficient Exploration

From logistics to the natural sciences, combinatorial optimisation on gr...
08/08/2021

On the Difficulty of Generalizing Reinforcement Learning Framework for Combinatorial Optimization

Combinatorial optimization problems (COPs) on the graph with real-life a...
01/14/2022

Reinforcement Learning to Solve NP-hard Problems: an Application to the CVRP

In this paper, we evaluate the use of Reinforcement Learning (RL) to sol...
06/06/2019

Combining Reinforcement Learning and Configuration Checking for Maximum k-plex Problem

The Maximum k-plex Problem is an important combinatorial optimization pr...
01/07/2021

Active Screening for Recurrent Diseases: A Reinforcement Learning Approach

Active screening is a common approach in controlling the spread of recur...