Reversible Action Design for Combinatorial Optimization with Reinforcement Learning

by   Fan Yao, et al.

Combinatorial optimization problem (COP) over graphs is a fundamental challenge in optimization. Reinforcement learning (RL) has recently emerged as a new framework to tackle these problems and has demonstrated promising results. However, most RL solutions employ a greedy manner to construct the solution incrementally, thus inevitably pose unnecessary dependency on action sequences and need a lot of problem-specific designs. We propose a general RL framework that not only exhibits state-of-the-art empirical performance but also generalizes to a variety class of COPs. Specifically, we define state as a solution to a problem instance and action as a perturbation to this solution. We utilize graph neural networks (GNN) to extract latent representations for given problem instances for state-action encoding, and then apply deep Q-learning to obtain a policy that gradually refines the solution by flipping or swapping vertex labels. Experiments are conducted on Maximum k-Cut and Traveling Salesman Problem and performance improvement is achieved against a set of learning-based and heuristic baselines.


page 1

page 2

page 3

page 4


Exploratory Combinatorial Optimization with Reinforcement Learning

Many real-world problems can be reduced to combinatorial optimization on...

Reinforcement Learning for Combinatorial Optimization: A Survey

Combinatorial optimization (CO) is the workhorse of numerous important a...

Learning to Solve Combinatorial Graph Partitioning Problems via Efficient Exploration

From logistics to the natural sciences, combinatorial optimisation on gr...

On the Difficulty of Generalizing Reinforcement Learning Framework for Combinatorial Optimization

Combinatorial optimization problems (COPs) on the graph with real-life a...

Reinforcement Learning to Solve NP-hard Problems: an Application to the CVRP

In this paper, we evaluate the use of Reinforcement Learning (RL) to sol...

Combining Reinforcement Learning and Configuration Checking for Maximum k-plex Problem

The Maximum k-plex Problem is an important combinatorial optimization pr...

Active Screening for Recurrent Diseases: A Reinforcement Learning Approach

Active screening is a common approach in controlling the spread of recur...