DeepAI AI Chat
Log In Sign Up

POMO: Policy Optimization with Multiple Optima for Reinforcement Learning

by   Yeong-Dae Kwon, et al.

In neural combinatorial optimization (CO), reinforcement learning (RL) can turn a deep neural net into a fast, powerful heuristic solver of NP-hard problems. This approach has a great potential in practical applications because it allows near-optimal solutions to be found without expert guides armed with substantial domain knowledge. We introduce Policy Optimization with Multiple Optima (POMO), an end-to-end approach for building such a heuristic solver. POMO is applicable to a wide range of CO problems. It is designed to exploit the symmetries in the representation of a CO solution. POMO uses a modified REINFORCE algorithm that forces diverse rollouts towards all optimal solutions. Empirically, the low-variance baseline of POMO makes RL training fast and stable, and it is more resistant to local minima compared to previous approaches. We also introduce a new augmentation-based inference method, which accompanies POMO nicely. We demonstrate the effectiveness of POMO by solving three popular NP-hard problems, namely, traveling salesman (TSP), capacitated vehicle routing (CVRP), and 0-1 knapsack (KP). For all three, our solver based on POMO shows a significant improvement in performance over all recent learned heuristics. In particular, we achieve the optimality gap of 0.14 while reducing inference time by more than an order of magnitude.


page 1

page 2

page 3

page 4


Population-Based Reinforcement Learning for Combinatorial Optimization

Applying reinforcement learning (RL) to combinatorial optimization probl...

Reinforcement Learning to Solve NP-hard Problems: an Application to the CVRP

In this paper, we evaluate the use of Reinforcement Learning (RL) to sol...

An interacting replica approach applied to the traveling salesman problem

We present a physics inspired heuristic method for solving combinatorial...

Evolutionary RL for Container Loading

Loading the containers on the ship from a yard, is an impor- tant part o...

Learning Vehicle Routing Problems using Policy Optimisation

Deep reinforcement learning (DRL) has been used to learn effective heuri...

Neural Airport Ground Handling

Airport ground handling (AGH) offers necessary operations to flights dur...

Complex matter field universal models with optimal scaling for solving combinatorial optimization problems

We develop a universal model based on the classical complex matter field...