SeaPearl: A Constraint Programming Solver guided by Reinforcement Learning

by   Félix Chalumeau, et al.

The design of efficient and generic algorithms for solving combinatorial optimization problems has been an active field of research for many years. Standard exact solving approaches are based on a clever and complete enumeration of the solution set. A critical and non-trivial design choice with such methods is the branching strategy, directing how the search is performed. The last decade has shown an increasing interest in the design of machine learning-based heuristics to solve combinatorial optimization problems. The goal is to leverage knowledge from historical data to solve similar new instances of a problem. Used alone, such heuristics are only able to provide approximate solutions efficiently, but cannot prove optimality nor bounds on their solution. Recent works have shown that reinforcement learning can be successfully used for driving the search phase of constraint programming (CP) solvers. However, it has also been shown that this hybridization is challenging to build, as standard CP frameworks do not natively include machine learning mechanisms, leading to some sources of inefficiencies. This paper presents the proof of concept for SeaPearl, a new CP solver implemented in Julia, that supports machine learning routines in order to learn branching decisions using reinforcement learning. Support for modeling the learning component is also provided. We illustrate the modeling and solution performance of this new solver on two problems. Although not yet competitive with industrial solvers, SeaPearl aims to provide a flexible and open-source framework in order to facilitate future research in the hybridization of constraint programming and machine learning.


page 1

page 2

page 3

page 4


Combining Reinforcement Learning and Constraint Programming for Combinatorial Optimization

Combinatorial optimization has found applications in numerous fields, fr...

An End-to-End Reinforcement Learning Approach for Job-Shop Scheduling Problems Based on Constraint Programming

Constraint Programming (CP) is a declarative programming paradigm that a...

Training a Deep Q-Learning Agent Inside a Generic Constraint Programming Solver

Constraint programming is known for being an efficient approach for solv...

Predictive Machine Learning of Objective Boundaries for Solving COPs

Solving Constraint Optimization Problems (COPs) can be dramatically simp...

Efficient Active Search for Combinatorial Optimization Problems

Recently numerous machine learning based methods for combinatorial optim...

Using machine learning to make constraint solver implementation decisions

Programs to solve so-called constraint problems are complex pieces of so...

Learning the Multiple Traveling Salesmen Problem with Permutation Invariant Pooling Networks

While there are optimal TSP solvers as well as recent learning-based app...

Please sign up or login with your details

Forgot password? Click here to reset