How to Evaluate Machine Learning Approaches for Combinatorial Optimization: Application to the Travelling Salesman Problem

by   Antoine François, et al.

Combinatorial optimization is the field devoted to the study and practice of algorithms that solve NP-hard problems. As Machine Learning (ML) and deep learning have popularized, several research groups have started to use ML to solve combinatorial optimization problems, such as the well-known Travelling Salesman Problem (TSP). Based on deep (reinforcement) learning, new models and architecture for the TSP have been successively developed and have gained increasing performances. At the time of writing, state-of-the-art models provide solutions to TSP instances of 100 cities that are roughly 1.33 from optimal solutions. However, despite these apparently positive results, the performances remain far from those that can be achieved using a specialized search procedure. In this paper, we address the limitations of ML approaches for solving the TSP and investigate two fundamental questions: (1) how can we measure the level of accuracy of the pure ML component of such methods; and (2) what is the impact of a search procedure plugged inside a ML model on the performances? To answer these questions, we propose a new metric, ratio of optimal decisions (ROD), based on a fair comparison with a parametrized oracle, mimicking a ML model with a controlled accuracy. All the experiments are carried out on four state-of-the-art ML approaches dedicated to solve the TSP. Finally, we made ROD open-source in order to ease future research in the field.


page 1

page 2

page 3

page 4


Graph Embedding for Combinatorial Optimization: A Survey

Graphs have been widely used to represent complex data in many applicati...

Matrix Encoding Networks for Neural Combinatorial Optimization

Machine Learning (ML) can help solve combinatorial optimization (CO) pro...

Boosting Combinatorial Problem Modeling with Machine Learning

In the past few years, the area of Machine Learning (ML) has witnessed t...

Learning with Combinatorial Optimization Layers: a Probabilistic Approach

Combinatorial optimization (CO) layers in machine learning (ML) pipeline...

Transferable Graph Optimizers for ML Compilers

Most compilers for machine learning (ML) frameworks need to solve many c...

Pretrained Cost Model for Distributed Constraint Optimization Problems

Distributed Constraint Optimization Problems (DCOPs) are an important su...

Learning the Quality of Machine Permutations in Job Shop Scheduling

In recent years, the power demonstrated by Machine Learning (ML) has inc...

Please sign up or login with your details

Forgot password? Click here to reset