Zero Training Overhead Portfolios for Learning to Solve Combinatorial Problems

02/05/2021
by   Yiwei Bai, et al.
6

There has been an increasing interest in harnessing deep learning to tackle combinatorial optimization (CO) problems in recent years. Typical CO deep learning approaches leverage the problem structure in the model architecture. Nevertheless, the model selection is still mainly based on the conventional machine learning setting. Due to the discrete nature of CO problems, a single model is unlikely to learn the problem entirely. We introduce ZTop, which stands for Zero Training Overhead Portfolio, a simple yet effective model selection and ensemble mechanism for learning to solve combinatorial problems. ZTop is inspired by algorithm portfolios, a popular CO ensembling strategy, particularly restart portfolios, which periodically restart a randomized CO algorithm, de facto exploring the search space with different heuristics. We have observed that well-trained models acquired in the same training trajectory, with similar top validation performance, perform well on very different validation instances. Following this observation, ZTop ensembles a set of well-trained models, each providing a unique heuristic with zero training overhead, and applies them, sequentially or in parallel, to solve the test instances. We show how ZTopping, i.e., using a ZTop ensemble strategy with a given deep learning approach, can significantly improve the performance of the current state-of-the-art deep learning approaches on three prototypical CO domains, the hardest unique-solution Sudoku instances, challenging routing problems, and the graph maximum cut problem, as well as on multi-label classification, a machine learning task with a large combinatorial label space.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/01/2022

On the Generalization of Neural Combinatorial Optimization Heuristics

Neural Combinatorial Optimization approaches have recently leveraged the...
research
11/21/2019

Neural Large Neighborhood Search for the Capacitated Vehicle Routing Problem

Learning how to automatically solve optimization problems has the potent...
research
06/09/2021

Efficient Active Search for Combinatorial Optimization Problems

Recently numerous machine learning based methods for combinatorial optim...
research
01/05/2020

Learning fine-grained search space pruning and heuristics for combinatorial optimization

Combinatorial optimization problems arise in a wide range of application...
research
05/14/2023

CREMP: Conformer-Rotamer Ensembles of Macrocyclic Peptides for Machine Learning

Computational and machine learning approaches to model the conformationa...
research
06/12/2020

Learning TSP Requires Rethinking Generalization

End-to-end training of neural network solvers for combinatorial problems...
research
06/01/2022

Neural Improvement Heuristics for Preference Ranking

In recent years, Deep Learning based methods have been a revolution in t...

Please sign up or login with your details

Forgot password? Click here to reset