SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems

10/22/2022
by   Aaron Ferber, et al.
7

Optimization problems with expensive nonlinear cost functions and combinatorial constraints appear in many real-world applications, but remain challenging to solve efficiently. Existing combinatorial solvers like Mixed Integer Linear Programming can be fast in practice but cannot readily optimize nonlinear cost functions, while general nonlinear optimizers like gradient descent often do not handle complex combinatorial structures, may require many queries of the cost function, and are prone to local optima. To bridge this gap, we propose SurCo that learns linear Surrogate costs which can be used by existing Combinatorial solvers to output good solutions to the original nonlinear combinatorial optimization problem, combining the flexibility of gradient-based methods with the structure of linear combinatorial optimization. We learn these linear surrogates end-to-end with the nonlinear loss by differentiating through the linear surrogate solver. Three variants of SurCo are proposed: SurCo-zero operates on individual nonlinear problems, SurCo-prior trains a linear surrogate predictor on distributions of problems, and SurCo-hybrid uses a model trained offline to warm start online solving for SurCo-zero. We analyze our method theoretically and empirically, showing smooth convergence and improved performance. Experiments show that compared to state-of-the-art approaches and expert-designed heuristics, SurCo obtains lower cost solutions with comparable or faster solve time for two realworld industry-level applications: embedding table sharding and inverse photonic design.

READ FULL TEXT
research
03/06/2022

A Survey for Solving Mixed Integer Programming via Machine Learning

This paper surveys the trend of leveraging machine learning to solve mix...
research
10/18/2019

Combinatorial Losses through Generalized Gradients of Integer Linear Programs

When samples have internal structure, we often see a mismatch between th...
research
11/10/2020

Discrete solution pools and noise-contrastive estimation for predict-and-optimize

Numerous real-life decision-making processes involve solving a combinato...
research
06/10/2022

Tensor Train for Global Optimization Problems in Robotics

The convergence of many numerical optimization techniques is highly sens...
research
07/18/2023

Landscape Surrogate: Learning Decision Losses for Mathematical Optimization Under Partial Information

Recent works in learning-integrated optimization have shown promise in s...
research
06/06/2021

Combinatorial Optimization for Panoptic Segmentation: An End-to-End Trainable Approach

We propose an end-to-end trainable architecture for simultaneous semanti...
research
03/09/2022

Surrogate "Level-Based" Lagrangian Relaxation for Mixed-Integer Linear Programming

Mixed-Integer Linear Programming problems, playing a prominent role in O...

Please sign up or login with your details

Forgot password? Click here to reset