An Efficient Combinatorial Optimization Model Using Learning-to-Rank Distillation

12/24/2021
by   Honguk Woo, et al.
0

Recently, deep reinforcement learning (RL) has proven its feasibility in solving combinatorial optimization problems (COPs). The learning-to-rank techniques have been studied in the field of information retrieval. While several COPs can be formulated as the prioritization of input items, as is common in the information retrieval, it has not been fully explored how the learning-to-rank techniques can be incorporated into deep RL for COPs. In this paper, we present the learning-to-rank distillation-based COP framework, where a high-performance ranking policy obtained by RL for a COP can be distilled into a non-iterative, simple model, thereby achieving a low-latency COP solver. Specifically, we employ the approximated ranking distillation to render a score-based ranking model learnable via gradient descent. Furthermore, we use the efficient sequence sampling to improve the inference performance with a limited delay. With the framework, we demonstrate that a distilled model not only achieves comparable performance to its respective, high-performance RL, but also provides several times faster inferences. We evaluate the framework with several COPs such as priority-based task scheduling and multidimensional knapsack, demonstrating the benefits of the framework in terms of inference latency and performance.

READ FULL TEXT
research
06/14/2022

Deep Reinforcement Learning for Exact Combinatorial Optimization: Learning to Branch

Branch-and-bound is a systematic enumerative method for combinatorial op...
research
11/09/2020

Geometric Deep Reinforcement Learning for Dynamic DAG Scheduling

In practice, it is quite common to face combinatorial optimization probl...
research
08/19/2023

Accelerating Exact Combinatorial Optimization via RL-based Initialization – A Case Study in Scheduling

Scheduling on dataflow graphs (also known as computation graphs) is an N...
research
08/12/2023

Accelerating Diffusion-based Combinatorial Optimization Solvers by Progressive Distillation

Graph-based diffusion models have shown promising results in terms of ge...
research
08/01/2023

Reinforcement Learning-based Non-Autoregressive Solver for Traveling Salesman Problems

The Traveling Salesman Problem (TSP) is a well-known problem in combinat...
research
11/12/2019

Combinatorial Optimization by Graph Pointer Networks and Hierarchical Reinforcement Learning

In this work, we introduce Graph Pointer Networks (GPNs) trained using r...
research
10/07/2022

Population-Based Reinforcement Learning for Combinatorial Optimization

Applying reinforcement learning (RL) to combinatorial optimization probl...

Please sign up or login with your details

Forgot password? Click here to reset