Combinatorial Optimization for Panoptic Segmentation: An End-to-End Trainable Approach

06/06/2021
by   Ahmed Abbas, et al.
0

We propose an end-to-end trainable architecture for simultaneous semantic and instance segmentation (a.k.a. panoptic segmentation) consisting of a convolutional neural network and an asymmetric multiway cut problem solver. The latter solves a combinatorial optimization problem that elegantly incorporates semantic and boundary predictions to produce a panoptic labeling. Our formulation allows to directly maximize a smooth surrogate of the panoptic quality metric by backpropagating the gradient through the optimization problem. Experimental evaluation shows improvement of end-to-end learning w.r.t. comparable approaches on Cityscapes and COCO datasets. Overall, our approach shows the utility of using combinatorial optimization in tandem with deep learning in a challenging large scale real-world problem and showcases benefits and insights into training such an architecture end-to-end.

READ FULL TEXT

page 4

page 9

page 19

research
04/24/2018

Learning to See the Invisible: End-to-End Trainable Amodal Instance Segmentation

Semantic amodal segmentation is a recently proposed extension to instanc...
research
03/25/2020

Deep Graph Matching via Blackbox Differentiation of Combinatorial Solvers

Building on recent progress at the intersection of combinatorial optimiz...
research
05/18/2023

Neural Bee Colony Optimization: A Case Study in Public Transit Network Design

In this work we explore the combination of metaheuristics and learned ne...
research
07/20/2020

DeepCO: Offline Combinatorial Optimization Framework Utilizing Deep Learning

Combinatorial optimization serves as an essential part in many modern in...
research
10/22/2022

SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problems

Optimization problems with expensive nonlinear cost functions and combin...
research
05/26/2022

A Hybrid Neural Autoencoder for Sensory Neuroprostheses and Its Applications in Bionic Vision

Sensory neuroprostheses are emerging as a promising technology to restor...
research
07/12/2019

Learning to Handle Parameter Perturbations in Combinatorial Optimization: an Application to Facility Location

We present an approach to couple the resolution of Combinatorial Optimiz...

Please sign up or login with your details

Forgot password? Click here to reset