DeepAI
Log In Sign Up

LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction

01/13/2020
by   Vlad Niculae, et al.
0

Structured prediction requires manipulating a large number of combinatorial structures, e.g., dependency trees or alignments, either as latent or output variables. Recently, the SparseMAP method has been proposed as a differentiable, sparse alternative to maximum a posteriori (MAP) and marginal inference. SparseMAP returns a combination of a small number of structures, a desirable property in some downstream applications. However, SparseMAP requires a tractable MAP inference oracle. This excludes, e.g., loopy graphical models or factor graphs with logic constraints, which generally require approximate inference. In this paper, we introduce LP-SparseMAP, an extension of SparseMAP that addresses this limitation via a local polytope relaxation. LP-SparseMAP uses the flexible and powerful domain specific language of factor graphs for defining and backpropagating through arbitrary hidden structure, supporting coarse decompositions, hard logic constraints, and higher-order correlations. We derive the forward and backward algorithms needed for using LP-SparseMAP as a hidden or output layer. Experiments in three structured prediction tasks show benefits compared to SparseMAP and Structured SVM.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/12/2018

SparseMAP: Differentiable Sparse Structured Inference

Structured prediction requires searching over a combinatorial number of ...
11/04/2015

Train and Test Tightness of LP Relaxations in Structured Prediction

Structured prediction is used in areas such as computer vision and natur...
12/17/2013

Constraint Reduction using Marginal Polytope Diagrams for MAP LP Relaxations

LP relaxation-based message passing algorithms provide an effective tool...
05/09/2019

MAP Inference via L2-Sphere Linear Program Reformulation

Maximum a posteriori (MAP) inference is an important task for graphical ...
04/14/2020

Exact MAP-Inference by Confining Combinatorial Search with LP Relaxation

We consider the MAP-inference problem for graphical models, which is a v...
12/28/2012

Alternating Directions Dual Decomposition

We propose AD3, a new algorithm for approximate maximum a posteriori (MA...
05/24/2016

Local Perturb-and-MAP for Structured Prediction

Conditional random fields (CRFs) provide a powerful tool for structured ...