Log In Sign Up

LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction

by   Vlad Niculae, et al.

Structured prediction requires manipulating a large number of combinatorial structures, e.g., dependency trees or alignments, either as latent or output variables. Recently, the SparseMAP method has been proposed as a differentiable, sparse alternative to maximum a posteriori (MAP) and marginal inference. SparseMAP returns a combination of a small number of structures, a desirable property in some downstream applications. However, SparseMAP requires a tractable MAP inference oracle. This excludes, e.g., loopy graphical models or factor graphs with logic constraints, which generally require approximate inference. In this paper, we introduce LP-SparseMAP, an extension of SparseMAP that addresses this limitation via a local polytope relaxation. LP-SparseMAP uses the flexible and powerful domain specific language of factor graphs for defining and backpropagating through arbitrary hidden structure, supporting coarse decompositions, hard logic constraints, and higher-order correlations. We derive the forward and backward algorithms needed for using LP-SparseMAP as a hidden or output layer. Experiments in three structured prediction tasks show benefits compared to SparseMAP and Structured SVM.


page 1

page 2

page 3

page 4


SparseMAP: Differentiable Sparse Structured Inference

Structured prediction requires searching over a combinatorial number of ...

Train and Test Tightness of LP Relaxations in Structured Prediction

Structured prediction is used in areas such as computer vision and natur...

Constraint Reduction using Marginal Polytope Diagrams for MAP LP Relaxations

LP relaxation-based message passing algorithms provide an effective tool...

MAP Inference via L2-Sphere Linear Program Reformulation

Maximum a posteriori (MAP) inference is an important task for graphical ...

Exact MAP-Inference by Confining Combinatorial Search with LP Relaxation

We consider the MAP-inference problem for graphical models, which is a v...

Alternating Directions Dual Decomposition

We propose AD3, a new algorithm for approximate maximum a posteriori (MA...

Local Perturb-and-MAP for Structured Prediction

Conditional random fields (CRFs) provide a powerful tool for structured ...