DeepAI AI Chat
Log In Sign Up

A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing

by   Alexander M. Rush, et al.

Dual decomposition, and more generally Lagrangian relaxation, is a classical method for combinatorial optimization; it has recently been applied to several inference problems in natural language processing (NLP). This tutorial gives an overview of the technique. We describe example algorithms, describe formal guarantees for the method, and describe practical issues in implementing the algorithms. While our examples are predominantly drawn from the NLP literature, the material should be of general relevance to inference problems in machine learning. A central theme of this tutorial is that Lagrangian relaxation is naturally applied in conjunction with a broad class of combinatorial algorithms, allowing inference in models that go significantly beyond previous work on Lagrangian relaxation for inference in graphical models.


page 1

page 2

page 3

page 4


A novel dual-decomposition method based on p-Lagrangian relaxation

In this paper, we propose the novel p-branch-and-bound method for solvin...

Submodular relaxation for inference in Markov random fields

In this paper we address the problem of finding the most probable state ...

The Transformer Network for the Traveling Salesman Problem

The Traveling Salesman Problem (TSP) is the most popular and most studie...

On the Lagrangian Biduality of Sparsity Minimization Problems

Recent results in Compressive Sensing have shown that, under certain con...

Generalized sequential tree-reweighted message passing

This paper addresses the problem of approximate MAP-MRF inference in gen...

Exact MAP-Inference by Confining Combinatorial Search with LP Relaxation

We consider the MAP-inference problem for graphical models, which is a v...