Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport

05/27/2020
by   Kyle Swanson, et al.
7

Selecting input features of top relevance has become a popular method for building self-explaining models. In this work, we extend this selective rationalization approach to text matching, where the goal is to jointly select and align text pieces, such as tokens or sentences, as a justification for the downstream prediction. Our approach employs optimal transport (OT) to find a minimal cost alignment between the inputs. However, directly applying OT often produces dense and therefore uninterpretable alignments. To overcome this limitation, we introduce novel constrained variants of the OT problem that result in highly sparse alignments with controllable sparsity. Our model is end-to-end differentiable using the Sinkhorn algorithm for OT and can be trained without any alignment annotations. We evaluate our model on the StackExchange, MultiNews, e-SNLI, and MultiRC datasets. Our model achieves very sparse rationale selections with high fidelity while preserving prediction accuracy compared to strong attention baseline models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/26/2020

Graph Optimal Transport for Cross-Domain Alignment

Cross-domain alignment between two sets of entities (e.g., objects in an...
research
05/30/2022

Neural Optimal Transport with General Cost Functionals

We present a novel neural-networks-based algorithm to compute optimal tr...
research
09/07/2023

Optimal Transport with Tempered Exponential Measures

In the field of optimal transport, two prominent subfields face each oth...
research
04/06/2022

The Self-Optimal-Transport Feature Transform

The Self-Optimal-Transport (SOT) feature transform is designed to upgrad...
research
07/14/2021

Scalable Optimal Transport in High Dimensions for Graph Distances, Embedding Alignment, and More

The current best practice for computing optimal transport (OT) is via en...
research
06/05/2023

Optimal transport for automatic alignment of untargeted metabolomic data

Untargeted metabolomic profiling through liquid chromatography-mass spec...
research
07/23/2019

Optimal Transport-based Alignment of Learned Character Representations for String Similarity

String similarity models are vital for record linkage, entity resolution...

Please sign up or login with your details

Forgot password? Click here to reset