Grounded Textual Entailment

06/14/2018
by   Hoa Trong Vu, et al.
0

Capturing semantic relations between sentences, such as entailment, is a long-standing challenge for computational semantics. Logic-based models analyse entailment in terms of possible worlds (interpretations, or situations) where a premise P entails a hypothesis H iff in all worlds where P is true, H is also true. Statistical models view this relationship probabilistically, addressing it in terms of whether a human would likely infer H from P. In this paper, we wish to bridge these two perspectives, by arguing for a visually-grounded version of the Textual Entailment task. Specifically, we ask whether models can perform better if, in addition to P and H, there is also an image (corresponding to the relevant "world" or "situation"). We use a multimodal version of the SNLI dataset (Bowman et al., 2015) and we compare "blind" and visually-augmented models of textual entailment. We show that visual information is beneficial, but we also conduct an in-depth error analysis that reveals that current multimodal models are not performing "grounding" in an optimal fashion.

READ FULL TEXT
research
06/10/2019

Multimodal Logical Inference System for Visual-Textual Entailment

A large amount of research about multimodal inference across text and vi...
research
09/21/2019

Visuallly Grounded Generation of Entailments from Premises

Natural Language Inference (NLI) is the task of determining the semantic...
research
01/04/2017

Textual Entailment with Structured Attentions and Composition

Deep learning techniques are increasingly popular in the textual entailm...
research
11/20/2019

Paraphrasing Verbs for Noun Compound Interpretation

An important challenge for the automatic analysis of English written tex...
research
06/10/2018

What Knowledge is Needed to Solve the RTE5 Textual Entailment Challenge?

This document gives a knowledge-oriented analysis of about 20 interestin...
research
08/13/2018

REGMAPR - A Recipe for Textual Matching

We propose a simple model for textual matching problems. Starting from a...
research
07/23/2019

Dr.Quad at MEDIQA 2019: Towards Textual Inference and Question Entailment using contextualized representations

This paper presents the submissions by Team Dr.Quad to the ACL-BioNLP 20...

Please sign up or login with your details

Forgot password? Click here to reset