On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference

04/25/2018
by   Adam Poliak, et al.
0

We propose a process for investigating the extent to which sentence representations arising from neural machine translation (NMT) systems encode distinct semantic phenomena. We use these representations as features to train a natural language inference (NLI) classifier based on datasets recast from existing semantic annotations. In applying this process to a representative NMT system, we find its encoder appears most suited to supporting inferences at the syntax-semantics interface, as compared to anaphora resolution requiring world-knowledge. We conclude with a discussion on the merits and potential deficiencies of the existing process, and how it may be improved and extended as a broader framework for evaluating semantic coverage.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/23/2018

Evaluating Layers of Representation in Neural Machine Translation on Part-of-Speech and Semantic Tagging Tasks

While neural machine translation (NMT) models provide improved translati...
research
05/28/2018

Inducing Grammars with and for Neural Machine Translation

Machine translation systems require semantic knowledge and grammatical u...
research
02/19/2019

Semantic Neural Machine Translation using AMR

It is intuitive that semantic representations can be useful for machine ...
research
06/21/2019

Neural Machine Translating from Natural Language to SPARQL

SPARQL is a highly powerful query language for an ever-growing number of...
research
04/23/2018

Towards a Unified Natural Language Inference Framework to Evaluate Sentence Representations

We present a large scale unified natural language inference (NLI) datase...
research
04/24/2019

Assessing the Tolerance of Neural Machine Translation Systems Against Speech Recognition Errors

Machine translation systems are conventionally trained on textual resour...
research
09/11/2019

DBPal: Weak Supervision for Learning a Natural Language Interface to Databases

This paper describes DBPal, a new system to translate natural language u...

Please sign up or login with your details

Forgot password? Click here to reset