Hypothesis Only Baselines in Natural Language Inference

05/02/2018
by   Adam Poliak, et al.
0

We propose a hypothesis only baseline for diagnosing Natural Language Inference (NLI). Especially when an NLI dataset assumes inference is occurring based purely on the relationship between a context and a hypothesis, it follows that assessing entailment relations while ignoring the provided context is a degenerate solution. Yet, through experiments on ten distinct NLI datasets, we find that this approach, which we refer to as a hypothesis-only model, is able to significantly outperform a majority class baseline across a number of NLI datasets. Our analysis suggests that statistical irregularities may allow a model to perform NLI in some datasets beyond what should be achievable without access to the context.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/19/2021

Exploring Lexical Irregularities in Hypothesis-Only Models of Natural Language Inference

Natural Language Inference (NLI) or Recognizing Textual Entailment (RTE)...
research
07/09/2019

Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference

Natural Language Inference (NLI) datasets often contain hypothesis-only ...
research
12/10/2021

A Puzzle-Based Dataset for Natural Language Inference

We provide here a dataset for tasks related to natural language understa...
research
07/30/2022

Smoothing Entailment Graphs with Language Models

The diversity and Zipfian frequency distribution of natural language pre...
research
05/14/2019

Misleading Failures of Partial-input Baselines

Recent work establishes dataset difficulty and removes annotation artifa...
research
04/16/2020

There is Strength in Numbers: Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training

Natural Language Inference (NLI) datasets contain annotation artefacts r...
research
05/04/2023

SemEval-2023 Task 7: Multi-Evidence Natural Language Inference for Clinical Trial Data

This paper describes the results of SemEval 2023 task 7 – Multi-Evidence...

Please sign up or login with your details

Forgot password? Click here to reset