AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreference

02/01/2023
by   Yuewei Yuan, et al.
0

Given a sentence "Abby told Brittney that she upset Courtney", one would struggle to understand who "she" refers to, and ask for clarification. However, if the word "upset" were replaced with "hugged", "she" unambiguously refers to Abby. We study if modern coreference resolution models are sensitive to such pronominal ambiguity. To this end, we construct AmbiCoref, a diagnostic corpus of minimal sentence pairs with ambiguous and unambiguous referents. Our examples generalize psycholinguistic studies of human perception of ambiguity around particular arrangements of verbs and their arguments. Analysis shows that (1) humans are less sure of referents in ambiguous AmbiCoref examples than unambiguous ones, and (2) most coreference models show little difference in output between ambiguous and unambiguous pairs. We release AmbiCoref as a diagnostic corpus for testing whether models treat ambiguity similarly to humans.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/12/2019

A Crowdsourced Frame Disambiguation Corpus with Ambiguity

We present a resource for the task of FrameNet semantic frame disambigua...
research
09/19/2000

Modeling Ambiguity in a Multi-Agent System

This paper investigates the formal pragmatics of ambiguous expressions b...
research
07/25/2023

Towards Resolving Word Ambiguity with Word Embeddings

Ambiguity is ubiquitous in natural language. Resolving ambiguous meaning...
research
07/27/2021

Ambiguity in Utopian XR-Games. Basic Principles for the Design of Virtual Worlds

Utopian images in XR-games are often ambiguous. How can ambiguity be con...
research
06/21/2022

TAPHSIR: Towards AnaPHoric Ambiguity Detection and ReSolution In Requirements

We introduce TAPHSIR, a tool for anaphoric ambiguity detection and anaph...
research
11/29/2022

Testing GLOM's ability to infer wholes from ambiguous parts

The GLOM architecture proposed by Hinton [2021] is a recurrent neural ne...
research
03/30/2022

Lay-it-out: Interactive Design of Layout-Sensitive Grammars

Layout-sensitive grammars have been adopted in many modern programming l...

Please sign up or login with your details

Forgot password? Click here to reset