Modeling Exemplification in Long-form Question Answering via Retrieval

05/19/2022
by   Shufan Wang, et al.
0

Exemplification is a process by which writers explain or clarify a concept by providing an example. While common in all forms of writing, exemplification is particularly useful in the task of long-form question answering (LFQA), where a complicated answer can be made more understandable through simple examples. In this paper, we provide the first computational study of exemplification in QA, performing a fine-grained annotation of different types of examples (e.g., hypotheticals, anecdotes) in three corpora. We show that not only do state-of-the-art LFQA models struggle to generate relevant examples, but also that standard evaluation metrics such as ROUGE are insufficient to judge exemplification quality. We propose to treat exemplification as a retrieval problem in which a partially-written answer is used to query a large set of human-written examples extracted from a corpus. Our approach allows a reliable ranking-type automatic metrics that correlates well with human evaluation. A human evaluation shows that our model's retrieved examples are more relevant than examples generated from a state-of-the-art LFQA model.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/12/2020

Fine-Grained Relevance Annotations for Multi-Task Document Ranking and Question Answering

There are many existing retrieval and question answering datasets. Howev...
research
09/07/2018

Trick Me If You Can: Adversarial Writing of Trivia Challenge Questions

Modern natural language processing systems have been touted as approachi...
research
01/06/2018

Analysis of Wikipedia-based Corpora for Question Answering

This paper gives comprehensive analyses of corpora based on Wikipedia fo...
research
03/19/2022

Clickbait Spoiling via Question Answering and Passage Retrieval

We introduce and study the task of clickbait spoiling: generating a shor...
research
05/29/2023

A Critical Evaluation of Evaluations for Long-form Question Answering

Long-form question answering (LFQA) enables answering a wide range of qu...
research
03/10/2021

Hurdles to Progress in Long-form Question Answering

The task of long-form question answering (LFQA) involves retrieving docu...
research
04/16/2020

A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation

In this paper, we introduce a novel methodology to efficiently construct...

Please sign up or login with your details

Forgot password? Click here to reset