DeepAI AI Chat
Log In Sign Up

A Simple Yet Strong Pipeline for HotpotQA

04/14/2020
by   Dirk Groeneveld, et al.
Indian Institute of Technology Delhi
Allen Institute for Artificial Intelligence
0

State-of-the-art models for multi-hop question answering typically augment large-scale language models like BERT with additional, intuitively useful capabilities such as named entity recognition, graph-based reasoning, and question decomposition. However, does their strong performance on popular multi-hop datasets really justify this added design complexity? Our results suggest that the answer may be no, because even our simple pipeline based on BERT, named Quark, performs surprisingly well. Specifically, on HotpotQA, Quark outperforms these models on both question answering and support identification (and achieves performance very close to a RoBERTa model). Our pipeline has three steps: 1) use BERT to identify potentially relevant sentences independently of each other; 2) feed the set of selected sentences as context into a standard BERT span prediction model to choose an answer; and 3) use the sentence selection model, now with the chosen answer, to produce supporting sentences. The strong performance of Quark resurfaces the importance of carefully exploring simple model designs before using popular benchmarks to justify the value of complex techniques.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/07/2019

Multi-hop Question Answering via Reasoning Chains

Multi-hop question answering requires models to gather information from ...
11/17/2019

Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering

We propose an unsupervised strategy for the selection of justification s...
03/01/2022

Semantic Sentence Composition Reasoning for Multi-Hop Question Answering

Due to the lack of insufficient data, existing multi-hop open domain que...
04/18/2021

Generative Context Pair Selection for Multi-hop Question Answering

Compositional reasoning tasks like multi-hop question answering, require...
11/09/2019

Hierarchical Graph Network for Multi-hop Question Answering

In this paper, we present Hierarchical Graph Network (HGN) for multi-hop...
04/20/2020

Fine-tuning Multi-hop Question Answering with Hierarchical Graph Network

In this paper, we present a two stage model for multi-hop question answe...
03/04/2020

A Study on Efficiency, Accuracy and Document Structure for Answer Sentence Selection

An essential task of most Question Answering (QA) systems is to re-rank ...