Rethinking the objectives of extractive question answering

by   Martin Fajcik, et al.

This paper describes two generally applicable approaches towards the significant improvement of the performance of state-of-the-art extractive question answering (EQA) systems. Firstly, contrary to a common belief, it demonstrates that using the objective with independence assumption for span probability P(a_s,a_e) = P(a_s)P(a_e) of span starting at position a_s and ending at position a_e may have adverse effects. Therefore we propose a new compound objective that models joint probability P(a_s,a_e) directly, while still keeping the objective with independency assumption as an auxiliary objective. Our second approach shows the beneficial effect of distantly semi-supervised shared-normalization objective known from (Clark and Gardner, 2017). We show that normalizing over a set of documents similar to the golden passage, and marginalizing over all ground-truth answer string positions leads to the improvement of results from smaller statistical models. Our results are supported via experiments with three QA models (BidAF, BERT, ALBERT) over six datasets. The proposed approaches do not use any additional data. Our code, analysis, pretrained models, and individual results will be available online.


page 1

page 2

page 3

page 4


HAS-QA: Hierarchical Answer Spans Model for Open-domain Question Answering

This paper is concerned with open-domain question answering (i.e., OpenQ...

Look at the First Sentence: Position Bias in Question Answering

Many extractive question answering models are trained to predict start a...

Few-Shot Question Answering by Pretraining Span Selection

In a number of question answering (QA) benchmarks, pretrained models hav...

Weakly-Supervised Open-Retrieval Conversational Question Answering

Recent studies on Question Answering (QA) and Conversational QA (ConvQA)...

Simple and Effective Multi-Paragraph Reading Comprehension

We consider the problem of adapting neural paragraph-level question answ...