Comparative Analysis of Neural QA models on SQuAD

06/18/2018
by   Soumya Wadhwa, et al.
0

The task of Question Answering has gained prominence in the past few decades for testing the ability of machines to understand natural language. Large datasets for Machine Reading have led to the development of neural models that cater to deeper language understanding compared to information retrieval tasks. Different components in these neural architectures are intended to tackle different challenges. As a first step towards achieving generalization across multiple domains, we attempt to understand and compare the peculiarities of existing end-to-end neural models on the Stanford Question Answering Dataset (SQuAD) by performing quantitative as well as qualitative analysis of the results attained by each of them. We observed that prediction errors reflect certain model-specific biases, which we further discuss in this paper.

READ FULL TEXT
research
08/20/2017

Learning to Paraphrase for Question Answering

Question answering (QA) systems are sensitive to the many different ways...
research
05/05/2020

MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models

Retrieval question answering (ReQA) is the task of retrieving a sentence...
research
09/17/2020

On the Transferability of Minimal Prediction Preserving Inputs in Question Answering

Recent work (Feng et al., 2018) establishes the presence of short, unint...
research
04/11/2020

Exploring The Spatial Reasoning Ability of Neural Models in Human IQ Tests

Although neural models have performed impressively well on various tasks...
research
11/08/2022

Toward a Neural Semantic Parsing System for EHR Question Answering

Clinical semantic parsing (SP) is an important step toward identifying t...
research
11/20/2020

What do we expect from Multiple-choice QA Systems?

The recent success of machine learning systems on various QA datasets co...
research
06/23/2019

Investigating Biases in Textual Entailment Datasets

The ability to understand logical relationships between sentences is an ...

Please sign up or login with your details

Forgot password? Click here to reset