-
Conversational Answer Generation and Factuality for Reading Comprehension Question-Answering
Question answering (QA) is an important use case on voice assistants. A ...
read it
-
Ask to Learn: A Study on Curiosity-driven Question Generation
We propose a novel text generation task, namely Curiosity-driven Questio...
read it
-
CoQA: A Conversational Question Answering Challenge
Humans gather information by engaging in conversations involving a serie...
read it
-
Context-Aware Answer Extraction in Question Answering
Extractive QA models have shown very promising performance in predicting...
read it
-
Right Answer for the Wrong Reason: Discovery and Mitigation
Exposing the weaknesses of neural models is crucial for improving their ...
read it
-
Towards Confident Machine Reading Comprehension
There has been considerable progress on academic benchmarks for the Read...
read it
-
Modeling Context in Answer Sentence Selection Systems on a Latency Budget
Answer Sentence Selection (AS2) is an efficient approach for the design ...
read it
A Wrong Answer or a Wrong Question? An Intricate Relationship between Question Reformulation and Answer Selection in Conversational Question Answering
The dependency between an adequate question formulation and correct answer selection is a very intriguing but still underexplored area. In this paper, we show that question rewriting (QR) of the conversational context allows to shed more light on this phenomenon and also use it to evaluate robustness of different answer selection approaches. We introduce a simple framework that enables an automated analysis of the conversational question answering (QA) performance using question rewrites, and present the results of this analysis on the TREC CAsT and QuAC (CANARD) datasets. Our experiments uncover sensitivity to question formulation of the popular state-of-the-art models for reading comprehension and passage ranking. Our results demonstrate that the reading comprehension model is insensitive to question formulation, while the passage ranking changes dramatically with a little variation in the input question. The benefit of QR is that it allows us to pinpoint and group such cases automatically. We show how to use this methodology to verify whether QA models are really learning the task or just finding shortcuts in the dataset, and better understand the frequent types of error they make.
READ FULL TEXT
Comments
There are no comments yet.