An Empirical Study of Content Understanding in Conversational Question Answering

09/24/2019
by   Ting-Rui Chiang, et al.
0

With a lot of work about context-free question answering systems, there is an emerging trend of conversational question answering models in the natural language processing field. Thanks to the recently collected datasets, including QuAC and CoQA, there has been more work on conversational question answering, and some models have achieved competitive performance on both datasets. However, to best of our knowledge, two important questions for conversational comprehension research have not been well studied: 1) How well can the benchmark dataset reflect models' content understanding? 2) Do the models utilize the conversation content well? To investigate the two questions, we design different training settings, testing settings, as well as an attack to verify the models' capability of content understanding on QuAC and CoQA. The results indicate some potential hazards of using QuAC and CoQA for conversational comprehension research. Our analysis also sheds some light one both models and datasets. Given deeper investigation of the task, it is believed that this work is beneficial to the future progress of conversation comprehension.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/30/2020

Question Rewriting for Conversational Question Answering

Conversational question answering (QA) requires answers conditioned on t...
research
04/23/2021

BERT-CoQAC: BERT-based Conversational Question Answering in Context

As one promising way to inquire about any particular information through...
research
01/22/2022

Question rewriting? Assessing its importance for conversational question answering

In conversational question answering, systems must correctly interpret t...
research
06/01/2020

Conversational Machine Comprehension: a Literature Review

Conversational Machine Comprehension (CMC) is a research track in conver...
research
12/10/2018

SDNet: Contextualized Attention-based Deep Network for Conversational Question Answering

Conversational question answering (CQA) is a novel QA task that requires...
research
04/13/2022

Can Question Rewriting Help Conversational Question Answering?

Question rewriting (QR) is a subtask of conversational question answerin...
research
10/06/2018

FlowQA: Grasping Flow in History for Conversational Machine Comprehension

Conversational machine comprehension requires a deep understanding of th...

Please sign up or login with your details

Forgot password? Click here to reset