DeepAI AI Chat
Log In Sign Up

ClarQ: A large-scale and diverse dataset for Clarification Question Generation

06/10/2020
by   Vaibhav Kumar, et al.
Carnegie Mellon University
12

Question answering and conversational systems are often baffled and need help clarifying certain ambiguities. However, limitations of existing datasets hinder the development of large-scale models capable of generating and utilising clarification questions. In order to overcome these limitations, we devise a novel bootstrapping framework (based on self-supervision) that assists in the creation of a diverse, large-scale dataset of clarification questions based on post-comment tuples extracted from stackexchange. The framework utilises a neural network based architecture for classifying clarification questions. It is a two-step method where the first aims to increase the precision of the classifier and second aims to increase its recall. We quantitatively demonstrate the utility of the newly created dataset by applying it to the downstream task of question-answering. The final dataset, ClarQ, consists of  2M examples distributed across 173 domains of stackexchange. We release this dataset in order to foster research into the field of clarification question generation with the larger goal of enhancing dialog and question answering systems.

READ FULL TEXT
09/23/2022

Conversational QA Dataset Generation with Answer Revision

Conversational question–answer generation is a task that automatically g...
03/14/2017

Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering

The last several years have seen intensive interest in exploring neural-...
08/09/2017

Learning to Disambiguate by Asking Discriminative Questions

The ability to ask questions is a powerful tool to gather information in...
11/16/2020

Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases

Existing studies on question answering on knowledge bases (KBQA) mainly ...
08/24/2022

FashionVQA: A Domain-Specific Visual Question Answering System

Humans apprehend the world through various sensory modalities, yet langu...
04/07/2020

Automated Utterance Generation

Conversational AI assistants are becoming popular and question-answering...
05/25/2021

Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting

This paper explores the task of Difficulty-Controllable Question Generat...

Code Repositories

ClarQ

Dataset of Clarification Questions


view repo