DeepAI
Log In Sign Up

ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)

09/23/2020
by   Mohammad Aliannejadi, et al.
0

This document presents a detailed description of the challenge on clarifying questions for dialogue systems (ClariQ). The challenge is organized as part of the Conversational AI challenge series (ConvAI3) at Search Oriented Conversational AI (SCAI) EMNLP workshop in 2020. The main aim of the conversational systems is to return an appropriate answer in response to the user requests. However, some user requests might be ambiguous. In IR settings such a situation is handled mainly thought the diversification of the search result page. It is however much more challenging in dialogue settings with limited bandwidth. Therefore, in this challenge, we provide a common evaluation framework to evaluate mixed-initiative conversations. Participants are asked to rank clarifying questions in an information-seeking conversations. The challenge is organized in two stages where in Stage 1 we evaluate the submissions in an offline setting and single-turn conversations. Top participants of Stage 1 get the chance to have their model tested by human annotators.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/31/2019

The Second Conversational Intelligence Challenge (ConvAI2)

We describe the setting and results of the ConvAI2 NeurIPS competition t...
11/07/2018

The RLLChatbot: a solution to the ConvAI Challenge

Current conversational systems can follow simple commands and answer bas...
09/13/2021

Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions

Enabling open-domain dialogue systems to ask clarifying questions when a...
01/25/2021

MICROS: Mixed-Initiative ConveRsatiOnal Systems Workshop

The 1st edition of the workshop on Mixed-Initiative ConveRsatiOnal Syste...
10/25/2022

Deploying a Retrieval based Response Model for Task Oriented Dialogues

Task-oriented dialogue systems in industry settings need to have high co...
04/06/2022

ConvSearch: A Open-Domain Conversational Search Behavior Dataset

Conversational Search has been paid much attention recently with the inc...
04/02/2022

Metaphorical User Simulators for Evaluating Task-oriented Dialogue Systems

Task-oriented dialogue systems (TDSs) are assessed mainly in an offline ...