TREC CAsT 2019: The Conversational Assistance Track Overview

03/30/2020
by   Jeffrey Dalton, et al.
0

The Conversational Assistance Track (CAsT) is a new track for TREC 2019 to facilitate Conversational Information Seeking (CIS) research and to create a large-scale reusable test collection for conversational search systems. The document corpus is 38,426,252 passages from the TREC Complex Answer Retrieval (CAR) and Microsoft MAchine Reading COmprehension (MARCO) datasets. Eighty information seeking dialogues (30 train, 50 test) are an average of 9 to 10 questions long. Relevance assessments are provided for 30 training topics and 20 test topics. This year 21 groups submitted a total of 65 runs using varying methods for conversational query understanding and ranking. Methods include traditional retrieval based methods, feature based learning-to-rank, neural models, and knowledge enhanced methods. A common theme through the runs is the use of BERT-based neural reranking methods. Leading methods also employed document expansion, conversational query expansion, and generative language models for conversational query rewriting (GPT-2). The results show a gap between automatic systems and those using the manually resolved utterances, with a 35 system.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/17/2021

Leveraging Query Resolution and Reading Comprehension for Conversational Passage Retrieval

This paper describes the participation of UvA.ILPS group at the TREC CAs...
research
03/31/2022

IITD-DBAI: Multi-Stage Retrieval with Pseudo-Relevance Feedback and Query Reformulation

Resolving the contextual dependency is one of the most challenging tasks...
research
05/05/2020

Query Reformulation using Query History for Passage Retrieval in Conversational Search

Passage retrieval in a conversational context is essential for many down...
research
11/07/2019

CROWN: Conversational Passage Ranking by Reasoning over Word Networks

Information needs around a topic cannot be satisfied in a single turn; u...
research
01/25/2023

From Baseline to Top Performer: A Reproducibility Study of Approaches at the TREC 2021 Conversational Assistance Track

This paper reports on an effort of reproducing the organizers' baseline ...
research
01/31/2018

ILPS at TREC 2017 Common Core Track

The TREC 2017 Common Core Track aimed at gathering a diverse set of part...
research
08/17/2023

Towards Filling the Gap in Conversational Search: From Passage Retrieval to Conversational Response Generation

Research on conversational search has so far mostly focused on query rew...

Please sign up or login with your details

Forgot password? Click here to reset