DeepAI
Log In Sign Up

Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrieval

03/07/2022
by   Dingkun Long, et al.
0

Passage retrieval is a fundamental task in information retrieval (IR) research, which has drawn much attention recently. In English field, the availability of large-scale annotated dataset (e.g, MS MARCO) and the emergence of deep pre-trained language models (e.g, BERT) have resulted in a substantial improvement of existing passage retrieval systems. However, in Chinese field, especially for specific domain, passage retrieval systems are still immature due to quality-annotated dataset being limited by scale. Therefore, in this paper, we present a novel multi-domain Chinese dataset for passage retrieval (Multi-CPR). The dataset is collected from three different domains, including E-commerce, Entertainment video and Medical. Each dataset contains millions of passages and a certain amount of human annotated query-passage related pairs. We implement various representative passage retrieval methods as baselines. We find that the performance of retrieval models trained on dataset from general domain will inevitably decrease on specific domain. Nevertheless, passage retrieval system built on in-domain annotated dataset can achieve significant improvement, which indeed demonstrates the necessity of domain labeled data for further optimization. We hope the release of the Multi-CPR dataset could benchmark Chinese passage retrieval task in specific domain and also make advances for future studies.

READ FULL TEXT

page 1

page 2

page 3

page 4

09/23/2022

Promptagator: Few-shot Dense Retrieval From 8 Examples

Much recent research on information retrieval has focused on how to tran...
08/26/2021

A Statutory Article Retrieval Dataset in French

Statutory article retrieval is the task of automatically retrieving law ...
10/24/2020

COUGH: A Challenge Dataset and Models for COVID-19 FAQ Retrieval

We present a large challenging dataset, COUGH, for COVID-19 FAQ retrieva...
03/19/2022

DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine

In this paper, we present DuReader_retrieval, a large-scale Chinese data...
10/12/2020

OCNLI: Original Chinese Natural Language Inference

Despite the tremendous recent progress on natural language inference (NL...
03/18/2022

RELIC: Retrieving Evidence for Literary Claims

Humanities scholars commonly provide evidence for claims that they make ...
10/30/2017

Open Set Logo Detection and Retrieval

Current logo retrieval research focuses on closed set scenarios. We argu...