Multi-Stage Document Ranking with BERT

10/31/2019
by   Rodrigo Nogueira, et al.
0

The advent of deep neural networks pre-trained via language modeling tasks has spurred a number of successful applications in natural language processing. This work explores one such popular model, BERT, in the context of document ranking. We propose two variants, called monoBERT and duoBERT, that formulate the ranking problem as pointwise and pairwise classification, respectively. These two models are arranged in a multi-stage ranking architecture to form an end-to-end search system. One major advantage of this design is the ability to trade off quality against latency by controlling the admission of candidates into each pipeline stage, and by doing so, we are able to find operating points that offer a good balance between these two competing metrics. On two large-scale datasets, MS MARCO and TREC CAR, experiments show that our model produces results that are either at or comparable to the state of the art. Ablation studies show the contributions of each component and characterize the latency/quality tradeoff space.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/17/2019

DocBERT: BERT for Document Classification

Pre-trained language representation models achieve remarkable state of t...
research
05/20/2021

Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking

An emerging recipe for achieving state-of-the-art effectiveness in neura...
research
04/16/2019

Understanding the Behaviors of BERT in Ranking

This paper studies the performances and behaviors of BERT in ranking tas...
research
08/06/2020

DeText: A Deep Text Ranking Framework with BERT

Ranking is the most important component in a search system. Mostsearch s...
research
08/16/2021

Deep Natural Language Processing for LinkedIn Search

Many search systems work with large amounts of natural language data, e....
research
03/18/2019

An Updated Duet Model for Passage Re-ranking

We propose several small modifications to Duet---a deep neural ranking m...
research
05/22/2020

L2R2: Leveraging Ranking for Abductive Reasoning

The abductive natural language inference task (αNLI) is proposed to eval...

Please sign up or login with your details

Forgot password? Click here to reset