DeepAI AI Chat
Log In Sign Up

Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval

by   Hongyin Tang, et al.

Recently, the retrieval models based on dense representations have been gradually applied in the first stage of the document retrieval tasks, showing better performance than traditional sparse vector space models. To obtain high efficiency, the basic structure of these models is Bi-encoder in most cases. However, this simple structure may cause serious information loss during the encoding of documents since the queries are agnostic. To address this problem, we design a method to mimic the queries on each of the documents by an iterative clustering process and represent the documents by multiple pseudo queries (i.e., the cluster centroids). To boost the retrieval process using approximate nearest neighbor search library, we also optimize the matching function with a two-step score calculation procedure. Experimental results on several popular ranking and QA datasets show that our model can achieve state-of-the-art results.


page 1

page 2

page 3

page 4


Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback

Dense retrieval systems conduct first-stage retrieval using embedded rep...

On the Value of Behavioral Representations for Dense Retrieval

We consider text retrieval within dense representational space in real-w...

Sparse, Dense, and Attentional Representations for Text Retrieval

Dual encoder architectures perform retrieval by encoding documents and q...

Are We There Yet? A Decision Framework for Replacing Term Based Retrieval with Dense Retrieval Systems

Recently, several dense retrieval (DR) models have demonstrated competit...

Multi-View Document Representation Learning for Open-Domain Dense Retrieval

Dense retrieval has achieved impressive advances in first-stage retrieva...

SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval

In neural Information Retrieval (IR), ongoing research is directed towar...

Contextualized Generative Retrieval

The text retrieval task is mainly performed in two ways: the bi-encoder ...