Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering

10/11/2022
by   Hao Cheng, et al.
0

Given its effectiveness on knowledge-intensive natural language processing tasks, dense retrieval models have become increasingly popular. Specifically, the de-facto architecture for open-domain question answering uses two isomorphic encoders that are initialized from the same pretrained model but separately parameterized for questions and passages. This bi-encoder architecture is parameter-inefficient in that there is no parameter sharing between encoders. Further, recent studies show that such dense retrievers underperform BM25 in various settings. We thus propose a new architecture, Task-aware Specialization for dense Retrieval (TASER), which enables parameter sharing by interleaving shared and specialized blocks in a single encoder. Our experiments on five question answering datasets show that can achieve superior accuracy, surpassing BM25, while using about 60 parameters as bi-encoder dense retrievers. In out-of-domain evaluations, TASER is also empirically more robust than bi-encoder dense retrievers.

READ FULL TEXT
research
10/04/2021

Encoder Adaptation of Dense Passage Retrieval for Open-Domain Question Answering

One key feature of dense passage retrievers (DPR) is the use of separate...
research
03/21/2022

Evaluating Token-Level and Passage-Level Dense Retrieval Models for Math Information Retrieval

With the recent success of dense retrieval methods based on bi-encoders,...
research
02/23/2023

MFBE: Leveraging Multi-Field Information of FAQs for Efficient Dense Retrieval

In the domain of question-answering in NLP, the retrieval of Frequently ...
research
09/17/2021

Simple Entity-Centric Questions Challenge Dense Retrievers

Open-domain question answering has exploded in popularity recently due t...
research
04/20/2021

Efficient Retrieval Optimized Multi-task Learning

Recently, there have been significant advances in neural methods for tac...
research
05/04/2022

Analysing the Robustness of Dual Encoders for Dense Retrieval Against Misspellings

Dense retrieval is becoming one of the standard approaches for document ...
research
04/30/2020

Progressively Pretrained Dense Corpus Index for Open-Domain Question Answering

To extract answers from a large corpus, open-domain question answering (...

Please sign up or login with your details

Forgot password? Click here to reset