You Only Need One Model for Open-domain Question Answering

12/14/2021
by   Haejun Lee, et al.
0

Recent works for Open-domain Question Answering refer to an external knowledge base using a retriever model, optionally rerank the passages with a separate reranker model and generate an answer using an another reader model. Despite performing related tasks, the models have separate parameters and are weakly-coupled during training. In this work, we propose casting the retriever and the reranker as hard-attention mechanisms applied sequentially within the transformer architecture and feeding the resulting computed representations to the reader. In this singular model architecture the hidden representations are progressively refined from the retriever to the reranker to the reader, which is more efficient use of model capacity and also leads to better gradient flow when we train it in an end-to-end manner. We also propose a pre-training methodology to effectively train this architecture. We evaluate our model on Natural Questions and TriviaQA open datasets and for a fixed parameter budget, our model outperforms the previous state-of-the-art model by 1.0 and 0.7 exact match scores.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/02/2021

End-to-End Training of Neural Retrievers for Open-Domain Question Answering

Recent work on training neural retrievers for open-domain question answe...
research
11/28/2017

Hyper-dimensional computing for a visual question-answering system that is trainable end-to-end

In this work we propose a system for visual question answering. Our arch...
research
10/16/2020

RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering

In open-domain question answering, dense passage retrieval has become a ...
research
11/18/2022

FiE: Building a Global Probability Space by Leveraging Early Fusion in Encoder for Open-Domain Question Answering

Generative models have recently started to outperform extractive models ...
research
10/22/2022

Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge

We propose a novel open-domain question answering (ODQA) framework for a...
research
06/22/2021

Fine-tune the Entire RAG Architecture (including DPR retriever) for Question-Answering

In this paper, we illustrate how to fine-tune the entire Retrieval Augme...
research
09/23/2022

Variational Open-Domain Question Answering

We introduce the Variational Open-Domain (VOD) framework for end-to-end ...

Please sign up or login with your details

Forgot password? Click here to reset