Mixture of Experts for Biomedical Question Answering

04/15/2022
by   Damai Dai, et al.
0

Biomedical Question Answering (BQA) has attracted increasing attention in recent years due to its promising application prospect. It is a challenging task because the biomedical questions are professional and usually vary widely. Existing question answering methods answer all questions with a homogeneous model, leading to various types of questions competing for the shared parameters, which will confuse the model decision for each single type of questions. In this paper, in order to alleviate the parameter competition problem, we propose a Mixture-of-Expert (MoE) based question answering method called MoEBQA that decouples the computation for different types of questions by sparse routing. To be specific, we split a pretrained Transformer model into bottom and top blocks. The bottom blocks are shared by all the examples, aiming to capture the general features. The top blocks are extended to an MoE version that consists of a series of independent experts, where each example is assigned to a few experts according to its underlying question type. MoEBQA automatically learns the routing strategy in an end-to-end manner so that each expert tends to deal with the question types it is expert in. We evaluate MoEBQA on three BQA datasets constructed based on real examinations. The results show that our MoE extension significantly boosts the performance of question answering models and achieves new state-of-the-art performance. In addition, we elaborately analyze our MoE modules to reveal how MoEBQA works and find that it can automatically group the questions into human-readable clusters.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/13/2023

Question Decomposition Tree for Answering Complex Questions over Knowledge Bases

Knowledge base question answering (KBQA) has attracted a lot of interest...
research
09/18/2019

Pre-trained Language Model for Biomedical Question Answering

The recent success of question answering systems is largely attributed t...
research
11/22/2019

An End-to-End Framework for Cold Question Routing in Community Question Answering Services

Routing newly posted questions (a.k.a cold questions) to potential answe...
research
11/04/2021

Medicines Question Answering System, MeQA

In this paper we present the first system in Spanish capable of answerin...
research
05/24/2023

Mixture of Prompt Experts for Generalizable and Interpretable Question Answering

One of the ultimate quests of question answering (QA) is to deploy a sys...
research
07/14/2020

XAlgo: Explaining the Internal States of Algorithms via Question Answering

Algorithms often appear as 'black boxes' to non-expert users. While prio...
research
06/28/2021

Overview of BioASQ 2020: The eighth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

In this paper, we present an overview of the eighth edition of the BioAS...

Please sign up or login with your details

Forgot password? Click here to reset