Can Language Models be Biomedical Knowledge Bases?

09/15/2021
by   Mujeen Sung, et al.
0

Pre-trained language models (LMs) have become ubiquitous in solving various natural language processing (NLP) tasks. There has been increasing interest in what knowledge these LMs contain and how we can extract that knowledge, treating LMs as knowledge bases (KBs). While there has been much work on probing LMs in the general domain, there has been little attention to whether these powerful LMs can be used as domain-specific KBs. To this end, we create the BioLAMA benchmark, which is comprised of 49K biomedical factual knowledge triples for probing biomedical LMs. We find that biomedical LMs with recently proposed probing methods can achieve up to 18.51 biomedical knowledge. Although this seems promising given the task difficulty, our detailed analyses reveal that most predictions are highly correlated with prompt templates without any subjects, hence producing similar results on each relation and hindering their capabilities to be used as domain-specific KBs. We hope that BioLAMA can serve as a challenging benchmark for biomedical factual probing.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/19/2021

ELECTRAMed: a new pre-trained language representation model for biomedical NLP

The overwhelming amount of biomedical scientific texts calls for the dev...
research
08/26/2022

Extracting Biomedical Factual Knowledge Using Pretrained Language Model and Electronic Health Record Context

Language Models (LMs) have performed well on biomedical natural language...
research
09/25/2021

Coreference Resolution for the Biomedical Domain: A Survey

Issues with coreference resolution are one of the most frequently mentio...
research
05/28/2023

Large Language Models, scientific knowledge and factuality: A systematic analysis in antibiotic discovery

Inferring over and extracting information from Large Language Models (LL...
research
03/30/2022

Position-based Prompting for Health Outcome Generation

Probing Pre-trained Language Models (PLMs) using prompts has indirectly ...
research
07/03/2023

Exploring the In-context Learning Ability of Large Language Model for Biomedical Concept Linking

The biomedical field relies heavily on concept linking in various areas ...
research
09/07/2022

On the Effectiveness of Compact Biomedical Transformers

Language models pre-trained on biomedical corpora, such as BioBERT, have...

Please sign up or login with your details

Forgot password? Click here to reset