Multi-label classification for biomedical literature: an overview of the BioCreative VII LitCovid Track for COVID-19 literature topic annotations

by   Qingyu Chen, et al.

The COVID-19 pandemic has been severely impacting global society since December 2019. Massive research has been undertaken to understand the characteristics of the virus and design vaccines and drugs. The related findings have been reported in biomedical literature at a rate of about 10,000 articles on COVID-19 per month. Such rapid growth significantly challenges manual curation and interpretation. For instance, LitCovid is a literature database of COVID-19-related articles in PubMed, which has accumulated more than 200,000 articles with millions of accesses each month by users worldwide. One primary curation task is to assign up to eight topics (e.g., Diagnosis and Treatment) to the articles in LitCovid. Despite the continuing advances in biomedical text mining methods, few have been dedicated to topic annotations in COVID-19 literature. To close the gap, we organized the BioCreative LitCovid track to call for a community effort to tackle automated topic annotation for COVID-19 literature. The BioCreative LitCovid dataset, consisting of over 30,000 articles with manually reviewed topics, was created for training and testing. It is one of the largest multilabel classification datasets in biomedical scientific literature. 19 teams worldwide participated and made 80 submissions in total. Most teams used hybrid systems based on transformers. The highest performing submissions achieved 0.8875, 0.9181, and 0.9394 for macro F1-score, micro F1-score, and instance-based F1-score, respectively. The level of participation and results demonstrate a successful track and help close the gap between dataset curation and method development. The dataset is publicly available via for benchmarking and further development.


page 1

page 2

page 3

page 4


LitMC-BERT: transformer-based multi-label classification of biomedical literature with an application on COVID-19 literature curation

The rapid growth of biomedical literature poses a significant challenge ...

Multi-label topic classification for COVID-19 literature with Bioformer

We describe Bioformer team's participation in the multi-label topic clas...

LitCovid in 2022: an information resource for the COVID-19 literature

LitCovid (, first lau...

A System for Worldwide COVID-19 Information Aggregation

The global pandemic of COVID-19 has made the public pay close attention ...

Neural language models for text classification in evidence-based medicine

The COVID-19 has brought about a significant challenge to the whole of h...

BioBERT Based SNP-traits Associations Extraction from Biomedical Literature

Scientific literature contains a considerable amount of information that...

Towards a corpus for credibility assessment in software practitioner blog articles

Blogs are a source of grey literature which are widely adopted by softwa...

Please sign up or login with your details

Forgot password? Click here to reset