CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets

by   Isabelle Mohr, et al.

Over the course of the COVID-19 pandemic, large volumes of biomedical information concerning this new disease have been published on social media. Some of this information can pose a real danger to people's health, particularly when false information is shared, for instance recommendations on how to treat diseases without professional medical advice. Therefore, automatic fact-checking resources and systems developed specifically for the medical domain are crucial. While existing fact-checking resources cover COVID-19-related information in news or quantify the amount of misinformation in tweets, there is no dataset providing fact-checked COVID-19-related Twitter posts with detailed annotations for biomedical entities, relations and relevant evidence. We contribute CoVERT, a fact-checked corpus of tweets with a focus on the domain of biomedicine and COVID-19-related (mis)information. The corpus consists of 300 tweets, each annotated with medical named entities and relations. We employ a novel crowdsourcing methodology to annotate all tweets with fact-checking labels and supporting evidence, which crowdworkers search for online. This methodology results in moderate inter-annotator agreement. Furthermore, we use the retrieved evidence extracts as part of a fact-checking pipeline, finding that the real-world evidence is more useful than the knowledge indirectly available in pretrained language models.


page 1

page 2

page 3

page 4


Claim Detection in Biomedical Twitter Posts

Social media contains unfiltered and unique information, which is potent...

An Entity-based Claim Extraction Pipeline for Real-world Biomedical Fact-checking

Existing fact-checking models for biomedical claims are typically traine...

An Empirical Methodology for Detecting and Prioritizing Needs during Crisis Events

In times of crisis, identifying the essential needs is a crucial step to...

The Role of the Crowd in Countering Misinformation: A Case Study of the COVID-19 Infodemic

Fact checking by professionals is viewed as a vital defense in the fight...

A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking

Automated fact-checking based on machine learning is a promising approac...

Crowdsourced Fact-Checking at Twitter: How Does the Crowd Compare With Experts?

Fact-checking is one of the effective solutions in fighting online misin...

Did They Really Tweet That? Querying Fact-Checking Sites and Politwoops to Determine Tweet Misattribution

Screenshots of social media posts have become common place on social med...

Please sign up or login with your details

Forgot password? Click here to reset