UniCausal: Unified Benchmark and Model for Causal Text Mining

08/19/2022
by   Fiona Anting Tan, et al.
0

Current causal text mining datasets vary in objectives, data coverage, and annotation schemes. These inconsistent efforts prevented modeling capabilities and fair comparisons of model performance. Few datasets include cause-effect span annotations, which are needed for end-to-end causal extraction. Therefore, we proposed UniCausal, a unified benchmark for causal text mining across three tasks: Causal Sequence Classification, Cause-Effect Span Detection and Causal Pair Classification. We consolidated and aligned annotations of six high quality human-annotated corpus, resulting in a total of 58,720, 12,144 and 69,165 examples for each task respectively. Since the definition of causality can be subjective, our framework was designed to allow researchers to work on some or all datasets and tasks. As an initial benchmark, we adapted BERT pre-trained models to our task and generated baseline scores. We achieved 70.10 Span Detection, and 84.68

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/07/2023

A Cross-Domain Evaluation of Approaches for Causal Knowledge Extraction

Causal knowledge extraction is the task of extracting relevant causes an...
research
10/31/2022

1Cademy @ Causal News Corpus 2022: Enhance Causal Span Detection via Beam-Search-based Position Selector

In this paper, we present our approach and empirical observations for Ca...
research
09/08/2022

IDIAPers @ Causal News Corpus 2022: Extracting Cause-Effect-Signal Triplets via Pre-trained Autoregressive Language Model

In this paper, we describe our shared task submissions for Subtask 2 in ...
research
07/21/2021

CausalBERT: Injecting Causal Knowledge Into Pre-trained Models with Minimal Supervision

Recent work has shown success in incorporating pre-trained models like B...
research
01/07/2020

Causal Mosaic: Cause-Effect Inference via Nonlinear ICA and Ensemble Method

We address the problem of distinguishing cause from effect in bivariate ...
research
06/06/2018

Open Domain Suggestion Mining: Problem Definition and Datasets

We propose a formal definition for the task of suggestion mining in the ...
research
06/17/2020

Modeling subjective assessments of guilt in newspaper crime narratives

Crime reporting is a prevalent form of journalism with the power to shap...

Please sign up or login with your details

Forgot password? Click here to reset