Transformer-Based Models for Automatic Identification of Argument Relations: A Cross-Domain Evaluation

11/26/2020
by   Ramon Ruiz-Dolz, et al.
0

Argument Mining is defined as the task of automatically identifying and extracting argumentative components (e.g., premises, claims, etc.) and detecting the existing relations among them (i.e., support, attack, rephrase, no relation). One of the main issues when approaching this problem is the lack of data, and the size of the publicly available corpora. In this work, we use the recently annotated US2016 debate corpus. US2016 is the largest existing argument annotated corpus, which allows exploring the benefits of the most recent advances in Natural Language Processing in a complex domain like Argument (relation) Mining. We present an exhaustive analysis of the behavior of transformer-based models (i.e., BERT, XLNET, RoBERTa, DistilBERT and ALBERT) when predicting argument relations. Finally, we evaluate the models in five different domains, with the objective of finding the less domain dependent model. We obtain a macro F1-score of 0.70 with the US2016 evaluation corpus, and a macro F1-score of 0.61 with the Moral Maze cross-domain corpus.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/18/2022

RuArg-2022: Argument Mining Evaluation

Argumentation analysis is a field of computational linguistics that stud...
research
09/17/2018

Categorization of Comparative Sentences for Argument Mining

We present the first work on domain-independent comparative argument min...
research
06/17/2020

An Exploratory Study of Argumentative Writing by Young Students: A Transformer-based Approach

We present a computational exploration of argument critique writing by y...
research
04/24/2017

What is the Essence of a Claim? Cross-Domain Claim Identification

Argument mining has become a popular research area in NLP. It typically ...
research
10/24/2022

Toward an Intelligent Tutoring System for Argument Mining in Legal Texts

We propose an adaptive environment (CABINET) to support caselaw analysis...
research
11/07/2020

NLP-CIC @ PRELEARN: Mastering prerequisites relations, from handcrafted features to embeddings

We present our systems and findings for the prerequisite relation learni...
research
02/18/2020

Annotating and Extracting Synthesis Process of All-Solid-State Batteries from Scientific Literature

The synthesis process is essential for achieving computational experimen...

Please sign up or login with your details

Forgot password? Click here to reset