Automatic Argument Quality Assessment -- New Datasets and Methods

09/03/2019
by   Assaf Toledo, et al.
0

We explore the task of automatic assessment of argument quality. To that end, we actively collected 6.3k arguments, more than a factor of five compared to previously examined data. Each argument was explicitly and carefully annotated for its quality. In addition, 14k pairs of arguments were annotated independently, identifying the higher quality argument in each pair. In spite of the inherent subjective nature of the task, both annotation schemes led to surprisingly consistent results. We release the labeled datasets to the community. Furthermore, we suggest neural methods based on a recently released language model, for argument ranking as well as for argument-pair classification. In the former task, our results are comparable to state-of-the-art; in the latter task our results significantly outperform earlier methods.

READ FULL TEXT
research
11/26/2019

A Large-scale Dataset for Argument Quality Ranking: Construction and Analysis

Identifying the quality of free-text arguments has become an important t...
research
05/13/2021

Thematic fit bits: Annotation quality and quantity for event participant representation

Modeling thematic fit (a verb–argument compositional semantics task) cur...
research
07/21/2019

Are You Convinced? Choosing the More Convincing Evidence with a Siamese Network

With the advancement in argument detection, we suggest to pay more atten...
research
11/03/2020

Creating a Domain-diverse Corpus for Theory-based Argument Quality Assessment

Computational models of argument quality (AQ) have focused primarily on ...
research
10/23/2020

Intrinsic Quality Assessment of Arguments

Several quality dimensions of natural language arguments have been inves...
research
11/05/2019

Discrete Argument Representation Learning for Interactive Argument Pair Identification

In this paper, we focus on extracting interactive argument pairs from tw...
research
02/19/2021

An Empirical Study on Measuring the Similarity of Sentential Arguments with Language Model Domain Adaptation

Measuring the similarity between two different sentential arguments is a...

Please sign up or login with your details

Forgot password? Click here to reset