Sklar's Omega: A Gaussian Copula-Based Framework for Assessing Agreement

03/07/2018
by   John Hughes, et al.
0

The statistical measurement of agreement is important in a number of fields, e.g., content analysis, education, computational linguistics, biomedical imaging. We propose Sklar's Omega, a Gaussian copula-based framework for measuring intra-coder, inter-coder, and inter-method agreement as well as agreement relative to a gold standard. We demonstrate the efficacy and advantages of our approach by applying it to both simulated and experimentally observed datasets, including data from two medical imaging studies. Application of our proposed methodology is supported by our open-source R package, sklarsomega, which is available for download from the Comprehensive R Archive Network.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/10/2018

sklarsomega: An R Package for Measuring Agreement Using Sklar's Omega Coefficient

R package sklarsomega provides tools for measuring agreement using Sklar...
research
03/22/2021

krippendorffsalpha: An R Package for Measuring Agreement Using Krippendorff's Alpha Coefficient

R package krippendorffsalpha provides tools for measuring agreement usin...
research
09/25/2018

Assessing Method Agreement for Paired Repeated Binary Measurements

Method comparison studies are essential for development in medical and c...
research
08/12/2022

Sparse Probability of Agreement

Measuring inter-annotator agreement is important for annotation tasks, b...
research
01/25/2023

Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement

We commonly use agreement measures to assess the utility of judgements m...
research
09/17/2022

DiPietro-Hazari Kappa: A Novel Metric for Assessing Labeling Quality via Annotation

Data is a key component of modern machine learning, but statistics for a...
research
05/28/2022

Building net-native agreement systems

Agreements and contracts are everywhere, but they are built on layers an...

Please sign up or login with your details

Forgot password? Click here to reset