Automatic coding of students' writing via Contrastive Representation Learning in the Wasserstein space

11/26/2020
by   Ruijie Jiang, et al.
0

Qualitative analysis of verbal data is of central importance in the learning sciences. It is labor-intensive and time-consuming, however, which limits the amount of data researchers can include in studies. This work is a step towards building a statistical machine learning (ML) method for achieving an automated support for qualitative analyses of students' writing, here specifically in score laboratory reports in introductory biology for sophistication of argumentation and reasoning. We start with a set of lab reports from an undergraduate biology course, scored by a four-level scheme that considers the complexity of argument structure, the scope of evidence, and the care and nuance of conclusions. Using this set of labeled data, we show that a popular natural language modeling processing pipeline, namely vector representation of words, a.k.a word embeddings, followed by Long Short Term Memory (LSTM) model for capturing language generation as a state-space model, is able to quantitatively capture the scoring, with a high Quadratic Weighted Kappa (QWK) prediction score, when trained in via a novel contrastive learning set-up. We show that the ML algorithm approached the inter-rater reliability of human analysis. Ultimately, we conclude, that machine learning (ML) for natural language processing (NLP) holds promise for assisting learning sciences researchers in conducting qualitative studies at much larger scales than is currently possible.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

11/01/2021

Interpretable contrastive word mover's embedding

This paper shows that a popular approach to the supervised embedding of ...
03/30/2020

QRMine: A python package for triangulation in Grounded Theory

Grounded theory (GT) is a qualitative research method for building theor...
03/22/2016

Comparing Human and Automated Evaluation of Open-Ended Student Responses to Questions of Evolution

Written responses can provide a wealth of data in understanding student ...
09/19/2021

Adversarial Training with Contrastive Learning in NLP

For years, adversarial training has been extensively studied in natural ...
09/13/2021

SCORE-IT: A Machine Learning-based Tool for Automatic Standardization of EEG Reports

Machine learning (ML)-based analysis of electroencephalograms (EEGs) is ...
10/21/2019

Two Case Studies of Experience Prototyping Machine Learning Systems in the Wild

Throughout the course of my Ph.D., I have been designing the user experi...
09/13/2021

ML Based Lineage in Databases

We track the lineage of tuples throughout their database lifetime. That ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.