Key Phrase Classification in Complex Assignments

03/16/2020
by   Manikandan Ravikiran, et al.
0

Complex assignments typically consist of open-ended questions with large and diverse content in the context of both classroom and online graduate programs. With the sheer scale of these programs comes a variety of problems in peer and expert feedback, including rogue reviews. As such with the hope of identifying important contents needed for the review, in this work we present a very first work on key phrase classification with a detailed empirical study on traditional and most recent language modeling approaches. From this study, we find that the task of classification of key phrases is ambiguous at a human level producing Cohen's kappa of 0.77 on a new data set. Both pretrained language models and simple TFIDF SVM classifiers produce similar results with a former producing average of 0.6 F1 higher than the latter. We finally derive practical advice from our extensive empirical and model interpretability results for those interested in key phrase classification from educational reports in the future.

READ FULL TEXT
research
04/03/2020

Finding Black Cat in a Coal Cellar – Keyphrase Extraction Keyphrase-Rubric Relationship Classification from Complex Assignments

Diversity in content and open-ended questions are inherent in complex as...
research
04/03/2020

Keyphrase Rubric Relationship Classification in Complex Assignments

Complex assignments are open-ended question with varying content irrespe...
research
04/03/2020

Unsupervised Keyphrase Rubric Relationship Classification in Complex Assignments

Complex assignments are open-ended question with varying content irrespe...
research
06/04/2019

Improving Neural Language Models by Segmenting, Attending, and Predicting the Future

Common language models typically predict the next word given the context...
research
04/15/2021

Linking open-source code commits and MOOC grades to evaluate massive online open peer review

Massive Open Online Courses (MOOCs) have been used by students as a low-...
research
01/25/2021

Unsupervised Key-phrase Extraction and Clustering for Classification Scheme in Scientific Publications

Several methods have been explored for automating parts of Systematic Ma...

Please sign up or login with your details

Forgot password? Click here to reset