An Attention-Based Model for Predicting Contextual Informativeness and Curriculum Learning Applications

04/21/2022
by   Sungjin Nam, et al.
0

Both humans and machines learn the meaning of unknown words through contextual information in a sentence, but not all contexts are equally helpful for learning. We introduce an effective method for capturing the level of contextual informativeness with respect to a given target word. Our study makes three main contributions. First, we develop models for estimating contextual informativeness, focusing on the instructional aspect of sentences. Our attention-based approach using pre-trained embeddings demonstrates state-of-the-art performance on our single-context dataset and an existing multi-sentence context dataset. Second, we show how our model identifies key contextual elements in a sentence that are likely to contribute most to a reader's understanding of the target word. Third, we examine how our contextual informativeness model, originally developed for vocabulary learning applications for students, can be used for developing better training curricula for word embedding models in batch learning and few-shot machine learning settings. We believe our results open new possibilities for applications that support language learning for both human and machine learners

READ FULL TEXT
research
04/13/2022

HIT at SemEval-2022 Task 2: Pre-trained Language Model for Idioms Detection

The same multi-word expressions may have different meanings in different...
research
07/11/2021

LexSubCon: Integrating Knowledge from Lexical Resources into Contextual Embeddings for Lexical Substitution

Lexical substitution is the task of generating meaningful substitutes fo...
research
11/23/2020

Sarcasm detection from user-generated noisy short text

Sentiment analysis of social media comments is very important for review...
research
09/01/2018

Contextual Encoding for Translation Quality Estimation

The task of word-level quality estimation (QE) consists of taking a sour...
research
06/27/2022

Few-Shot Stance Detection via Target-Aware Prompt Distillation

Stance detection aims to identify whether the author of a text is in fav...
research
07/23/2022

Context based lemmatizer for Polish language

Lemmatization is the process of grouping together the inflected forms of...

Please sign up or login with your details

Forgot password? Click here to reset