CoSimLex: A Resource for Evaluating Graded Word Similarity in Context

12/11/2019
by   Carlos Santos Armendariz, et al.
0

State of the art natural language processing tools are built on context-dependent word embeddings, but no direct method for evaluating these representations currently exists. Standard tasks and datasets for intrinsic evaluation of embeddings are based on judgements of similarity, but ignore context; standard tasks for word sense disambiguation take account of context but do not provide continuous measures of meaning similarity. This paper describes an effort to build a new dataset, CoSimLex, intended to fill this gap. Building on the standard pairwise similarity task of SimLex-999, it provides context-dependent similarity measures; covers not only discrete differences in word sense but more subtle, graded changes in meaning; and covers not only a well-resourced language (English) but a number of less-resourced languages. We define the task and evaluation metrics, outline the dataset collection methodology, and describe the status of the dataset so far.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/20/2016

Learning Word Embeddings from Intrinsic and Extrinsic Views

While word embeddings are currently predominant for natural language pro...
research
03/17/2017

Construction of a Japanese Word Similarity Dataset

An evaluation of distributed word representation is generally conducted ...
research
10/13/2020

BRUMS at SemEval-2020 Task 3: Contextualised Embeddings forPredicting the (Graded) Effect of Context in Word Similarity

This paper presents the team BRUMS submission to SemEval-2020 Task 3: Gr...
research
10/25/2016

EmojiNet: Building a Machine Readable Sense Inventory for Emoji

Emoji are a contemporary and extremely popular way to enhance electronic...
research
11/29/2022

Measuring the Measuring Tools: An Automatic Evaluation of Semantic Metrics for Text Corpora

The ability to compare the semantic similarity between text corpora is i...
research
09/17/2018

Unsupervised Sense-Aware Hypernymy Extraction

In this paper, we show how unsupervised sense representations can be use...
research
10/06/2019

Measuring Sentences Similarity: A Survey

This study is to review the approaches used for measuring sentences simi...

Please sign up or login with your details

Forgot password? Click here to reset