DeepAI AI Chat
Log In Sign Up

SuperSim: a test set for word similarity and relatedness in Swedish

by   Simon Hengchen, et al.

Language models are notoriously difficult to evaluate. We release SuperSim, a large-scale similarity and relatedness test set for Swedish built with expert human judgments. The test set is composed of 1,360 word-pairs independently judged for both relatedness and similarity by five annotators. We evaluate three different models (Word2Vec, fastText, and GloVe) trained on two separate Swedish datasets, namely the Swedish Gigaword corpus and a Swedish Wikipedia dump, to provide a baseline for future comparison. We release the fully annotated test set, code, baseline models, and data.


page 1

page 2

page 3

page 4


Learning to Learn to be Right for the Right Reasons

Improving model generalization on held-out data is one of the core objec...

MSC: A Dataset for Macro-Management in StarCraft II

Macro-management is an important problem in StarCraft, which has been st...

Detecting a single fault in a deterministic finite automaton

Given a deterministic finite automaton and its implementation with at mo...

Composer's Assistant: Interactive Transformers for Multi-Track MIDI Infilling

We consider the task of multi-track MIDI infilling when arbitrary (track...

IStego100K: Large-scale Image Steganalysis Dataset

In order to promote the rapid development of image steganalysis technolo...

Using Synthetic Images To Uncover Population Biases In Facial Landmarks Detection

In order to analyze a trained model performance and identify its weak sp...