Slovene SuperGLUE Benchmark: Translation and Evaluation

02/10/2022
by   Aleš Žagar, et al.
0

We present a Slovene combined machine-human translated SuperGLUE benchmark. We describe the translation process and problems arising due to differences in morphology and grammar. We evaluate the translated datasets in several modes: monolingual, cross-lingual, and multilingual, taking into account differences between machine and human translated training sets. The results show that the monolingual Slovene SloBERTa model is superior to massively multilingual and trilingual BERT models, but these also show a good cross-lingual performance on certain tasks. The performance of Slovene models still lags behind the best English models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/22/2023

L3Cube-IndicSBERT: A simple approach for learning cross-lingual sentence representations using multilingual BERT

The multilingual Sentence-BERT (SBERT) models map different languages to...
research
10/25/2019

On the Cross-lingual Transferability of Monolingual Representations

State-of-the-art unsupervised multilingual models (e.g., multilingual BE...
research
05/23/2023

Revisiting Machine Translation for Cross-lingual Classification

Machine Translation (MT) has been widely used for cross-lingual classifi...
research
09/16/2023

Contextual Label Projection for Cross-Lingual Structure Extraction

Translating training data into target languages has proven beneficial fo...
research
06/23/2019

Cross-lingual Data Transformation and Combination for Text Classification

Text classification is a fundamental task for text data mining. In order...
research
08/30/2019

PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification

Most existing work on adversarial data generation focuses on English. Fo...
research
06/20/2023

On Evaluating Multilingual Compositional Generalization with Translated Datasets

Compositional generalization allows efficient learning and human-like in...

Please sign up or login with your details

Forgot password? Click here to reset