SlovakBERT: Slovak Masked Language Model

09/30/2021
by   Matúš Pikuliak, et al.
0

We introduce a new Slovak masked language model called SlovakBERT in this paper. It is the first Slovak-only transformers-based model trained on a sizeable corpus. We evaluate the model on several NLP tasks and achieve state-of-the-art results. We publish the masked language model, as well as the subsequently fine-tuned models for part-of-speech tagging, sentiment analysis and semantic textual similarity.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/13/2020

Transferring Monolingual Model to Low-Resource Language: The Case of Tigrinya

In recent years, transformer models have achieved great success in natur...
research
02/18/2022

From FreEM to D'AlemBERT: a Large Corpus and a Language Model for Early Modern French

Language models for historical states of language are becoming increasin...
research
09/20/2023

BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model

We introduce the Bittensor Language Model, called "BTLM-3B-8K", a new st...
research
06/28/2023

Taqyim: Evaluating Arabic NLP Tasks Using ChatGPT Models

Large language models (LLMs) have demonstrated impressive performance on...
research
06/07/2019

Analyzing the Structure of Attention in a Transformer Language Model

The Transformer is a fully attention-based alternative to recurrent netw...
research
11/26/2019

Single Headed Attention RNN: Stop Thinking With Your Head

The leading approaches in language modeling are all obsessed with TV sho...
research
07/14/2023

MorphPiece : Moving away from Statistical Language Representation

Tokenization is a critical part of modern NLP pipelines. However, contem...

Please sign up or login with your details

Forgot password? Click here to reset