NorBench – A Benchmark for Norwegian Language Models

05/06/2023

∙

We present NorBench: a streamlined suite of NLP tasks and probes for evaluating Norwegian language models (LMs) on standardized data splits and evaluation metrics. We also introduce a range of new Norwegian language models (both encoder and encoder-decoder based). Finally, we compare and analyze their performance, along with other existing LMs, across the different benchmark tests of NorBench.

READ FULL TEXT

NorBench – A Benchmark for Norwegian Language Models

Can Large Language Models design a Robot?

Large-Scale Contextualised Language Modelling for Norwegian

Task-agnostic Distillation of Encoder-Decoder Language Models

MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Types

Holistic Evaluation of Language Models

Negation, Coordination, and Quantifiers in Contextualized Language Models

Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views

NorBench – A Benchmark for Norwegian Language Models

Related Research

Can Large Language Models design a Robot?

Large-Scale Contextualised Language Modelling for Norwegian

Task-agnostic Distillation of Encoder-Decoder Language Models

MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Types

Holistic Evaluation of Language Models

Negation, Coordination, and Quantifiers in Contextualized Language Models

Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views