research
∙
05/26/2023
Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages
Multilingual language models have recently gained attention as a promisi...
share
research
∙
10/13/2022
You Can Have Your Data and Balance It Too: Towards Balanced and Efficient Multilingual Models
Multilingual models have been widely used for cross-lingual transfer to ...
research
∙
06/21/2022
Don't Forget About Pronouns: Removing Gender Bias in Language Models Without Losing Factual Gender Information
The representations in large language models contain multiple types of g...
research
∙
05/09/2022
A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank
We show that the choice of pretraining languages affects downstream cros...
research
∙
09/10/2021
Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes
State-of-the-art contextual embeddings are obtained from large language ...
research
∙
12/30/2020
Introducing Orthogonal Constraint in Structural Probes
With the recent success of pre-trained models in NLP, a significant focu...
research
∙
10/12/2020
Gender Coreference and Bias Evaluation at WMT 2020
Gender bias in machine translation can manifest when choosing gender inf...
research
∙
10/02/2020
Syntax Representation in Word Embeddings and Neural Networks – A Survey
Neural networks trained on natural language processing tasks capture syn...
research
∙
04/30/2020