research
∙
03/21/2022
Word Order Does Matter (And Shuffled Language Models Know It)
Recent studies have shown that language models pretrained and/or fine-tu...
share
research
∙
09/11/2021
The Impact of Positional Encodings on Multilingual Compression
In order to preserve word-order information in a non-autoregressive sett...
research
∙
01/26/2021
Attention Can Reflect Syntactic Structure (If You Let It)
Since the popularization of the Transformer as a general-purpose feature...
research
∙
05/04/2020
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations
Large-scale pretrained language models are the major driving force behin...
research
∙
05/01/2020
From Zero to Hero: On the Limitations of Zero-Shot Cross-Lingual Transfer with Multilingual Transformers
Massively multilingual transformers pretrained with language modeling ob...
research
∙
04/29/2020
Do Neural Language Models Show Preferences for Syntactic Formalisms?
Recent work on the interpretability of deep neural language models has c...
research
∙
02/19/2020
Hierarchical models vs. transfer learning for document-level sentiment classification
Documents are composed of smaller pieces - paragraphs, sentences, and to...
research
∙
06/12/2019
Probing Multilingual Sentence Representations With X-Probe
This paper extends the task of probing sentence representations for ling...
research
∙
08/29/2018