research
∙
06/01/2023
Learning Transformer Programs
Recent research in mechanistic interpretability has attempted to reverse...
share
research
∙
05/24/2023
Adapting Language Models to Compress Contexts
Transformer-based language models (LMs) are powerful and widely-applicab...
research
∙
10/20/2022
Finding Dataset Shortcuts with Grammar Induction
Many NLP datasets have been found to contain shortcuts: simple decision ...
research
∙
10/11/2022
A Kernel-Based View of Language Model Fine-Tuning
It has become standard to solve NLP tasks by fine-tuning pre-trained lan...
research
∙
02/16/2022
Should You Mask 15
Masked language models conventionally use a masking rate of 15 belief th...
research
∙
09/16/2021