research
∙
03/30/2023
oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes
In this paper, we introduce the range of oBERTa language models, an easy...
share
research
∙
05/25/2022
Sparse*BERT: Sparse Models are Robust
Large Language Models have become the core architecture upon which most ...
research
∙
03/14/2022
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models
Pre-trained Transformer-based language models have become a key building...
research
∙
11/26/2021