research
∙
07/18/2023
Llama 2: Open Foundation and Fine-Tuned Chat Models
In this work, we develop and release Llama 2, a collection of pretrained...
share
research
∙
04/19/2023
A Theory on Adam Instability in Large-Scale Machine Learning
We present a theory for the previously unexplained divergent behavior no...
research
∙
12/22/2022
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
Recent work has shown that fine-tuning large pre-trained language models...
research
∙
05/02/2022
OPT: Open Pre-trained Transformer Language Models
Large language models, which are often trained for hundreds of thousands...
research
∙
12/20/2021
Efficient Large Scale Language Modeling with Mixtures of Experts
Mixture of Experts layers (MoEs) enable efficient scaling of language mo...
research
∙
12/20/2021
