LLaMA: Open and Efficient Foundation Language Models

02/27/2023

∙

We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.

READ FULL TEXT

LLaMA: Open and Efficient Foundation Language Models

Code Llama: Open Foundation Models for Code

Under the Skin of Foundation NFT Auctions

Can Foundation Models Talk Causality?

Examination and Extension of Strategies for Improving Personalized Language Modeling via Interpolation

SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality

Baselines for Identifying Watermarked Large Language Models

Toward Reproducing Network Research Results Using Large Language Models

LLaMA: Open and Efficient Foundation Language Models

Related Research

Code Llama: Open Foundation Models for Code

Under the Skin of Foundation NFT Auctions

Can Foundation Models Talk Causality?

Examination and Extension of Strategies for Improving Personalized Language Modeling via Interpolation

SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality

Baselines for Identifying Watermarked Large Language Models

Toward Reproducing Network Research Results Using Large Language Models