research
∙
01/31/2023
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
We study the design decisions of publicly available instruction tuning m...
share
research
∙
11/03/2022
Inverse scaling can become U-shaped
Although scaling language models improves performance on a range of task...
research
∙
10/06/2022
Language Models are Multilingual Chain-of-Thought Reasoners
We evaluate the reasoning abilities of large language models in multilin...
research
∙
11/25/2021
PolyViT: Co-training Vision Transformers on Images, Videos and Audio
Can we train a single transformer model capable of processing multiple m...
research
∙
06/23/2021
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
State-of-the-art models in natural language processing rely on separate ...
research
∙
11/08/2020
Long Range Arena: A Benchmark for Efficient Transformers
Transformers do not scale very well to long sequence lengths largely bec...
research
∙
09/14/2020
Efficient Transformers: A Survey
Transformer model architectures have garnered immense interest lately du...
research
∙
05/02/2020
Synthesizer: Rethinking Self-Attention in Transformer Models
The dot product self-attention is known to be central and indispensable ...
research
∙
09/05/2018
