research
∙
05/25/2023
Scaling Data-Constrained Language Models
The current trend of scaling language models involves increasing both pa...
share
research
∙
05/09/2023
StarCoder: may the source be with you!
The BigCode community, an open-scientific collaboration working on the r...
research
∙
02/06/2023
Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning
Recent works successfully leveraged Large Language Models' (LLM) abiliti...
research
∙
12/09/2022
BigScience: A Case Study in the Social Construction of a Multilingual Large Language Model
The BigScience Workshop was a value-driven initiative that spanned one a...
research
∙
11/20/2022
The Stack: 3 TB of permissively licensed source code
Large Language Models (LLMs) play an ever-increasing role in the field o...
research
∙
07/07/2022
Training Transformers Together
The infrastructure necessary for training state-of-the-art models is bec...
research
∙
10/15/2021
Multitask Prompted Training Enables Zero-Shot Task Generalization
Large language models have recently been shown to attain reasonable zero...
research
∙
09/07/2021
Datasets: A Community Library for Natural Language Processing
The scale, variety, and quantity of publicly-available NLP datasets has ...
research
∙
06/21/2021
VIMPAC: Video Pre-Training via Masked Token Prediction and Contrastive Learning
Video understanding relies on perceiving the global content and modeling...
research
∙
06/18/2021
Distributed Deep Learning in Open Collaborations
Modern deep learning applications require increasingly more compute to t...
research
∙
12/02/2020
Learning from others' mistakes: Avoiding dataset biases without modeling them
State-of-the-art natural language processing (NLP) models often learn to...
research
∙
05/15/2020
Movement Pruning: Adaptive Sparsity by Fine-Tuning
Magnitude pruning is a widely used strategy for reducing model size in p...
research
∙
03/26/2020
TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation
Natural Language Generation (NLG) models are prone to generating repetit...
research
∙
10/09/2019
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Recent advances in modern Natural Language Processing (NLP) research hav...
research
∙
10/09/2019
Transformers: State-of-the-art Natural Language Processing
Recent advances in modern Natural Language Processing (NLP) research hav...
research
∙
10/02/2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
As Transfer Learning from large-scale pre-trained models becomes more pr...
research
∙
01/23/2019
TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents
We introduce a new approach to generative data-driven dialogue systems (...
research
∙
11/14/2018
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks
Much efforts has been devoted to evaluate whether multi-task learning ca...
research
∙
05/15/2018
Continuous Learning in a Hierarchical Multiscale Neural Network
We reformulate the problem of encoding a multi-scale representation of a...
research
∙
03/28/2018
Meta-Learning a Dynamical Language Model
We consider the task of word-level language modeling and study the possi...
research
∙
03/15/2018
Studying Invariances of Trained Convolutional Neural Networks
Convolutional Neural Networks (CNNs) define an exceptionally powerful cl...
research
∙
08/02/2016
Rational Solutions of Underdetermined Polynomial Equations
In this paper we report on an application of computer algebra in which m...
research
∙
08/21/2011
A Dynamical Systems Approach for Static Evaluation in Go
In the paper arguments are given why the concept of static evaluation ha...
research
∙
02/02/2003