research
∙
06/30/2023
Should you marginalize over possible tokenizations?
Autoregressive language models (LMs) map token sequences to probabilitie...
share
research
∙
03/08/2023
disco: a toolkit for Distributional Control of Generative Models
Pre-trained language models and other generative models have revolutioni...
research
∙
02/16/2023
Aligning Language Models with Preferences through f-divergence Minimization
Aligning language models with preferences can be posed as approximating ...
research
∙
06/01/2022
On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting
The availability of large pre-trained models is changing the landscape o...
research
∙
12/10/2021
Sampling from Discrete Energy-Based Models with Quality/Efficiency Trade-offs
Energy-Based Models (EBMs) allow for extremely flexible specifications o...
research
∙
12/01/2021
Controlling Conditional Language Models with Distributional Policy Gradients
Machine learning is shifting towards general-purpose pretrained generati...
research
∙
11/04/2021
Unsupervised and Distributional Detection of Machine-Generated Text
The power of natural language generation models has provoked a flurry of...
research
∙
06/09/2021
Energy-Based Models for Code Generation under Compilability Constraints
Neural language models can be successfully trained on source code, leadi...
research
∙
03/15/2021
Emergence of self-reproducing metabolisms as recursive algorithms in an Artificial Chemistry
Researching the conditions for the emergence of life – not necessarily a...
research
∙
04/07/2020
Class-Agnostic Continual Learning of Alternating Languages and Domains
Continual Learning has been often framed as the problem of training a mo...
research
∙
03/17/2020
Combinatory Chemistry: Towards a Simple Model of Emergent Evolution
Researching the conditions for the emergence of life – not necessarily a...
research
∙
03/18/2019
The emergence of number and syntax units in LSTM language models
Recent work has shown that LSTMs trained on a generic language modeling ...
research
∙
02/25/2019
Cooperative Learning of Disjoint Syntax and Semantics
There has been considerable attention devoted to models that learn to jo...
research
∙
09/17/2018
The Fast and the Flexible: training neural networks to learn to follow instructions from small data
Learning to follow human instructions is a challenging task because whil...
research
∙
05/20/2018
Learning compositionally through attentive guidance
In this paper, we introduce Attentive Guidance (AG), a new mechanism to ...
research
∙
05/03/2018
What you can cram into a single vector: Probing sentence embeddings for linguistic properties
Although much effort has recently been devoted to training high-quality ...
research
∙
02/18/2018
Memorize or generalize? Searching for a compositional RNN in a haystack
Neural networks are very powerful learning systems, but they do not read...
research
∙
01/31/2017
CommAI: Evaluating the first steps towards a useful general AI
With machine learning successfully applied to new daunting problems almo...
research
∙
06/20/2016