research
∙
05/31/2023
Let's Verify Step by Step
In recent years, large language models have greatly improved in their ab...
share
research
∙
12/06/2022
Robust Speech Recognition via Large-Scale Weak Supervision
We study the capabilities of speech processing systems trained simply to...
research
∙
02/03/2022
Formal Mathematics Statement Curriculum Learning
We explore the use of expert iteration in the context of language modeli...
research
∙
12/20/2021
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
Diffusion models have recently been shown to generate high-quality synth...
research
∙
10/11/2021
Unsupervised Neural Machine Translation with Generative Language Models Only
We show how to derive state-of-the-art unsupervised neural machine trans...
research
∙
07/07/2021
Evaluating Large Language Models Trained on Code
We introduce Codex, a GPT language model fine-tuned on publicly availabl...
research
∙
02/26/2021
Learning Transferable Visual Models From Natural Language Supervision
State-of-the-art computer vision systems are trained to predict a fixed ...
research
∙
02/24/2021
Zero-Shot Text-to-Image Generation
Text-to-image generation has traditionally focused on finding better mod...
research
∙
09/07/2020
Generative Language Modeling for Automated Theorem Proving
We explore the application of transformer-based language models to autom...
research
∙
05/28/2020
Language Models are Few-Shot Learners
Recent work has demonstrated substantial gains on many NLP tasks and ben...
research
∙
04/30/2020
Jukebox: A Generative Model for Music
We introduce Jukebox, a model that generates music with singing in the r...
research
∙
12/13/2019
Dota 2 with Large Scale Deep Reinforcement Learning
On April 13th, 2019, OpenAI Five became the first AI system to defeat th...
research
∙
12/04/2019
Deep Double Descent: Where Bigger Models and More Data Hurt
We show that a variety of modern deep learning tasks exhibit a "double-d...
research
∙
04/23/2019
Generating Long Sequences with Sparse Transformers
Transformers are powerful sequence models, but require time and memory t...
research
∙
10/02/2018
FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
A promising class of generative models maps points from a simple distrib...
research
∙
06/02/2018
GamePad: A Learning Environment for Theorem Proving
In this paper, we introduce a system called GamePad that can be used to ...
research
∙
03/03/2018
Some Considerations on Learning to Explore via Meta-Reinforcement Learning
We consider the problem of exploration in meta reinforcement learning. T...
research
∙
10/10/2017
Emergent Complexity via Multi-Agent Competition
Reinforcement learning algorithms can train agents that solve problems i...
research
∙
10/10/2017
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
Ability to continuously learn and adapt from limited experience in nonst...
research
∙
06/16/2017
An online sequence-to-sequence model for noisy speech recognition
Generative models have long been the dominant approach for speech recogn...
research
∙
04/05/2017
Learning to Generate Reviews and Discovering Sentiment
We explore the properties of byte-level recurrent language models. When ...
research
∙
03/21/2017
One-Shot Imitation Learning
Imitation learning has been commonly applied to solve different tasks in...
research
∙
03/10/2017
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
We explore the use of Evolution Strategies (ES), a class of black box op...
research
∙
03/06/2017
Third-Person Imitation Learning
Reinforcement learning (RL) makes it possible to train agents capable of...
research
∙
11/09/2016
RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning
Deep reinforcement learning (deep RL) has been successful in learning so...
research
∙
11/08/2016
Variational Lossy Autoencoder
Representation learning seeks to expose certain aspects of observed data...
research
∙
11/02/2016
Extensions and Limitations of the Neural GPU
The Neural GPU is a recent model that can learn algorithms such as multi...
research
∙
08/03/2016
Learning Online Alignments with Continuous Rewards Policy Gradient
Sequence-to-sequence models with soft attention had significant success ...
research
∙
06/15/2016
Improving Variational Inference with Inverse Autoregressive Flow
The framework of normalizing flows provides a general strategy for flexi...
research
∙
06/12/2016
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
This paper describes InfoGAN, an information-theoretic extension to the ...
research
∙
03/14/2016
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
TensorFlow is an interface for expressing machine learning algorithms, a...
research
∙
03/02/2016
Continuous Deep Q-Learning with Model-based Acceleration
Model-free reinforcement learning has been successfully applied to a ran...
research
∙
11/25/2015
Neural GPUs Learn Algorithms
Learning an algorithm from examples is a fundamental problem that has be...
research
∙
11/21/2015
Adding Gradient Noise Improves Learning for Very Deep Networks
Deep feedforward and recurrent networks have achieved impressive results...
research
∙
11/19/2015
Towards Principled Unsupervised Learning
General unsupervised learning is a long-standing conceptual problem in m...
research
∙
11/19/2015
Neural Random-Access Machines
In this paper, we propose and investigate a new neural network architect...
research
∙
11/19/2015
Multi-task Sequence to Sequence Learning
Sequence to sequence learning has recently emerged as a new paradigm in ...
research
∙
11/16/2015
MuProp: Unbiased Backpropagation for Stochastic Neural Networks
Deep neural networks are powerful parametric models that can be trained ...
research
∙
11/16/2015
A Neural Transducer
Sequence-to-sequence models have achieved impressive results on various ...
research
∙
11/16/2015
Neural Programmer: Inducing Latent Programs with Gradient Descent
Deep neural networks have achieved impressive supervised classification ...
research
∙
05/04/2015
Reinforcement Learning Neural Turing Machines - Revised
The Neural Turing Machine (NTM) is more expressive than all previously c...
research
∙
12/23/2014
Grammar as a Foreign Language
Syntactic constituency parsing is a fundamental problem in natural langu...
research
∙
12/20/2014
Move Evaluation in Go Using Deep Convolutional Neural Networks
The game of Go is more challenging than other board games, due to the di...
research
∙
10/30/2014
Addressing the Rare Word Problem in Neural Machine Translation
Neural Machine Translation (NMT) is a new approach to machine translatio...
research
∙
10/17/2014
Learning to Execute
Recurrent Neural Networks (RNNs) with Long Short-Term Memory units (LSTM...
research
∙
09/10/2014
Sequence to Sequence Learning with Neural Networks
Deep Neural Networks (DNNs) are powerful models that have achieved excel...
research
∙
09/08/2014
Recurrent Neural Network Regularization
We present a simple regularization technique for Recurrent Neural Networ...
research
∙
12/21/2013
Intriguing properties of neural networks
Deep neural networks are highly expressive models that have recently ach...
research
∙
12/16/2013
Learning Factored Representations in a Deep Mixture of Experts
Mixtures of Experts combine the outputs of several "expert" networks, ea...
research
∙
10/16/2013