
-
Generative Language Modeling for Automated Theorem Proving
We explore the application of transformer-based language models to autom...
read it
-
Language Models are Few-Shot Learners
Recent work has demonstrated substantial gains on many NLP tasks and ben...
read it
-
Jukebox: A Generative Model for Music
We introduce Jukebox, a model that generates music with singing in the r...
read it
-
Dota 2 with Large Scale Deep Reinforcement Learning
On April 13th, 2019, OpenAI Five became the first AI system to defeat th...
read it
-
Deep Double Descent: Where Bigger Models and More Data Hurt
We show that a variety of modern deep learning tasks exhibit a "double-d...
read it
-
Generating Long Sequences with Sparse Transformers
Transformers are powerful sequence models, but require time and memory t...
read it
-
FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
A promising class of generative models maps points from a simple distrib...
read it
-
GamePad: A Learning Environment for Theorem Proving
In this paper, we introduce a system called GamePad that can be used to ...
read it
-
Some Considerations on Learning to Explore via Meta-Reinforcement Learning
We consider the problem of exploration in meta reinforcement learning. T...
read it
-
Emergent Complexity via Multi-Agent Competition
Reinforcement learning algorithms can train agents that solve problems i...
read it
-
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
Ability to continuously learn and adapt from limited experience in nonst...
read it
-
An online sequence-to-sequence model for noisy speech recognition
Generative models have long been the dominant approach for speech recogn...
read it
-
Learning to Generate Reviews and Discovering Sentiment
We explore the properties of byte-level recurrent language models. When ...
read it
-
One-Shot Imitation Learning
Imitation learning has been commonly applied to solve different tasks in...
read it
-
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
We explore the use of Evolution Strategies (ES), a class of black box op...
read it
-
Third-Person Imitation Learning
Reinforcement learning (RL) makes it possible to train agents capable of...
read it
-
RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning
Deep reinforcement learning (deep RL) has been successful in learning so...
read it
-
Variational Lossy Autoencoder
Representation learning seeks to expose certain aspects of observed data...
read it
-
Extensions and Limitations of the Neural GPU
The Neural GPU is a recent model that can learn algorithms such as multi...
read it
-
Learning Online Alignments with Continuous Rewards Policy Gradient
Sequence-to-sequence models with soft attention had significant success ...
read it
-
Improving Variational Inference with Inverse Autoregressive Flow
The framework of normalizing flows provides a general strategy for flexi...
read it
-
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
This paper describes InfoGAN, an information-theoretic extension to the ...
read it
-
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
TensorFlow is an interface for expressing machine learning algorithms, a...
read it
-
Continuous Deep Q-Learning with Model-based Acceleration
Model-free reinforcement learning has been successfully applied to a ran...
read it
-
Neural GPUs Learn Algorithms
Learning an algorithm from examples is a fundamental problem that has be...
read it
-
Adding Gradient Noise Improves Learning for Very Deep Networks
Deep feedforward and recurrent networks have achieved impressive results...
read it
-
Towards Principled Unsupervised Learning
General unsupervised learning is a long-standing conceptual problem in m...
read it
-
Neural Random-Access Machines
In this paper, we propose and investigate a new neural network architect...
read it
-
Multi-task Sequence to Sequence Learning
Sequence to sequence learning has recently emerged as a new paradigm in ...
read it
-
MuProp: Unbiased Backpropagation for Stochastic Neural Networks
Deep neural networks are powerful parametric models that can be trained ...
read it
-
A Neural Transducer
Sequence-to-sequence models have achieved impressive results on various ...
read it
-
Neural Programmer: Inducing Latent Programs with Gradient Descent
Deep neural networks have achieved impressive supervised classification ...
read it
-
Reinforcement Learning Neural Turing Machines - Revised
The Neural Turing Machine (NTM) is more expressive than all previously c...
read it
-
Grammar as a Foreign Language
Syntactic constituency parsing is a fundamental problem in natural langu...
read it
-
Move Evaluation in Go Using Deep Convolutional Neural Networks
The game of Go is more challenging than other board games, due to the di...
read it
-
Addressing the Rare Word Problem in Neural Machine Translation
Neural Machine Translation (NMT) is a new approach to machine translatio...
read it
-
Learning to Execute
Recurrent Neural Networks (RNNs) with Long Short-Term Memory units (LSTM...
read it
-
Sequence to Sequence Learning with Neural Networks
Deep Neural Networks (DNNs) are powerful models that have achieved excel...
read it
-
Recurrent Neural Network Regularization
We present a simple regularization technique for Recurrent Neural Networ...
read it
-
Intriguing properties of neural networks
Deep neural networks are highly expressive models that have recently ach...
read it
-
Learning Factored Representations in a Deep Mixture of Experts
Mixtures of Experts combine the outputs of several "expert" networks, ea...
read it
-
Distributed Representations of Words and Phrases and their Compositionality
The recently introduced continuous Skip-gram model is an efficient metho...
read it
-
Exploiting Similarities among Languages for Machine Translation
Dictionaries and phrase tables are the basis of modern statistical machi...
read it
-
Improving neural networks by preventing co-adaptation of feature detectors
When a large feedforward neural network is trained on a small training s...
read it
-
Estimating the Hessian by Back-propagating Curvature
In this work we develop Curvature Propagation (CP), a general technique ...
read it