Ilya Sutskever

is this you? claim profile

∙ 2 followers
Google ∙ OpenAI

Ilya Sutskever is an informatics scientist who works in the field of machine learning and currently serves as OpenAI’s Chief Scientist. He is the co-inventor of the renowned neural network AlexNet. He and Oriol Vinyals and Quoc Le invented Sequence to Sequence Learning. Sutskever is also AlphaGo and TensorFlow co-inventor.

Sutskever obtained his B.Sc, M.Sc, and Ph.D. in computer science from the Department of Computer Science at the University of Toronto, under Geoffrey Hinton’s supervision.

After graduating in 2012, Sutskever spent two months at Stanford University as a postdoc with Andrew Ng. He then returned to the University of Toronto and joined the new Hinton research group DNNResearch. Google acquired DNNResearch four months later, in March 2013 and employed Sutskever as a Google Brain research scientist. At Google Brain, Sutskever worked with Oriol Vinyals and Quoc Le on sequence by sequence learning algorithms.

In 2015, Sutskever had been nominated for the MIT Technology Review 35 Innovators Under 35. At the end of 2015, Sutskever left Google to be the Director of the newly founded OpenAI Institute.

Featured Co-authors

Sergey Levine
252 publications
Pieter Abbeel
171 publications
Xi Chen
170 publications
Max Welling
136 publications
Quoc V. Le
90 publications
Kai Chen
82 publications
Dawn Song
82 publications
Oriol Vinyals
81 publications
Joan Bruna
71 publications
Jie Tang
70 publications
Peter L. Bartlett
51 publications

Learning Transferable Visual Models From Natural Language Supervision

State-of-the-art computer vision systems are trained to predict a fixed ...
02/26/2021 ∙ by Alec Radford, et al. ∙ 8 ∙ share

read it
Zero-Shot Text-to-Image Generation

Text-to-image generation has traditionally focused on finding better mod...
02/24/2021 ∙ by Aditya Ramesh, et al. ∙ 10 ∙ share

read it
Generative Language Modeling for Automated Theorem Proving

We explore the application of transformer-based language models to autom...
09/07/2020 ∙ by Stanislas Polu, et al. ∙ 83 ∙ share

read it
Language Models are Few-Shot Learners

Recent work has demonstrated substantial gains on many NLP tasks and ben...
05/28/2020 ∙ by Tom B. Brown, et al. ∙ 33 ∙ share

read it
Jukebox: A Generative Model for Music

We introduce Jukebox, a model that generates music with singing in the r...
04/30/2020 ∙ by Prafulla Dhariwal, et al. ∙ 4 ∙ share

read it
Dota 2 with Large Scale Deep Reinforcement Learning

On April 13th, 2019, OpenAI Five became the first AI system to defeat th...
12/13/2019 ∙ by OpenAI, et al. ∙ 13 ∙ share

read it
Deep Double Descent: Where Bigger Models and More Data Hurt

We show that a variety of modern deep learning tasks exhibit a "double-d...
12/04/2019 ∙ by Preetum Nakkiran, et al. ∙ 13 ∙ share

read it
Generating Long Sequences with Sparse Transformers

Transformers are powerful sequence models, but require time and memory t...
04/23/2019 ∙ by Rewon Child, et al. ∙ 12 ∙ share

read it
FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models

A promising class of generative models maps points from a simple distrib...
10/02/2018 ∙ by Will Grathwohl, et al. ∙ 6 ∙ share

read it
GamePad: A Learning Environment for Theorem Proving

In this paper, we introduce a system called GamePad that can be used to ...
06/02/2018 ∙ by Daniel Huang, et al. ∙ 0 ∙ share

read it
Some Considerations on Learning to Explore via Meta-Reinforcement Learning

We consider the problem of exploration in meta reinforcement learning. T...
03/03/2018 ∙ by Bradly C. Stadie, et al. ∙ 0 ∙ share

read it
Emergent Complexity via Multi-Agent Competition

Reinforcement learning algorithms can train agents that solve problems i...
10/10/2017 ∙ by Trapit Bansal, et al. ∙ 0 ∙ share

read it
Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments

Ability to continuously learn and adapt from limited experience in nonst...
10/10/2017 ∙ by Maruan Al-Shedivat, et al. ∙ 0 ∙ share

read it
An online sequence-to-sequence model for noisy speech recognition

Generative models have long been the dominant approach for speech recogn...
06/16/2017 ∙ by Chung-Cheng Chiu, et al. ∙ 0 ∙ share

read it
Learning to Generate Reviews and Discovering Sentiment

We explore the properties of byte-level recurrent language models. When ...
04/05/2017 ∙ by Alec Radford, et al. ∙ 0 ∙ share

read it
One-Shot Imitation Learning

Imitation learning has been commonly applied to solve different tasks in...
03/21/2017 ∙ by Yan Duan, et al. ∙ 0 ∙ share

read it
Evolution Strategies as a Scalable Alternative to Reinforcement Learning

We explore the use of Evolution Strategies (ES), a class of black box op...
03/10/2017 ∙ by Tim Salimans, et al. ∙ 0 ∙ share

read it
Third-Person Imitation Learning

Reinforcement learning (RL) makes it possible to train agents capable of...
03/06/2017 ∙ by Bradly C. Stadie, et al. ∙ 0 ∙ share

read it
RL^2: Fast Reinforcement Learning via Slow Reinforcement Learning

Deep reinforcement learning (deep RL) has been successful in learning so...
11/09/2016 ∙ by Yan Duan, et al. ∙ 0 ∙ share

read it
Variational Lossy Autoencoder

Representation learning seeks to expose certain aspects of observed data...
11/08/2016 ∙ by Xi Chen, et al. ∙ 0 ∙ share

read it
Extensions and Limitations of the Neural GPU

The Neural GPU is a recent model that can learn algorithms such as multi...
11/02/2016 ∙ by Eric Price, et al. ∙ 0 ∙ share

read it
Learning Online Alignments with Continuous Rewards Policy Gradient

Sequence-to-sequence models with soft attention had significant success ...
08/03/2016 ∙ by Yuping Luo, et al. ∙ 0 ∙ share

read it
Improving Variational Inference with Inverse Autoregressive Flow

The framework of normalizing flows provides a general strategy for flexi...
06/15/2016 ∙ by Diederik P. Kingma, et al. ∙ 0 ∙ share

read it
InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets

This paper describes InfoGAN, an information-theoretic extension to the ...
06/12/2016 ∙ by Xi Chen, et al. ∙ 0 ∙ share

read it
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems

TensorFlow is an interface for expressing machine learning algorithms, a...
03/14/2016 ∙ by Martín Abadi, et al. ∙ 0 ∙ share

read it
Continuous Deep Q-Learning with Model-based Acceleration

Model-free reinforcement learning has been successfully applied to a ran...
03/02/2016 ∙ by Shixiang Gu, et al. ∙ 0 ∙ share

read it
Neural GPUs Learn Algorithms

Learning an algorithm from examples is a fundamental problem that has be...
11/25/2015 ∙ by Łukasz Kaiser, et al. ∙ 0 ∙ share

read it
Adding Gradient Noise Improves Learning for Very Deep Networks

Deep feedforward and recurrent networks have achieved impressive results...
11/21/2015 ∙ by Arvind Neelakantan, et al. ∙ 0 ∙ share

read it
Towards Principled Unsupervised Learning

General unsupervised learning is a long-standing conceptual problem in m...
11/19/2015 ∙ by Ilya Sutskever, et al. ∙ 0 ∙ share

read it
Neural Random-Access Machines

In this paper, we propose and investigate a new neural network architect...
11/19/2015 ∙ by Karol Kurach, et al. ∙ 0 ∙ share

read it
Multi-task Sequence to Sequence Learning

Sequence to sequence learning has recently emerged as a new paradigm in ...
11/19/2015 ∙ by Minh-Thang Luong, et al. ∙ 0 ∙ share

read it
MuProp: Unbiased Backpropagation for Stochastic Neural Networks

Deep neural networks are powerful parametric models that can be trained ...
11/16/2015 ∙ by Shixiang Gu, et al. ∙ 0 ∙ share

read it
A Neural Transducer

Sequence-to-sequence models have achieved impressive results on various ...
11/16/2015 ∙ by Navdeep Jaitly, et al. ∙ 0 ∙ share

read it
Neural Programmer: Inducing Latent Programs with Gradient Descent

Deep neural networks have achieved impressive supervised classification ...
11/16/2015 ∙ by Arvind Neelakantan, et al. ∙ 0 ∙ share

read it
Reinforcement Learning Neural Turing Machines - Revised

The Neural Turing Machine (NTM) is more expressive than all previously c...
05/04/2015 ∙ by Wojciech Zaremba, et al. ∙ 0 ∙ share

read it
Grammar as a Foreign Language

Syntactic constituency parsing is a fundamental problem in natural langu...
12/23/2014 ∙ by Oriol Vinyals, et al. ∙ 0 ∙ share

read it
Move Evaluation in Go Using Deep Convolutional Neural Networks

The game of Go is more challenging than other board games, due to the di...
12/20/2014 ∙ by Chris J. Maddison, et al. ∙ 0 ∙ share

read it
Addressing the Rare Word Problem in Neural Machine Translation

Neural Machine Translation (NMT) is a new approach to machine translatio...
10/30/2014 ∙ by Minh-Thang Luong, et al. ∙ 0 ∙ share

read it
Learning to Execute

Recurrent Neural Networks (RNNs) with Long Short-Term Memory units (LSTM...
10/17/2014 ∙ by Wojciech Zaremba, et al. ∙ 0 ∙ share

read it
Sequence to Sequence Learning with Neural Networks

Deep Neural Networks (DNNs) are powerful models that have achieved excel...
09/10/2014 ∙ by Ilya Sutskever, et al. ∙ 0 ∙ share

read it
Recurrent Neural Network Regularization

We present a simple regularization technique for Recurrent Neural Networ...
09/08/2014 ∙ by Wojciech Zaremba, et al. ∙ 0 ∙ share

read it
Intriguing properties of neural networks

Deep neural networks are highly expressive models that have recently ach...
12/21/2013 ∙ by Christian Szegedy, et al. ∙ 0 ∙ share

read it
Learning Factored Representations in a Deep Mixture of Experts

Mixtures of Experts combine the outputs of several "expert" networks, ea...
12/16/2013 ∙ by David Eigen, et al. ∙ 0 ∙ share

read it
Distributed Representations of Words and Phrases and their Compositionality

The recently introduced continuous Skip-gram model is an efficient metho...
10/16/2013 ∙ by Tomas Mikolov, et al. ∙ 0 ∙ share

read it
Exploiting Similarities among Languages for Machine Translation

Dictionaries and phrase tables are the basis of modern statistical machi...
09/17/2013 ∙ by Tomas Mikolov, et al. ∙ 0 ∙ share

read it
Improving neural networks by preventing co-adaptation of feature detectors

When a large feedforward neural network is trained on a small training s...
07/03/2012 ∙ by Geoffrey E. Hinton, et al. ∙ 0 ∙ share

read it
Estimating the Hessian by Back-propagating Curvature

In this work we develop Curvature Propagation (CP), a general technique ...
06/27/2012 ∙ by James Martens, et al. ∙ 0 ∙ share

read it