
Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes
There is a previously identified equivalence between wide fully connecte...
10/11/2018 ∙ by Roman Novak, et al. ∙ 18 ∙ shareread it

Adversarial Reprogramming of Neural Networks
Deep neural networks are susceptible to adversarial attacks. In computer...
06/28/2018 ∙ by Gamaleldin F. Elsayed, et al. ∙ 10 ∙ shareread it

Learned optimizers that outperform SGD on wallclock and test loss
Deep learning has shown that learned functions can dramatically outperfo...
10/24/2018 ∙ by Luke Metz, et al. ∙ 8 ∙ shareread it

A RAD approach to deep mixture models
Flow based models such as Real NVP are an extremely powerful approach to...
03/18/2019 ∙ by Laurent Dinh, et al. ∙ 4 ∙ shareread it

Guided evolutionary strategies: escaping the curse of dimensionality in random search
Many applications in machine learning require optimizing a function whos...
06/26/2018 ∙ by Niru Maheswaranathan, et al. ∙ 2 ∙ shareread it

Learned optimizers that outperform SGD on wallclock and validation loss
Deep learning has shown that learned functions can dramatically outperfo...
10/24/2018 ∙ by Luke Metz, et al. ∙ 2 ∙ shareread it

Deep Neural Networks as Gaussian Processes
A deep fullyconnected neural network with an i.i.d. prior over its para...
11/01/2017 ∙ by Jaehoon Lee, et al. ∙ 0 ∙ shareread it

A Correspondence Between Random Neural Networks and Statistical Field Theory
A number of recent papers have provided evidence that practical design q...
10/18/2017 ∙ by Samuel S. Schoenholz, et al. ∙ 0 ∙ shareread it

Survey of Expressivity in Deep Neural Networks
We survey results on neural network expressivity described in "On the Ex...
11/24/2016 ∙ by Maithra Raghu, et al. ∙ 0 ∙ shareread it

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
We propose a new technique, Singular Vector Canonical Correlation Analys...
06/19/2017 ∙ by Maithra Raghu, et al. ∙ 0 ∙ shareread it

Capacity and Trainability in Recurrent Neural Networks
Two potential bottlenecks on the expressiveness of recurrent neural netw...
11/29/2016 ∙ by Jasmine Collins, et al. ∙ 0 ∙ shareread it

Analyzing noise in autoencoders and deep networks
Autoencoders have emerged as a useful framework for unsupervised learnin...
06/06/2014 ∙ by Ben Poole, et al. ∙ 0 ∙ shareread it

REBAR: Lowvariance, unbiased gradient estimates for discrete latent variable models
Learning in models with discrete latent variables is challenging due to ...
03/21/2017 ∙ by George Tucker, et al. ∙ 0 ∙ shareread it

Learned Optimizers that Scale and Generalize
Learning to learn has emerged as an important direction for achieving ar...
03/14/2017 ∙ by Olga Wichrowska, et al. ∙ 0 ∙ shareread it

Unrolled Generative Adversarial Networks
We introduce a method to stabilize Generative Adversarial Networks (GANs...
11/07/2016 ∙ by Luke Metz, et al. ∙ 0 ∙ shareread it

Deep Information Propagation
We study the behavior of untrained neural networks whose weights and bia...
11/04/2016 ∙ by Samuel S. Schoenholz, et al. ∙ 0 ∙ shareread it

Deep Knowledge Tracing
Knowledge tracingwhere a machine models the knowledge of a student as...
06/19/2015 ∙ by Chris Piech, et al. ∙ 0 ∙ shareread it

Exponential expressivity in deep neural networks through transient chaos
We combine Riemannian geometry with the mean field theory of high dimens...
06/16/2016 ∙ by Ben Poole, et al. ∙ 0 ∙ shareread it

On the Expressive Power of Deep Neural Networks
We propose a new approach to the problem of neural network expressivity,...
06/16/2016 ∙ by Maithra Raghu, et al. ∙ 0 ∙ shareread it

Density estimation using Real NVP
Unsupervised learning of probabilistic models is a central yet challengi...
05/27/2016 ∙ by Laurent Dinh, et al. ∙ 0 ∙ shareread it

A universal tradeoff between power, precision and speed in physical communication
Maximizing the speed and precision of communication while minimizing pow...
03/24/2016 ∙ by Subhaneil Lahiri, et al. ∙ 0 ∙ shareread it

A Markov Jump Process for More Efficient Hamiltonian Monte Carlo
In most sampling algorithms, including Hamiltonian Monte Carlo, transiti...
09/13/2015 ∙ by Andrew B. Berger, et al. ∙ 0 ∙ shareread it

Deep Unsupervised Learning using Nonequilibrium Thermodynamics
A central problem in machine learning involves modeling complex dataset...
03/12/2015 ∙ by Jascha SohlDickstein, et al. ∙ 0 ∙ shareread it

Efficient Methods for Unsupervised Learning of Probabilistic Models
In this thesis I develop a variety of techniques to train, evaluate, and...
05/19/2012 ∙ by Jascha SohlDickstein, et al. ∙ 0 ∙ shareread it

The Natural Gradient by Analogy to Signal Whitening, and Recipes and Tricks for its Use
The natural gradient allows for more efficient gradient descent by remov...
05/08/2012 ∙ by Jascha SohlDickstein, et al. ∙ 0 ∙ shareread it

An Unsupervised Algorithm For Learning Lie Group Transformations
We present several theoretical contributions which allow Lie groups to b...
01/07/2010 ∙ by Jascha SohlDickstein, et al. ∙ 0 ∙ shareread it

Input Switched Affine Networks: An RNN Architecture Designed for Interpretability
There exist many problem domains where the interpretability of neural ne...
11/28/2016 ∙ by Jakob N. Foerster, et al. ∙ 0 ∙ shareread it

Generalizing Hamiltonian Monte Carlo with Neural Networks
We present a generalpurpose method to train Markov chain Monte Carlo ke...
11/25/2017 ∙ by Daniel Lévy, et al. ∙ 0 ∙ shareread it

Adversarial Examples that Fool both Human and Computer Vision
Machine learning models are vulnerable to adversarial examples: small ch...
02/22/2018 ∙ by Gamaleldin F. Elsayed, et al. ∙ 0 ∙ shareread it

Sensitivity and Generalization in Neural Networks: an Empirical Study
In practice it is often found that large overparameterized neural netwo...
02/23/2018 ∙ by Roman Novak, et al. ∙ 0 ∙ shareread it

Learning Unsupervised Learning Rules
A major goal of unsupervised learning is to discover data representation...
03/31/2018 ∙ by Luke Metz, et al. ∙ 0 ∙ shareread it

Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000Layer Vanilla Convolutional Neural Networks
In recent years, stateoftheart methods in computer vision have utiliz...
06/14/2018 ∙ by Lechao Xiao, et al. ∙ 0 ∙ shareread it

PCA of high dimensional random walks with comparison to neural network training
One technique to visualize the training of neural networks is to perform...
06/22/2018 ∙ by Joseph M. Antognini, et al. ∙ 0 ∙ shareread it

Stochastic natural gradient descent draws posterior samples in function space
Natural gradient descent (NGD) minimises the cost function on a Riemanni...
06/25/2018 ∙ by Samuel L. Smith, et al. ∙ 0 ∙ shareread it

Measuring the Effects of Data Parallelism on Neural Network Training
Recent hardware developments have made unprecedented amounts of data par...
11/08/2018 ∙ by Christopher J. Shallue, et al. ∙ 0 ∙ shareread it

Eliminating all bad Local Minima from Loss Landscapes without even adding an Extra Unit
Recent work has noted that all bad local minima can be removed from neur...
01/12/2019 ∙ by Jascha SohlDickstein, et al. ∙ 0 ∙ shareread it

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent
A longstanding goal in deep learning research has been to precisely char...
02/18/2019 ∙ by Jaehoon Lee, et al. ∙ 0 ∙ shareread it

A Mean Field Theory of Batch Normalization
We develop a mean field theory for batch normalization in fullyconnecte...
02/21/2019 ∙ by Greg Yang, et al. ∙ 0 ∙ shareread it

The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study
We investigate how the final parameters found by stochastic gradient des...
05/09/2019 ∙ by Daniel S. Park, et al. ∙ 0 ∙ shareread it

Using learned optimizers to make models robust to input noise
Stateofthe art vision models can achieve superhuman performance on ima...
06/08/2019 ∙ by Luke Metz, et al. ∙ 0 ∙ shareread it

Neural reparameterization improves structural optimization
Structural optimization is a popular method for designing objects such a...
09/10/2019 ∙ by Stephan Hoyer, et al. ∙ 0 ∙ shareread it
Jascha SohlDickstein
is this you? claim profile
Staff Research Scientist in the Brain group at Google, Academic Resident at Khan Academy, Visiting scholar in Surya Ganguli's lab at Stanford University, PhD in 2012 in the Redwood Center for Theoretical Neuroscience at UC Berkeley, in Bruno Olshausen's lab.