
Tilting the playing field: Dynamical loss functions for machine learning
We show that learning can be improved by using loss functions that evolv...
read it

Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible
Machine learning is predicated on the concept of generalization: a model...
read it

Finite Versus Infinite Neural Networks: an Empirical Study
We perform a careful, thorough, and large scale empirical study of the c...
read it

On the infinite width limit of neural networks with a standard parameterization
There are currently two parameterizations used to derive fixed kernels c...
read it

Disentangling trainability and generalization in deep learning
A fundamental goal in deep learning is the characterization of trainabil...
read it

JAX, M.D.: EndtoEnd Differentiable, Hardware Accelerated, Molecular Dynamics in Pure Python
A large fraction of computational science involves simulating the dynami...
read it

Neural Tangents: Fast and Easy Infinite Neural Networks in Python
Neural Tangents is a library designed to enable research into infinitew...
read it

A Mean Field Theory of Batch Normalization
We develop a mean field theory for batch normalization in fullyconnecte...
read it

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent
A longstanding goal in deep learning research has been to precisely char...
read it

Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs
Training recurrent neural networks (RNNs) on long sequence tasks is plag...
read it

PeptideSpectra Matching from Weak Supervision
As in many other scientific domains, we face a fundamental problem when ...
read it

Dynamical Isometry and a Mean Field Theory of RNNs: Gating Enables Signal Propagation in Recurrent Neural Networks
Recurrent neural networks have gained widespread use in modeling sequenc...
read it

Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000Layer Vanilla Convolutional Neural Networks
In recent years, stateoftheart methods in computer vision have utiliz...
read it

The Emergence of Spectral Universality in Deep Networks
Recent work has shown that tight concentration of the entire spectrum of...
read it

Adversarial Spheres
State of the art computer vision models have been shown to be vulnerable...
read it

Mean Field Residual Networks: On the Edge of Chaos
We study randomly initialized residual networks using mean field theory ...
read it

Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice
It is well known that the initialization of weights in deep neural netwo...
read it

Intriguing Properties of Adversarial Examples
It is becoming increasingly clear that many machine learning classifiers...
read it

Deep Neural Networks as Gaussian Processes
A deep fullyconnected neural network with an i.i.d. prior over its para...
read it

A Correspondence Between Random Neural Networks and Statistical Field Theory
A number of recent papers have provided evidence that practical design q...
read it

Combining Machine Learning and Physics to Understand Glassy Systems
Our understanding of supercooled liquids and glasses has lagged signific...
read it

Deep Information Propagation
We study the behavior of untrained neural networks whose weights and bia...
read it
Samuel S. Schoenholz
is this you? claim profile