
Implicit Acceleration and Feature Learning in Infinitely Wide Neural Networks with Bottlenecks
We analyze the learning dynamics of infinitely wide neural networks with...
read it

Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamics
Yang (2020a) recently showed that the Neural Tangent Kernel (NTK) at ini...
read it

Feature Learning in InfiniteWidth Neural Networks
As its width tends to infinity, a deep neural network's behavior under g...
read it

Tensor Programs III: Neural Matrix Laws
In a neural network (NN), weight matrices linearly transform inputs into...
read it

Tensor Programs II: Neural Tangent Kernel for Any Architecture
We prove that a randomly initialized neural network of *any architecture...
read it

Improved Image Wasserstein Attacks and Defenses
Robustness against image perturbations bounded by a ℓ_p ball have been w...
read it

Blackbox Smoothing: A Provable Defense for Pretrained Classifiers
We present a method for provably defending any pretrained image classifi...
read it

Randomized Smoothing of All Shapes and Sizes
Randomized smoothing is a recently proposed defense against adversarial ...
read it

Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes
Wide neural networks with random weights and biases are Gaussian process...
read it

Free resolutions of function classes via order complexes
Function classes are collections of Boolean functions on a finite set, w...
read it

A FineGrained Spectral Perspective on Neural Networks
Are neural networks biased toward simple functions? Does depth always he...
read it

Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
Recent works have shown the effectiveness of randomized smoothing as a s...
read it

A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
Verification of neural networks enables us to gauge their robustness aga...
read it

A Convex Relaxation Barrier to Tight Robust Verification of Neural Networks
Verification of neural networks enables us to gauge their robustness aga...
read it

A Mean Field Theory of Batch Normalization
We develop a mean field theory for batch normalization in fullyconnecte...
read it

Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation
Several recent trends in machine learning theory and practice, from the ...
read it

NAIL: A General Interactive Fiction Agent
Interactive Fiction (IF) games are complex textual decision making probl...
read it

Dynamical Isometry and a Mean Field Theory of LSTMs and GRUs
Training recurrent neural networks (RNNs) on long sequence tasks is plag...
read it

Mean Field Residual Networks: On the Edge of Chaos
We study randomly initialized residual networks using mean field theory ...
read it

LieAccess Neural Turing Machines
External neural memory structures have recently become a popular tool fo...
read it

Lie Access Neural Turing Machine
Following the recent trend in explicit neural memory structures, we pres...
read it
Greg Yang
is this you? claim profile