research
∙
08/04/2023
Multi-attacks: Many images + the same adversarial attack → many target labels
We show that we can easily design a single adversarial perturbation P th...
share
research
∙
12/15/2022
Constitutional AI: Harmlessness from AI Feedback
As AI systems become more capable, we would like to enlist their help to...
research
∙
11/04/2022
Measuring Progress on Scalable Oversight for Large Language Models
Developing safe and useful general-purpose AI systems will require us to...
research
∙
10/11/2022
What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundaries
Deep neural network classifiers partition input space into high confiden...
research
∙
08/23/2022
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
We describe our early efforts to red team language models in order to si...
research
∙
07/11/2022
Language Models (Mostly) Know What They Know
We study whether language models can evaluate the validity of their own ...
research
∙
04/12/2022
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
We apply preference modeling and reinforcement learning from human feedb...
research
∙
02/15/2022
Predictability and Surprise in Large Generative Models
Large-scale pre-training has recently emerged as a technique for creatin...
research
∙
01/18/2022
Adversarial vulnerability of powerful near out-of-distribution detection
There has been a significant progress in detecting out-of-distribution (...
research
∙
07/13/2021
How many degrees of freedom do we need to train deep networks: a loss landscape perspective
A variety of recent works, spanning pruning, lottery tickets, and traini...
research
∙
06/16/2021
A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
Mahalanobis distance (MD) is a simple and popular post-processing method...
research
∙
06/06/2021
Exploring the Limits of Out-of-Distribution Detection
Near out-of-distribution detection (OOD) is a major challenge for deep n...
research
∙
05/27/2021
Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error
In computer vision, it is standard practice to draw a single sample from...
research
∙
04/22/2021
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes
Linear interpolation between initial neural network parameters and conve...
research
∙
10/28/2020
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel
In suitably initialized wide networks, small learning rates transform de...
research
∙
10/13/2020
Training independent subnetworks for robust prediction
Recent approaches to efficiently ensemble neural networks have shown tha...
research
∙
02/21/2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks
The early phase of training of deep neural networks is critical for thei...
research
∙
12/05/2019
Deep Ensembles: A Loss Landscape Perspective
Deep ensembles have been empirically shown to be a promising approach fo...
research
∙
10/14/2019
Emergent properties of the local geometry of neural loss landscapes
The local geometry of high dimensional neural network loss landscapes ca...
research
∙
06/11/2019
Large Scale Structure of Neural Network Loss Landscapes
There are many surprising and perhaps counter-intuitive properties of op...
research
∙
01/28/2019
Stiffness: A New Perspective on Generalization in Neural Networks
We investigate neural network training and generalization using the conc...
research
∙
12/17/2018
Adaptive Quantum State Tomography with Neural Networks
Quantum State Tomography is the task of determining an unknown quantum s...
research
∙
07/06/2018
The Goldilocks zone: Towards better understanding of neural network loss landscapes
We explore the loss landscape of fully-connected neural networks using r...
research
∙
12/02/2017
Towards understanding feedback from supermassive black holes using convolutional neural networks
Supermassive black holes at centers of clusters of galaxies strongly int...
research
∙
08/09/2017