research
∙
09/15/2023
Replacing softmax with ReLU in Vision Transformers
Previous research observed accuracy degradation when replacing the atten...
share
research
∙
06/12/2023
Benchmarking Neural Network Training Algorithms
Training algorithms, broadly construed, are an essential part of every d...
research
∙
09/23/2022
Do Current Multi-Task Optimization Methods in Deep Learning Even Help?
Recent research has proposed a series of specialized optimization algori...
research
∙
07/29/2022
Adaptive Gradient Methods at the Edge of Stability
Very little is known about the training dynamics of adaptive gradient me...
research
∙
07/07/2022
Pre-training helps Bayesian optimization too
Bayesian optimization (BO) has become a popular strategy for global opti...
research
∙
03/18/2022
AI system for fetal ultrasound in low-resource settings
Despite considerable progress in maternal healthcare, maternal and perin...
research
∙
12/15/2021
Predicting the utility of search spaces for black-box optimization: a simple, budget-aware approach
Black box optimization requires specifying a search space to explore for...
research
∙
10/08/2021
A Loss Curvature Perspective on Training Instability in Deep Learning
In this work, we study the evolution of the loss Hessian across many cla...
research
∙
09/16/2021
Automatic prior selection for meta Bayesian optimization with a case study on tuning deep neural network optimizers
The performance of deep neural networks can be highly sensitive to the c...
research
∙
06/29/2020
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
We introduce three new robustness benchmarks consisting of naturally occ...
research
∙
12/05/2019
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
Modern deep neural networks can achieve high accuracy when the training ...
research
∙
06/21/2019
A Fourier Perspective on Model Robustness in Computer Vision
Achieving robustness to distributional shift is a longstanding and chall...
research
∙
06/06/2019
Improving Robustness Without Sacrificing Accuracy with Patch Gaussian Augmentation
Deploying machine learning systems in the real world requires both high ...
research
∙
06/05/2019
MNIST-C: A Robustness Benchmark for Computer Vision
We introduce the MNIST-C dataset, a comprehensive suite of 15 corruption...
research
∙
01/29/2019
Adversarial Examples Are a Natural Consequence of Test Error in Noise
Over the last few years, the phenomenon of adversarial examples --- mali...
research
∙
10/08/2018
Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values
Explaining the output of a complicated machine learning model like a dee...
research
∙
10/08/2018
Sanity Checks for Saliency Maps
Saliency methods have emerged as a popular tool to highlight features in...
research
∙
07/18/2018
Motivating the Rules of the Game for Adversarial Example Research
Advances in machine learning have led to broad deployment of systems wit...
research
∙
06/04/2018
Relational inductive biases, deep learning, and graph networks
Artificial intelligence (AI) has undergone a renaissance recently, makin...
research
∙
01/09/2018
Adversarial Spheres
State of the art computer vision models have been shown to be vulnerable...
research
∙
12/27/2017
Adversarial Patch
We present a method to create universal, robust, targeted adversarial im...
research
∙
11/30/2017
TCAV: Relative concept importance testing with Linear Concept Activation Vectors
Neural networks commonly offer high utility but remain difficult to inte...
research
∙
06/19/2017
SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
We propose a new technique, Singular Vector Canonical Correlation Analys...
research
∙
11/28/2016
Input Switched Affine Networks: An RNN Architecture Designed for Interpretability
There exist many problem domains where the interpretability of neural ne...
research
∙
11/04/2016