research
∙
06/12/2023
Benchmarking Neural Network Training Algorithms
Training algorithms, broadly construed, are an essential part of every d...
share
research
∙
07/29/2022
Adaptive Gradient Methods at the Edge of Stability
Very little is known about the training dynamics of adaptive gradient me...
research
∙
07/07/2022
Pre-training helps Bayesian optimization too
Bayesian optimization (BO) has become a popular strategy for global opti...
research
∙
03/18/2022
AI system for fetal ultrasound in low-resource settings
Despite considerable progress in maternal healthcare, maternal and perin...
research
∙
12/15/2021
Predicting the utility of search spaces for black-box optimization: a simple, budget-aware approach
Black box optimization requires specifying a search space to explore for...
research
∙
09/16/2021
Automatic prior selection for meta Bayesian optimization with a case study on tuning deep neural network optimizers
The performance of deep neural networks can be highly sensitive to the c...
research
∙
04/05/2021
What Will it Take to Fix Benchmarking in Natural Language Understanding?
Evaluation for many natural language understanding (NLU) tasks is broken...
research
∙
02/12/2021
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes
Recently the LARS and LAMB optimizers have been proposed for training ne...
research
∙
10/11/2019
On Empirical Comparisons of Optimizers for Deep Learning
Selecting an optimizer is a central step in the contemporary deep learni...
research
∙
07/12/2019
Faster Neural Network Training with Data Echoing
In the twilight of Moore's law, GPUs and other specialized hardware acce...
research
∙
07/09/2019
Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model
Increasing the batch size is a popular way to speed up neural network tr...
research
∙
11/08/2018
Measuring the Effects of Data Parallelism on Neural Network Training
Recent hardware developments have made unprecedented amounts of data par...
research
∙
08/23/2018
The Importance of Generation Order in Language Modeling
Neural language models are a critical component of state-of-the-art syst...
research
∙
07/18/2018
Motivating the Rules of the Game for Adversarial Example Research
Advances in machine learning have led to broad deployment of systems wit...
research
∙
06/12/2018
Embedding Text in Hyperbolic Spaces
Natural language text exhibits hierarchical structure in a variety of re...
research
∙
05/25/2018
Parallel Architecture and Hyperparameter Search via Successive Halving and Classification
We present a simple and powerful algorithm for parallel black box optimi...
research
∙
04/09/2018
Large scale distributed neural network training through online distillation
Techniques such as ensembling and distillation promise model quality imp...
research
∙
03/03/2017
Detecting Cancer Metastases on Gigapixel Pathology Images
Each year, the treatment decisions for more than 230,000 breast cancer p...
research
∙
08/09/2014
Incorporating Side Information in Probabilistic Matrix Factorization with Gaussian Processes
Probabilistic matrix factorization (PMF) is a powerful method for modeli...
research
∙
06/04/2014
Multi-task Neural Networks for QSAR Predictions
Although artificial neural networks have occasionally been used for Quan...
research
∙
09/05/2013
Improvements to deep convolutional neural networks for LVCSR
Deep Convolutional Neural Networks (CNNs) are more powerful than Deep Ne...
research
∙
02/25/2012