Ethan Dyer | DeepAI

DeepAI AI Chat

Featured Co-authors

Jascha Sohl-Dickstein
68 publications
Behnam Neyshabur
42 publications
Yuhuai Wu
40 publications
Samuel S. Schoenholz
26 publications
Henryk Michalewski
26 publications
Jaehoon Lee
26 publications
Ekin D. Cubuk
23 publications
Jared Kaplan
21 publications
Jojo Yun
20 publications
Xiaoxia Wu
19 publications
Daniel Duckworth
14 publications

∙ 07/11/2022

Exploring Length Generalization in Large Language Models

The ability to extrapolate from short problem instances to longer ones i...

0 Cem Anil, et al. ∙

∙ 06/29/2022

Solving Quantitative Reasoning Problems with Language Models

Language models have achieved remarkable performance on a wide range of ...

13 Aitor Lewkowycz, et al. ∙

∙ 03/11/2022

Block-Recurrent Transformers

We introduce the Block-Recurrent Transformer, which applies a transforme...

0 DeLesley Hutchins, et al. ∙

∙ 02/12/2021

Explaining Neural Scaling Laws

The test loss of well-trained neural networks often follows precise powe...

0 Yasaman Bahri, et al. ∙

∙ 12/05/2020

When Do Curricula Work?

Inspired by human learning, researchers have proposed ordering examples ...

0 Xiaoxia Wu, et al. ∙

∙ 08/19/2020

Asymptotics of Wide Convolutional Neural Networks

Wide neural networks have proven to be a rich class of architectures for...

0 Anders Andreassen, et al. ∙

∙ 08/17/2020

Whitening and second order optimization both destroy information about the dataset, and can make generalization impossible

Machine learning is predicated on the concept of generalization: a model...

0 Neha S. Wadia, et al. ∙

∙ 07/14/2020

Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics

A central challenge in developing versatile machine learning systems is ...

0 Vinay V. Ramasesh, et al. ∙

∙ 03/04/2020

The large learning rate phase of deep learning: the catapult mechanism

The choice of initial learning rate can have a profound effect on the pe...

0 Aitor Lewkowycz, et al. ∙

∙ 02/20/2020

Affinity and Diversity: Quantifying Mechanisms of Data Augmentation

Though data augmentation has become a standard component of deep neural ...

0 Raphael Gontijo Lopes, et al. ∙

∙ 09/25/2019

Asymptotics of Wide Networks from Feynman Diagrams

Understanding the asymptotic behavior of wide networks is of considerabl...

0 Ethan Dyer, et al. ∙

∙ 12/12/2018

Gradient Descent Happens in a Tiny Subspace

We show that in a variety of large-scale deep learning scenarios the gra...

0 Guy Gur-Ari, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world