Ari S. Morcos | DeepAI

AI Chat

Featured Co-authors

Pieter Abbeel
263 publications
Dhruv Batra
141 publications
Joelle Pineau
116 publications
Kurt Keutzer
112 publications
Ali Farhadi
106 publications
Douwe Kiela
70 publications
Yuandong Tian
69 publications
Michael Auli
68 publications
Pascal Vincent
59 publications
Irfan Essa
58 publications
Surya Ganguli
56 publications

∙ 08/23/2023

D4: Improving LLM Pretraining via Document De-Duplication and Diversification

Over recent years, an increasing amount of compute and data has been pou...

0 Kushal Tirumala, et al. ∙

∙ 08/08/2023

PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning

Synthetic image datasets offer unmatched advantages for designing and ev...

0 Florian Bordes, et al. ∙

∙ 05/27/2023

On the special role of class-selective neurons in early training

It is commonly observed that deep networks trained for classification ex...

0 Omkar Ranadive, et al. ∙

∙ 03/16/2023

SemDeDup: Data-efficient learning at web-scale through semantic deduplication

Progress in machine learning has been driven in large part by massive in...

0 Amro Abbas, et al. ∙

∙ 01/30/2023

Emergence of Maps in the Memories of Blind Navigation Agents

Animal navigation research posits that organisms build and maintain inte...

0 Erik Wijmans, et al. ∙

∙ 10/19/2022

lo-fi: distributed fine-tuning without communication

When fine-tuning large neural networks, it is common to use multiple nod...

5 Mitchell Wortsman, et al. ∙

∙ 06/29/2022

Beyond neural scaling laws: beating power law scaling via data pruning

Widely observed neural scaling laws, in which error falls off as a power...

12 Ben Sorscher, et al. ∙

∙ 03/10/2022

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

The conventional recipe for maximizing model accuracy is to (1) train mu...

10 Mitchell Wortsman, et al. ∙

∙ 06/09/2021

Grounding inductive biases in natural images:invariance stems from variations in data

To perform well on unseen and potentially out-of-distribution samples, i...

0 Diane Bouchacourt, et al. ∙

∙ 04/24/2021

Width Transfer: On the (In)variance of Width Optimization

Optimizing the channel counts for different layers of a CNN has shown gr...

3 Ting-Wu Chin, et al. ∙

∙ 03/23/2021

Leveraging background augmentations to encourage semantic focus in self-supervised contrastive learning

Unsupervised representation learning is an important challenge in comput...

1 Chaitanya K. Ryali, et al. ∙

∙ 12/30/2020

Reservoir Transformer

We demonstrate that transformers obtain impressive performance even when...

23 Sheng Shen, et al. ∙

∙ 10/13/2020

Are all negatives created equal in contrastive instance discrimination?

Self-supervised learning has recently begun to rival supervised learning...

0 Tiffany, et al. ∙

∙ 07/23/2020

PareCO: Pareto-aware Channel Optimization for Slimmable Neural Networks

Slimmable neural networks provide a flexible trade-off front between pre...

0 Ting-Wu Chin, et al. ∙

∙ 07/08/2020

On the relationship between class selectivity, dimensionality, and robustness

While the relative trade-offs between sparse and distributed representat...

0 Matthew L. Leavitt, et al. ∙

∙ 05/07/2020

Plan2Vec: Unsupervised Representation Learning by Latent Plans

In this paper we introduce plan2vec, an unsupervised representation lear...

8 Ge Yang, et al. ∙

∙ 02/29/2020

Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs

Batch normalization (BatchNorm) has become an indispensable tool for tra...

0 Jonathan Frankle, et al. ∙

∙ 02/24/2020

The Early Phase of Neural Network Training

Recent studies have shown that many important aspects of neural network ...

27 Jonathan Frankle, et al. ∙

∙ 06/09/2019

The Generalization-Stability Tradeoff in Neural Network Pruning

Pruning neural network parameters to reduce model size is an area of muc...

0 Brian R. Bartoldson, et al. ∙

∙ 06/06/2019

One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers

The success of lottery ticket initializations (Frankle and Carbin, 2019)...

5 Ari S. Morcos, et al. ∙

∙ 06/06/2019

Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP

The lottery ticket hypothesis proposes that over-parameterization of dee...

0 Haonan Yu, et al. ∙

∙ 01/31/2019

Learning to Make Analogies by Contrasting Abstract Relational Structure

Analogical reasoning has been a principal focus of various waves of AI r...

0 Felix Hill, et al. ∙

∙ 10/31/2018

Analyzing biological and artificial neural networks: challenges with opportunities for synergy?

Deep neural networks (DNNs) transform stimuli across multiple processing...

0 David G. T. Barrett, et al. ∙

∙ 07/11/2018

Measuring abstract reasoning in neural networks

Whether neural networks can learn abstract reasoning or whether they mer...

0 David G. T. Barrett, et al. ∙

∙ 07/03/2018

Human-level performance in first-person multiplayer games with population-based deep reinforcement learning

Recent progress in artificial intelligence through reinforcement learnin...

2 Max Jaderberg, et al. ∙

∙ 06/14/2018

Insights on representational similarity in neural networks with canonical correlation

Comparing different neural network representations and determining how r...

0 Ari S. Morcos, et al. ∙

∙ 04/12/2018

Learned Deformation Stability in Convolutional Neural Networks

Conventional wisdom holds that interleaved pooling layers in convolution...

0 Avraham Ruderman, et al. ∙

∙ 03/19/2018

On the importance of single directions for generalization

Despite their ability to memorize large datasets, deep neural networks o...

0 Ari S. Morcos, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world