research
∙
09/12/2023
A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale
Shampoo is an online and stochastic optimization algorithm belonging to ...
share
research
∙
04/14/2023
DINOv2: Learning Robust Visual Features without Supervision
The recent breakthroughs in natural language processing for model pretra...
research
∙
01/19/2023
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
This paper demonstrates an approach for learning highly semantic image r...
research
∙
10/19/2022
lo-fi: distributed fine-tuning without communication
When fine-tuning large neural networks, it is common to use multiple nod...
research
∙
10/14/2022
Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning
An oft-cited challenge of federated learning is the presence of heteroge...
research
∙
10/13/2022
The Hidden Uniform Cluster Prior in Self-Supervised Learning
A successful paradigm in representation learning is to perform self-supe...
research
∙
06/30/2022
Where to Begin? Exploring the Impact of Pre-Training and Initialization in Federated Learning
An oft-cited challenge of federated learning is the presence of data het...
research
∙
04/27/2022
FedShuffle: Recipes for Better Use of Local Work in Federated Learning
The practice of applying several local updates before aggregation across...
research
∙
04/14/2022
Masked Siamese Networks for Label-Efficient Learning
We propose Masked Siamese Networks (MSN), a self-supervised learning fra...
research
∙
04/08/2022
Federated Learning with Partial Model Personalization
We consider two federated learning algorithms for training partially per...
research
∙
06/22/2021
Stochastic Polyak Stepsize with a Moving Target
We propose a new stochastic gradient method that uses recorded past loss...
research
∙
06/11/2021
Federated Learning with Buffered Asynchronous Aggregation
Federated Learning (FL) trains a shared model across distributed devices...
research
∙
04/28/2021
Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples
This paper proposes a novel method of learning by predicting view assign...
research
∙
10/06/2020
A Closer Look at Codistillation for Distributed Training
Codistillation has been proposed as a mechanism to share knowledge among...
research
∙
06/24/2020
Advances in Asynchronous Parallel and Distributed Optimization
Motivated by large-scale optimization problems arising in the context of...
research
∙
06/18/2020
Recovering Petaflops in Contrastive Semi-Supervised Learning of Visual Representations
We investigate a strategy for improving the computational efficiency of ...
research
∙
02/27/2020
On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings
We study Nesterov's accelerated gradient method in the stochastic approx...
research
∙
01/06/2020
Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge
Purpose: To advance research in the field of machine learning for MR ima...
research
∙
10/01/2019
SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
Distributed optimization is essential for training large models on large...
research
∙
11/28/2018
A Graph-CNN for 3D Point Cloud Classification
Graph convolutional neural networks (Graph-CNNs) extend traditional CNNs...
research
∙
11/27/2018
Stochastic Gradient Push for Distributed Deep Learning
Large mini-batch parallel SGD is commonly used for distributed training ...
research
∙
11/21/2018
fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
Accelerating Magnetic Resonance Imaging (MRI) by taking fewer measuremen...
research
∙
10/31/2018
Provably Accelerated Randomized Gossip Algorithms
In this work we present novel provably accelerated gossip algorithms for...
research
∙
10/26/2018
TarMAC: Targeted Multi-Agent Communication
We explore a collaborative multi-agent reinforcement learning setting wh...
research
∙
06/03/2018
Learning Graphs from Data: A Signal Representation Perspective
The construction of a meaningful graph topology plays a crucial role in ...
research
∙
03/23/2018
Asynchronous Subgradient-Push
We consider a multi-agent framework for distributed optimization where e...
research
∙
12/10/2014
Memory vectors for similarity search in high-dimensional spaces
We study an indexing architecture to store and search in a database of h...
research
∙
09/01/2014
Storing sequences in binary tournament-based neural networks
An extension to a recently introduced architecture of clique-based neura...
research
∙
08/27/2013
Improving Sparse Associative Memories by Escaping from Bogus Fixed Points
The Gripon-Berrou neural network (GBNN) is a recently invented recurrent...
research
∙
05/05/2011
GANC: Greedy Agglomerative Normalized Cut
This paper describes a graph clustering algorithm that aims to minimize ...
research
∙
09/09/2009