b'Martin Jaggi'

research

∙ 06/16/2022

Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning

We study the asynchronous stochastic gradient descent algorithm for dist...

Anastasia Koloskova, et al. ∙

research

∙ 06/07/2022

Beyond spectral gap: The role of the topology in decentralized learning

In data-parallel optimization of machine learning models, workers collab...

Thijs Vogels, et al. ∙

research

∙ 05/30/2022

On Avoiding Local Minima Using Gradient Descent With Large Learning Rates

It has been widely observed in training of neural networks that when app...

Amirkeivan Mohtashami, et al. ∙

research

∙ 05/17/2022

SKILL: Structured Knowledge Infusion for Large Language Models

Large language models (LLMs) have demonstrated human-level performance o...

Fedor Moiseev, et al. ∙

research

∙ 04/13/2022

Data-heterogeneity-aware Mixing for Decentralized Learning

Decentralized learning provides an effective framework to train machine ...

Yatin Dandi, et al. ∙

research

∙ 02/11/2022

Improving Generalization via Uncertainty Driven Perturbations

Recently Shah et al., 2020 pointed out the pitfalls of the simplicity bi...

Matteo Pagliardini, et al. ∙

research

∙ 02/09/2022

Agree to Disagree: Diversity through Disagreement for Better Transferability

Gradient-based learning algorithms have an implicit simplicity bias whic...

Matteo Pagliardini, et al. ∙

research

∙ 02/03/2022

Characterizing Finding Good Data Orderings for Fast Convergence of Sequential Gradient Methods

While SGD, which samples from the data with replacement is widely studie...

Amirkeivan Mohtashami, et al. ∙

research

∙ 02/03/2022

Byzantine-Robust Decentralized Learning via Self-Centered Clipping

In this paper, we study the challenging task of Byzantine-robust decentr...

Lie He, et al. ∙

research

∙ 12/16/2021

Understanding Memorization from the Perspective of Optimization via Efficient Influence Estimation

Over-parameterized deep neural networks are able to achieve excellent tr...

Futong Liu, et al. ∙

research

∙ 11/16/2021

Interpreting Language Models Through Knowledge Graph Extraction

Transformer-based language models trained on large text corpora have enj...

Vinitra Swamy, et al. ∙

research

∙ 11/10/2021

Linear Speedup in Personalized Collaborative Learning

Personalization in federated learning can improve the accuracy of a mode...

El Mahdi Chayti, et al. ∙

research

∙ 10/25/2021

Optimal Model Averaging: Towards Personalized Collaborative Learning

In federated learning, differences in the data or objectives between the...

Felix Grimberg, et al. ∙

research

∙ 10/13/2021

WAFFLE: Weighted Averaging for Personalized Federated Learning

In collaborative or federated learning, model personalization can be a v...

Martin Beaussart, et al. ∙

research

∙ 10/08/2021

RelaySum for Decentralized Deep Learning on Heterogeneous Data

In decentralized machine learning, workers compute model updates on thei...

Thijs Vogels, et al. ∙

research

∙ 09/06/2021

On Second-order Optimization Methods for Federated Learning

We consider federated learning (FL), where the training data is distribu...

Sebastian Bischoff, et al. ∙

research

∙ 08/18/2021

Semantic Perturbations with Normalizing Flows for Improved Generalization

Data augmentation is a widely adopted technique for avoiding overfitting...

Oğuz Kaan Yüksel, et al. ∙

research

∙ 07/14/2021

IFedAvg: Interpretable Data-Interoperability for Federated Learning

Recently, the ever-growing demand for privacy-oriented machine learning ...

David Roschewitz, et al. ∙

research

∙ 06/25/2021

Implicit Gradient Alignment in Distributed and Federated Learning

A major obstacle to achieving global convergence in distributed and fede...

Yatin Dandi, et al. ∙

research

∙ 06/16/2021

Simultaneous Training of Partially Masked Neural Networks

For deploying deep learning models to lower end devices, it is necessary...

Amirkeivan Mohtashami, et al. ∙

research

∙ 06/08/2021

Obtaining Better Static Word Embeddings Using Contextual Embedding Models

The advent of contextual word embeddings – representations of words whic...

Prakhar Gupta, et al. ∙

research

∙ 05/28/2021

Lightweight Cross-Lingual Sentence Representation Learning

Large-scale models for learning fixed-dimensional cross-lingual sentence...

Zhuoyuan Mao, et al. ∙

research

∙ 04/15/2021

Federated Learning for Malware Detection in IoT Devices

This work investigates the possibilities enabled by federated learning c...

Valerian Rey, et al. ∙

research

∙ 03/03/2021

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

It has been experimentally observed that the efficiency of distributed t...

Sebastian U. Stich, et al. ∙

research

∙ 02/09/2021

Consensus Control for Decentralized Deep Learning

Decentralized training of deep learning models enables on-device learnin...

Lingjing Kong, et al. ∙

research

∙ 02/09/2021

Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Decentralized training of deep learning models is a key element for enab...

Tao Lin, et al. ∙

research

∙ 02/05/2021

Exact Optimization of Conformal Predictors via Incremental and Decremental Learning

Conformal Predictors (CP) are wrappers around ML methods, providing erro...

Giovanni Cherubin, et al. ∙

research

∙ 12/18/2020

Learning from History for Byzantine Robust Optimization

Byzantine robustness has received significant attention recently given i...

Sai Praneeth Karimireddy, et al. ∙

research

∙ 11/03/2020

A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!

Decentralized optimization methods enable on-device training of machine ...

Dmitry Kovalev, et al. ∙

research

∙ 09/19/2020

Sparse Communication for Training Deep Networks

Synchronous stochastic gradient descent (SGD) is the most common method ...

Negar Foroutan Eghlidi, et al. ∙

research

∙ 08/08/2020

Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning

Federated learning is a challenging optimization problem due to the hete...

Sai Praneeth Karimireddy, et al. ∙

research

∙ 08/04/2020

PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning

Lossy gradient compression has become a practical tool to overcome the c...

Thijs Vogels, et al. ∙

research

∙ 06/29/2020

Multi-Head Attention: Collaborate Instead of Concatenate

Attention layers are widely used in natural language processing (NLP) an...

Jean-Baptiste Cordonnier, et al. ∙

research

∙ 06/25/2020

Taming GANs with Lookahead

Generative Adversarial Networks are notoriously challenging to train. Th...

Tatjana Chavdarova, et al. ∙

research

∙ 06/16/2020

Byzantine-Robust Learning on Heterogeneous Datasets via Resampling

In Byzantine robust distributed optimization, a central server wants to ...

Lie He, et al. ∙

research

∙ 06/12/2020

Dynamic Model Pruning with Feedback

Deep neural networks often have millions of parameters. This can hinder ...

Tao Lin, et al. ∙

research

∙ 06/12/2020

Ensemble Distillation for Robust Model Fusion in Federated Learning

Federated Learning (FL) is a machine learning setting where many devices...

Tao Lin, et al. ∙

research

∙ 06/10/2020

Extrapolation for Large-batch Training in Deep Learning

Deep learning networks are typically trained by Stochastic Gradient Desc...

Tao Lin, et al. ∙

research

∙ 06/08/2020

Secure Byzantine-Robust Machine Learning

Increasingly machine learning systems are being deployed to edge servers...

Lie He, et al. ∙

research

∙ 04/26/2020

Masking as an Efficient Alternative to Finetuning for Pretrained Language Models

We present an efficient method of utilizing pretrained language models, ...

Mengjie Zhao, et al. ∙

research

∙ 03/25/2020

Data Parallelism in Training Sparse Neural Networks

Network pruning is an effective methodology to compress large neural net...

Namhoon Lee, et al. ∙

research

∙ 03/23/2020

A Unified Theory of Decentralized SGD with Changing Topology and Local Updates

Decentralized stochastic optimization methods have gained a lot of atten...

Anastasia Koloskova, et al. ∙

research

∙ 12/28/2019

Robust Cross-lingual Embeddings from Parallel Sentences

Recent advances in cross-lingual word embeddings have primarily relied o...

Ali Sabet, et al. ∙

research

∙ 12/10/2019

Advances and Open Problems in Federated Learning

Federated learning (FL) is a machine learning setting where many clients...

Peter Kairouz, et al. ∙

research

∙ 11/08/2019

On the Relationship between Self-Attention and Convolutional Layers

Recent trends of incorporating attention mechanisms in vision have led r...

Jean-Baptiste Cordonnier, et al. ∙

research

∙ 10/25/2019

On the Tunability of Optimizers in Deep Learning

There is no consensus yet on the question whether adaptive gradient meth...

Prabhu Teja Sivaprasad, et al. ∙

research

∙ 10/12/2019

Model Fusion via Optimal Transport

Combining different models is a widely used paradigm in machine learning...

Sidak Pal Singh, et al. ∙

research

∙ 07/22/2019

Decentralized Deep Learning with Arbitrary Communication Compression

Decentralized training of deep learning models is a key element for enab...

Anastasia Koloskova, et al. ∙

research

∙ 06/13/2019

Correlating Twitter Language with Community-Level Health Outcomes

We study how language on social media is linked to diseases such as athe...

Arno Schneuwly, et al. ∙

research

∙ 05/31/2019

PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization

We study gradient compression methods to alleviate the communication bot...

Thijs Vogels, et al. ∙

Martin Jaggi

Featured Co-authors