Martin Jaggi

is this you? claim profile

∙ 0 followers
EPFL ∙ ETH Zurich

Tenure-Track Assistant Professor at EPFL (École polytechnique fédérale de Lausanne)

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates

It has been experimentally observed that the efficiency of distributed t...
03/03/2021 ∙ by Sebastian U. Stich, et al. ∙ 0 ∙ share

read it
Consensus Control for Decentralized Deep Learning

Decentralized training of deep learning models enables on-device learnin...
02/09/2021 ∙ by Lingjing Kong, et al. ∙ 0 ∙ share

read it
Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Decentralized training of deep learning models is a key element for enab...
02/09/2021 ∙ by Tao Lin, et al. ∙ 41 ∙ share

read it
Exact Optimization of Conformal Predictors via Incremental and Decremental Learning

Conformal Predictors (CP) are wrappers around ML methods, providing erro...
02/05/2021 ∙ by Giovanni Cherubin, et al. ∙ 0 ∙ share

read it
Learning from History for Byzantine Robust Optimization

Byzantine robustness has received significant attention recently given i...
12/18/2020 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!

Decentralized optimization methods enable on-device training of machine ...
11/03/2020 ∙ by Dmitry Kovalev, et al. ∙ 22 ∙ share

read it
Sparse Communication for Training Deep Networks

Synchronous stochastic gradient descent (SGD) is the most common method ...
09/19/2020 ∙ by Negar Foroutan Eghlidi, et al. ∙ 0 ∙ share

read it
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning

Federated learning is a challenging optimization problem due to the hete...
08/08/2020 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning

Lossy gradient compression has become a practical tool to overcome the c...
08/04/2020 ∙ by Thijs Vogels, et al. ∙ 0 ∙ share

read it
Multi-Head Attention: Collaborate Instead of Concatenate

Attention layers are widely used in natural language processing (NLP) an...
06/29/2020 ∙ by Jean-Baptiste Cordonnier, et al. ∙ 0 ∙ share

read it
Taming GANs with Lookahead

Generative Adversarial Networks are notoriously challenging to train. Th...
06/25/2020 ∙ by Tatjana Chavdarova, et al. ∙ 12 ∙ share

read it
Byzantine-Robust Learning on Heterogeneous Datasets via Resampling

In Byzantine robust distributed optimization, a central server wants to ...
06/16/2020 ∙ by Lie He, et al. ∙ 0 ∙ share

read it
Dynamic Model Pruning with Feedback

Deep neural networks often have millions of parameters. This can hinder ...
06/12/2020 ∙ by Tao Lin, et al. ∙ 8 ∙ share

read it
Ensemble Distillation for Robust Model Fusion in Federated Learning

Federated Learning (FL) is a machine learning setting where many devices...
06/12/2020 ∙ by Tao Lin, et al. ∙ 1 ∙ share

read it
Extrapolation for Large-batch Training in Deep Learning

Deep learning networks are typically trained by Stochastic Gradient Desc...
06/10/2020 ∙ by Tao Lin, et al. ∙ 13 ∙ share

read it
Secure Byzantine-Robust Machine Learning

Increasingly machine learning systems are being deployed to edge servers...
06/08/2020 ∙ by Lie He, et al. ∙ 0 ∙ share

read it
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models

We present an efficient method of utilizing pretrained language models, ...
04/26/2020 ∙ by Mengjie Zhao, et al. ∙ 0 ∙ share

read it
Data Parallelism in Training Sparse Neural Networks

Network pruning is an effective methodology to compress large neural net...
03/25/2020 ∙ by Namhoon Lee, et al. ∙ 13 ∙ share

read it
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates

Decentralized stochastic optimization methods have gained a lot of atten...
03/23/2020 ∙ by Anastasia Koloskova, et al. ∙ 0 ∙ share

read it
Robust Cross-lingual Embeddings from Parallel Sentences

Recent advances in cross-lingual word embeddings have primarily relied o...
12/28/2019 ∙ by Ali Sabet, et al. ∙ 0 ∙ share

read it
Advances and Open Problems in Federated Learning

Federated learning (FL) is a machine learning setting where many clients...
12/10/2019 ∙ by Peter Kairouz, et al. ∙ 33 ∙ share

read it
On the Relationship between Self-Attention and Convolutional Layers

Recent trends of incorporating attention mechanisms in vision have led r...
11/08/2019 ∙ by Jean-Baptiste Cordonnier, et al. ∙ 0 ∙ share

read it
On the Tunability of Optimizers in Deep Learning

There is no consensus yet on the question whether adaptive gradient meth...
10/25/2019 ∙ by Prabhu Teja Sivaprasad, et al. ∙ 8 ∙ share

read it
Model Fusion via Optimal Transport

Combining different models is a widely used paradigm in machine learning...
10/12/2019 ∙ by Sidak Pal Singh, et al. ∙ 0 ∙ share

read it
Decentralized Deep Learning with Arbitrary Communication Compression

Decentralized training of deep learning models is a key element for enab...
07/22/2019 ∙ by Anastasia Koloskova, et al. ∙ 0 ∙ share

read it
Correlating Twitter Language with Community-Level Health Outcomes

We study how language on social media is linked to diseases such as athe...
06/13/2019 ∙ by Arno Schneuwly, et al. ∙ 0 ∙ share

read it
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization

We study gradient compression methods to alleviate the communication bot...
05/31/2019 ∙ by Thijs Vogels, et al. ∙ 0 ∙ share

read it
On Linear Learning with Manycore Processors

A new generation of manycore processors is on the rise that offers dozen...
05/02/2019 ∙ by Eliza Wszola, et al. ∙ 0 ∙ share

read it
Better Word Embeddings by Disentangling Contextual n-Gram Information

Pre-trained word vectors are ubiquitous in Natural Language Processing a...
04/10/2019 ∙ by Prakhar Gupta, et al. ∙ 0 ∙ share

read it
Crosslingual Document Embedding as Reduced-Rank Ridge Regression

There has recently been much interest in extending vector-based word rep...
04/08/2019 ∙ by Martin Josifoski, et al. ∙ 0 ∙ share

read it
SysML: The New Frontier of Machine Learning Systems

Machine learning (ML) techniques are enjoying rapidly increasing adoptio...
03/29/2019 ∙ by Alexander Ratner, et al. ∙ 0 ∙ share

read it
Structure Tree-LSTM: Structure-aware Attentional Document Encoders

We propose a method to create document representations that reflect thei...
02/26/2019 ∙ by Khalil Mrini, et al. ∙ 0 ∙ share

read it
Forecasting intracranial hypertension using multi-scale waveform metrics

Objective: Intracranial hypertension is an important risk factor of seco...
02/25/2019 ∙ by Matthias Hüser, et al. ∙ 0 ∙ share

read it
Overcoming Multi-Model Forgetting

We identify a phenomenon, which we refer to as multi-model forgetting, t...
02/21/2019 ∙ by Yassine Benyahia, et al. ∙ 0 ∙ share

read it
Evaluating the Search Phase of Neural Architecture Search

Neural Architecture Search (NAS) aims to facilitate the design of deep n...
02/21/2019 ∙ by Christian Sciuto, et al. ∙ 0 ∙ share

read it
Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication

We consider decentralized stochastic optimization with the objective fun...
02/01/2019 ∙ by Anastasia Koloskova, et al. ∙ 0 ∙ share

read it
Unsupervised Scalable Representation Learning for Multivariate Time Series

Time series constitute a challenging data type for machine learning algo...
01/30/2019 ∙ by Jean-Yves Franceschi, et al. ∙ 0 ∙ share

read it
Error Feedback Fixes SignSGD and other Gradient Compression Schemes

Sign-based algorithms (e.g. signSGD) have been proposed as a biased grad...
01/28/2019 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
Efficient Greedy Coordinate Descent for Composite Problems

Coordinate descent with random coordinate selection is the current state...
10/16/2018 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
Sparsified SGD with Memory

Huge scale machine learning problems are nowadays tackled by distributed...
09/20/2018 ∙ by Sebastian U. Stich, et al. ∙ 0 ∙ share

read it
Wasserstein is all you need

We propose a unified framework for building unsupervised representations...
08/29/2018 ∙ by Sidak Pal Singh, et al. ∙ 0 ∙ share

read it
Don't Use Large Mini-Batches, Use Local SGD

Mini-batch stochastic gradient methods are the current state of the art ...
08/22/2018 ∙ by Tao Lin, et al. ∙ 0 ∙ share

read it
COLA: Communication-Efficient Decentralized Linear Learning

Decentralized machine learning is a promising emerging paradigm in view ...
08/13/2018 ∙ by Lie He, et al. ∙ 0 ∙ share

read it
A Distributed Second-Order Algorithm You Can Trust

Due to the rapid growth of data and computational resources, distributed...
06/20/2018 ∙ by Celestine Dünner, et al. ∙ 2 ∙ share

read it
Global linear convergence of Newton's method without strong-convexity or Lipschitz gradients

We show that Newton's method converges globally at a linear rate for obj...
06/01/2018 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
Training DNNs with Hybrid Block Floating Point

The wide adoption of DNNs has given birth to unrelenting computing requi...
04/04/2018 ∙ by Mario Drumond, et al. ∙ 0 ∙ share

read it
End-to-End DNN Training with Block Floating Point Arithmetic

DNNs are ubiquitous datacenter workloads, requiring orders of magnitude ...
04/04/2018 ∙ by Mario Drumond, et al. ∙ 0 ∙ share

read it
Revisiting First-Order Convex Optimization Over Linear Spaces

Two popular examples of first-order optimization methods over linear spa...
03/26/2018 ∙ by Francesco Locatello, et al. ∙ 0 ∙ share

read it
EmbedRank: Unsupervised Keyphrase Extraction using Sentence Embeddings

Keyphrase extraction is the task of automatically selecting a small set ...
01/13/2018 ∙ by Kamil Bennani-Smires, et al. ∙ 0 ∙ share

read it
Efficient Use of Limited-Memory Accelerators for Linear Learning on Heterogeneous Systems

We propose a generic algorithmic building block to accelerate training o...
08/17/2017 ∙ by Celestine Dünner, et al. ∙ 0 ∙ share

read it

Martin Jaggi

Featured Co-authors