Martin Jaggi

is this you? claim profile

∙ 0 followers
EPFL ∙ ETH Zurich

Tenure-Track Assistant Professor at EPFL (École polytechnique fédérale de Lausanne)

A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!

Decentralized optimization methods enable on-device training of machine ...
11/03/2020 ∙ by Dmitry Kovalev, et al. ∙ 22 ∙ share

read it
Sparse Communication for Training Deep Networks

Synchronous stochastic gradient descent (SGD) is the most common method ...
09/19/2020 ∙ by Negar Foroutan Eghlidi, et al. ∙ 0 ∙ share

read it
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning

Federated learning is a challenging optimization problem due to the hete...
08/08/2020 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
PowerGossip: Practical Low-Rank Communication Compression in Decentralized Deep Learning

Lossy gradient compression has become a practical tool to overcome the c...
08/04/2020 ∙ by Thijs Vogels, et al. ∙ 0 ∙ share

read it
Multi-Head Attention: Collaborate Instead of Concatenate

Attention layers are widely used in natural language processing (NLP) an...
06/29/2020 ∙ by Jean-Baptiste Cordonnier, et al. ∙ 0 ∙ share

read it
Taming GANs with Lookahead

Generative Adversarial Networks are notoriously challenging to train. Th...
06/25/2020 ∙ by Tatjana Chavdarova, et al. ∙ 12 ∙ share

read it
Byzantine-Robust Learning on Heterogeneous Datasets via Resampling

In Byzantine robust distributed optimization, a central server wants to ...
06/16/2020 ∙ by Lie He, et al. ∙ 0 ∙ share

read it
Dynamic Model Pruning with Feedback

Deep neural networks often have millions of parameters. This can hinder ...
06/12/2020 ∙ by Tao Lin, et al. ∙ 8 ∙ share

read it
Ensemble Distillation for Robust Model Fusion in Federated Learning

Federated Learning (FL) is a machine learning setting where many devices...
06/12/2020 ∙ by Tao Lin, et al. ∙ 1 ∙ share

read it
Extrapolation for Large-batch Training in Deep Learning

Deep learning networks are typically trained by Stochastic Gradient Desc...
06/10/2020 ∙ by Tao Lin, et al. ∙ 13 ∙ share

read it
Secure Byzantine-Robust Machine Learning

Increasingly machine learning systems are being deployed to edge servers...
06/08/2020 ∙ by Lie He, et al. ∙ 0 ∙ share

read it
Masking as an Efficient Alternative to Finetuning for Pretrained Language Models

We present an efficient method of utilizing pretrained language models, ...
04/26/2020 ∙ by Mengjie Zhao, et al. ∙ 0 ∙ share

read it
Data Parallelism in Training Sparse Neural Networks

Network pruning is an effective methodology to compress large neural net...
03/25/2020 ∙ by Namhoon Lee, et al. ∙ 13 ∙ share

read it
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates

Decentralized stochastic optimization methods have gained a lot of atten...
03/23/2020 ∙ by Anastasia Koloskova, et al. ∙ 0 ∙ share

read it
Robust Cross-lingual Embeddings from Parallel Sentences

Recent advances in cross-lingual word embeddings have primarily relied o...
12/28/2019 ∙ by Ali Sabet, et al. ∙ 0 ∙ share

read it
Advances and Open Problems in Federated Learning

Federated learning (FL) is a machine learning setting where many clients...
12/10/2019 ∙ by Peter Kairouz, et al. ∙ 33 ∙ share

read it
On the Relationship between Self-Attention and Convolutional Layers

Recent trends of incorporating attention mechanisms in vision have led r...
11/08/2019 ∙ by Jean-Baptiste Cordonnier, et al. ∙ 0 ∙ share

read it
On the Tunability of Optimizers in Deep Learning

There is no consensus yet on the question whether adaptive gradient meth...
10/25/2019 ∙ by Prabhu Teja Sivaprasad, et al. ∙ 8 ∙ share

read it
Model Fusion via Optimal Transport

Combining different models is a widely used paradigm in machine learning...
10/12/2019 ∙ by Sidak Pal Singh, et al. ∙ 0 ∙ share

read it
Decentralized Deep Learning with Arbitrary Communication Compression

Decentralized training of deep learning models is a key element for enab...
07/22/2019 ∙ by Anastasia Koloskova, et al. ∙ 0 ∙ share

read it
Correlating Twitter Language with Community-Level Health Outcomes

We study how language on social media is linked to diseases such as athe...
06/13/2019 ∙ by Arno Schneuwly, et al. ∙ 0 ∙ share

read it
PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization

We study gradient compression methods to alleviate the communication bot...
05/31/2019 ∙ by Thijs Vogels, et al. ∙ 0 ∙ share

read it
On Linear Learning with Manycore Processors

A new generation of manycore processors is on the rise that offers dozen...
05/02/2019 ∙ by Eliza Wszola, et al. ∙ 0 ∙ share

read it
Better Word Embeddings by Disentangling Contextual n-Gram Information

Pre-trained word vectors are ubiquitous in Natural Language Processing a...
04/10/2019 ∙ by Prakhar Gupta, et al. ∙ 0 ∙ share

read it
Crosslingual Document Embedding as Reduced-Rank Ridge Regression

There has recently been much interest in extending vector-based word rep...
04/08/2019 ∙ by Martin Josifoski, et al. ∙ 0 ∙ share

read it
SysML: The New Frontier of Machine Learning Systems

Machine learning (ML) techniques are enjoying rapidly increasing adoptio...
03/29/2019 ∙ by Alexander Ratner, et al. ∙ 0 ∙ share

read it
Structure Tree-LSTM: Structure-aware Attentional Document Encoders

We propose a method to create document representations that reflect thei...
02/26/2019 ∙ by Khalil Mrini, et al. ∙ 0 ∙ share

read it
Forecasting intracranial hypertension using multi-scale waveform metrics

Objective: Intracranial hypertension is an important risk factor of seco...
02/25/2019 ∙ by Matthias Hüser, et al. ∙ 0 ∙ share

read it
Overcoming Multi-Model Forgetting

We identify a phenomenon, which we refer to as multi-model forgetting, t...
02/21/2019 ∙ by Yassine Benyahia, et al. ∙ 0 ∙ share

read it
Evaluating the Search Phase of Neural Architecture Search

Neural Architecture Search (NAS) aims to facilitate the design of deep n...
02/21/2019 ∙ by Christian Sciuto, et al. ∙ 0 ∙ share

read it
Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication

We consider decentralized stochastic optimization with the objective fun...
02/01/2019 ∙ by Anastasia Koloskova, et al. ∙ 0 ∙ share

read it
Unsupervised Scalable Representation Learning for Multivariate Time Series

Time series constitute a challenging data type for machine learning algo...
01/30/2019 ∙ by Jean-Yves Franceschi, et al. ∙ 0 ∙ share

read it
Error Feedback Fixes SignSGD and other Gradient Compression Schemes

Sign-based algorithms (e.g. signSGD) have been proposed as a biased grad...
01/28/2019 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
Efficient Greedy Coordinate Descent for Composite Problems

Coordinate descent with random coordinate selection is the current state...
10/16/2018 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
Sparsified SGD with Memory

Huge scale machine learning problems are nowadays tackled by distributed...
09/20/2018 ∙ by Sebastian U. Stich, et al. ∙ 0 ∙ share

read it
Wasserstein is all you need

We propose a unified framework for building unsupervised representations...
08/29/2018 ∙ by Sidak Pal Singh, et al. ∙ 0 ∙ share

read it
Don't Use Large Mini-Batches, Use Local SGD

Mini-batch stochastic gradient methods are the current state of the art ...
08/22/2018 ∙ by Tao Lin, et al. ∙ 0 ∙ share

read it
COLA: Communication-Efficient Decentralized Linear Learning

Decentralized machine learning is a promising emerging paradigm in view ...
08/13/2018 ∙ by Lie He, et al. ∙ 0 ∙ share

read it
A Distributed Second-Order Algorithm You Can Trust

Due to the rapid growth of data and computational resources, distributed...
06/20/2018 ∙ by Celestine Dünner, et al. ∙ 2 ∙ share

read it
Global linear convergence of Newton's method without strong-convexity or Lipschitz gradients

We show that Newton's method converges globally at a linear rate for obj...
06/01/2018 ∙ by Sai Praneeth Karimireddy, et al. ∙ 0 ∙ share

read it
Training DNNs with Hybrid Block Floating Point

The wide adoption of DNNs has given birth to unrelenting computing requi...
04/04/2018 ∙ by Mario Drumond, et al. ∙ 0 ∙ share

read it
End-to-End DNN Training with Block Floating Point Arithmetic

DNNs are ubiquitous datacenter workloads, requiring orders of magnitude ...
04/04/2018 ∙ by Mario Drumond, et al. ∙ 0 ∙ share

read it
Revisiting First-Order Convex Optimization Over Linear Spaces

Two popular examples of first-order optimization methods over linear spa...
03/26/2018 ∙ by Francesco Locatello, et al. ∙ 0 ∙ share

read it
EmbedRank: Unsupervised Keyphrase Extraction using Sentence Embeddings

Keyphrase extraction is the task of automatically selecting a small set ...
01/13/2018 ∙ by Kamil Bennani-Smires, et al. ∙ 0 ∙ share

read it
Efficient Use of Limited-Memory Accelerators for Linear Learning on Heterogeneous Systems

We propose a generic algorithmic building block to accelerate training o...
08/17/2017 ∙ by Celestine Dünner, et al. ∙ 0 ∙ share

read it
Learning Aerial Image Segmentation from Online Maps

This study deals with semantic segmentation of high-resolution (aerial) ...
07/21/2017 ∙ by Pascal Kaiser, et al. ∙ 0 ∙ share

read it
Unsupervised robust nonparametric learning of hidden community properties

We consider learning of fundamental properties of communities in large n...
07/11/2017 ∙ by Mikhail A. Langovoy, et al. ∙ 0 ∙ share

read it
Greedy Algorithms for Cone Constrained Optimization with Convergence Guarantees

Greedy optimization methods such as Matching Pursuit (MP) and Frank-Wolf...
05/31/2017 ∙ by Francesco Locatello, et al. ∙ 0 ∙ share

read it
Generating Steganographic Text with LSTMs

Motivated by concerns for user privacy, we design a steganographic syste...
05/30/2017 ∙ by Tina Fang, et al. ∙ 0 ∙ share

read it
Faster Coordinate Descent via Adaptive Importance Sampling

Coordinate descent methods employ random partial updates of decision var...
03/07/2017 ∙ by Dmytro Perekrestenko, et al. ∙ 0 ∙ share

read it

Martin Jaggi

Featured Co-authors