DeepAI AI Chat
Log In Sign Up

Improving all-reduce collective operations for imbalanced process arrival patterns

04/15/2018
by   Jerzy Proficz, et al.
Okręgowa Izba Radców Prawnych w Gdańsku
0

Two new algorithms for the all-reduce operation, optimized for imbalanced process arrival patterns (PAPs) are presented: (i) sorted linear tree (SLT), (ii) pre-reduced ring (PRR) as well as a new way of on-line PAP detection, including process arrival time (PAT) estimations and their distribution between cooperating processes was introduced. The idea, pseudo-code, implementation details, benchmark for performance evaluation and a real case example for machine learning are provided. The results of the experiments were described and analyzed, showing that the proposed solution has high scalability and improved performance in comparison with the usually used ring and Rabenseifner algorithms.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/09/2021

A Note on Comparison of F-measures

We comment on a recent TKDE paper "Linear Approximation of F-measure for...
12/06/2020

TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture

Federated learning has emerged as a new paradigm of collaborative machin...
04/20/2020

A Generalization of the Allreduce Operation

Allreduce is one of the most frequently used MPI collective operations, ...
05/31/2020

Staffing for many-server systems facing non-standard arrival processes

Arrival processes to service systems often display (i) larger than antic...
02/14/2018

Tackling Multilabel Imbalance through Label Decoupling and Data Resampling Hybridization

The learning from imbalanced data is a deeply studied problem in standar...
05/31/2023

Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust

Watermarking the outputs of generative models is a crucial technique for...