Sparbit: a new logarithmic-cost and data locality-aware MPI Allgather algorithm

09/17/2021
by   Wilton Jaciel Loch, et al.
0

The collective operations are considered critical for improving the performance of exascale-ready and high-performance computing applications. On this paper we focus on the Message-Passing Interface (MPI) Allgather many to many collective, which is amongst the most called and time-consuming operations. Each MPI algorithm for this call suffers from different operational and performance limitations, that might include only working for restricted cases, requiring linear amounts of communication steps with the growth in number of processes, memory copies and shifts to assure correct data organization, and non-local data exchange patterns, most of which negatively contribute to the total operation time. All these characteristics create an environment where there is no algorithm which is the best for all cases and this consequently implies that careful choices of alternatives must be made to execute the call. Considering such aspects, we propose the Stripe Parallel Binomial Trees (Sparbit) algorithm, which has optimal latency and bandwidth time costs with no usage restrictions. It also maintains a much more local communication pattern that minimizes the delays due to long range exchanges, allowing the extraction of more performance from current systems when compared with asymptotically equivalent alternatives. On its best scenario, Sparbit surpassed the traditional MPI algorithms on 46.43 (median) improvements of 34.7

READ FULL TEXT

page 1

page 2

page 7

research
07/22/2020

Collectives in hybrid MPI+MPI code: design, practice and performance

The use of hybrid scheme combining the message passing programming model...
research
06/07/2022

A Locality-Aware Bruck Allgather

Collective algorithms are an essential part of MPI, allowing application...
research
10/22/2017

Lightweight MPI Communicators with Applications to Perfectly Balanced Schizophrenic Quicksort

MPI uses the concept of communicators to connect groups of processes. It...
research
04/20/2020

A Generalization of the Allreduce Operation

Allreduce is one of the most frequently used MPI collective operations, ...
research
04/23/2020

Accurate runtime selection of optimal MPI collective algorithms using analytical performance modelling

The performance of collective operations has been a critical issue since...
research
06/23/2020

Optimised allgatherv, reduce_scatter and allreduce communication in message-passing systems

Collective communications, namely the patterns allgatherv, reduce_scatte...
research
11/28/2022

RAMP: A Flat Nanosecond Optical Network and MPI Operations for Distributed Deep Learning Systems

Distributed deep learning (DDL) systems strongly depend on network perfo...

Please sign up or login with your details

Forgot password? Click here to reset