Pipeflow: An Efficient Task-Parallel Pipeline Programming Framework using Modern C++

02/01/2022
by   Cheng-Hsiang Chiu, et al.
0

Pipeline is a fundamental parallel programming pattern. Mainstream pipeline programming frameworks count on data abstractions to perform pipeline scheduling. This design is convenient for data-centric pipeline applications but inefficient for algorithms that only exploit task parallelism in pipeline. As a result, we introduce a new task-parallel pipeline programming framework called Pipeflow. Pipeflow does not design yet another data abstraction but focuses on the pipeline scheduling itself, enabling more efficient implementation of task-parallel pipeline algorithms than existing frameworks. We have evaluated Pipeflow on both micro-benchmarks and real-world applications. As an example, Pipeflow outperforms oneTBB 24 a VLSI placement and a timing analysis workloads that adopt pipeline parallelism to speed up runtimes, respectively.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/21/2020

torchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models

We design and implement a ready-to-use library in PyTorch for performing...
research
04/02/2017

Structured Parallel Programming for Monte Carlo Tree Search

In this paper, we present a new algorithm for parallel Monte Carlo tree ...
research
10/09/2019

PipeMare: Asynchronous Pipeline Parallel DNN Training

Recently there has been a flurry of interest around using pipeline paral...
research
04/22/2023

Pipeline MoE: A Flexible MoE Implementation with Pipeline Parallelism

The Mixture of Experts (MoE) model becomes an important choice of large ...
research
06/01/2023

Bridging Control-Centric and Data-Centric Optimization

With the rise of specialized hardware and new programming languages, cod...
research
06/03/2018

An Efficient Dispatcher for Large Scale GraphProcessing on OpenCL-based FPGAs

High parallel framework has been proved to be very suitable for graph pr...
research
07/12/2022

The OpenMP Cluster Programming Model

Despite the various research initiatives and proposed programming models...

Please sign up or login with your details

Forgot password? Click here to reset