DeepAI AI Chat
Log In Sign Up

UCX Programming Interface for Remote Function Injection and Invocation

by   Luis E. Peña, et al.
Los Alamos National Laboratory
Stony Brook University

Network library APIs have historically been developed with the emphasis on data movement, placement, and communication semantics. Many communication semantics are available across a large variety of network libraries, such as send-receive, data streaming, put/get/atomic, RPC, active messages, collective communication, etc. In this work we introduce new compute and data movement APIs that overcome the constraints of the single-program, multiple-data (SPMD) programming model by allowing users to send binary executable code between processing elements. Our proof-of-concept implementation of the API is based on the UCX communication framework and leverages the RDMA network for fast compute migration. We envision the API being used to dispatch user functions from a host CPU to a SmartNIC (DPU), computational storage drive (CSD), or remote servers. In addition, the API can be used by large-scale irregular applications (such as semantic graph analysis), composed of many coordinating tasks operating on a data set so big that it has to be stored on many physical devices. In such cases, it may be more efficient to dynamically choose where code runs as the applications progresses.


page 1

page 2

page 3

page 4


Two-Chains: High Performance Framework for Function Injection and Execution

Some important problems, such as semantic graph analysis, require large-...

Bring the BitCODE – Moving Compute and Data in Distributed Heterogeneous Systems

In this paper, we present a framework for moving compute and data betwee...

SOAR: A Synthesis Approach for Data Science API Refactoring

With the growth of the open-source data science community, both the numb...

Recommending Analogical APIs via Knowledge Graph Embedding

Library migration, which re-implements the same software behavior by usi...

Weld: Rethinking the Interface Between Data-Intensive Applications

Data analytics applications combine multiple functions from different li...

Safe and Efficient Remote Application Code Execution on Disaggregated NVM Storage with eBPF

With rapid improvements in NVM storage devices, the performance bottlene...

P4RROT: Generating P4 Code for the Application Layer

Throughput and latency critical applications could often benefit of perf...