Predicting Performance of Heterogeneous AI Systems with Discrete-Event Simulations

04/07/2022
by   Vyacheslav Zhdanovskiy, et al.
0

In recent years, artificial intelligence (AI) technologies have found industrial applications in various fields. AI systems typically possess complex software and heterogeneous CPU/GPU hardware architecture, making it difficult to answer basic questions considering performance evaluation and software optimization. Where is the bottleneck impeding the system? How does the performance scale with the workload? How the speed-up of a specific module would contribute to the whole system? Finding the answers to these questions through experiments on the real system could require a lot of computational, human, financial, and time resources. A solution to cut these costs is to use a fast and accurate simulation model preparatory to implementing anything in the real system. In this paper, we propose a discrete-event simulation model of a high-load heterogeneous AI system in the context of video analytics. Using the proposed model, we estimate: 1) the performance scalability with the increasing number of cameras; 2) the performance impact of integrating a new module; 3) the performance gain from optimizing a single module. We show that the performance estimation accuracy of the proposed model is higher than 90 also demonstrate, that the considered system possesses a counter-intuitive relationship between workload and performance, which nevertheless is correctly inferred by the proposed simulation model.

READ FULL TEXT
research
06/26/2020

Could regulating the creators deliver trustworthy AI?

Is a new regulated profession, such as Artificial Intelligence (AI) Arch...
research
05/20/2015

Towards a Simulation-Based Programming Paradigm for AI applications

We present initial ideas for a programming paradigm based on simulation ...
research
12/07/2022

SAIH: A Scalable Evaluation Methodology for Understanding AI Performance Trend on HPC Systems

Novel artificial intelligence (AI) technology has expedited various scie...
research
08/17/2020

AIPerf: Automated machine learning as an AI-HPC benchmark

The plethora of complex artificial intelligence (AI) algorithms and avai...
research
08/21/2019

How Good is Artificial Intelligence at Automatically Answering Consumer Questions Related to Alzheimer's Disease?

Alzheimer's Disease (AD) is the most common type of dementia, comprising...
research
03/21/2023

AI-in-the-Loop – The impact of HMI in AI-based Application

Artificial intelligence (AI) and human-machine interaction (HMI) are two...
research
03/17/2023

VPU-EM: An Event-based Modeling Framework to Evaluate NPU Performance and Power Efficiency at Scale

State-of-art NPUs are typically architected as a self-contained sub-syst...

Please sign up or login with your details

Forgot password? Click here to reset