MLPerf Inference Benchmark

11/06/2019
by   Vijay Janapa Reddi, et al.
27

Machine-learning (ML) hardware and software system demand is burgeoning. Driven by ML applications, the number of different ML inference systems has exploded. Over 100 organizations are building ML inference chips, and the systems that incorporate existing models span at least three orders of magnitude in power consumption and four orders of magnitude in performance; they range from embedded devices to data-center solutions. Fueling the hardware are a dozen or more software frameworks and libraries. The myriad combinations of ML hardware and ML software make assessing ML-system performance in an architecture-neutral, representative, and reproducible manner challenging. There is a clear need for industry-wide standard ML benchmarking and evaluation criteria. MLPerf Inference answers that call. Driven by more than 30 organizations as well as more than 200 ML engineers and practitioners, MLPerf implements a set of rules and practices to ensure comparability across systems with wildly differing architectures. In this paper, we present the method and design principles of the initial MLPerf Inference release. The first call for submissions garnered more than 600 inference-performance measurements from 14 organizations, representing over 30 systems that show a range of capabilities.

READ FULL TEXT
research
02/04/2020

Adversarial Machine Learning – Industry Perspectives

Based on interviews with 28 organizations, we found that industry practi...
research
09/12/2023

The Grand Illusion: The Myth of Software Portability and Implications for ML Progress

Pushing the boundaries of machine learning often requires exploring diff...
research
04/20/2016

CLAASIC: a Cortex-Inspired Hardware Accelerator

This work explores the feasibility of specialized hardware implementing ...
research
02/02/2021

Pick the Right Edge Device: Towards Power and Performance Estimation of CUDA-based CNNs on GPGPUs

The emergence of Machine Learning (ML) as a powerful technique has been ...
research
10/26/2022

Desiderata for next generation of ML model serving

Inference is a significant part of ML software infrastructure. Despite t...
research
08/24/2019

Neural Network Inference on Mobile SoCs

The ever-increasing demand from mobile Machine Learning (ML) application...
research
05/09/2022

Towards a multi-stakeholder value-based assessment framework for algorithmic systems

In an effort to regulate Machine Learning-driven (ML) systems, current a...

Please sign up or login with your details

Forgot password? Click here to reset