Benchmarking Contemporary Deep Learning Hardware and Frameworks:A Survey of Qualitative Metrics

07/05/2019
by   Wei Dai, et al.
0

This paper surveys benchmarking principles, machine learning devices including GPUs, FPGAs, and ASICs, and deep learning software frameworks. It also reviews these technologies with respect to benchmarking from the perspectives of a 6-metric approach to frameworks and an 11-metric approach to hardware platforms. Because MLPerf is a benchmark organization working with industry and academia, and offering deep learning benchmarks that evaluate training and inference on deep learning hardware devices, the survey also mentions MLPerf benchmark results, benchmark metrics, datasets, deep learning frameworks and algorithms. We summarize seven benchmarking principles, differential characteristics of mainstream AI devices, and qualitative comparison of deep learning hardware and frameworks.

READ FULL TEXT

page 1

page 4

research
07/05/2019

Benchmarking Deep Learning Hardware and Frameworks: Qualitative Metrics

Previous survey papers offer knowledge of deep learning hardware devices...
research
10/23/2017

BENCHIP: Benchmarking Intelligence Processors

The increasing attention on deep learning has tremendously spurred the d...
research
10/02/2019

MLPerf Training Benchmark

Machine learning is experiencing an explosion of software and hardware s...
research
10/12/2021

Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking for Everyone

Obtaining standardized crowdsourced benchmark of computational methods i...
research
07/05/2019

Qualitative Benchmarking of Deep Learning Hardware and Frameworks: Review and Tutorial

Previous survey papers offer knowledge of deep learning hardware devices...
research
01/29/2019

A Modular Benchmarking Infrastructure for High-Performance and Reproducible Deep Learning

We introduce Deep500: the first customizable benchmarking infrastructure...
research
12/03/2020

MLPerf Mobile Inference Benchmark: Why Mobile AI Benchmarking Is Hard and What to Do About It

MLPerf Mobile is the first industry-standard open-source mobile benchmar...

Please sign up or login with your details

Forgot password? Click here to reset