End-to-end 100-TOPS/W Inference With Analog In-Memory Computing: Are We There Yet?

09/03/2021

∙

In-Memory Acceleration (IMA) promises major efficiency improvements in deep neural network (DNN) inference, but challenges remain in the integration of IMA within a digital system. We propose a heterogeneous architecture coupling 8 RISC-V cores with an IMA in a shared-memory cluster, analyzing the benefits and trade-offs of in-memory computing on the realistic use case of a MobileNetV2 bottleneck layer. We explore several IMA integration strategies, analyzing performance, area, and energy efficiency. We show that while pointwise layers achieve significant speed-ups over software implementation, on depthwise layer the inability to efficiently map parameters on the accelerator leads to a significant trade-off between throughput and area. We propose a hybrid solution where pointwise convolutions are executed on IMA while depthwise on the cluster cores, achieving a speed-up of 3x over SW execution while saving 50 when compared to an all-in IMA solution with similar performance.

READ FULL TEXT

End-to-end 100-TOPS/W Inference With Analog In-Memory Computing: Are We There Yet?

A Heterogeneous In-Memory Computing Cluster For Flexible End-to-End Inference of Real-World Deep Neural Networks

End-to-End DNN Inference on a Massively Parallel Analog In Memory Computing Architecture

ENOS: Energy-Aware Network Operator Search for Hybrid Digital and Compute-in-Memory DNN Accelerators

Shared Memory-contention-aware Concurrent DNN Execution for Diversely Heterogeneous System-on-Chips

Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics

Are We There Yet? Product Quantization and its Hardware Acceleration

Stream Semantic Registers: A Lightweight RISC-V ISA Extension Achieving Full Compute Utilization in Single-Issue Cores

End-to-end 100-TOPS/W Inference With Analog In-Memory Computing: Are We There Yet?

Related Research

A Heterogeneous In-Memory Computing Cluster For Flexible End-to-End Inference of Real-World Deep Neural Networks

End-to-End DNN Inference on a Massively Parallel Analog In Memory Computing Architecture

ENOS: Energy-Aware Network Operator Search for Hybrid Digital and Compute-in-Memory DNN Accelerators

Shared Memory-contention-aware Concurrent DNN Execution for Diversely Heterogeneous System-on-Chips

Jointly Optimizing Preprocessing and Inference for DNN-based Visual Analytics

Are We There Yet? Product Quantization and its Hardware Acceleration

Stream Semantic Registers: A Lightweight RISC-V ISA Extension Achieving Full Compute Utilization in Single-Issue Cores