Breaking the Memory Wall for AI Chip with a New Dimension

09/28/2020
by   Eugene Tam, et al.
56

Recent advancements in deep learning have led to the widespread adoption of artificial intelligence (AI) in applications such as computer vision and natural language processing. As neural networks become deeper and larger, AI modeling demands outstrip the capabilities of conventional chip architectures. Memory bandwidth falls behind processing power. Energy consumption comes to dominate the total cost of ownership. Currently, memory capacity is insufficient to support the most advanced NLP models. In this work, we present a 3D AI chip, called Sunrise, with near-memory computing architecture to address these three challenges. This distributed, near-memory computing architecture allows us to tear down the performance-limiting memory wall with an abundance of data bandwidth. We achieve the same level of energy efficiency on 40nm technology as competing chips on 7nm technology. By moving to similar technologies as other AI chips, we project to achieve more than ten times the energy efficiency, seven times the performance of the current state-of-the-art chips, and twenty times of memory capacity as compared with the best chip in each benchmark.

READ FULL TEXT

page 2

page 3

page 4

05/19/2020

In-memory Implementation of On-chip Trainable and Scalable ANN for AI/ML Applications

Traditional von Neumann architecture based processors become inefficient...
05/28/2019

Towards Efficient Neural Networks On-a-chip: Joint Hardware-Algorithm Approaches

Machine learning algorithms have made significant advances in many appli...
12/07/2019

Dissecting the Graphcore IPU Architecture via Microbenchmarking

This report focuses on the architecture and performance of the Intellige...
08/17/2021

Edge AI without Compromise: Efficient, Versatile and Accurate Neurocomputing in Resistive Random-Access Memory

Realizing today's cloud-level artificial intelligence functionalities di...
03/26/2021

RCT: Resource Constrained Training for Edge AI

Neural networks training on edge terminals is essential for edge AI comp...
06/12/2016

Application-Driven Near-Data Processing for Similarity Search

Similarity search is a key to a variety of applications including conten...