DeepAI AI Chat
Log In Sign Up

Supporting Massive DLRM Inference Through Software Defined Memory

10/21/2021
by   Ehsan K. Ardestani, et al.
Facebook
0

Deep Learning Recommendation Models (DLRM) are widespread, account for a considerable data center footprint, and grow by more than 1.5x per year. With model size soon to be in terabytes range, leveraging Storage ClassMemory (SCM) for inference enables lower power consumption and cost. This paper evaluates the major challenges in extending the memory hierarchy to SCM for DLRM, and presents different techniques to improve performance through a Software Defined Memory. We show how underlying technologies such as Nand Flash and 3DXP differentiate, and relate to real world scenarios, enabling from 5 power savings.

READ FULL TEXT

page 4

page 7

07/31/2018

Data Center Interconnects at 400G and Beyond

Current trends in Data Center Interconnectivity are considered in the li...
05/24/2016

An Analysis of Deep Neural Network Models for Practical Applications

Since the emergence of Deep Neural Networks (DNNs) as a prominent techni...
09/11/2018

Characterizing the Power Cost of Virtualization Environments

Virtualization is a key building block of next-generation mobile network...
09/01/2020

Survey of Machine Learning Accelerators

New machine learning accelerators are being announced and released each ...
04/18/2020

Efficient Synthesis of Compact Deep Neural Networks

Deep neural networks (DNNs) have been deployed in myriad machine learnin...
12/15/2017

Improved Ahead-of-Time Compilation of Stack-Based JVM Bytecode on Resource-Constrained Devices

Many virtual machines exist for sensor nodes with only a few KB RAM and ...
01/02/2020

SmartWatts: Self-Calibrating Software-Defined Power Meter for Containers

Fine-grained power monitoring of software activities becomes unavoidable...