Low-Cost Floating-Point Processing in ReRAM for Scientific Computing

11/06/2020
by   Linghao Song, et al.
0

We propose ReFloat, a principled approach for low-cost floating-point processing in ReRAM. The exponent offsets based on a base are stored by a flexible and fine-grained floating-point number representation. The key motivation is that, while the number of exponent bits must be reduced due to the exponential relation to the computation latency and hardware cost, the convergence still requires sufficient accuracy for exponents. Our design reconciles the conflicting goals by storing the exponent offsets from a common base among matrix values in a block, which is the granularity of computation in ReRAM. Due to the value locality, the differences among the exponents in a block are small, thus the offsets require much less number of bits to represent exponents. In essence, ReFloat enables the principled local fine-tuning of floating-point representation. Based on the idea, we define a flexible ReFloat format that specifies matrix block size, and the number of bits for exponent and fraction. To determine the base for each block, we propose an optimization method that minimizes the difference between the exponents of the original matrix block and the converted block. We develop the conversion scheme from default double-precision floating-point format to ReFloat format, the computation procedure, and the low-cost floating-point processing architecture in ReRAM.

READ FULL TEXT

Authors

page 9

04/15/2021

All-You-Can-Fit 8-Bit Flexible Floating-Point Format for Accurate and Memory-Efficient Inference of Deep Neural Networks

Modern deep neural network (DNN) models generally require a huge amount ...
02/04/2021

EFloat: Entropy-coded Floating Point Format for Deep Learning

We describe the EFloat floating-point number format with 4 to 6 addition...
04/10/2021

Fixed-Posit: A Floating-Point Representation for Error-Resilient Applications

Today, almost all computer systems use IEEE-754 floating point to repres...
07/09/2019

Template-Based Posit Multiplication for Training and Inferring in Neural Networks

The posit number system is arguably the most promising and discussed top...
04/09/2020

Efficient Reasoning About Stencil Programs Using Selective Direct Evaluation

We present FPDetect, a low overhead approach for detecting logical error...
04/09/2020

FPDetect: Efficient Reasoning About Stencil Programs Using Selective Direct Evaluation

We present FPDetect, a low overhead approach for detecting logical error...
08/30/2020

Floating-Point Multiplication Using Neuromorphic Computing

Neuromorphic computing describes the use of VLSI systems to mimic neuro-...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.