Deep Learning Training on the Edge with Low-Precision Posits

07/30/2019

∙

Recently, the posit numerical format has shown promise for DNN data representation and compute with ultra-low precision ([5..8]-bit). However, majority of studies focus only on DNN inference. In this work, we propose DNN training using posits and compare with the floating point training. We evaluate on both MNIST and Fashion MNIST corpuses, where 16-bit posits outperform 16-bit floating point for end-to-end DNN training.

READ FULL TEXT

Deep Learning Training on the Edge with Low-Precision Posits

Cheetah: Mixed Low-Precision Hardware Software Co-Design Framework for DNNs on the Edge

Minimizing Area and Energy of Deep Learning Hardware Design Using Collective Low Precision and Structured Compression

BRAMAC: Compute-in-BRAM Architectures for Multiply-Accumulate on FPGAs

Reduced Precision Floating-Point Optimization for Deep Neural Network On-Device Learning on MicroControllers

Accuracy Boosters: Epoch-Driven Mixed-Mantissa Block Floating-Point for DNN Training

Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode

PocketNN: Integer-only Training and Inference of Neural Networks via Direct Feedback Alignment and Pocket Activations in Pure C++

Deep Learning Training on the Edge with Low-Precision Posits

Related Research

Cheetah: Mixed Low-Precision Hardware Software Co-Design Framework for DNNs on the Edge

Minimizing Area and Energy of Deep Learning Hardware Design Using Collective Low Precision and Structured Compression

BRAMAC: Compute-in-BRAM Architectures for Multiply-Accumulate on FPGAs

Reduced Precision Floating-Point Optimization for Deep Neural Network On-Device Learning on MicroControllers

Accuracy Boosters: Epoch-Driven Mixed-Mantissa Block Floating-Point for DNN Training

Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode

PocketNN: Integer-only Training and Inference of Neural Networks via Direct Feedback Alignment and Pocket Activations in Pure C++