A MAC-less Neural Inference Processor Supporting Compressed, Variable Precision Weights

12/10/2020

∙

This paper introduces two architectures for the inference of convolutional neural networks (CNNs). Both architectures exploit weight sparsity and compression to reduce computational complexity and bandwidth. The first architecture uses multiply-accumulators (MACs) but avoids unnecessary multiplications by skipping zero weights. The second architecture exploits weight sparsity at the level of their bit representation by substituting resource-intensive MACs with much smaller Bit Layer Multiply Accumulators (BLMACs). The use of BLMACs also allows variable precision weights as variable size integers and even floating points. Some details of an implementation of the second architecture are given. Weight compression with arithmetic coding is also discussed as well as bandwidth implications. Finally, some implementation results for a pathfinder design and various technologies are presented.

READ FULL TEXT

A MAC-less Neural Inference Processor Supporting Compressed, Variable Precision Weights

Bit-serial Weight Pools: Compression and Arbitrary Precision Execution of Neural Networks on Resource Constrained Processors

HOBFLOPS CNNs: Hardware Optimized Bitsliced Floating-Point Operations Convolutional Neural Networks

Bit-balance: Model-Hardware Co-design for Accelerating NNs by Exploiting Bit-level Sparsity

Neural gradients are lognormally distributed: understanding sparse and quantized training

Pyramid Vector Quantization and Bit Level Sparsity in Weights for Efficient Neural Networks Inference

Learnable Mixed-precision and Dimension Reduction Co-design for Low-storage Activation

At-Scale Evaluation of Weight Clustering to Enable Energy-Efficient Object Detection

A MAC-less Neural Inference Processor Supporting Compressed, Variable Precision Weights

Related Research

Bit-serial Weight Pools: Compression and Arbitrary Precision Execution of Neural Networks on Resource Constrained Processors

HOBFLOPS CNNs: Hardware Optimized Bitsliced Floating-Point Operations Convolutional Neural Networks

Bit-balance: Model-Hardware Co-design for Accelerating NNs by Exploiting Bit-level Sparsity

Neural gradients are lognormally distributed: understanding sparse and quantized training

Pyramid Vector Quantization and Bit Level Sparsity in Weights for Efficient Neural Networks Inference

Learnable Mixed-precision and Dimension Reduction Co-design for Low-storage Activation

At-Scale Evaluation of Weight Clustering to Enable Energy-Efficient Object Detection