DeepAI AI Chat
Log In Sign Up

Learned Step Size Quantization

by   Steven K. Esser, et al.

We present here Learned Step Size Quantization, a method for training deep networks such that they can run at inference time using low precision integer matrix multipliers, which offer power and space advantages over high precision alternatives. The essence of our approach is to learn the step size parameter of a uniform quantizer by backpropagation of the training loss, applying a scaling factor to its learning rate, and computing its associated loss gradient by ignoring the discontinuity present in the quantizer. This quantization approach can be applied to activations or weights, using different levels of precision as needed for a given system, and requiring only a simple modification of existing training code. As demonstrated on the ImageNet dataset, our approach achieves better accuracy than all previous published methods for creating quantized networks on several ResNet network architectures at 2-, 3- and 4-bits of precision.


UNIQ: Uniform Noise Injection for the Quantization of Neural Networks

We present a novel method for training deep neural network amenable to i...

LG-LSQ: Learned Gradient Linear Symmetric Quantization

Deep neural networks with lower precision weights and operations at infe...

Differentiable Quantization of Deep Neural Networks

We propose differentiable quantization (DQ) for efficient deep neural ne...

Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point

We propose a cluster-based quantization method to convert pre-trained fu...

Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference

To realize the promise of ubiquitous embedded deep network inference, it...

Autoencoder based image compression: can the learning be quantization independent?

This paper explores the problem of learning transforms for image compres...

Optimized learned entropy coding parameters for practical neural-based image and video compression

Neural-based image and video codecs are significantly more power-efficie...

Code Repositories


Unofficial implementation of LSQ-Net, a neural network quantization framework

view repo


This is an implementation of YOLO using LSQ network quantization method.

view repo


This is an unofficial implementation of the paper - LEARNED STEP SIZE QUANTIZATION at ICLR 2020

view repo


FakeQuantize with Learned Step Size(LSQ+) as Observer in PyTorch

view repo