Pyramid Vector Quantization for Deep Learning

04/10/2017
by   Vincenzo Liguori, et al.
0

This paper explores the use of Pyramid Vector Quantization (PVQ) to reduce the computational cost for a variety of neural networks (NNs) while, at the same time, compressing the weights that describe them. This is based on the fact that the dot product between an N dimensional vector of real numbers and an N dimensional PVQ vector can be calculated with only additions and subtractions and one multiplication. This is advantageous since tensor products, commonly used in NNs, can be re-conduced to a dot product or a set of dot products. Finally, it is stressed that any NN architecture that is based on an operation that can be re-conduced to a dot product can benefit from the techniques described here.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/24/2019

Pyramid Vector Quantization and Bit Level Sparsity in Weights for Efficient Neural Networks Inference

This paper discusses three basic blocks for the inference of convolution...
research
03/30/2016

Vector Quantization for Machine Vision

This paper shows how to reduce the computational cost for a variety of c...
research
05/24/2018

Backpropagation with N-D Vector-Valued Neurons Using Arbitrary Bilinear Products

Vector-valued neural learning has emerged as a promising direction in de...
research
02/09/2017

Energy Saving Additive Neural Network

In recent years, machine learning techniques based on neural networks fo...
research
11/12/2019

Norm-Explicit Quantization: Improving Vector Quantization for Maximum Inner Product Search

Vector quantization (VQ) techniques are widely used in similarity search...
research
06/20/2023

Scalable Evaluation of Hadamard Products with Tensor Product Basis for Entropy-Stable High-Order Methods

A sum-factorization form for the evaluation of Hadamard products with a ...
research
06/29/2017

Music Signal Processing Using Vector Product Neural Networks

We propose a novel neural network model for music signal processing usin...

Please sign up or login with your details

Forgot password? Click here to reset