Sigma-Delta and Distributed Noise-Shaping Quantization Methods for Random Fourier Features

06/04/2021
by   Jinjie Zhang, et al.
0

We propose the use of low bit-depth Sigma-Delta and distributed noise-shaping methods for quantizing the Random Fourier features (RFFs) associated with shift-invariant kernels. We prove that our quantized RFFs – even in the case of 1-bit quantization – allow a high accuracy approximation of the underlying kernels, and the approximation error decays at least polynomially fast as the dimension of the RFFs increases. We also show that the quantized RFFs can be further compressed, yielding an excellent trade-off between memory use and accuracy. Namely, the approximation error now decays exponentially as a function of the bits used. Moreover, we empirically show by testing the performance of our methods on several machine learning tasks that our method compares favorably to other state of the art quantization methods in this context.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/26/2019

Smart Ternary Quantization

Neural network models are resource hungry. Low bit quantization such as ...
research
10/31/2018

Low-Precision Random Fourier Features for Memory-Constrained Kernel Approximation

We investigate how to train kernel approximation methods that generalize...
research
06/27/2023

On the reconstruction of bandlimited signals from random samples quantized via noise-shaping

Noise-shaping quantization techniques are widely used for converting ban...
research
01/20/2020

Adaptive Dithering Using Curved Markov-Gaussian Noise in the Quantized Domain for Mapping SDR to HDR Image

High Dynamic Range (HDR) imaging is gaining increased attention due to i...
research
09/11/2023

Quantized Fourier and Polynomial Features for more Expressive Tensor Network Models

In the context of kernel machines, polynomial and Fourier features are c...
research
04/14/2020

Breaking the waves: asymmetric random periodic features for low-bitrate kernel machines

Many signal processing and machine learning applications are built from ...
research
03/08/2018

Analysis of Decimation on Finite Frames with Sigma-Delta Quantization

In Analog-to-digital (A/D) conversion on oversampled bandlimited functio...

Please sign up or login with your details

Forgot password? Click here to reset