Hard Sample Matters a Lot in Zero-Shot Quantization

03/24/2023
by   Huantong Li, et al.
0

Zero-shot quantization (ZSQ) is promising for compressing and accelerating deep neural networks when the data for training full-precision models are inaccessible. In ZSQ, network quantization is performed using synthetic samples, thus, the performance of quantized models depends heavily on the quality of synthetic samples. Nonetheless, we find that the synthetic samples constructed in existing ZSQ methods can be easily fitted by models. Accordingly, quantized models obtained by these methods suffer from significant performance degradation on hard samples. To address this issue, we propose HArd sample Synthesizing and Training (HAST). Specifically, HAST pays more attention to hard samples when synthesizing samples and makes synthetic samples hard to fit when training quantized models. HAST aligns features extracted by full-precision and quantized models to ensure the similarity between features extracted by these two models. Extensive experiments show that HAST significantly outperforms existing ZSQ methods, achieving performance comparable to models that are quantized with real data.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/09/2022

Genie: Show Me the Data for Quantization

Zero-shot quantization is a promising approach for developing lightweigh...
research
03/29/2021

Zero-shot Adversarial Quantization

Model quantization is a promising approach to compress deep neural netwo...
research
03/31/2022

It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teacher

Model quantization is considered as a promising method to greatly reduce...
research
11/17/2021

IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization

Learning to synthesize data has emerged as a promising direction in zero...
research
03/01/2021

Diversifying Sample Generation for Accurate Data-Free Quantization

Quantization has emerged as one of the most prevalent approaches to comp...
research
03/13/2023

Adaptive Data-Free Quantization

Data-free quantization (DFQ) recovers the performance of quantized netwo...
research
01/21/2021

Generative Zero-shot Network Quantization

Convolutional neural networks are able to learn realistic image priors f...

Please sign up or login with your details

Forgot password? Click here to reset