DeepAI AI Chat
Log In Sign Up

A Layer-wise Adversarial-aware Quantization Optimization for Improving Robustness

by   Chang Song, et al.

Neural networks are getting better accuracy with higher energy and computational cost. After quantization, the cost can be greatly saved, and the quantized models are more hardware friendly with acceptable accuracy loss. On the other hand, recent research has found that neural networks are vulnerable to adversarial attacks, and the robustness of a neural network model can only be improved with defense methods, such as adversarial training. In this work, we find that adversarially-trained neural networks are more vulnerable to quantization loss than plain models. To minimize both the adversarial and the quantization losses simultaneously and to make the quantized model robust, we propose a layer-wise adversarial-aware quantization method, using the Lipschitz constant to choose the best quantization parameter settings for a neural network. We theoretically derive the losses and prove the consistency of our metric selection. The experiment results show that our method can effectively and efficiently improve the robustness of quantized adversarially-trained neural networks.


Improving Adversarial Robustness in Weight-quantized Neural Networks

Neural networks are getting deeper and more computation-intensive nowada...

Defensive Quantization: When Efficiency Meets Robustness

Neural network quantization is becoming an industry standard to efficien...

ODG-Q: Robust Quantization via Online Domain Generalization

Quantizing neural networks to low-bitwidth is important for model deploy...

Quantization-aware Interval Bound Propagation for Training Certifiably Robust Quantized Neural Networks

We study the problem of training and certifying adversarially robust qua...

QNNRepair: Quantized Neural Network Repair

We present QNNRepair, the first method in the literature for repairing q...

RobustMQ: Benchmarking Robustness of Quantized Models

Quantization has emerged as an essential technique for deploying deep ne...

Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs

Quantized neural networks (NN) are the common standard to efficiently de...