research
∙
08/13/2023
Token-Scaled Logit Distillation for Ternary Weight Generative Language Models
Generative Language Models (GLMs) have shown impressive performance in t...
share
research
∙
05/12/2023
PillarAcc: Sparse PointPillars Accelerator for Real-Time Point Cloud 3D Object Detection on Edge Devices
3D object detection using point cloud (PC) data is vital for autonomous ...
research
∙
02/23/2023
Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers
Pre-trained Transformer models such as BERT have shown great success in ...
research
∙
01/29/2023
Exploring Attention Map Reuse for Efficient Transformer Neural Networks
Transformer-based deep neural networks have achieved great success in va...
research
∙
12/21/2022
Automatic Network Adaptation for Ultra-Low Uniform-Precision Quantization
Uniform-precision neural network quantization has gained popularity sinc...
research
∙
11/20/2022
Understanding and Improving Knowledge Distillation for Quantization-Aware Training of Large Transformer Encoders
Knowledge distillation (KD) has been a ubiquitous method for model compr...
research
∙
02/11/2022
Learning from distinctive candidates to optimize reduced-precision convolution program on tensor cores
Convolution is one of the fundamental operations of deep neural networks...
research
∙
12/03/2021
NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference
Non-linear operations such as GELU, Layer normalization, and Softmax are...
research
∙
10/07/2021
Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling
While Transformer-based models have shown impressive language modeling p...
research
∙
01/04/2021
Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead
Machine Learning (ML) techniques have been rapidly adopted by smart Cybe...
research
∙
09/30/2020
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks
The quantization of deep neural networks (QDNNs) has been actively studi...
research
∙
01/19/2019
Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks
Efforts to reduce the numerical precision of computations in deep learni...
research
∙
12/19/2018
Training Deep Neural Networks with 8-bit Floating Point Numbers
The state-of-the-art hardware platforms for training Deep Neural Network...
research
∙
11/05/2018
Workload-aware Automatic Parallelization for Multi-GPU DNN Training
Deep neural networks (DNNs) have emerged as successful solutions for var...
research
∙
07/17/2018
Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)
Deep learning algorithms achieve high classification accuracy at the exp...
research
∙
05/16/2018
PACT: Parameterized Clipping Activation for Quantized Neural Networks
Deep learning algorithms achieve high classification accuracy at the exp...
research
∙
12/07/2017