research
∙
06/12/2023
Efficient Quantization-aware Training with Adaptive Coreset Selection
The expanding model size and computation of deep neural networks (DNNs) ...
share
research
∙
06/08/2023
Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts
Weight-sharing supernet has become a vital component for performance est...
research
∙
06/02/2023
Binary and Ternary Natural Language Generation
Ternary and binary neural networks enable multiplication-free computatio...
research
∙
05/29/2023
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
Several post-training quantization methods have been applied to large la...
research
∙
03/22/2023
EBSR: Enhanced Binary Neural Network for Image Super-Resolution
While the performance of deep convolutional neural networks for image su...
research
∙
02/04/2023
Oscillation-free Quantization for Low-bit Vision Transformers
Weight oscillation is an undesirable side effect of quantization-aware t...
research
∙
05/25/2022
BiT: Robustly Binarized Multi-distilled Transformer
Modern pre-trained transformers have rapidly advanced the state-of-the-a...
research
∙
01/03/2022
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space
This paper explores the feasibility of finding an optimal sub-model from...
research
∙
12/03/2021
Data-Free Neural Architecture Search via Recursive Label Calibration
This paper aims to explore the feasibility of neural architecture search...
research
∙
11/29/2021
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
The nonuniform quantization strategy for compressing neural networks usu...
research
∙
11/09/2021
Sliced Recursive Transformer
We present a neat yet effective recursive operation on vision transforme...
research
∙
06/21/2021
How Do Adam and Training Strategies Help BNNs Optimization?
The best performing Binary Neural Networks (BNNs) are usually attained u...
research
∙
04/16/2021
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization
Batch normalization (BN) is a key facilitator and considered essential f...
research
∙
04/01/2021
Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study
This work aims to empirically clarify a recently discovered perspective ...
research
∙
02/17/2021
S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration
Previous studies dominantly target at self-supervised learning on real-v...
research
∙
02/08/2021
Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning
The goal of few-shot learning is to learn a classifier that can recogniz...
research
∙
11/29/2020
Conditional Link Prediction of Category-Implicit Keypoint Detection
Keypoints of objects reflect their concise abstractions, while the corre...
research
∙
07/04/2020
Weight-dependent Gates for Network Pruning
In this paper, we propose a simple and effective network pruning framewo...
research
∙
05/18/2020
Joint Multi-Dimension Pruning
We present joint multi-dimension pruning (named as JointPruning), a new ...
research
∙
05/13/2020
Binarizing MobileNet via Evolution-based Searching
Binary Neural Networks (BNNs), known to be one among the effectively com...
research
∙
03/29/2020
Attentive CutMix: An Enhanced Data Augmentation Approach for Deep Learning Based Image Classification
Convolutional neural networks (CNN) are capable of learning robust repre...
research
∙
03/11/2020
Rethinking Image Mixture for Unsupervised Visual Representation Learning
In supervised learning, smoothing label/prediction distribution in neura...
research
∙
03/07/2020
ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions
In this paper, we propose several ideas for enhancing a binary network t...
research
∙
06/05/2019
Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization
Optimization of Binarized Neural Networks (BNNs) currently relies on rea...
research
∙
03/31/2019
Single Path One-Shot Neural Architecture Search with Uniform Sampling
One-shot method is a powerful Neural Architecture Search (NAS) framework...
research
∙
03/25/2019
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
In this paper, we propose a novel meta learning approach for automatic c...
research
∙
11/04/2018
Bi-Real Net: Binarizing Deep Network Towards Real-Network Performance
In this paper, we study 1-bit convolutional neural networks (CNNs), of w...
research
∙
08/01/2018
