research
∙
07/17/2023
Cumulative Spatial Knowledge Distillation for Vision Transformers
Distilling knowledge from convolutional neural networks (CNNs) is a doub...
share
research
∙
07/17/2023
DOT: A Distillation-Oriented Trainer
Knowledge distillation transfers knowledge from a large model to a small...
research
∙
03/20/2023
Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data
Semi-supervised learning (SSL) has attracted enormous attention due to i...
research
∙
07/26/2022
Efficient One Pass Self-distillation with Zipf's Label Smoothing
Self-distillation exploits non-uniform soft supervision from itself duri...
research
∙
03/16/2022
Decoupled Knowledge Distillation
State-of-the-art distillation methods are mainly based on distilling dee...
research
∙
08/04/2020