research
∙
07/31/2023
HAGRID: A Human-LLM Collaborative Dataset for Generative Information-Seeking with Attribution
The rise of large language models (LLMs) had a transformative impact on ...
share
research
∙
01/27/2023
Improved knowledge distillation by utilizing backward pass knowledge in neural networks
Knowledge distillation (KD) is one of the prominent techniques for model...
research
∙
12/12/2022
Continuation KD: Improved Knowledge Distillation through the Lens of Continuation Optimization
Knowledge Distillation (KD) has been extensively used for natural langua...
research
∙
05/25/2022
Towards Understanding Label Regularization for Fine-tuning Pre-trained Language Models
Knowledge Distillation (KD) is a prominent neural model compression tech...
research
∙
10/16/2021
Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher
With ever growing scale of neural models, knowledge distillation (KD) at...
research
∙
09/13/2021
How to Select One Among All? An Extensive Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
Knowledge Distillation (KD) is a model compression algorithm that helps ...
research
∙
04/14/2021
Annealing Knowledge Distillation
Significant memory and computational requirements of large deep neural n...
research
∙
06/30/2020