Lion: Adversarial Distillation of Closed-Source Large Language Model

05/22/2023
by   Yuxin Jiang, et al.
0

The practice of transferring knowledge from a sophisticated, closed-source large language model (LLM) to a compact, open-source LLM has garnered considerable attention. Previous works have focused on a unidirectional knowledge distillation way by aligning the responses of the student model with those of the teacher model to a set of instructions. Nevertheless, they overlooked the possibility of incorporating any reciprocal "feedback"–identifying challenging instructions where the student model's performance falls short–to boost the student model's proficiency iteratively. To this end, we propose a novel adversarial distillation framework for a more efficient knowledge transfer. Leveraging the versatile role adaptability of LLMs, we prompt the closed-source model to identify "hard" instructions and generate new "hard" instructions for the student model, creating a three-stage adversarial loop of imitation, discrimination, and generation. By applying this adversarial framework, we successfully transfer knowledge from ChatGPT to a 7B student model (named Lion), achieving nearly 95 a mere 70k training data. We aspire that this proposed model may serve as the baseline to reflect the performance of ChatGPT, especially the open-source instruction-following language model baseline for our community.

READ FULL TEXT

page 5

page 8

research
10/22/2022

Hard Gate Knowledge Distillation – Leverage Calibration for Robust and Reliable Language Model

In knowledge distillation, a student model is trained with supervisions ...
research
05/29/2022

AutoDisc: Automatic Distillation Schedule for Large Language Model Compression

Driven by the teacher-student paradigm, knowledge distillation is one of...
research
04/29/2021

Spirit Distillation: A Model Compression Method with Multi-domain Knowledge Transfer

Recent applications pose requirements of both cross-domain knowledge tra...
research
08/27/2023

Detecting Language Model Attacks with Perplexity

A novel hack involving Large Language Models (LLMs) has emerged, leverag...
research
08/10/2023

RTLLM: An Open-Source Benchmark for Design RTL Generation with Large Language Model

Inspired by the recent success of large language models (LLMs) like Chat...
research
05/25/2023

The False Promise of Imitating Proprietary LLMs

An emerging method to cheaply improve a weaker language model is to fine...
research
06/15/2023

Propagating Knowledge Updates to LMs Through Distillation

Modern language models have the capacity to store and use immense amount...

Please sign up or login with your details

Forgot password? Click here to reset