Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation

05/23/2023
by   Da Yin, et al.
0

Instruction tuning has emerged to enhance the capabilities of large language models (LLMs) in providing appropriate outputs based on input instructions. However, existing methods for collecting instruction-tuning data suffer from limitations in scalability and affordability. In this paper, we propose Dynosaur, a dynamic growth paradigm for instruction-tuning data curation. Built upon the metadata of existing NLP datasets, we generate multiple task instructions applicable to various NLP datasets and determine the relevant data fields for constructing instruction-tuning data with LLMs. Dynosaur offers several advantages: 1) lower generation costs (less than 12 for generating 800K instruction-tuning data), 2) good quality of instruction-tuning data (better performance than Alpaca and Instruction GPT-4 on Super-NI with comparable data sizes), and 3) the ability to grow dynamically by incorporating new datasets from Huggingface Datasets Platform. We further investigate continual learning as an approach to learning with the ever-growing instruction-tuning dataset. We demonstrate that replay methods not only help mitigate forgetting issues but help generalize to unseen tasks better. As a novel continual learning scenario for instruction tuning, selecting tasks based on instruction representations can be an effective replaying strategy. Code and data are released at <https://github.com/WadeYin9712/Dynosaur>.

READ FULL TEXT

page 4

page 13

research
05/24/2023

Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models

Instruction-tuned models are trained on crowdsourcing datasets with task...
research
04/17/2023

LongForm: Optimizing Instruction Tuning for Long Text Generation with Corpus Extraction

Instruction tuning enables language models to generalize more effectivel...
research
03/16/2022

ConTinTin: Continual Learning from Task Instructions

The mainstream machine learning paradigms for NLP often work with two un...
research
07/28/2023

Exploring Format Consistency for Instruction Tuning

Instruction tuning has emerged as a promising approach to enhancing larg...
research
08/10/2023

A Preliminary Study of the Intrinsic Relationship between Complexity and Alignment

Training large language models (LLMs) with open-domain instruction data ...
research
05/23/2023

Robust Instruction Optimization for Large Language Models with Distribution Shifts

Large Language Models have demonstrated significant ability in accomplis...
research
06/16/2023

Differentiable Instruction Optimization for Cross-Task Generalization

Instruction tuning has been attracting much attention to achieve general...

Please sign up or login with your details

Forgot password? Click here to reset