Gradual Fine-Tuning for Low-Resource Domain Adaptation

03/03/2021

∙

Fine-tuning is known to improve NLP models by adapting an initial model trained on more plentiful but less domain-salient examples to data in a target domain. Such domain adaptation is typically done using one stage of fine-tuning. We demonstrate that gradually fine-tuning in a multi-stage process can yield substantial further gains and can be applied without modifying the model or learning objective.

READ FULL TEXT

Gradual Fine-Tuning for Low-Resource Domain Adaptation

On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation

Efficient Domain Adaptation of Sentence Embeddings using Adapters

First U-Net Layers Contain More Domain Specific Information Than The Last Ones

Super-model ecosystem: A domain-adaptation perspective

Automatic Data Augmentation for Domain Adapted Fine-Tuning of Self-Supervised Speech Representations

Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models

Gradual Fine-Tuning for Low-Resource Domain Adaptation

Related Research

On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation

Efficient Domain Adaptation of Sentence Embeddings using Adapters

First U-Net Layers Contain More Domain Specific Information Than The Last Ones

Super-model ecosystem: A domain-adaptation perspective

Automatic Data Augmentation for Domain Adapted Fine-Tuning of Self-Supervised Speech Representations

Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Models