Fine-tuning of Language Models with Discriminator

11/12/2018
by   Vadim Popov, et al.
0

Cross-entropy loss is a common choice when it comes to multiclass classification tasks and language modeling in particular. Minimizing this loss results in language models of very good quality. We show that it is possible to fine-tune these models and make them perform even better if they are fine-tuned with sum of cross-entropy loss and reverse Kullback-Leibler divergence. The latter is estimated using discriminator network that we train in advance. During fine-tuning we can use this discriminator to figure out if probabilities of some words are overestimated and reduce them in this case. The novel approach that we propose allows us to reach state-of-the-art quality on Penn TreeBank: perplexity of the fine-tuned model drops down by more than 0.5 and is now below 54.0 in standard evaluation setting; however, in dynamic evaluation framework the improvement is much less perceptible. Our fine-tuning algorithm is rather fast and requires almost no hyperparameter tuning. We test it on different datasets including WikiText-2 and large-scale dataset. In the former case we also reach state-of-the-art results.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/15/2021

Applying SoftTriple Loss for Supervised Language Model Fine Tuning

We introduce a new loss function TripleEntropy, to improve classificatio...
research
11/28/2022

Distance Metric Learning Loss Functions in Few-Shot Scenarios of Supervised Language Models Fine-Tuning

This paper presents an analysis regarding an influence of the Distance M...
research
02/07/2023

Capturing Topic Framing via Masked Language Modeling

Differential framing of issues can lead to divergent world views on impo...
research
04/18/2023

Stochastic Parrots Looking for Stochastic Parrots: LLMs are Easy to Fine-Tune and Hard to Detect with other LLMs

The self-attention revolution allowed generative language models to scal...
research
09/14/2023

PerPLM: Personalized Fine-tuning of Pretrained Language Models via Writer-specific Intermediate Learning and Prompts

The meanings of words and phrases depend not only on where they are used...
research
08/26/2023

Adversarial Fine-Tuning of Language Models: An Iterative Optimisation Approach for the Generation and Detection of Problematic Content

In this paper, we tackle the emerging challenge of unintended harmful co...
research
08/29/2022

Assessing, testing and estimating the amount of fine-tuning by means of active information

A general framework is introduced to estimate how much external informat...

Please sign up or login with your details

Forgot password? Click here to reset