Ranking Creative Language Characteristics in Small Data Scenarios

10/23/2020
by   Julia Siekiera, et al.
0

The ability to rank creative natural language provides an important general tool for downstream language understanding and generation. However, current deep ranking models require substantial amounts of labeled data that are difficult and expensive to obtain for different domains, languages and creative characteristics. A recent neural approach, the DirectRanker, promises to reduce the amount of training data needed but its application to text isn't fully explored. We therefore adapt the DirectRanker to provide a new deep model for ranking creative language with small data. We compare DirectRanker with a Bayesian approach, Gaussian process preference learning (GPPL), which has previously been shown to work well with sparse data. Our experiments with sparse training data show that while the performance of standard neural ranking approaches collapses with small training datasets, DirectRanker remains effective. We find that combining DirectRanker with GPPL increases performance across different settings by leveraging the complementary benefits of both models. Our combined approach outperforms the previous state-of-the-art on humor and metaphor novelty tasks, increasing Spearman's ρ by 14 on average.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/01/2016

Domain Adaptation of Recurrent Neural Networks for Natural Language Understanding

The goal of this paper is to use multi-task learning to efficiently scal...
research
11/04/2021

CLUES: Few-Shot Learning Evaluation in Natural Language Understanding

Most recent progress in natural language understanding (NLU) has been dr...
research
06/06/2023

Evaluating the Effectiveness of Natural Language Inference for Hate Speech Detection in Languages with Limited Labeled Data

Most research on hate speech detection has focused on English where a si...
research
06/06/2018

Finding Convincing Arguments Using Scalable Bayesian Preference Learning

We introduce a scalable Bayesian preference learning method for identify...
research
04/03/2019

Cross-lingual transfer learning for spoken language understanding

Typically, spoken language understanding (SLU) models are trained on ann...
research
03/31/2022

Domain Adaptation for Sparse-Data Settings: What Do We Gain by Not Using Bert?

The practical success of much of NLP depends on the availability of trai...
research
10/05/2019

Content-Based Features to Rank Influential Hidden Services of the Tor Darknet

The unevenness importance of criminal activities in the onion domains of...

Please sign up or login with your details

Forgot password? Click here to reset