Evaluation of Text Generation: A Survey

06/26/2020
by   Asli Celikyilmaz, et al.
0

The paper surveys evaluation methods of natural language generation (NLG) systems that have been developed in the last few years. We group NLG evaluation methods into three categories: (1) human-centric evaluation metrics, (2) automatic metrics that require no training, and (3) machine-learned metrics. For each category, we discuss the progress that has been made and the challenges still being faced, with a focus on the evaluation of recently proposed NLG tasks and neural NLG models. We then present two case studies of automatic text summarization and long text generation, and conclude the paper by proposing future research directions.

READ FULL TEXT

page 27

page 36

research
01/17/2021

GENIE: A Leaderboard for Human-in-the-Loop Evaluation of Text Generation

Leaderboards have eased model development for many NLP datasets by stand...
research
10/16/2021

FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metricsfor Automatic Text Generation

Fast and reliable evaluation metrics are key to R D progress. While tr...
research
10/19/2019

Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation

Neural conditional text generation systems have achieved significant pro...
research
02/14/2022

Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text

Evaluation practices in natural language generation (NLG) have many know...
research
03/06/2022

Recent Advances in Neural Text Generation: A Task-Agnostic Survey

In recent years much effort has been devoted to applying neural models t...
research
07/03/2022

An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics

Long documents such as academic articles and business reports have been ...
research
03/06/2023

Models See Hallucinations: Evaluating the Factuality in Video Captioning

Video captioning aims to describe events in a video with natural languag...

Please sign up or login with your details

Forgot password? Click here to reset