AI Chat AI Image Generator AI Video Text to Speech

Shared Task on Evaluating Accuracy in Natural Language Generation

06/22/2020

∙

by Ehud Reiter, et al.

∙

∙

We propose a shared task on methodologies and algorithms for evaluating the accuracy of generated texts. Participants will measure the accuracy of basketball game summaries produced by NLG systems from basketball box score data.

Ehud Reiter
17 publications
Craig Thomson
4 publications

page 1

page 2

page 3

research

∙ 08/12/2021

Generation Challenges: Results of the Accuracy Evaluation Shared Task

The Shared Task on Evaluating Accuracy focused on techniques (both manua...

0 Craig Thomson, et al. ∙

research

∙ 04/09/2021

Towards objectively evaluating the quality of generated medical summaries

We propose a method for evaluating the quality of generated text by aski...

0 Francesco Moramarco, et al. ∙

research

∙ 11/08/2020

A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems

Most Natural Language Generation systems need to produce accurate texts....

0 Craig Thomson, et al. ∙

research

∙ 09/21/2023

SemEval-2022 Task 7: Identifying Plausible Clarifications of Implicit and Underspecified Phrases in Instructional Texts

We describe SemEval-2022 Task 7, a shared task on rating the plausibilit...

0 Michael Roth, et al. ∙

research

∙ 11/21/2020

Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference

A major challenge in evaluating data-to-text (D2T) generation is measuri...

0 Ondřej Dušek, et al. ∙

research

∙ 05/23/2023

Evaluating Factual Consistency of Summaries with Large Language Models

Detecting factual errors in summaries has been an important and challeng...

0 Shiqi Chen, et al. ∙

research

∙ 01/23/2019

Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge

This paper provides a detailed summary of the first shared task on End-t...

0 Ondřej Dušek, et al. ∙