research
∙
05/22/2023
SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation
Reliable automatic evaluation of summarization systems is challenging du...
share
research
∙
11/02/2022
Dialect-robust Evaluation of Generated Text
Evaluation metrics that are not robust to dialect variation make it impo...
research
∙
10/12/2022
SQuId: Measuring Speech Naturalness in Many Languages
Much of text-to-speech research relies on human evaluation, which incurs...
research
∙
02/14/2022
Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text
Evaluation practices in natural language generation (NLG) have many know...
research
∙
10/12/2021
Learning Compact Metrics for MT
Recent developments in machine translation and multilingual text generat...
research
∙
06/30/2021
The MultiBERTs: BERT Reproductions for Robustness Analysis
Experiments with pretrained models such as BERT are often based on a sin...
research
∙
02/02/2021
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
We introduce GEM, a living benchmark for natural language Generation (NL...
research
∙
10/08/2020
Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task
The quality of machine translation systems has dramatically improved ove...
research
∙
04/09/2020
BLEURT: Learning Robust Metrics for Text Generation
Text generation has made significant advances in the last few years. Yet...
research
∙
02/07/2020
A Multilingual View of Unsupervised Machine Translation
We present a probabilistic framework for multilingual neural machine tra...
research
∙
10/19/2019
Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation
Neural conditional text generation systems have achieved significant pro...
research
∙
08/13/2018
DeepBase: Deep Inspection of Neural Networks
Although deep learning models perform remarkably across a range of tasks...
research
∙
11/30/2017