SelF-Eval: Self-supervised Fine-grained Dialogue Evaluation

08/17/2022
by   Longxuan Ma, et al.
0

This paper introduces a novel Self-supervised Fine-grained Dialogue Evaluation framework (SelF-Eval). The core idea is to model the correlation between turn quality and the entire dialogue quality. We first propose a novel automatic data construction method that can automatically assign fine-grained scores for arbitrarily dialogue data. Then we train SelF-Eval with a multi-level contrastive learning schema which helps to distinguish different score levels. Experimental results on multiple benchmarks show that SelF-Eval is highly consistent with human evaluations and better than the state-of-the-art models. We give a detailed analysis of the experiments in this paper. Our code is available on GitHub.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/25/2022

FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation

Recent model-based reference-free metrics for open-domain dialogue evalu...
research
07/29/2021

Self-Supervised Learning for Fine-Grained Image Classification

Fine-grained image classification involves identifying different subcate...
research
04/18/2021

Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learning

Aiming to generate a set of keyphrases, Keyphrase Generation (KG) is a c...
research
12/14/2021

MDD-Eval: Self-Training on Augmented Data for Multi-Domain Dialogue Evaluation

Chatbots are designed to carry out human-like conversations across diffe...
research
06/17/2021

Open Data and the Status Quo – A Fine-Grained Evaluation Framework for Open Data Quality and an Analysis of Open Data portals in Germany

This paper presents a framework for assessing data and metadata quality ...
research
04/09/2021

Towards Fine-grained Visual Representations by Combining Contrastive Learning with Image Reconstruction and Attention-weighted Pooling

This paper presents Contrastive Reconstruction, ConRec - a self-supervis...
research
08/03/2022

Convolutional Fine-Grained Classification with Self-Supervised Target Relation Regularization

Fine-grained visual classification can be addressed by deep representati...

Please sign up or login with your details

Forgot password? Click here to reset