Log In Sign Up

Can Visual Dialogue Models Do Scorekeeping? Exploring How Dialogue Representations Incrementally Encode Shared Knowledge

by   Brielen Madureira, et al.

Cognitively plausible visual dialogue models should keep a mental scoreboard of shared established facts in the dialogue context. We propose a theory-based evaluation method for investigating to what degree models pretrained on the VisDial dataset incrementally build representations that appropriately do scorekeeping. Our conclusion is that the ability to make the distinction between shared and privately known statements along the dialogue is moderately present in the analysed models, but not always incrementally consistent, which may partially be due to the limited need for grounding interactions in the original task.


page 2

page 10

page 14


Training an adaptive dialogue policy for interactive learning of visually grounded word meanings

We present a multi-modal dialogue system for interactive learning of per...

A Natural Language Corpus of Common Grounding under Continuous and Partially-Observable Context

Common grounding is the process of creating, repairing and updating mutu...

The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue

This paper introduces the PhotoBook dataset, a large-scale collection of...

Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding

Dialogue systems powered by large pre-trained language models (LM) exhib...

Grounded Agreement Games: Emphasizing Conversational Grounding in Visual Dialogue Settings

Where early work on dialogue in Computational Linguistics put much empha...

A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generation

Towards building intelligent dialogue agents, there has been a growing i...

Visual Dialogue without Vision or Dialogue

We characterise some of the quirks and shortcomings in the exploration o...