Pragmatically Informative Color Generation by Grounding Contextual Modifiers

10/09/2020
by   Zhengxuan Wu, et al.
0

Grounding language in contextual information is crucial for fine-grained natural language understanding. One important task that involves grounding contextual modifiers is color generation. Given a reference color "green", and a modifier "bluey", how does one generate a color that could represent "bluey green"? We propose a computational pragmatics model that formulates this color generation task as a recursive game between speakers and listeners. In our model, a pragmatic speaker reasons about the inferences that a listener would make, and thus generates a modified color that is maximally informative to help the listener recover the original referents. In this paper, we show that incorporating pragmatic information provides significant improvements in performance compared with other state-of-the-art deep learning models where pragmatic inference and flexibility in representing colors from a large continuous space are lacking. Our model has an absolute 98 performance for the test cases where the reference colors are unseen during training, and an absolute 40 both the reference colors and the modifiers are unseen during training.

READ FULL TEXT

page 6

page 7

research
09/17/2019

Grounding learning of modifier dynamics: An application to color naming

Grounding is crucial for natural language understanding. An important su...
research
03/29/2017

Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding

We present a model of pragmatic referring expression interpretation in a...
research
09/27/2017

Learning Distributions of Meant Color

When a speaker says the name of a color, the color they picture is not n...
research
06/21/2017

Comicolorization: Semi-Automatic Manga Colorization

We developed "Comicolorization", a semi-automatic colorization system fo...
research
05/19/2022

Voxel-informed Language Grounding

Natural language applied to natural 2D images describes a fundamentally ...
research
12/06/2018

Guided Zoom: Questioning Network Evidence for Fine-grained Classification

We propose Guided Zoom, an approach that utilizes spatial grounding to m...
research
10/13/2022

Understanding the Effect of Smartphone Cameras on Estimating Munsell Soil Colors from Imagery

The Munsell soil color chart (MSCC) is a in laboratories under controlle...

Please sign up or login with your details

Forgot password? Click here to reset