DeepAI AI Chat
Log In Sign Up

Pragmatically Informative Color Generation by Grounding Contextual Modifiers

by   Zhengxuan Wu, et al.

Grounding language in contextual information is crucial for fine-grained natural language understanding. One important task that involves grounding contextual modifiers is color generation. Given a reference color "green", and a modifier "bluey", how does one generate a color that could represent "bluey green"? We propose a computational pragmatics model that formulates this color generation task as a recursive game between speakers and listeners. In our model, a pragmatic speaker reasons about the inferences that a listener would make, and thus generates a modified color that is maximally informative to help the listener recover the original referents. In this paper, we show that incorporating pragmatic information provides significant improvements in performance compared with other state-of-the-art deep learning models where pragmatic inference and flexibility in representing colors from a large continuous space are lacking. Our model has an absolute 98 performance for the test cases where the reference colors are unseen during training, and an absolute 40 both the reference colors and the modifiers are unseen during training.


page 6

page 7


Grounding learning of modifier dynamics: An application to color naming

Grounding is crucial for natural language understanding. An important su...

Colors in Context: A Pragmatic Neural Model for Grounded Language Understanding

We present a model of pragmatic referring expression interpretation in a...

Learning Distributions of Meant Color

When a speaker says the name of a color, the color they picture is not n...

Comicolorization: Semi-Automatic Manga Colorization

We developed "Comicolorization", a semi-automatic colorization system fo...

Voxel-informed Language Grounding

Natural language applied to natural 2D images describes a fundamentally ...

Guided Zoom: Questioning Network Evidence for Fine-grained Classification

We propose Guided Zoom, an approach that utilizes spatial grounding to m...

Understanding the Effect of Smartphone Cameras on Estimating Munsell Soil Colors from Imagery

The Munsell soil color chart (MSCC) is a in laboratories under controlle...