DeepAI AI Chat
Log In Sign Up

Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color

by   Mostafa Abdou, et al.

Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases – (Paris, Capital, France). However, simple relations of this type can often be recovered heuristically and the extent to which models implicitly reflect topological structure that is grounded in world, such as perceptual structure, is unknown. To explore this question, we conduct a thorough case study on color. Namely, we employ a dataset of monolexemic color terms and color chips represented in CIELAB, a color space with a perceptually meaningful distance metric. Using two methods of evaluating the structural alignment of colors in this space with text-derived color term representations, we find significant correspondence. Analyzing the differences in alignment across the color spectrum, we find that warmer colors are, on average, better aligned to the perceptual color space than cooler ones, suggesting an intriguing connection to findings from recent work on efficient communication in color naming. Further analysis suggests that differences in alignment are, in part, mediated by collocationality and differences in syntactic usage, posing questions as to the relationship between color perception and usage and context.


page 14

page 15

page 16

page 17

page 18

page 19

page 20

page 21


Improving Perceptual Color Difference using Basic Color Terms

We suggest a new color distance based on two observations. First, percep...

What Language Reveals about Perception: Distilling Psychophysical Knowledge from Large Language Models

Understanding the extent to which the perceptual world can be recovered ...

Color naming reflects both perceptual structure and communicative need

Gibson et al. (2017) argued that color naming is shaped by patterns of c...

The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Color

Recent work has raised concerns about the inherent limitations of text-o...

Exploration of the Usage of Color Terms by Color-blind Participants in Online Discussion Platforms

Prominent questions about the role of sensory vs. linguistic input in th...

Grounding learning of modifier dynamics: An application to color naming

Grounding is crucial for natural language understanding. An important su...

Semantic Discriminability for Visual Communication

To interpret information visualizations, observers must determine how vi...