Emergent Graphical Conventions in a Visual Communication Game

11/28/2021
by   Shuwen Qiu, et al.
0

Humans communicate with graphical sketches apart from symbolic languages. While recent studies of emergent communication primarily focus on symbolic languages, their settings overlook the graphical sketches existing in human communication; they do not account for the evolution process through which symbolic sign systems emerge in the trade-off between iconicity and symbolicity. In this work, we take the very first step to model and simulate such an evolution process via two neural agents playing a visual communication game; the sender communicates with the receiver by sketching on a canvas. We devise a novel reinforcement learning method such that agents are evolved jointly towards successful communication and abstract graphical conventions. To inspect the emerged conventions, we carefully define three key properties – iconicity, symbolicity, and semanticity – and design evaluation methods accordingly. Our experimental results under different controls are consistent with the observation in studies of human graphical conventions. Of note, we find that evolved sketches can preserve the continuum of semantics under proper environmental pressures. More interestingly, co-evolved agents can switch between conventionalized and iconic communication based on their familiarity with referents. We hope the present research can pave the path for studying emergent communication with the unexplored modality of sketches.

READ FULL TEXT

page 2

page 3

page 5

page 6

page 9

page 10

page 11

page 12

research
03/31/2021

A Signal-Centric Perspective on the Evolution of Symbolic Communication

The evolution of symbolic communication is a longstanding open research ...
research
11/13/2019

Learning to Communicate in Multi-Agent Reinforcement Learning : A Review

We consider the issue of multiple agents learning to communicate through...
research
09/24/2019

Paying Attention to Function Words

All natural languages exhibit a distinction between content words (like ...
research
01/25/2021

Emergent Communication under Competition

The literature in modern machine learning has only negative results for ...
research
12/28/2022

Towards Learning Abstractions via Reinforcement Learning

In this paper we take the first steps in studying a new approach to synt...
research
06/04/2021

Emergent Communication of Generalizations

To build agents that can collaborate effectively with others, recent res...
research
07/18/2020

ESCELL: Emergent Symbolic Cellular Language

We present ESCELL, a method for developing an emergent symbolic language...

Please sign up or login with your details

Forgot password? Click here to reset