Emergent Bartering Behaviour in Multi-Agent Reinforcement Learning

by   Michael Bradley Johanson, et al.

Advances in artificial intelligence often stem from the development of new environments that abstract real-world situations into a form where research can be done conveniently. This paper contributes such an environment based on ideas inspired by elementary Microeconomics. Agents learn to produce resources in a spatially complex world, trade them with one another, and consume those that they prefer. We show that the emergent production, consumption, and pricing behaviors respond to environmental conditions in the directions predicted by supply and demand shifts in Microeconomics. We also demonstrate settings where the agents' emergent prices for goods vary over space, reflecting the local abundance of goods. After the price disparities emerge, some agents then discover a niche of transporting goods between regions with different prevailing prices – a profitable strategy because they can buy goods where they are cheap and sell them where they are expensive. Finally, in a series of ablation experiments, we investigate how choices in the environmental rewards, bartering actions, agent architecture, and ability to consume tradable goods can either aid or inhibit the emergence of this economic behavior. This work is part of the environment development branch of a research program that aims to build human-like artificial general intelligence through multi-agent interactions in simulated societies. By exploring which environment features are needed for the basic phenomena of elementary microeconomics to emerge automatically from learning, we arrive at an environment that differs from those studied in prior multi-agent reinforcement learning work along several dimensions. For example, the model incorporates heterogeneous tastes and physical abilities, and agents negotiate with one another as a grounded form of communication.


page 1

page 26

page 42


Multi-Agent Interplay in a Competitive Survival Environment

Solving hard-exploration environments in an important challenge in Reinf...

Understanding the World to Solve Social Dilemmas Using Multi-Agent Reinforcement Learning

Social dilemmas are situations where groups of individuals can benefit f...

Signifiers as a First-class Abstraction in Hypermedia Multi-Agent Systems

Hypermedia APIs enable the design of reusable hypermedia clients that di...

Emergence of Adaptive Circadian Rhythms in Deep Reinforcement Learning

Adapting to regularities of the environment is critical for biological o...

An Unified Intelligence-Communication Model for Multi-Agent System Part-I: Overview

Motivated by Shannon's model and recent rehabilitation of self-supervise...

A path to AI

To build a safe system that would replicate and perhaps transcend human-...

Please sign up or login with your details

Forgot password? Click here to reset