Social Network Structure Shapes Innovation: Experience-sharing in RL with SAPIENS

by   Eleni Nisioti, et al.

The human cultural repertoire relies on innovation: our ability to continuously and hierarchically explore how existing elements can be combined to create new ones. Innovation is not solitary, it relies on collective accumulation and merging of previous solutions. Machine learning approaches commonly assume that fully connected multi-agent networks are best suited for innovation. However, human laboratory and field studies have shown that hierarchical innovation is more robustly achieved by dynamic communication topologies. In dynamic topologies, humans oscillate between innovating individually or in small clusters, and then sharing outcomes with others. To our knowledge, the role of multi-agent topology on innovation has not been systematically studied in machine learning. It remains unclear a) which communication topologies are optimal for which innovation tasks, and b) which properties of experience sharing improve multi-level innovation. Here we use a multi-level hierarchical problem setting (WordCraft), with three different innovation tasks. We systematically design networks of DQNs sharing experiences from their replay buffers in varying topologies (fully connected, small world, dynamic, ring). Comparing the level of innovation achieved by different experience-sharing topologies across different tasks shows that, first, consistent with human findings, experience sharing within a dynamic topology achieves the highest level of innovation across tasks. Second, experience sharing is not as helpful when there is a single clear path to innovation. Third, two metrics we propose, conformity and diversity of shared experience, can explain the success of different topologies on different tasks. These contributions can advance our understanding of optimal AI-AI, human-human, and human-AI collaborative networks, inspiring future tools for fostering collective innovation in large organizations.


page 6

page 16

page 17

page 18

page 19


How to Organize your Deep Reinforcement Learning Agents: The Importance of Communication Topology

In this empirical paper, we investigate how learning agents can be arran...

Communication Topologies Between Learning Agents in Deep Reinforcement Learning

A common technique to improve speed and robustness of learning in deep r...

Capturing the Production of the Innovative Ideas: An Online Social Network Experiment and "Idea Geography" Visualization

Collective design and innovation are crucial in organizations. To invest...

Innovation and informal knowledge exchanges between firms

Firm clusters are seen as having a positive effect on innovations, what ...

Decentralized core-periphery structure in social networks accelerates cultural innovation in agent-based model

Previous investigations into creative and innovation networks have sugge...

Improved Learning in Evolution Strategies via Sparser Inter-Agent Network Topologies

We draw upon a previously largely untapped literature on human collectiv...

Rethinking Machine Learning Collective Communication as a Multi-Commodity Flow Problem

We show communication schedulers' recent work proposed for ML collective...

Please sign up or login with your details

Forgot password? Click here to reset