Stigmergic Independent Reinforcement Learning for Multi-Agent Collaboration
share
With the rapid evolution of wireless mobile devices, it emerges stronger incentive to design proper collaboration mechanisms among the intelligent agents. Following their individual observations, multiple intelligent agents could cooperate and gradually approach the final collective objective through continuously learning from the environment. In that regard, independent reinforcement learning (IRL) is often deployed within the multi-agent collaboration to alleviate the dilemma of non-stationary learning environment. However, behavioral strategies of the intelligent agents in IRL could only be formulated upon their local individual observations of the global environment, and appropriate communication mechanisms must be introduced to reduce their behavioral localities. In this paper, we tackle the communication problem among the intelligent agents in IRL by jointly adopting two mechanisms with different scales. For the large scale, we introduce the stigmergy mechanism as an indirect communication bridge among the independent learning agents and carefully design a mathematical representation to indicate the impact of digital pheromone. For the small scale, we propose a conflict-avoidance mechanism between adjacent agents by implementing an additionally embedded neural network to provide more opportunities for participants with higher action priorities. Besides, we also present a federal training method to effectively optimize the neural networks within each agent in a decentralized manner. Finally, we establish a simulation scenario where a number of mobile agents in a certain area move automatically to form a specified target shape, and demonstrate the superiorities of our proposed methods through extensive simulations.
READ FULL TEXT
