Towards Efficient Multi-Agent Learning Systems
share
Multi-Agent Reinforcement Learning (MARL) is an increasingly important research field that can model and control multiple large-scale autonomous systems. Despite its achievements, existing multi-agent learning methods typically involve expensive computations in terms of training time and power arising from large observation-action space and a huge number of training steps. Therefore, a key challenge is understanding and characterizing the computationally intensive functions in several popular classes of MARL algorithms during their training phases. Our preliminary experiments reveal new insights into the key modules of MARL algorithms that limit the adoption of MARL in real-world systems. We explore neighbor sampling strategy to improve cache locality and observe performance improvement ranging from 26.66 agents) to 27.39 sampling phase. Additionally, we demonstrate that improving the locality leads to an end-to-end training time reduction of 10.2 existing multi-agent algorithms without significant degradation in the mean reward.
READ FULL TEXT