Know your Enemy: Investigating Monte-Carlo Tree Search with Opponent Models in Pommerman

by   Jannis Weil, et al.
Technische Universität Darmstadt

In combination with Reinforcement Learning, Monte-Carlo Tree Search has shown to outperform human grandmasters in games such as Chess, Shogi and Go with little to no prior domain knowledge. However, most classical use cases only feature up to two players. Scaling the search to an arbitrary number of players presents a computational challenge, especially if decisions have to be planned over a longer time horizon. In this work, we investigate techniques that transform general-sum multiplayer games into single-player and two-player games that consider other agents to act according to given opponent models. For our evaluation, we focus on the challenging Pommerman environment which involves partial observability, a long time horizon and sparse rewards. In combination with our search methods, we investigate the phenomena of opponent modeling using heuristics and self-play. Overall, we demonstrate the effectiveness of our multiplayer search variants both in a supervised learning and reinforcement learning setting.


page 2

page 7


Single-Agent Optimization Through Policy Iteration Using Monte-Carlo Tree Search

The combination of Monte-Carlo Tree Search (MCTS) and deep reinforcement...

StarCraft II Build Order Optimization using Deep Reinforcement Learning and Monte-Carlo Tree Search

The real-time strategy game of StarCraft II has been posed as a challeng...

Image Transformation Sequence Retrieval with General Reinforcement Learning

In this work, the novel Image Transformation Sequence Retrieval (ITSR) t...

Dual Monte Carlo Tree Search

AlphaZero, using a combination of Deep Neural Networks and Monte Carlo T...

Does it matter how well I know what you're thinking? Opponent Modelling in an RTS game

Opponent Modelling tries to predict the future actions of opponents, and...

Re-determinizing Information Set Monte Carlo Tree Search in Hanabi

This technical report documents the winner of the Computational Intellig...

An AlphaZero-Inspired Approach to Solving Search Problems

AlphaZero and its extension MuZero are computer programs that use machin...

Please sign up or login with your details

Forgot password? Click here to reset