Convex Hull Monte-Carlo Tree Search

03/09/2020
by   Michael Painter, et al.
0

This work investigates Monte-Carlo planning for agents in stochastic environments, with multiple objectives. We propose the Convex Hull Monte-Carlo Tree-Search (CHMCTS) framework, which builds upon Trial Based Heuristic Tree Search and Convex Hull Value Iteration (CHVI), as a solution to multi-objective planning in large environments. Moreover, we consider how to pose the problem of approximating multiobjective planning solutions as a contextual multi-armed bandits problem, giving a principled motivation for how to select actions from the view of contextual regret. This leads us to the use of Contextual Zooming for action selection, yielding Zooming CHMCTS. We evaluate our algorithm using the Generalised Deep Sea Treasure environment, demonstrating that Zooming CHMCTS can achieve a sublinear contextual regret and scales better than CHVI on a given computational budget.

READ FULL TEXT
research
06/08/2021

Measurable Monte Carlo Search Error Bounds

Monte Carlo planners can often return sub-optimal actions, even if they ...
research
12/15/2019

Multi-Object Rearrangement with Monte Carlo Tree Search:A Case Study on Planar Nonprehensile Sorting

In this work, we address a planar non-prehensile sorting task. Here, a r...
research
06/08/2023

Habits of Mind: Reusing Action Sequences for Efficient Planning

When we exercise sequences of actions, their execution becomes more flue...
research
05/16/2023

Scale-Adaptive Balancing of Exploration and Exploitation in Classical Planning

Balancing exploration and exploitation has been an important problem in ...
research
04/22/2023

Recomputing Solutions to Perturbed Multi-Commodity Pickup and Delivery Vehicle Routing Problems using Monte Carlo Tree Search

The Multi-Commodity Pickup and Delivery Vehicle Routing Problem aims to ...
research
06/12/2021

Planning Spatial Networks

We tackle the problem of goal-directed graph construction: given a start...
research
06/08/2021

Vector Quantized Models for Planning

Recent developments in the field of model-based RL have proven successfu...

Please sign up or login with your details

Forgot password? Click here to reset