Multi-objective Reinforcement Learning with Continuous Pareto Frontier Approximation Supplementary Material

06/13/2014
by   Matteo Pirotta, et al.
0

This document contains supplementary material for the paper "Multi-objective Reinforcement Learning with Continuous Pareto Frontier Approximation", published at the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI-15). The paper is about learning a continuous approximation of the Pareto frontier in Multi-Objective Markov Decision Problems (MOMDPs). We propose a policy-based approach that exploits gradient information to generate solutions close to the Pareto ones. Differently from previous policy-gradient multi-objective algorithms, where n optimization routines are use to have n solutions, our approach performs a single gradient-ascent run that at each step generates an improved continuous approximation of the Pareto frontier. The idea is to exploit a gradient-based approach to optimize the parameters of a function that defines a manifold in the policy parameter space so that the corresponding image in the objective space gets as close as possible to the Pareto frontier. Besides deriving how to compute and estimate such gradient, we will also discuss the non-trivial issue of defining a metric to assess the quality of the candidate Pareto frontiers. Finally, the properties of the proposed approach are empirically evaluated on two interesting MOMDPs.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/15/2023

Latent-Conditioned Policy Gradient for Multi-Objective Deep Reinforcement Learning

Sequential decision making in the real world often requires finding a go...
research
03/02/2022

Pareto Frontier Approximation Network (PA-Net) to Solve Bi-objective TSP

Travelling salesperson problem (TSP) is a classic resource allocation pr...
research
12/19/2022

Taming Lagrangian Chaos with Multi-Objective Reinforcement Learning

We consider the problem of two active particles in 2D complex flows with...
research
09/17/2020

Multi-objective dynamic programming with limited precision

This paper addresses the problem of approximating the set of all solutio...
research
10/07/2021

Multi-objective Optimization by Learning Space Partitions

In contrast to single-objective optimization (SOO), multi-objective opti...
research
06/17/2015

Hybrid Algorithm for Multi-Objective Optimization by Greedy Hypervolume Maximization

This paper introduces a high-performance hybrid algorithm, called Hybrid...
research
01/19/2021

Dynamic Bicycle Dispatching of Dockless Public Bicycle-sharing Systems using Multi-objective Reinforcement Learning

As a new generation of Public Bicycle-sharing Systems (PBS), the dockles...

Please sign up or login with your details

Forgot password? Click here to reset