Probabilistic Planning with Partially Ordered Preferences over Temporal Goals

09/25/2022
by   Hazhar Rahmani, et al.
0

In this paper, we study planning in stochastic systems, modeled as Markov decision processes (MDPs), with preferences over temporally extended goals. Prior work on temporal planning with preferences assumes that the user preferences form a total order, meaning that every pair of outcomes are comparable with each other. In this work, we consider the case where the preferences over possible outcomes are a partial order rather than a total order. We first introduce a variant of deterministic finite automaton, referred to as a preference DFA, for specifying the user's preferences over temporally extended goals. Based on the order theory, we translate the preference DFA to a preference relation over policies for probabilistic planning in a labeled MDP. In this treatment, a most preferred policy induces a weak-stochastic nondominated probability distribution over the finite paths in the MDP. The proposed planning algorithm hinges on the construction of a multi-objective MDP. We prove that a weak-stochastic nondominated policy given the preference specification is Pareto-optimal in the constructed multi-objective MDP, and vice versa. Throughout the paper, we employ a running example to demonstrate the proposed preference specification and solution approaches. We show the efficacy of our algorithm using the example with detailed analysis, and then discuss possible future directions.

READ FULL TEXT
research
03/26/2021

Probabilistic Planning with Preferences over Temporal Goals

We present a formal language for specifying qualitative preferences over...
research
03/25/2022

Opportunistic Qualitative Planning in Stochastic Systems with Preferences over Temporal Logic Objectives

Preferences play a key role in determining what goals/constraints to sat...
research
10/04/2022

Opportunistic Qualitative Planning in Stochastic Systems with Incomplete Preferences over Reachability Objectives

Preferences play a key role in determining what goals/constraints to sat...
research
04/23/2023

Probabilistic Planning with Prioritized Preferences over Temporal Logic Objectives

This paper studies temporal planning in probabilistic environments, mode...
research
05/10/2017

Solving Multi-Objective MDP with Lexicographic Preference: An application to stochastic planning with multiple quantile objective

In most common settings of Markov Decision Process (MDP), an agent evalu...
research
05/10/2021

Multi-Objective Controller Synthesis with Uncertain Human Preferences

Multi-objective controller synthesis concerns the problem of computing a...
research
06/02/2010

Learning Probabilistic Hierarchical Task Networks to Capture User Preferences

We propose automatically learning probabilistic Hierarchical Task Networ...

Please sign up or login with your details

Forgot password? Click here to reset