Multi-Objective Controller Synthesis with Uncertain Human Preferences

05/10/2021
by   Shenghui Chen, et al.
0

Multi-objective controller synthesis concerns the problem of computing an optimal controller subject to multiple (possibly conflicting) objective properties. The relative importance of objectives is often specified by human decision-makers. However, there is inherent uncertainty in human preferences (e.g., due to different preference elicitation methods). In this paper, we formalize the notion of uncertain human preferences and present a novel approach that accounts for uncertain human preferences in the multi-objective controller synthesis for Markov decision processes (MDPs). Our approach is based on mixed-integer linear programming (MILP) and synthesizes a sound, optimally permissive multi-strategy with respect to a multi-objective property and an uncertain set of human preferences. Experimental results on a range of large case studies show that our MILP-based approach is feasible and scalable to synthesize sound, optimally permissive multi-strategies with varying MDP model sizes and uncertainty levels of human preferences. Evaluation via an online user study also demonstrates the quality and benefits of synthesized (multi-)strategies.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/27/2023

Inferring Preferences from Demonstrations in Multi-objective Reinforcement Learning: A Dynamic Weight-based Approach

Many decision-making problems feature multiple objectives. In such probl...
research
10/07/2020

Quantifying the multi-objective cost of uncertainty

Various real-world applications involve modeling complex systems with im...
research
09/25/2022

Probabilistic Planning with Partially Ordered Preferences over Temporal Goals

In this paper, we study planning in stochastic systems, modeled as Marko...
research
10/20/2017

Multi-Objective Approaches to Markov Decision Processes with Uncertain Transition Parameters

Markov decision processes (MDPs) are a popular model for performance ana...
research
05/26/2023

MULTIGAIN 2.0: MDP controller synthesis for multiple mean-payoff, LTL and steady-state constraints

We present MULTIGAIN 2.0, a major extension to the controller synthesis ...
research
10/04/2022

Opportunistic Qualitative Planning in Stochastic Systems with Incomplete Preferences over Reachability Objectives

Preferences play a key role in determining what goals/constraints to sat...
research
01/16/2020

Optimal by Design: Model-Driven Synthesis of Adaptation Strategies for Autonomous Systems

Many software systems have become too large and complex to be managed ef...

Please sign up or login with your details

Forgot password? Click here to reset