Optimal sequential decision making with probabilistic digital twins

by   Christian Agrell, et al.

Digital twins are emerging in many industries, typically consisting of simulation models and data associated with a specific physical system. One of the main reasons for developing a digital twin, is to enable the simulation of possible consequences of a given action, without the need to interfere with the physical system itself. Physical systems of interest, and the environments they operate in, do not always behave deterministically. Moreover, information about the system and its environment is typically incomplete or imperfect. Probabilistic representations of systems and environments may therefore be called for, especially to support decisions in application areas where actions may have severe consequences. In this paper we introduce the probabilistic digital twin (PDT). We will start by discussing how epistemic uncertainty can be treated using measure theory, by modelling epistemic information via σ-algebras. Based on this, we give a formal definition of how epistemic uncertainty can be updated in a PDT. We then study the problem of optimal sequential decision making. That is, we consider the case where the outcome of each decision may inform the next. Within the PDT framework, we formulate this optimization problem. We discuss how this problem may be solved (at least in theory) via the maximum principle method or the dynamic programming principle. However, due to the curse of dimensionality, these methods are often not tractable in practice. To mend this, we propose a generic approximate solution using deep reinforcement learning together with neural networks defined on sets. We illustrate the method on a practical problem, considering optimal information gathering for the estimation of a failure probability.


page 1

page 2

page 3

page 4


On the Complexity of Decision Making in Possibilistic Decision Trees

When the information about uncertainty cannot be quantified in a simple,...

Exploration via Epistemic Value Estimation

How to efficiently explore in reinforcement learning is an open problem....

Risk-Sensitive and Robust Model-Based Reinforcement Learning and Planning

Many sequential decision-making problems that are currently automated, s...

Information Avoidance and Overvaluation in Sequential Decision Making under Epistemic Constraints

Decision makers involved in the management of civil assets and systems u...

Multi-Target Decision Making under Conditions of Severe Uncertainty

The quality of consequences in a decision making problem under (severe) ...

Decision-Making Under Uncertainty: Beyond Probabilities

This position paper reflects on the state-of-the-art in decision-making ...

An Application of a Runtime Epistemic Probabilistic Event Calculus to Decision-making in e-Health Systems

We present and discuss a runtime architecture that integrates sensorial ...

Please sign up or login with your details

Forgot password? Click here to reset