## I Introduction

*Motivation:* Safety is a desirable and necessary design
constraint for any control system; specially when operated in a shared
environment with a decision maker (DM).
Arguably, most environments have associated spatial risks, whose
source can vary from hard constraints (e.g. moving obstacles) to softer
constraints (e.g. wind conditions). Different DMs can perceive these
risks differently, leading to notions of perceived risks and perceived
safety from these risks.

It is well known from psychophysics [1] and behavioral economics [2] research that humans as DMs have fundamental non-linear perception leading to non-rational decision making in risky situations. In such cases, existing methods assuming perfect knowledge or rational and coherent treatment (as in expected risk and Conditional Value at Risk (CVaR)) of risks may not suffice, which can lead to loss of trust or discomfort among DMs. This motivates the need of richer and more inclusive modeling of risk perception to capture a variety of DMs and use them for safe control design. This work aims to bridge the gap between behavioral decision making and safety using Cumulative Prospect Theory (CPT) as a risk perception model, and Control Barrier Functions (CBFs) for safe control design.

*Related Work:* Safe control system design has been tackled using
various frameworks such as artificial potential
functions [3], barrier certificates [4] and, more
recently, control barrier
functions (CBFs) [5]. CBFs have gained popularity due to their Lyapunov-like properties,
rigorous safety guarantees and ease of
application. They have been successfully used in
optimization [5],
stabilization [6] and data-driven control
frameworks [7]. CBFs were traditionally used in
static scenarios, more recently, they have been used to deal with
moving obstacles [8] and multi-agent
systems [9].

Uncertainty has been mainly handled using robustness measures [10], stochastic control [11], or chance constraints [12]. Very few works have considered the notion of risk perception explicitly in a control system [13, 14]. All these works use CVaR to quantify risk perception, which only captures linear and rational risk-averse behavior. CPT on the other hand is a more expressive (see [15]), non-linear and non-rational perception theory which is yet to be applied in the context of safety for a control system. Moreover, CPT has been successfully used in engineering applications like path planning [15], traffic routing [16], and network protection [17].

*Contributions:* We first adapt the notion of non-rational risk
perception to the context of safety for control systems. With this, we
capture a larger spectrum of DM’s risk profile, extending the existing
literature. We support this claim theoretically by defining the notion
of “inclusiveness” and proving that CPT is the most inclusive risk
perception model out of the other popular models: CVaR and ER.
We then use the CPT value function to construct a class of CBFs to
guarantee safety according to a DM’s perceived risk and define the
notion of perceived safety. Additionally, we find sufficient geometric
conditions on the control input to maintain the validity of our
proposed RPA CBF and compare them among the three risk perception models (RPMs). Then, we design a QP-based RPA controller to guide
an agent to a desired goal safely w.r.t. perceived risks. Thus we
extend the literature with more inclusive safe control
design. Practically, we consider 2D simulations with moving obstacles
and show the effectiveness of the proposed RPA controller along with
the practical translation of the inclusiveness heirarchy.

This work provides a framework to incorporate and compare a wide range of RPMs to generate a variety of RPA controls. We would also like to clarify that the validation of CPT models using user studies for typical control scenarios is beyond the scope of this work.

## Ii Risk perception formalism and problem setup

Here, we introduce some notation^{1}^{1}1The
Euclidean norm in is denoted by . We use

as the expectation operator on a random variable. The set

is a ball of radius centered at . and a formal notion of risk perception, starting with a concise description of CPT and CVaR (see [18] and [19] for more details). Later, we describe our problem statement.*Risk Perception:* By risk perception, we refer to the notion of
attaching a value (risk) to a random cost output. Formally, let

be a discrete sample space endowed with a probability distribution

. We model environmental cost via a real-valued, discrete random variable

, taking possible values, , , and such that , with . We Let be the set of such random cost variables and a value function which associates a value (risk) to a random cost variable.A value function can be defined in many ways, resulting in
different risk perceptions. Here, an RPM is characterized as a
parameterized family of
value functions.
In what follows, we consider three popular RPMs: Expected Risk
(ER) Conditional Value at Risk^{2}^{2}2The CVaR model uses a class of
value functions parameterized by to represent expectation over a fraction () of the worst-case
outcomes.
Thus the CVaR value with
is the worst-case outcome of , .
While, with
CVaR value equals ER ().
(CVaR) [19]
and Cumulative Prospect Theory (CPT) [2].

CPT captures non-rational decision making, and was introduced in [18, 20]. In CPT, outcomes are first weighed using a non-linear utility function , with , modeling a DM’s perceived cost. The parameters

represent “risk aversion” and “risk sensitivity”, respectively. In addition, a non-linear probability weighing function

, given by and , is used to model uncertainty perception. Here, uncertainty sensitivity is tuned via the parameters . CPT also suggests that probabilities are perceived via decision weights , which are calculated in a cumulative fashion. Defining a partial sum function as , and , we haveWith this, assigning the parameter for CVaR and for CPT, the value functions of ER (), CVaR () and CPT () of a DM are defined as:

(1a) | ||||

(1b) | ||||

(1c) |

In CPT, can be varied to generate different value functions pertaining to various risk profiles of DMs (from risk-taker to risk-averse). We refer to [15, 18] for more details on the parameter choices in CPT. Risky Environment: Consider a compact state space containing dynamic spatial sources of risk at and an agent or robot at a state . The relative state space is . Our starting point is an uncertain cost field , that aims to quantify objectively the (negative) consequences of being at relative to a known risk source at . More precisely, is a discrete RV which can take possible values, , for . We assume that

has associated mean and standard deviation functions

and , respectively. We assume that are continuously differentiable in their domains. Given , an associated spatial-risk function is given by , , where belongs to any of the previous RPMs defined in (1) above. When clear from the context, we will identify . The larger is at , the higher the perceived risk of being at*Dynamic systems:* We aim to control an agent modeled as a control-affine dynamic system:

(2) |

where and and are locally Lipschitz. We also consider a dynamic risk

(3) |

with a locally Lipschitz . We focus on moving obstacles as the source of risk, but the approach can be extended to other scenarios. We also assume that a asymptotically stable controller has been designed to guide the agent to a goal state in the absence of risk sources. We wish to drive the agent to a goal safely, while avoiding risky areas. Formally, we define safety considering a perceived spatial risk function as follows:

###### Definition 1

We now state the problems we address in this work:

###### Problem 1

(RPA safe sets) Given a risky environment , endowed with an uncertain cost , design perceived safety sets considering RPMs from (1). Characterize and constrast the properties of these sets among the three RPMs.

###### Problem 2

(RPA safe controls) Under previous conditions, design a controller , nominally deviating from a stable state feedback controller , such that the agent reaches the goal safely (Definition 1) and examine feasibility of .

## Iii Perceived Safety using various RPMs

This section compares various RPMs, solving Problem 1. Given an uncertain field cost , we apply the different risk perception models (see Section II) to obtain the corresponding fields, . With this, let us define the following sets:

(4a) | ||||

(4b) |

In particular, these sets depend on the choice of from
(1). Given , we define the range set
associated with
wrt as the set
^{3}^{3}3When clear from the
context, we will just denote ..
Fix a model and a risk source at
. The total safe set of wrt is
given as (resp. the total risky
set of wrt is
).
Thus, given , the set
(resp. covers all
the states in that safe (resp. unsafe) according to a RPM
.

###### Definition 2

(Inclusiveness and Strict Inclusiveness). Consider two RPMs and , a threshold , and a risk source at . Let the sets and be the total safe and risky sets of and wrt and a spatial cost , respectively. We say that is more inclusive than () if either and holds, or and holds, for all and costs . If and both hold, then is strictly more inclusive than ().

In particular, if , then results into a wider range of safety and risky sets for a given environment than .

Now we compare the inclusiveness of CPT, CVaR and ER via their respective value functions. We start by comparing the range space of these RPMs.

###### Lemma 1

Consider a threshold , a risk source at , and two RPMs with range spaces , respectively. If , and if there exists an such that or for any , and any , then . In addition, if there are such that and , , and any , then .

Fix . Since , , there is s.t. , . Thus, and . Assume s.t. or hold for all . This implies either or . Inclusiveness follows from Definition 2. In parallel, .

###### Lemma 2

Consider the CPT, CVaR and ER risk models, with associated range sets and . Then, it holds that , .

Fix . Note that . By choosing with and we have , . Note that only if then for all . When , with any other valid choice of parameters in CVaR we obtain . We can find such that , . Hence, and .

For CVaR, and , where is the worst-case outcome of . Since increases in , . Choosing for , leads to . Taking , with , we get ; hence, .

The previous results now lead to the following.

###### Theorem 1

Let be a discrete random field cost. Consider the ER, CVaR and CPT risk perception models with risk value functions , , and , respectively. For any threshold and risk source , and holds. If the cost outcomes are strictly lower-bounded by 1, then and . If in fact , then .

From Lemma 2, and . As in Lemma 2, take , for some . Choosing , with , we get , for any , and . Thus, from Lemma 1, we have and . Now assume for all . Taking with , we get for any and . Now, take with , we have . Since , , then , implying ando , . From Lemma 1, we get and .

Finally, assume . There is such that . Since the lower bound of is , there is no s.t. . Hence from Lemma 1 and the first part of this result, we get . The above arguments show CPT can produce a larger variety of safe and risky sets leading to richer risk perception. This is illustrated via simulations in Section V.

*Additional properties of RPMs: *
In addition to the notion of inclusiveness, we now characterize
the versatility of a RPM in the context of perceived safety.

###### Definition 3

(Versatility of a RPM).
Consider a compact space , a risk source
, and a discrete random
field cost , with range in . Let
be a
compact interval. An RPM is said
to be versatile if for any for a given . If , then
is *most versatile* in
.

The above definition implies that an RPM is versatile,
if it has a risk-perception functionthat perceives any states having costs less than as safe,
. Further, is *most versatile*
when it contains risk-perception functions that capture a range of
perceptions from most risk averse (only states having costs are safe) to the least risk-sensitive (every state
including states having the highest cost as safe).
With this, we will look at versatility of the three RPMs.

###### Lemma 3

Consider a compact space , with a risk source , and associated discrete random field cost . Then, CPT can capture the most risk averse perception, i.e. the set is considered safe.

###### Proposition 1

Under the setting of Lemma 3, CPT can
capture the least risk sensitive perception (the set is
considered safe), if , for , and
over . Consequently, CPT is
*most versatile* in .

For with we
have . Now,
choosing , since
, , and , we get
. Thus, from
(4a) and Definition 3,
the first result follows. Take now
and . Observe that is
continuous in and is
continuous in . By the intermediate
value theorem s.t. , and a
s.t. . Hence, from Lemma 3, CPT is *most
versatile* in .

###### Lemma 4

Under the assumptions of Lemma 3, with
and , CVaR is versatile and ER is
versatile. Hence neither are *most
versatile* RPMs.

This result trivially follows from the range spaces and in the proof of Lemma 2.

## Iv Control design with Risk-Perception-Aware-CBFs

Here, we address Problem 2 and design controls for an agent subject to (2), to ensure perceived safety (Definition 1). To do this, we formally adapt CBFs (see [5]) to our setting.

###### Definition 4 (Rpa-Cbf)

The existence of according to Definition 4 implies that the superlevel set is forward invariant under (2). We specify via given as

(6) |

where is a extended class function. Since is non-decreasing, implies and from (4a), . Thus, indicates that is perceived as safe w.r.t. .

The RPA control input can be now computed via:

(7a) | |||

(7b) |

The above problem captures the notion of minimally modifying a stable controller to ensure safety of the system. Next we will analyze the feasibility conditions for the proposed controller and compare it across the proposed models.

*Feasibility analysis and comparison:* We first describe a
construction of finite outcomes of from and
called “truncated-Gaussian cost” which will be used for
analysis. Assume that is distributed as a truncated
Gaussian^{4}^{4}4This truncation reassigns the probability
mass s.t. using an appropriate
re-normalization constant.
. Then, given , we approximate by means of discrete values ,
, with probability calculated from the
CDF of at each . That is, , and , for .
Now, we show conditions on for the set to be
non-empty for a given risk function . We first define a few
constants and variables to help us compare the feasibility conditions
of the three RPMs. Let be the relative
angle^{5}^{5}5

recall angle between two vectors

is given by between and , and and . Now define , and . Also we define constants , and . Consider , and . The following holds.###### Proposition 2

Let an agent and risk source be subject to (2) and (3), respectively. Consider cost build from a truncated Gaussian field. If there is a s.t.:

(8) |

then defined according to (6) is a valid RPA-CBF for any and (7) is feasible. Specifically, with , the RHS of the above inequality reduces to , , and for ER, CVaR and CPT, respectively.

For first part, rearranging terms in (8) we get:

(9) |

where . For the RPA-CBF to be valid, the set needs to be non-empty. Due to the dynamics of the agent and obstacle, and have dynamics:

(10) |

For the last part, the expressions are obtained by substituting the respective risk functions and evaluating the partial derivatives and (part of ). Thus we need to show the following hold true:

(12a) | ||||

(12b) | ||||

(12c) |

For ER we get and . For CVaR, since

is assumed to belong to a truncated Gaussian distribution, we can use the closed form expression of CVaR (

13) for a Gaussian distribution to calculate the partials and .(13) |

From (13), it is easy to see that CVaR is linear in and . With this, we get and .

Substituting these derivatives in (11) correspondingly for ER and CVaR, and using (10) we obtain the results.

For CPT, the expression is obtained by substituting the CPT risk function and evaluating the partial derivatives and . Constructing truncated Gaussian costs from and , we get outcomes and corresponding probabilities resulting in constant throughout. In this way, from (1c), the CPT value of a random cost with mean and is given by:

(14) |

With this expression, we can proceed to calculate the partial derivatives and . From (14), we get

(15a) | ||||

(15b) |

From (8), observe that the RHS is independent of and the LHS is independent of and the RPM. This separation makes it easier to compare various RPMs and their associated feasibility conditions.

Next, we remark on the uncertainty perception of each RPM, which will be used in the subsequent proposition to compare the size of control sets respectively generated by each of the RPMs.

###### Remark 1 (Uncertainty perception among RPMs)

The ER model is insensitive to uncertainty as . In this way, CVaR is
averse to uncertainty as for all . With
CPT, can be tuned to get both uncertainty insensitive
and uncertainty averse behavior, additionally, it can also
produce uncertainty liking behavior (when ). ^{6}^{6}6The first two properties follow by choosing
as in Theorem 1. The latter
property can be obtained by tuning the uncertainty
perception parameters and . Since the chosen
distribution is symmetric, we can examine the relation
between and for . If we have
(for example when is concave) or
(when is convex), then we have , or
,
respectively. A concave ()
implies that unlikely outcomes are viewed to be more
probable compared with the more certain outcomes. This
results into an “uncertainty averse behavior”, which is
reflected in the positive sign of . Conversely, a convex
() leads to an “uncertainty
liking behavior” with ..

We finally compare the the flexibility provided by each model via the corresponding control sets .

###### Proposition 3

In order to compare the feasibility of the sets from (5) for the three RPMs, we can compare their respective feasibility conditions (8). Consider , and . Since the LHS in (8) remains the same for any RPM and its parameter choice, to prove the proposition, it is sufficient to show that and . These inequalities follow from the choice of

Comments

There are no comments yet.