DeepAI
Log In Sign Up

Safe and Stable Control Synthesis for Uncertain System Models via Distributionally Robust Optimization

10/04/2022
by   Kehan Long, et al.
University of California, San Diego
0

This paper considers enforcing safety and stability of dynamical systems in the presence of model uncertainty. Safety and stability constraints may be specified using a control barrier function (CBF) and a control Lyapunov function (CLF), respectively. To take model uncertainty into account, robust and chance formulations of the constraints are commonly considered. However, this requires known error bounds or a known distribution for the model uncertainty, and the resulting formulations may suffer from over-conservatism or over-confidence. In this paper, we assume that only a finite set of model parametric uncertainty samples is available and formulate a distributionally robust chance-constrained program (DRCCP) for control synthesis with CBF safety and CLF stability guarantees. To enable the efficient computation of control inputs during online execution, we provide a reformulation of the DRCCP as a second-order cone program (SOCP). Our formulation is evaluated in an adaptive cruise control example in comparison to 1) a baseline CLF-CBF quadratic programming approach, 2) a robust approach that assumes known error bounds of the system uncertainty, and 3) a chance-constrained approach that assumes a known Gaussian Process distribution of the uncertainty.

READ FULL TEXT VIEW PDF
02/19/2022

Safe Control Synthesis with Uncertain Dynamics and Constraints

This paper considers safe control synthesis for dynamical systems in the...
12/03/2022

Distributionally Robust Lyapunov Function Search Under Uncertainty

This paper develops methods for proving Lyapunov stability of dynamical ...
06/13/2021

Pointwise Feasibility of Gaussian Process-based Safety-Critical Control under Model Uncertainty

Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) a...
11/01/2021

Safe Learning of Linear Time-Invariant Systems

We consider safety in simultaneous learning and control of discrete-time...
03/18/2019

A Control Lyapunov Perspective on Episodic Learning via Projection to State Stability

The goal of this paper is to understand the impact of learning on contro...
12/29/2020

Control Barriers in Bayesian Learning of System Dynamics

This paper focuses on learning a model of system dynamics online while s...

I Introduction

With the increasing deployment of automatic control systems and robotic platforms in unstructured real-world environments, it is crucial to develop feedback controllers with safety and stability guarantees in the presence of model uncertainty. Enforcing safety by utilizing set invariance properties has become a mainstream approach for constrained control synthesis. Inspired by the property of control Lyapunov functions (CLFs) [sontag1989universal] to yield invariant level sets, control barrier functions (CBFs) [wieland2007] were introduced as a tool to verify that a desired safe subset of the state space is invariant. Stability and safety can be considered simultaneously by introducing CLF and CBF constraints on the control input in a quadratic program (QP) formulation for control synthesis [ames2014cdc, ames2016control]. The reliability and efficiency of CLF-CBF-QP control synthesis has been evidenced in several robotic applications, including multi-agent systems [Wang2017TRO], aerial robots [Wu2016ACC], and walking robots [nguyen2016cdc].

The notion of safety in the presence of system model uncertainty has been mainly described in two ways: using robust constraints [freeman_robust, Petersen2014RobustCO] or chance constraints [Masahiro_ccdp, Zhu2019ChanceConstrainedCA]. Studies have also considered system uncertainty when pairing safety with stability in the CLF-CBF-QP formulation. Regarding robust formulations, Choi et al. [Jason2020cdc] consider model disturbances with a compact and convex support set and propose a robust control barrier value function to ensure safety. Similarly, [Nguyen2022RobustSC] assumes bounded model uncertainty and reformulates the original safety and stability constraints as min-max constraints. Regarding probabilistic formulations, [dhiman2020control, Long2022RAL] assume a Gaussian Process distribution for the model uncertainty and propose probabilistic versions of the CLF stability and CBF safety constraints. All these approaches require known error bounds or known distributions of the uncertainty. In addition, robust formulations may suffer from over-conservatism due to the worst-case error bounds, while chance-constrained formulations may suffer from over-confidence due to a distributional shift at deployment time.

To tackle such scenarios, here we rely on a body of work from the literature on stochastic programming [AS-DD-AR:14] that considers distributionally robust versions of stochastic optimization problems, see e.g. [AB-LEG-AN:09, AS:17]

. In particular, distributionally robust chance-constrained programs (DRCCP) deal with uncertain variables in the constraints when only finitely many samples are available. The main idea is to construct an ambiguity ball centered at the empirical distribution obtained from the observed samples and with radius defined using a probability distance function, such as Kullback–Leibler divergence

[Jiang2016DatadrivenCC] or Wasserstein distance [Esfahani2018DatadrivenDR, Chen2018DataDrivenCC, Xie2021OnDR, Hota2019DataDrivenCC, DB-JC-SM:21-tac]. In DRCCP, the desired constraints must be satisfied with high probability for all distributions in the constructed ambiguity set. The advantage over standard chance-constrained program formulations is the powerful guarantee on out-of-sample performance.

The contributions of this work are summarized as follows. First, we relax the assumption for safe and stable control synthesis that known error bounds or known distribution of model uncertainty are available by formulating distributionally robust safety and stability constraints using offline model uncertainty samples. Second, we show that the DRCCP control synthesis problem can be reformulated as a second-order cone program (SOCP) in two cases: when there is no restriction on the uncertainty support set and when the uncertainty support set is polyhedral. We demonstrate on an adaptive cruise control problem how our DRCCP SOCP guarantees safety in scenarios with incorrect model uncertainty error bounds or uncertainty distribution shift, in contrast with the vanilla CLF-CBF-QP approach, a robust approach, and a chance-constrained approach.

Ii Preliminaries

This section reviews control Lyapunov and control barrier functions, distributionally robust modeling, and chance-constrained programming.

Ii-a Optimization-based Control Synthesis

Consider a non-linear control-affine system111Notation. The sets of real, non-negative real, and natural numbers are denoted by , , and , respectively. For , we let

. We denote the distribution and expectation of a random variable

by and , respectively. We use and to denote the

-dimensional vector with all entries equal to

and , respectively. For scalar , we define . The norm for a vector is denoted by . We denote by

the identity matrix and by

the Kronecker product. We use to denote the vectorization of , obtained by stacking its columns. The gradient of a differentiable function is denoted by , while its Lie derivative along a vector field by . A continuous function is of class if it is strictly increasing and . A continuous function is of extended class if it is of class and .:

(1)

where is the state and is the control input. Assume and are locally Lipschitz. We start by recalling the notions of CLF [sontag1989universal] and CBF [ames2016control], which play a key role in the synthesis of stable and safe controllers, respectively.

Definition II.1.

A positive-definite continuously differentiable function is a control Lyapunov function (CLF) on for system (1) if there exists a class function such that:

(2)

where the control Lyapunov condition (CLC) is:

(3)

The existence of a CLF simplifies the stabilization problem considerably because a stabilizing feedback control law can be obtained in terms of the derivatives of the CLF [sontag1989universal].

In addition to stability, it is often necessary to ensure that the closed-loop system trajectories remain within a safe set . To facilitate safe control synthesis, the safe set is specified as the zero superlevel set, , of a function .

Definition II.2.

A continuously differentiable function is a control barrier function (CBF) on for system (1) if there exists an extended class function such that:

(4)

where the control barrier condition (CBC) is:

(5)

Noting that the CLF stability requirement in (2) and the CBF safety requirement in (4) are affine in , they can be enforced as constraints in an optimization problem. Given a baseline controller , the following QP modifies the controller to guarantee safety and encourage stability:

(6)

where is a slack variable that relaxes the CLF constraints to ensure the feasibility of the QP, controlled by the scaling factor .

We are interested in the control synthesis problem in (6) when the system dynamics in (1) are not perfectly known. Considering probabilistic uncertainty in the system model requires probabilistic versions of the safety and stability constraints in (6). We investigate how to handle model uncertainty using samples rather than a known distribution and whether the uncertainty-aware versions of the constraints in (6) remain convex and tractable.

Ii-B Distributionally Robust Chance-constrained Programming

To handle probabilistic constraints, we begin by reviewing chance-constrained programming. Throughout the paper we consider a complete separable metric space with metric and associate with it a Borel -algebra and the set of Borel probability measures on . A chance-constrained program (CCP) takes the form:

(7)

with closed convex set and uncertainty set . The constraint function depends both on the decision vector and an uncertainty vector , whose distribution is supported on , and is a user-specified risk tolerance. The feasible set defined by the chance constraint in (7) is not convex in general. Nemirovski and Shapiro [Nemirovski2006ConvexAO] proposed a conservative convex approximation [Nemirovski2006ConvexAO] of the feasible set in (7), which consists of replacing the chance constraint by a conditional value-at-risk (CVaR) constraint:

(8)

The feasible set of (8) is a subset of the feasible set of (7). The following paragraph describes a way of defining CVaR.

Value-at-risk (VaR) at confidence level for is defined as for a random variable with distribution . VaR does not provide information about the right tail of the distribution, and optimization programs involving VaR variables are intractable in general [Mausser_1999]. To address this, Rockafellar and Uryasev [Rockafellar00optimizationof] introduced conditional value-at-risk (CVaR), defined as . CVaR can be also formulated as a convex program:

(9)

Both the formulations in (7) and (8) assume that , the true distribution of , is known. When this is not the case, one can instead resort to distributionally robust formulations [Esfahani2018DatadrivenDR, Xie2021OnDR]. Assume we only have access to samples from the true distribution of . We describe a way of constructing an ambiguity set of distributions that could have potentially generated such samples. Let be the set of Borel probability measures with finite

-th moment for

. The -Wasserstein distance [Hota2019DataDrivenCC] between two probability measures , in is:

(10)

where denotes the measures on with marginals and on the first and second factors, and denotes the metric in the space .

Let denote the discrete empirical distribution constructed from the observed samples . Using the Wasserstein distance (10), one can define a Wasserstein ambiguity set of radius centered at :

(11)

and, in turn, a distributionally robust chance-constrained program (DRCCP):

(12)

The constraint in (12) is equivalent to . Thus, mimicking the convexification for CCP in (8), one can use CVaR to obtain a convex approximation of (12):

(13)

Iii Problem Formulation

We study the problem of enforcing safety and stability of control-affine dynamical systems with model uncertainty. Critically, we do not assume that the probability distribution or error bounds for the model uncertainty are known. We model the uncertainty in the system in (

1) using a nominal model and a linear combination of perturbations:

(14)

For , we use to denote the possible model perturbations, and to denote the corresponding unknown weight. We let .

We assume that offline observations about the uncertainty are available. Many control applications require safety and stability guarantees for an uncertain system under online error realizations. This motivates us to consider a distributionally robust formulation for online control synthesis.

Problem 1 (Distributionally Robust Safety and Stability for Uncertain Systems).

Consider a nominal model and perturbation matrices , for the system dynamics in (1). Given observations of the model uncertainty with support set , design a feedback controller with a risk-tolerance parameter such that, for each :

(15)

where , are Wasserstein ambiguity sets with user-specified radii and .

We consider two cases based on the information available about the support set . In the first case, we consider a unbounded support set ; in the second case, we assume a compact polyhedron set .

Inspired by the CLF-CBF-QP in (6), we consider the following DRCCP formulation to enforce safety and stability with high probability and out-of-sample errors by leveraging the CVaR approximations (13) and optimization definition of CVaR (9),

(16)

Although the constraints in (16) are convex, the program is intractable [Hota2019DataDrivenCC, Esfahani2018DatadrivenDR] due to the search of suprema over the Wasserstein ambiguity set. In the following sections, we discuss our approach to identify tractable reformulations of (16) and enable online stable and safe control synthesis.

Iv Tractable Reformulation of Control Synthesis With Model Uncertainty

This section presents our approach for solving (16). To simplify the notation, we use the vectorization of ,

(17)

where

Observe that the CBC expression in (5) is affine in both and . Using the Kronecker product property and in (17), we have:

(18)

We can also write with similar definitions. Since , , , and are known and deterministic, both and are affine in .

We consider a general optimization program:

(19)

where may represent a safety or stability constraint that is affine in :

(20)

We write as for brevity. Depending on the information available about the uncertainty space , we propose two reformulations of (19). In either case, we assume the metric of is the Euclidean distance.

Iv-a Reformulation with Unbounded Uncertainty Space

First, we consider the case with no prior knowledge of , meaning that . We show that the constraints in (19) can be reformulated as second-order cone constraints.

Proposition IV.1 (DRCCP formulation with unbounded support set).

Consider the optimization problem in (19) with in (20), -Wasserstein distance with , and . Then, the following SOCP is equivalent to (19):

(21)
Proof.

We start by considering the following program:

(22)

Based on [Hota2019DataDrivenCC, Lemma V.8] and assuming , the supremum over the Wasserstein ambiguity set (i.e. the constraint in (19)) can be written equivalently as the sample average and a regularization term , where denotes the Lipschitz constant of in .

As defined in (20), for each , we can define the convex function by

(23)

Then, the function is Lipschitz in with constant s for fixing (assuming ). This is because the Lipschitz constant of a differentiable affine function equals the dual-norm of its gradient [shai_convex], and the dual norm of the norm is itself. This implies that (22) is equivalent to (19).

Next, we show that the bi-level optimization in (22) is equivalent to:

(24)

For , let denote an optimal solution to (24) and an optimal solution to (22), with the optimizer for the terms in the constraint of (22).

Given , we have and

(25)

Thus, if is replaced by in (25), we conclude that the constraint in (22) is satisfied with and . This implies that is also a solution to (22), and the cost satisfies .

Given and , for every , we choose . This implies , , and the first constraints in (24) is satisfied since

Thus, is also a solution to (24). Furthermore, the cost satisfies since is an optimal solution to (24). Therefore, both costs are equal, and (24) and (22) are equivalent.

Finally, by reformulating the objective function of (24) as a linear objective with an SOC constraint [Long2022RAL, Proposition IV.3], we conclude the SOCP (21) is equivalent to (19). ∎

Proposition IV.1 allows control synthesis with distributionally robust safety and stability constraints without prior knowledge about the uncertainty support set . The SOCP in (21) can be solved efficiently online using an off-the-shelf solver (e.g. [mosek]).

Iv-B Reformulation with Bounded Uncertainty Space

Assuming no prior knowledge about the uncertainty set may result in an overly conservative controller. This motivates us to also consider the case that the uncertainty support set is a compact polyhedron.

Proposition IV.2 (DRCCP formulation with bounded polyhedron support set).

Consider the optimization problem in (19) with in (20), -Wasserstein distance with , and compact , where and for some . Then, the following SOCP is equivalent to (19),

(26)
Proof.

Based on [Hota2019DataDrivenCC, Proposition V.1] and [Esfahani2018DatadrivenDR, Corollary 5.1], we know the following program is equivalent to (19),

(27)

Next, we aim to rewrite (27

) as a SOCP. The ReLU-type inequality

can be written equivalently as two constraints: and . Following the same technique as in Proposition IV.1, we conclude that (26) is equivalent to (19). ∎

Remark IV.3 (Comparison between the two formulations).

If and in Proposition IV.2, then and the SOCP in (26) reduces to (21).

Remark IV.4 (Different choice of metric ).

If instead of norm, we take the metric of to be the norm, then the optimization problems in Propositions IV.1 and IV.2 become QPs. Details are provided in Appendix A.

V Evaluation

We evaluate the proposed distributionally robust approach for safe and stable control synthesis in an adaptive cruise control problem introduced in [ames2014cdc].

V-a Cruise Control Model

Consider a simplified adaptive cruise control model that consists of two vehicles, one leading vehicle traveling at a constant speed and one following vehicle using our control synthesis methodology. The objective is have the following vehicle achieve a desired speed while keeping a safe distance from the leading vehicle. The system model is:

(28)

where and are the velocities of the following and leading vehicles, respectively, is the air drag, is the following vehicle position, and is the distance to the leading vehicle. The input is constrained by , where and denote the factor of for deceleration and acceleration, respectively. We define a CLF, , where is the desired speed of the following vehicle. The safety requirements is specified by the CBF . We assume that the system (28) is uncertain with the following parametric uncertainty,

(29)

where , and:

where , , and represent the model perturbations in the drag, input force, and leading vehicle distance, respectively. Table I reports the parameter values used in the simulation.

Variable Description Value
Gravitational acceleration 9.81
Mass of vehicle 1650
Coefficient in 0.1
Coefficient in 5
Coefficient in 0.25
Desired speed 35
Speed of leading vehicle 20
Max accelerate constant 0.3
Max decelerate constant -0.3
TABLE I: Parameters used in the simulation results

V-B Results

We evaluate our distributionally robust control synthesis approach and illustrate its versatility in handling model uncertainty. We report simulation results from the unbounded uncertainty formulation (Proposition IV.1) and the bounded uncertainty formulation (Proposition IV.2). For comparison, we include results from the CLF-CBF-QP (which takes no model uncertainty into account) formulation in [ames2014cdc] with baseline controller , the robust (which requires prior knowledge on the error bound) and the chance-constrained (which assumes the uncertainty distribution to be Gaussian) formulations in [Long2022RAL]

. In the simulation, the error bounds are provided by the support set information and the Gaussian parameters are estimated via offline uncertainty samples. In all cases, we use the value of the CBF as a measure of the safety ensured by the corresponding approach. We consider different choices of Wasserstein radius

, confidence level , support set , offline uncertainty samples , and online true uncertainty realization . To demonstrate that our formulation ensures safety for out-of-sample uncertainty, we use different distributions for sampling offline observations and a true online uncertainty realization .

We consider cases with different parameter choices, where and

denote normal and beta distributions, respectively. For each case, we conduct 50 simulations with the same

and different .

Case 1 (Gaussian Distribution):

.
Case 2 (Confident in Sample): .
Case 3 (Out of Sample): .
Case 4 (Baseline Radius and Confidence): .
Case 5 (Larger Radius): .
Case 6 (Higher Confidence): .
Case 7 (Larger Radius and Higher Confidence): .
Case 8 (Out of Support): .

In Table II, we report the failure rate and the average CBF values for the cases above. In Cases 1 and 2, under Gaussian uncertainty in the dynamics model, all formulations ensure safety except the CLF-CBF-QP. When we set the Wasserstein radius small (), meaning that we are confident in the offline uncertainty samples, the unbounded DRCCP and bounded DRCCP formulations have the same mean CBF values. In Case 3, we verify that if the uncertainty distribution shifts during the online phase (e.g., the online uncertainty no longer from a Gaussian distribution), then the Gaussian CLF-CBF-SOCP formulation fails, while the other three formulations ensure safety. Cases 4 to 7 demonstrate the effects of the Wasserstein distance and confidence level in our bounded and unbounded DRCCP formulations. On the one hand, the unbounded DRCCP formulations tend to be more conservative if we increase the Wasserstein radius and/or the confidence level, as shown in Fig. 1. On the other hand, only increasing the confidence level makes the bounded DRCCP controller more conservative, since support information provides a tighter bound than the Wasserstein radius. In Case 8, we see that the unbounded DRCCP formulation works well even with out-of-support uncertainty, while the robust CLF-CBF-SOCP and bounded DRCCP both fail due to the provided incorrect support set information, as Fig. 2 shows.

Generally, the controller provided by the bounded DRCCP formulation has the best performance in ensuring safety while not being too conservative (smaller average CBF values). However, if one fails to provide reliable support set information, then the controller provided by the unbounded DRCCP formulation is the safe choice.

Fig. 1: CBF value of one of the 50 simulations corresponding to Case 5. Both the unbounded and bounded DRCCP formulations ensure safety while the CLF-CBF-QP and the Gaussian formulation fail. This demonstrates that either the bounded or unbounded DRCCP formulation ensures safety for out-of-sample uncertainty. The unbounded DRCCP formulation is more conservative since it does not take the uncertainty support set information into account.
Fig. 2: CBF value of one of the 50 simulations corresponding to Case 8. The offline uncertainty distribution is set to be within the uncertainty support set , while the online uncertainty distribution is outside of . The controller obtained with the bounded DRCCP formulation (26) fails to guarantee safety because the assumptions in Proposition IV.2 are violated. However, the controller obtained with the unbounded DRCCP formulation (21) still guarantees safety.
Case Unbounded Bounded Robust Gaussian Original
DRCCP DRCCP CLF-CBF-SOCP CLF-CBF-SOCP CLF-CBF-QP
1 0% 2.11 0% 1.01 0% 0.55 100% -0.48
2 0% 1.01 0% 0.56 100% -0.48
3 0% 1.23 0% 0.96 98% -0.17 100% -0.50
4 0% 0.56 0% 0.89 100% -0.92 100% -0.53
5 0% 1.42 0% 1.01 100% -0.88 100% -0.48
6 0% 1.94 0% 1.01 100% -0.65 100% -0.48
7 0% 4.49 0% 1.01 100% -0.64 100% -0.49
8 98% -0.26 84% -0.16 100% -0.57 100% -0.51
TABLE II: Failure rate and average CBF values. The results are shown in the following format: , where denotes the violation rate of each formulation: (simulations with unsafe state)/(total simulations), and denotes the average value of CBF over all simulations. The average CBF value is computed based on stabilized CBF values, e.g., for in Fig. 2.

Vi Conclusions

We considered the problem of enforcing safety and stability of uncertain control-affine systems. Compared with previous approaches, we derive new distributionally robust chance constrained formulations of safe and stable control synthesis that do not require any prior knowledge of error bounds or uncertainty distributions. Using only offline model uncertainty samples, we show that our formulations ensure safety and stability with out-of-sample errors during online execution. Future work will consider deploying the algorithms on real autonomous systems and learning the perturbation matrices and uncertainty samples from offline state-control sequences.

References

Appendix A Different Choice of Metric

We show that when the metric of the uncertainty support set is the norm (instead of the Euclidean norm as in Propositions IV.1 and IV.2), then (19) becomes a QP for both the cases of unbounded and bounded uncertainty sets.

Proposition A.1 (DRCCP formulation with unbounded support set under norm).

Consider the optimization problem in (19) with in (20), -Wasserstein distance with , and with metric . Then, the following QP is equivalent to (19):

(30)
Proposition A.2 (DRCCP formulation with bounded polyhedron set under norm).

Consider the optimization problem in (19) with in (20), -Wasserstein distance with , and compact with