Finite-Data Performance Guarantees for the Output-Feedback Control of an Unknown System

03/25/2018 ∙ by Ross Boczar, et al. ∙ 0

As the systems we control become more complex, first-principle modeling becomes either impossible or intractable, motivating the use of machine learning techniques for the control of systems with continuous action spaces. As impressive as the empirical success of these methods have been, strong theoretical guarantees of performance, safety, or robustness are few and far between. This paper takes a step towards such providing such guarantees by establishing finite-data performance guarantees for the robust output-feedback control of an unknown FIR SISO system. In particular, we introduce the "Coarse-ID control" pipeline, which is composed of a system identification step followed by a robust controller synthesis procedure, and analyze its end-to-end performance, providing quantitative bounds on the performance degradation suffered due to model uncertainty as a function of the number of experiments run to identify the system. We conclude with numerical examples demonstrating the effectiveness of our method.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

There have been many recent results (see for example [1, 2, 3, 4, 5, 6] and the references within) that apply state-of-the-art machine learning techniques to the control of systems with continuous action spaces. As the systems we control become ever more complex, be it in their dynamics, their scale, or their interaction with the environment, moving to a data-driven approach will be inevitable: in these settings, first-principle modeling becomes either impossible or intractable. However, as promising and exciting as recent empirical demonstrations of these techniques have been, they have, for the most part, lacked the rigorous stability, safety and robustness guarantees that the controls community has always prided itself in providing. Indeed, such guarantees are not only desirable, but necessary when such techniques are being proposed for the control of safety critical systems or infrastructures.

This paper can be seen as a step towards providing such guarantees, albeit in a simplified setting, wherein we establish rigorous baselines of robustness and performance when controlling a single-input-single-output (SISO) system with an unknown transfer function. To do so, we combine contemporary approaches to system identification and robust control into what we term the “Coarse-ID control” pipeline. In particular, we leverage the results developed in [7]

to provide finite-sample guarantees on optimally (in a certain sense) estimating a stable single-input single-output linear time-invariant (SISO LTI) system, using input-output data pairs.

111We note that there have been recent results in the system identification literature (for example [8, 9]) that also seek to provide non-asymptotic guarantees of model estimation quality. Such finite-data guarantees are not only in stark contrast to classical system identification results, which typically only provide asymptotic guarantees of model fidelity (see [10] for an overview), but also necessary for the principled integration of these techniques with robust control, as they allow us to quantify the amount of uncertainty that our controller must contend with. We then formulate a robust control problem using the recently developed system-level synthesis (SLS) procedure [11], which exploits a novel parameterization of stabilizing controllers for LTI systems that allows us to quantify performance degradation in terms of the amount of uncertainty affecting the system [12]. Again this is in contrast to classical methods from robust control [13] that are only able to provide robust stability guarantees for a prescribed amount of uncertainty.

Main contribution

A feature of “Coarse-ID control,” as described above, is that we can analyze the end-to-end performance of this pipeline in a non-asymptotic setting. Specifically, we show that the difference in cost between the optimal cost for the true system (an FIR SISO system of length ) and the realized cost induced by instead solving a robust SLS procedure for the approximate system is . Here, we assume that the approximate system was estimated using the “optimal” coarse-grained system identification procedure described in Tu et al. [7], with

the measurement noise variance and

the number of experiments conducted in order to construct an estimate of the system. Finally, this paper should be viewed as a step towards generalizing the results in [6], which provides finite-data end-to-end performance guarantees for the classical LQR optimal control problem, to the output-feedback setting.

Paper organization

In Section 2 we fix notation and quickly outline the structure used by common robust control problems. Section 3 then gives an overview of the system-level synthesis framework and how it can be used to solve these problems. Finally, in Section 4 we combine this framework with recent work on coarse-grained identification to provide quantitative bounds on how the performance of a robust controller synthesized using the SLS framework degrades when the plant to be controlled is only approximately identified. We conclude in Section 5 with computational examples.

2 Preliminaries

Notation

We use boldface to denote frequency domain signals and transfer functions. The

-th standard basis vector is given by

. A discrete-time dynamical system

can be represented compactly by or the tuple (with implying ). The set of stable real-rational proper transfer matrices is denoted . Unless otherwise noted, represents the -norm (the induced norm) for elements in (this reduces to the spectral norm for constant matrices).

2.1 The standard robust control problem

We first introduce a standard form for generic robust and optimal control problems, and then show how simple disturbance attenuation and reference tracking problems can be cast into this standard form. We work with discrete-time LTI systems, but unless stated otherwise, all results extend naturally to the continuous-time setting. A system in standard form can be described by the following equations:

(2.1)

where is the regulated output (e.g., deviations of the system state from a desired set-point), is the measured output available to the controller to compute the control action , and is the exogenous disturbance. We further assume that the full plant admits a joint realization222We assume throughout that is strictly proper—it follows that is a necesssary and sufficient condition for internal stability of the closed loop system shown in Figure 1 [13]., i.e.

(2.2)

where .

The standard optimal control problem of minimizing the gain from exogenous disturbance to regulated output , subject to internal stability of the closed loop system shown in Figure 1, can then be posed as

(2.3)
Figure 1: The standard optimal control problem (2.3) for the plant as defined in equations (2.1) and (2.2).

Disturbance rejection

Consider the feedback system shown in Figure 2, wherein a controller is in feedback with a SISO plant , with input disturbance and measurement noise . We can then define the disturbances and outputs as

respectively, where . Furthermore, let the plant have a state-space realization . We then have that

from which it follows that the generalized plant admits the joint realization

Figure 2: The disturbance rejection problem for a SISO plant with input disturbance and measurement noise .

Reference tracking

Now, consider the feedback system shown in Figure 3, wherein a controller is in feedback with a SISO plant , with input disturbance and reference signal . We can then define the disturbances and outputs as

respectively, where . Furthermore, let the plant have a state-space realization . We then have that

from which it follows that the full plant admits the joint realization

Figure 3: The reference tracking problem for a SISO plant with disturbance and reference .

Specialization to FIR plant G

Suppose that is strictly proper and has a finite impulse response (FIR) of order , i.e., that for a collection of real scalars . Defining , the plant admits the state-space realization where is the right-shift operator (i.e., a matrix with ones one the sub-diagonal and zeros elsewhere). Given the examples presented thus far, going forward we assume that

(2.4)

as well as the standard assumption that . Additionally, given that we are considering SISO systems, we can without loss of generality (by suitably rescaling ) assume that

(2.5)

2.2 Coarse-grained identification

As our aim is to provide end-to-end guarantees for robust control problems, we must first have a scheme to acquire an approximate plant model . Toward that end, Coarse-grained identification, as defined in Tu et al. [7], describes the following procedure:

  1. carefully choose a series of inputs , where , and collect noisy outputs where with

  2. form a least-squares estimate of the impulse response of using .

We refer to each such pair as an experiment.

In [7], upper and lower bounds are shown on the resulting error between and for different sets . This built on the work of Goldenschluger [14], who derived estimation rates for -constrained inputs. We make slight modifications to the results in [7] to instead provide error bounds on the impulse response coefficients, as these are more natural for our problem. One concern is that we will need bounds on the impulse response error, and these bounds are in term of the -norm. However, while they can conservatively be plugged in verbatim (as ), we will instead modify their proofs slightly to fit our application.

3 System-Level Synthesis

The System-Level Synthesis (SLS) framework, proposed by Wang et. al [11], provides a parameterization of stabilizing controllers that achieve specified responses between disturbances and outputs. We briefly review here the SLS framework, and later show in Section 4.1 how it can be modified to solve a robust optimal control problem subject to bounded uncertainty on the FIR coefficients .

For an LTI system with dynamics described by (2.2), we define a system response to be the maps satisfying

(3.1)

where is the process noise, and is the measurement noise.

We call a system response stable and achievable with respect to a plant if there exists an internally stabilizing controller such that the control rule leads to closed loop behavior consistent with (3.1). It was shown in [11] that the parameterization of all stable and achievable system responses is defined by the following affine space:

(3.2a)
(3.2b)
(3.2c)

We call equations (3.2a) - (3.2c) the SLS constraints. The parameterization of all internally stabilizing controllers is given by the following theorem. [Theorem 2 in [11]] Suppose that a system response satisfies the SLS constraints (3.2a) - (3.2c). Then, is an internally stabilizing controller for the plant (2.2) that yields the desired system response (3.1). Furthermore, the solutions of (3.2a) - (3.2c) with the implementation parameterize all internally stabilizing controllers for the plant (2.2). Using this parameterization, we can recast the standard optimal control problem (2.3) as the SLS problem

(3.3)

In the FIR case, we use the abbreviated notation for the case where is the plant . Although we focus on the optimal control problem posed in equation (3.3), the results that follow carry over naturally to (LQG) and optimal control problems as well.

4 Sample Complexity Bounds

We now provide finite-data performance guarantees for a controller synthesized using the system identification and robust synthesis procedures described in the previous sections. Prior to stating our main results, we recall the problem set up and Coarse-ID Control pipeline.

We consider the identification and control of the system , which is assumed to be FIR of order . We begin with the simplified setting that the order of the true system is known, and we use the Coarse-grained identification procedure described in Section 2.2 to identify an approximate system , also of order , using a series of experiments. We then use this approximate system

, as well as high-probability bounds on the estimation error

, in a robust SLS problem (see (4.6) in Section 4.1) to compute a controller with provable suboptimality guarantees, as formalized in the following theorem. 333Here, hides universal constants: see Lemma 4.3 for an explicit characterization. Let be the optimal solution of the SLS problem (3.3) for the plant , and let be an estimate of obtained using coarse-grained identification (-variance output noise only) with experiments, where . Let be the optimal solution to the robust SLS problem (4.6) for , and let be the response achieved on the true system by the synthesized controller . Then, if , with probability at least , the controller stabilizes the true system and has a suboptimality gap bounded by

Assume that we are in the setting of Theorem 4, and further let there be process noise with variance that enters the system via the same channel as the control input (i.e., ) and measurement noise with variance . Then, Theorem 4 holds with .

We can further generalize these results to the setting where the order of the underlying system is not known, and that the true system is approximated by a length- FIR filter with coefficients where . In this case, applying the triangle equality

gives a similar sample complexity bound, albeit one where the cost difference does not tend to zero as the number of experiments tends to infinity. Assume that we are in the setting of Theorem 4, except let be a length- (where ) FIR estimate of obtained using the prescribed coarse-grained identification. Furthermore, assume that . Then, if

with probability at least the controller stabilizes the true system and has a suboptimal cost bounded by

To prove the above results, we first derive a robust variant of the SLS framework presented in Section 3, and then show how it can be used to pose a robust synthesis problem that admits suboptimality guarantees. In particular, these guarantees characterize the degradation in performance of the synthesized controller as a function of the size of the uncertainty on the transfer function coefficients . We then combine this characterization of performance degradation with high-probability bounds on the estimation error produced by the coarse-grained identification procedure to provide an end-to-end analysis of the Coarse-ID control procedure.

4.1 Robust SLS

As we only have access to approximately identified plants, we need a robust variant of Theorem 3.2. First, we introduce a robust version of (3.2b),

(4.1)

We call equations (3.2a), (4.1), and (3.2c) the robust SLS constraints. We now have the ingredients needed to connect the main and robust SLS constraints. The proof is mostly algebraic and is thus deferred to the Appendix. [Robust Equivalence] Consider system reponses and , where the latter is given by

(4.2)

where by assumption exists and is in . Let be a given plant, and consider the following statements.

  1. satisfies the robust SLS constraints for .

  2. satisfies the SLS constraints for .

Under the assumptions, . Furthermore, let , and let

Then, is equivalent to a third statement : satisfies the SLS constraints for .

A chain of corollaries follow from Lemma 4.1 that will be useful in quantifying the performance achieved on the true system of a controller designed using an approximate system model. Unless otherwise noted, let be defined as in Lemma 4.1.

Suppose that satisfies the robust SLS constraints for the system (2.2). Then, the controller stabilizes the system (2.2) and achieves the closed-loop system response if and only if .

Proof.

First note that the robust SLS constraints imply that . Next, assume . By Lemma 4.1, satisfies the SLS constraints for (2.2). Thus, by Theorem 3.2, is stabilizing and achieves the closed-loop response . Moreover, is precisely equal to .

Conversely, assume exists but is not in (if it does not exist the system response (4.2) is obviously not well-defined). It then follows that is not in as is square and invertible. ∎

This immediately gives us a sufficient condition for robustness of the SLS procedure. Suppose that satisfies the robust SLS constraints for the system (2.2). A sufficient condition for the controller to stabilize the system (2.2) and achieve closed-loop response is that , for any induced norm .

Proof.

It follows from the small-gain theorem [13] that , and thus Corollary 4.1 applies. ∎

We now specialize our results to the case where the plant , as defined in (2.2), is FIR. In this case, the modeling error arises only in the coefficient vector defining the impulse response. To that end, we define the the estimated plant with the realization and note that the resulting error arises only in the and terms of the corresponding estimated plant , where these state-space parameters are defined as in (2.4). To that end, we define the estimation error vector allowing us to further specialize Corollary 4.1. Suppose satisfies the SLS constraints for the estimated system . If for any induced norm , then the controller stabilizes the true system and achieves the closed-loop response as specified in (4.2). Additionally, if the induced norm is either the or norm, the response simplifies to

(4.3)
Proof.

By Lemma 4.1, satsisfies the robust SLS constraints for . The sufficient condition then follows by applying Corollary 4.1.Furthermore, if the induced norm used in Corollary 4.1 and Corollary 4.1 is either the or norm, it follows from Hölder’s inequality that implies that on . Hence, we can use the Sherman-Morrison identity,

to simplify the closed loop response (4.2) achieved by the approximate controller to the expression (4.3). ∎

We now use this robust parameterization to formulate a robust SLS problem that yields a controller with stability and performance guarantees.

We will use these two facts without fanfare in the following sections. Let . Then . for all .

Define to be the performance (i.e. the objective in (3.3)) of the controller induced by when placed in closed-loop with the FIR plant specified by impulse response coefficients . Now, assume we design a response , with corresponding controller , that satisfies the SLS constraints specified by the estimate system . We saw in the previous section that under suitable conditions, the response on the true system is given by , as specified in Corollary 4.1. By the triangle inequality, Corollary 4.1, and our parametric assumption (2.4)-(2.5), we can then bound the difference between expectation and reality as follows:

where we assume for the bound to be valid.

For any estimated response satisfying , it then follows that

(4.4)

for any satisfying

(4.5)

noting that , which implies , is equivalent to . We denote the right-hand side of this bound as

The bound (4.4) then suggests the following robust controller synthesis procedure, which balances between solving for the optimal controller for the approximate system and controlling a perturbative term. We call this problem the robust SLS problem for .

(4.6)

Although this problem is not jointly convex in and the system responses , one can use a golden section search on in practice. Moreover, the sum of norms can be split into two norm constraints using an epigraph formulation (see [15], Ch. 3).

4.2 Sub-optimality guarantees for robust SLS

We now show a bound on the change in the optimal control cost when the controller is synthesized using the robust SLS problem (4.6).

Let and , as well as , be defined as in Theorem (4), and let . If , we have that

(4.7)

To prove this proposition, we require a technical lemma that ensures that the true controller stabilizes the estimate system specified by the FIR coefficients , i.e. that the optimal system response can be used to construct a feasible solution to the approximate SLS synthesis problem (4.6). Let and its induced controller be as defined in Theorem 4, and let , with . Then

(4.8)

is strictly positive, and the controller is stabilizing for the estimate system specified by and achieves the system response defined by

(4.9)

Furthermore, are feasible solutions to the approximate SLS synthesis problem (4.6).

Proof.

Both of these points are conditional on existing in :

  • By assumption, satisfies the SLS constraints for ; by Lemma 4.1, it equivalently satisfies the robust SLS constraints for , where and (note the switched roles of and ). By Corollary 4.1, then stabilizes and achieves the system response .

  • By Lemma 4.1, satisfies the SLS constraints for , and is thus part of a feasible point for the approximate SLS synthesis problem (4.6). Now, we need to check that the corresponding is also part of a feasible solution. Toward that end, by the Sherman-Morrison identity, we see that

    Furthermore,

    Therefore,

    the final feasibility condition of (4.6).

It therefore remains to verify that exists and is in . As we have seen, a sufficient condition is , and this condition is implied by the assumption that . ∎

Proof of Proposition 4.2.

We immediately invoke Lemma 4.2 by noting that our assumption on ensures , and we are assured that is a feasible point for the approximate SLS synthesis problem (4.6). From inequality (4.4), we then have that

(4.10)

where the second inequality follows from the optimality of , and the final inequality from the definitions of and . Now, we repeat the argument used to derive (4.4) with expectation and reality reversed: this time we assume our design expectation was but our reality is . This is a valid analogy as satisfies the SLS equations for . With the true and estimated parameters reversed, we can thus bound by

(4.11)

Finally, combining bounds (4.10) and (4.11) and plugging in gives

4.3 Coarse-grained ID and the proof of Theorem 4

First, to prove the sample complexity of synthesizing a stabilizing controller based on an approximate system, we require an intermediary lemma on how well coarse-grained identification can identify the true system. The proof of the lemma (and the related change necessary for Corollary 4) is deferred to the Appendix. [] Assume we estimate the system by a length- FIR system using coarse-grained identification (output noise only) on experiments, where the inputs are constrained to lie in a unit ball. Then, with probability at least ,

Taking large enough such that (implied by taking ), we have

Finally, we show that is stabilizing for the true system . Since is optimal for the approximate SLS synthesis problem for , it is feasible, and thus allows us to invoke Corollary 4.1, as we have that .

5 Experiments

The robust SLS procedure analyzed in the previous section requires solving an infinite-dimensional optimization problem as the responses are not required to be FIR. However, as an approximation, we limit them to be FIR responses of a prescribed length . By making this restriction, the resulting optimization problem is then finite-dimensional and admits an efficient solution using off-the-shelf convex optimization solvers444Code for these computations can be found at https://github.com/rjboczar/OF-end-to-end-CDC.

Figure 4 shows a quantification of this approximation. In this experiment, for each

, we chose random FIR plants with impulse response coefficients uniformly distributed in

. We then computed the smallest such that the robust performance returned by the SLS program was within 2% relative error of the performance calculated by MATLAB’s hinfsyn with relative tolerance . Figure 4 also shows this calculation when each plant was normalized to have unit -norm.

Figure 4: FIR approximation length required to achieve small relative error in the robust performance objective. For each plant length

, the boxes denote the middle quartiles for

random plants, and the whiskers show the extents of the data.

5.1 Optimization Model

Let , , and be the (static) identity transfer function. Furthermore, appealing to the SDP characterization of -bounded FIR systems ([16] Thm. 5.8), define

Then, under the approximate assumption that are FIR of length , and using the notation for the -th block of , we can write the full optimization problem of solving (4.6) for a fixed :