1 Introduction
There have been many recent results (see for example [1, 2, 3, 4, 5, 6] and the references within) that apply stateoftheart machine learning techniques to the control of systems with continuous action spaces. As the systems we control become ever more complex, be it in their dynamics, their scale, or their interaction with the environment, moving to a datadriven approach will be inevitable: in these settings, firstprinciple modeling becomes either impossible or intractable. However, as promising and exciting as recent empirical demonstrations of these techniques have been, they have, for the most part, lacked the rigorous stability, safety and robustness guarantees that the controls community has always prided itself in providing. Indeed, such guarantees are not only desirable, but necessary when such techniques are being proposed for the control of safety critical systems or infrastructures.
This paper can be seen as a step towards providing such guarantees, albeit in a simplified setting, wherein we establish rigorous baselines of robustness and performance when controlling a singleinputsingleoutput (SISO) system with an unknown transfer function. To do so, we combine contemporary approaches to system identification and robust control into what we term the “CoarseID control” pipeline. In particular, we leverage the results developed in [7]
to provide finitesample guarantees on optimally (in a certain sense) estimating a stable singleinput singleoutput linear timeinvariant (SISO LTI) system, using inputoutput data pairs.
^{1}^{1}1We note that there have been recent results in the system identification literature (for example [8, 9]) that also seek to provide nonasymptotic guarantees of model estimation quality. Such finitedata guarantees are not only in stark contrast to classical system identification results, which typically only provide asymptotic guarantees of model fidelity (see [10] for an overview), but also necessary for the principled integration of these techniques with robust control, as they allow us to quantify the amount of uncertainty that our controller must contend with. We then formulate a robust control problem using the recently developed systemlevel synthesis (SLS) procedure [11], which exploits a novel parameterization of stabilizing controllers for LTI systems that allows us to quantify performance degradation in terms of the amount of uncertainty affecting the system [12]. Again this is in contrast to classical methods from robust control [13] that are only able to provide robust stability guarantees for a prescribed amount of uncertainty.Main contribution
A feature of “CoarseID control,” as described above, is that we can analyze the endtoend performance of this pipeline in a nonasymptotic setting. Specifically, we show that the difference in cost between the optimal cost for the true system (an FIR SISO system of length ) and the realized cost induced by instead solving a robust SLS procedure for the approximate system is . Here, we assume that the approximate system was estimated using the “optimal” coarsegrained system identification procedure described in Tu et al. [7], with
the measurement noise variance and
the number of experiments conducted in order to construct an estimate of the system. Finally, this paper should be viewed as a step towards generalizing the results in [6], which provides finitedata endtoend performance guarantees for the classical LQR optimal control problem, to the outputfeedback setting.Paper organization
In Section 2 we fix notation and quickly outline the structure used by common robust control problems. Section 3 then gives an overview of the systemlevel synthesis framework and how it can be used to solve these problems. Finally, in Section 4 we combine this framework with recent work on coarsegrained identification to provide quantitative bounds on how the performance of a robust controller synthesized using the SLS framework degrades when the plant to be controlled is only approximately identified. We conclude in Section 5 with computational examples.
2 Preliminaries
Notation
We use boldface to denote frequency domain signals and transfer functions. The
th standard basis vector is given by
. A discretetime dynamical systemcan be represented compactly by or the tuple (with implying ). The set of stable realrational proper transfer matrices is denoted . Unless otherwise noted, represents the norm (the induced norm) for elements in (this reduces to the spectral norm for constant matrices).
2.1 The standard robust control problem
We first introduce a standard form for generic robust and optimal control problems, and then show how simple disturbance attenuation and reference tracking problems can be cast into this standard form. We work with discretetime LTI systems, but unless stated otherwise, all results extend naturally to the continuoustime setting. A system in standard form can be described by the following equations:
(2.1)  
where is the regulated output (e.g., deviations of the system state from a desired setpoint), is the measured output available to the controller to compute the control action , and is the exogenous disturbance. We further assume that the full plant admits a joint realization^{2}^{2}2We assume throughout that is strictly proper—it follows that is a necesssary and sufficient condition for internal stability of the closed loop system shown in Figure 1 [13]., i.e.
(2.2) 
where .
The standard optimal control problem of minimizing the gain from exogenous disturbance to regulated output , subject to internal stability of the closed loop system shown in Figure 1, can then be posed as
(2.3)  
Disturbance rejection
Consider the feedback system shown in Figure 2, wherein a controller is in feedback with a SISO plant , with input disturbance and measurement noise . We can then define the disturbances and outputs as
respectively, where . Furthermore, let the plant have a statespace realization . We then have that
from which it follows that the generalized plant admits the joint realization
Reference tracking
Now, consider the feedback system shown in Figure 3, wherein a controller is in feedback with a SISO plant , with input disturbance and reference signal . We can then define the disturbances and outputs as
respectively, where . Furthermore, let the plant have a statespace realization . We then have that
from which it follows that the full plant admits the joint realization
Specialization to FIR plant G
Suppose that is strictly proper and has a finite impulse response (FIR) of order , i.e., that for a collection of real scalars . Defining , the plant admits the statespace realization where is the rightshift operator (i.e., a matrix with ones one the subdiagonal and zeros elsewhere). Given the examples presented thus far, going forward we assume that
(2.4) 
as well as the standard assumption that . Additionally, given that we are considering SISO systems, we can without loss of generality (by suitably rescaling ) assume that
(2.5) 
2.2 Coarsegrained identification
As our aim is to provide endtoend guarantees for robust control problems, we must first have a scheme to acquire an approximate plant model . Toward that end, Coarsegrained identification, as defined in Tu et al. [7], describes the following procedure:

carefully choose a series of inputs , where , and collect noisy outputs where with

form a leastsquares estimate of the impulse response of using .
We refer to each such pair as an experiment.
In [7], upper and lower bounds are shown on the resulting error between and for different sets . This built on the work of Goldenschluger [14], who derived estimation rates for constrained inputs. We make slight modifications to the results in [7] to instead provide error bounds on the impulse response coefficients, as these are more natural for our problem. One concern is that we will need bounds on the impulse response error, and these bounds are in term of the norm. However, while they can conservatively be plugged in verbatim (as ), we will instead modify their proofs slightly to fit our application.
3 SystemLevel Synthesis
The SystemLevel Synthesis (SLS) framework, proposed by Wang et. al [11], provides a parameterization of stabilizing controllers that achieve specified responses between disturbances and outputs. We briefly review here the SLS framework, and later show in Section 4.1 how it can be modified to solve a robust optimal control problem subject to bounded uncertainty on the FIR coefficients .
For an LTI system with dynamics described by (2.2), we define a system response to be the maps satisfying
(3.1) 
where is the process noise, and is the measurement noise.
We call a system response stable and achievable with respect to a plant if there exists an internally stabilizing controller such that the control rule leads to closed loop behavior consistent with (3.1). It was shown in [11] that the parameterization of all stable and achievable system responses is defined by the following affine space:
(3.2a)  
(3.2b)  
(3.2c) 
We call equations (3.2a)  (3.2c) the SLS constraints. The parameterization of all internally stabilizing controllers is given by the following theorem. [Theorem 2 in [11]] Suppose that a system response satisfies the SLS constraints (3.2a)  (3.2c). Then, is an internally stabilizing controller for the plant (2.2) that yields the desired system response (3.1). Furthermore, the solutions of (3.2a)  (3.2c) with the implementation parameterize all internally stabilizing controllers for the plant (2.2). Using this parameterization, we can recast the standard optimal control problem (2.3) as the SLS problem
(3.3)  
In the FIR case, we use the abbreviated notation for the case where is the plant . Although we focus on the optimal control problem posed in equation (3.3), the results that follow carry over naturally to (LQG) and optimal control problems as well.
4 Sample Complexity Bounds
We now provide finitedata performance guarantees for a controller synthesized using the system identification and robust synthesis procedures described in the previous sections. Prior to stating our main results, we recall the problem set up and CoarseID Control pipeline.
We consider the identification and control of the system , which is assumed to be FIR of order . We begin with the simplified setting that the order of the true system is known, and we use the Coarsegrained identification procedure described in Section 2.2 to identify an approximate system , also of order , using a series of experiments. We then use this approximate system
, as well as highprobability bounds on the estimation error
, in a robust SLS problem (see (4.6) in Section 4.1) to compute a controller with provable suboptimality guarantees, as formalized in the following theorem. ^{3}^{3}3Here, hides universal constants: see Lemma 4.3 for an explicit characterization. Let be the optimal solution of the SLS problem (3.3) for the plant , and let be an estimate of obtained using coarsegrained identification (variance output noise only) with experiments, where . Let be the optimal solution to the robust SLS problem (4.6) for , and let be the response achieved on the true system by the synthesized controller . Then, if , with probability at least , the controller stabilizes the true system and has a suboptimality gap bounded byAssume that we are in the setting of Theorem 4, and further let there be process noise with variance that enters the system via the same channel as the control input (i.e., ) and measurement noise with variance . Then, Theorem 4 holds with .
We can further generalize these results to the setting where the order of the underlying system is not known, and that the true system is approximated by a length FIR filter with coefficients where . In this case, applying the triangle equality
gives a similar sample complexity bound, albeit one where the cost difference does not tend to zero as the number of experiments tends to infinity. Assume that we are in the setting of Theorem 4, except let be a length (where ) FIR estimate of obtained using the prescribed coarsegrained identification. Furthermore, assume that . Then, if
with probability at least the controller stabilizes the true system and has a suboptimal cost bounded by
To prove the above results, we first derive a robust variant of the SLS framework presented in Section 3, and then show how it can be used to pose a robust synthesis problem that admits suboptimality guarantees. In particular, these guarantees characterize the degradation in performance of the synthesized controller as a function of the size of the uncertainty on the transfer function coefficients . We then combine this characterization of performance degradation with highprobability bounds on the estimation error produced by the coarsegrained identification procedure to provide an endtoend analysis of the CoarseID control procedure.
4.1 Robust SLS
As we only have access to approximately identified plants, we need a robust variant of Theorem 3.2. First, we introduce a robust version of (3.2b),
(4.1) 
We call equations (3.2a), (4.1), and (3.2c) the robust SLS constraints. We now have the ingredients needed to connect the main and robust SLS constraints. The proof is mostly algebraic and is thus deferred to the Appendix. [Robust Equivalence] Consider system reponses and , where the latter is given by
(4.2)  
where by assumption exists and is in . Let be a given plant, and consider the following statements.

satisfies the robust SLS constraints for .

satisfies the SLS constraints for .
Under the assumptions, . Furthermore, let , and let
Then, is equivalent to a third statement : satisfies the SLS constraints for .
A chain of corollaries follow from Lemma 4.1 that will be useful in quantifying the performance achieved on the true system of a controller designed using an approximate system model. Unless otherwise noted, let be defined as in Lemma 4.1.
Suppose that satisfies the robust SLS constraints for the system (2.2). Then, the controller stabilizes the system (2.2) and achieves the closedloop system response if and only if .
Proof.
First note that the robust SLS constraints imply that . Next, assume . By Lemma 4.1, satisfies the SLS constraints for (2.2). Thus, by Theorem 3.2, is stabilizing and achieves the closedloop response . Moreover, is precisely equal to .
Conversely, assume exists but is not in (if it does not exist the system response (4.2) is obviously not welldefined). It then follows that is not in as is square and invertible. ∎
This immediately gives us a sufficient condition for robustness of the SLS procedure. Suppose that satisfies the robust SLS constraints for the system (2.2). A sufficient condition for the controller to stabilize the system (2.2) and achieve closedloop response is that , for any induced norm .
We now specialize our results to the case where the plant , as defined in (2.2), is FIR. In this case, the modeling error arises only in the coefficient vector defining the impulse response. To that end, we define the the estimated plant with the realization and note that the resulting error arises only in the and terms of the corresponding estimated plant , where these statespace parameters are defined as in (2.4). To that end, we define the estimation error vector allowing us to further specialize Corollary 4.1. Suppose satisfies the SLS constraints for the estimated system . If for any induced norm , then the controller stabilizes the true system and achieves the closedloop response as specified in (4.2). Additionally, if the induced norm is either the or norm, the response simplifies to
(4.3) 
Proof.
By Lemma 4.1, satsisfies the robust SLS constraints for . The sufficient condition then follows by applying Corollary 4.1.Furthermore, if the induced norm used in Corollary 4.1 and Corollary 4.1 is either the or norm, it follows from Hölder’s inequality that implies that on . Hence, we can use the ShermanMorrison identity,
to simplify the closed loop response (4.2) achieved by the approximate controller to the expression (4.3). ∎
We now use this robust parameterization to formulate a robust SLS problem that yields a controller with stability and performance guarantees.
We will use these two facts without fanfare in the following sections. Let . Then . for all .
Define to be the performance (i.e. the objective in (3.3)) of the controller induced by when placed in closedloop with the FIR plant specified by impulse response coefficients . Now, assume we design a response , with corresponding controller , that satisfies the SLS constraints specified by the estimate system . We saw in the previous section that under suitable conditions, the response on the true system is given by , as specified in Corollary 4.1. By the triangle inequality, Corollary 4.1, and our parametric assumption (2.4)(2.5), we can then bound the difference between expectation and reality as follows:
where we assume for the bound to be valid.
For any estimated response satisfying , it then follows that
(4.4) 
for any satisfying
(4.5) 
noting that , which implies , is equivalent to . We denote the righthand side of this bound as
The bound (4.4) then suggests the following robust controller synthesis procedure, which balances between solving for the optimal controller for the approximate system and controlling a perturbative term. We call this problem the robust SLS problem for .
(4.6)  
Although this problem is not jointly convex in and the system responses , one can use a golden section search on in practice. Moreover, the sum of norms can be split into two norm constraints using an epigraph formulation (see [15], Ch. 3).
4.2 Suboptimality guarantees for robust SLS
We now show a bound on the change in the optimal control cost when the controller is synthesized using the robust SLS problem (4.6).
Let and , as well as , be defined as in Theorem (4), and let . If , we have that
(4.7)  
To prove this proposition, we require a technical lemma that ensures that the true controller stabilizes the estimate system specified by the FIR coefficients , i.e. that the optimal system response can be used to construct a feasible solution to the approximate SLS synthesis problem (4.6). Let and its induced controller be as defined in Theorem 4, and let , with . Then
(4.8) 
is strictly positive, and the controller is stabilizing for the estimate system specified by and achieves the system response defined by
(4.9) 
Furthermore, are feasible solutions to the approximate SLS synthesis problem (4.6).
Proof.
Both of these points are conditional on existing in :

By Lemma 4.1, satisfies the SLS constraints for , and is thus part of a feasible point for the approximate SLS synthesis problem (4.6). Now, we need to check that the corresponding is also part of a feasible solution. Toward that end, by the ShermanMorrison identity, we see that
Furthermore,
Therefore,
the final feasibility condition of (4.6).
It therefore remains to verify that exists and is in . As we have seen, a sufficient condition is , and this condition is implied by the assumption that . ∎
Proof of Proposition 4.2.
We immediately invoke Lemma 4.2 by noting that our assumption on ensures , and we are assured that is a feasible point for the approximate SLS synthesis problem (4.6). From inequality (4.4), we then have that
(4.10) 
where the second inequality follows from the optimality of , and the final inequality from the definitions of and . Now, we repeat the argument used to derive (4.4) with expectation and reality reversed: this time we assume our design expectation was but our reality is . This is a valid analogy as satisfies the SLS equations for . With the true and estimated parameters reversed, we can thus bound by
(4.11) 
Finally, combining bounds (4.10) and (4.11) and plugging in gives
∎
4.3 Coarsegrained ID and the proof of Theorem 4
First, to prove the sample complexity of synthesizing a stabilizing controller based on an approximate system, we require an intermediary lemma on how well coarsegrained identification can identify the true system. The proof of the lemma (and the related change necessary for Corollary 4) is deferred to the Appendix. [] Assume we estimate the system by a length FIR system using coarsegrained identification (output noise only) on experiments, where the inputs are constrained to lie in a unit ball. Then, with probability at least ,
Taking large enough such that (implied by taking ), we have
Finally, we show that is stabilizing for the true system . Since is optimal for the approximate SLS synthesis problem for , it is feasible, and thus allows us to invoke Corollary 4.1, as we have that .
5 Experiments
The robust SLS procedure analyzed in the previous section requires solving an infinitedimensional optimization problem as the responses are not required to be FIR. However, as an approximation, we limit them to be FIR responses of a prescribed length . By making this restriction, the resulting optimization problem is then finitedimensional and admits an efficient solution using offtheshelf convex optimization solvers^{4}^{4}4Code for these computations can be found at https://github.com/rjboczar/OFendtoendCDC.
Figure 4 shows a quantification of this approximation. In this experiment, for each
, we chose random FIR plants with impulse response coefficients uniformly distributed in
. We then computed the smallest such that the robust performance returned by the SLS program was within 2% relative error of the performance calculated by MATLAB’s hinfsyn with relative tolerance . Figure 4 also shows this calculation when each plant was normalized to have unit norm.5.1 Optimization Model
Let , , and be the (static) identity transfer function. Furthermore, appealing to the SDP characterization of bounded FIR systems ([16] Thm. 5.8), define
Then, under the approximate assumption that are FIR of length , and using the notation for the th block of , we can write the full optimization problem of solving (4.6) for a fixed :
Comments
There are no comments yet.