There have been many recent results (see for example [1, 2, 3, 4, 5, 6] and the references within) that apply state-of-the-art machine learning techniques to the control of systems with continuous action spaces. As the systems we control become ever more complex, be it in their dynamics, their scale, or their interaction with the environment, moving to a data-driven approach will be inevitable: in these settings, first-principle modeling becomes either impossible or intractable. However, as promising and exciting as recent empirical demonstrations of these techniques have been, they have, for the most part, lacked the rigorous stability, safety and robustness guarantees that the controls community has always prided itself in providing. Indeed, such guarantees are not only desirable, but necessary when such techniques are being proposed for the control of safety critical systems or infrastructures.
This paper can be seen as a step towards providing such guarantees, albeit in a simplified setting, wherein we establish rigorous baselines of robustness and performance when controlling a single-input-single-output (SISO) system with an unknown transfer function. To do so, we combine contemporary approaches to system identification and robust control into what we term the “Coarse-ID control” pipeline. In particular, we leverage the results developed in 
to provide finite-sample guarantees on optimally (in a certain sense) estimating a stable single-input single-output linear time-invariant (SISO LTI) system, using input-output data pairs.111We note that there have been recent results in the system identification literature (for example [8, 9]) that also seek to provide non-asymptotic guarantees of model estimation quality. Such finite-data guarantees are not only in stark contrast to classical system identification results, which typically only provide asymptotic guarantees of model fidelity (see  for an overview), but also necessary for the principled integration of these techniques with robust control, as they allow us to quantify the amount of uncertainty that our controller must contend with. We then formulate a robust control problem using the recently developed system-level synthesis (SLS) procedure , which exploits a novel parameterization of stabilizing controllers for LTI systems that allows us to quantify performance degradation in terms of the amount of uncertainty affecting the system . Again this is in contrast to classical methods from robust control  that are only able to provide robust stability guarantees for a prescribed amount of uncertainty.
A feature of “Coarse-ID control,” as described above, is that we can analyze the end-to-end performance of this pipeline in a non-asymptotic setting. Specifically, we show that the difference in cost between the optimal cost for the true system (an FIR SISO system of length ) and the realized cost induced by instead solving a robust SLS procedure for the approximate system is . Here, we assume that the approximate system was estimated using the “optimal” coarse-grained system identification procedure described in Tu et al. , with
the measurement noise variance andthe number of experiments conducted in order to construct an estimate of the system. Finally, this paper should be viewed as a step towards generalizing the results in , which provides finite-data end-to-end performance guarantees for the classical LQR optimal control problem, to the output-feedback setting.
In Section 2 we fix notation and quickly outline the structure used by common robust control problems. Section 3 then gives an overview of the system-level synthesis framework and how it can be used to solve these problems. Finally, in Section 4 we combine this framework with recent work on coarse-grained identification to provide quantitative bounds on how the performance of a robust controller synthesized using the SLS framework degrades when the plant to be controlled is only approximately identified. We conclude in Section 5 with computational examples.
We use boldface to denote frequency domain signals and transfer functions. The
-th standard basis vector is given by. A discrete-time dynamical system
can be represented compactly by or the tuple (with implying ). The set of stable real-rational proper transfer matrices is denoted . Unless otherwise noted, represents the -norm (the induced norm) for elements in (this reduces to the spectral norm for constant matrices).
2.1 The standard robust control problem
We first introduce a standard form for generic robust and optimal control problems, and then show how simple disturbance attenuation and reference tracking problems can be cast into this standard form. We work with discrete-time LTI systems, but unless stated otherwise, all results extend naturally to the continuous-time setting. A system in standard form can be described by the following equations:
where is the regulated output (e.g., deviations of the system state from a desired set-point), is the measured output available to the controller to compute the control action , and is the exogenous disturbance. We further assume that the full plant admits a joint realization222We assume throughout that is strictly proper—it follows that is a necesssary and sufficient condition for internal stability of the closed loop system shown in Figure 1 ., i.e.
The standard optimal control problem of minimizing the gain from exogenous disturbance to regulated output , subject to internal stability of the closed loop system shown in Figure 1, can then be posed as
Consider the feedback system shown in Figure 2, wherein a controller is in feedback with a SISO plant , with input disturbance and measurement noise . We can then define the disturbances and outputs as
respectively, where . Furthermore, let the plant have a state-space realization . We then have that
from which it follows that the generalized plant admits the joint realization
Now, consider the feedback system shown in Figure 3, wherein a controller is in feedback with a SISO plant , with input disturbance and reference signal . We can then define the disturbances and outputs as
respectively, where . Furthermore, let the plant have a state-space realization . We then have that
from which it follows that the full plant admits the joint realization
Specialization to FIR plant G
Suppose that is strictly proper and has a finite impulse response (FIR) of order , i.e., that for a collection of real scalars . Defining , the plant admits the state-space realization where is the right-shift operator (i.e., a matrix with ones one the sub-diagonal and zeros elsewhere). Given the examples presented thus far, going forward we assume that
as well as the standard assumption that . Additionally, given that we are considering SISO systems, we can without loss of generality (by suitably rescaling ) assume that
2.2 Coarse-grained identification
As our aim is to provide end-to-end guarantees for robust control problems, we must first have a scheme to acquire an approximate plant model . Toward that end, Coarse-grained identification, as defined in Tu et al. , describes the following procedure:
carefully choose a series of inputs , where , and collect noisy outputs where with
form a least-squares estimate of the impulse response of using .
We refer to each such pair as an experiment.
In , upper and lower bounds are shown on the resulting error between and for different sets . This built on the work of Goldenschluger , who derived estimation rates for -constrained inputs. We make slight modifications to the results in  to instead provide error bounds on the impulse response coefficients, as these are more natural for our problem. One concern is that we will need bounds on the impulse response error, and these bounds are in term of the -norm. However, while they can conservatively be plugged in verbatim (as ), we will instead modify their proofs slightly to fit our application.
3 System-Level Synthesis
The System-Level Synthesis (SLS) framework, proposed by Wang et. al , provides a parameterization of stabilizing controllers that achieve specified responses between disturbances and outputs. We briefly review here the SLS framework, and later show in Section 4.1 how it can be modified to solve a robust optimal control problem subject to bounded uncertainty on the FIR coefficients .
For an LTI system with dynamics described by (2.2), we define a system response to be the maps satisfying
where is the process noise, and is the measurement noise.
We call a system response stable and achievable with respect to a plant if there exists an internally stabilizing controller such that the control rule leads to closed loop behavior consistent with (3.1). It was shown in  that the parameterization of all stable and achievable system responses is defined by the following affine space:
We call equations (3.2a) - (3.2c) the SLS constraints. The parameterization of all internally stabilizing controllers is given by the following theorem. [Theorem 2 in ] Suppose that a system response satisfies the SLS constraints (3.2a) - (3.2c). Then, is an internally stabilizing controller for the plant (2.2) that yields the desired system response (3.1). Furthermore, the solutions of (3.2a) - (3.2c) with the implementation parameterize all internally stabilizing controllers for the plant (2.2). Using this parameterization, we can recast the standard optimal control problem (2.3) as the SLS problem
In the FIR case, we use the abbreviated notation for the case where is the plant . Although we focus on the optimal control problem posed in equation (3.3), the results that follow carry over naturally to (LQG) and optimal control problems as well.
4 Sample Complexity Bounds
We now provide finite-data performance guarantees for a controller synthesized using the system identification and robust synthesis procedures described in the previous sections. Prior to stating our main results, we recall the problem set up and Coarse-ID Control pipeline.
We consider the identification and control of the system , which is assumed to be FIR of order . We begin with the simplified setting that the order of the true system is known, and we use the Coarse-grained identification procedure described in Section 2.2 to identify an approximate system , also of order , using a series of experiments. We then use this approximate system
, as well as high-probability bounds on the estimation error, in a robust SLS problem (see (4.6) in Section 4.1) to compute a controller with provable suboptimality guarantees, as formalized in the following theorem. 333Here, hides universal constants: see Lemma 4.3 for an explicit characterization. Let be the optimal solution of the SLS problem (3.3) for the plant , and let be an estimate of obtained using coarse-grained identification (-variance output noise only) with experiments, where . Let be the optimal solution to the robust SLS problem (4.6) for , and let be the response achieved on the true system by the synthesized controller . Then, if , with probability at least , the controller stabilizes the true system and has a suboptimality gap bounded by
Assume that we are in the setting of Theorem 4, and further let there be process noise with variance that enters the system via the same channel as the control input (i.e., ) and measurement noise with variance . Then, Theorem 4 holds with .
We can further generalize these results to the setting where the order of the underlying system is not known, and that the true system is approximated by a length- FIR filter with coefficients where . In this case, applying the triangle equality
gives a similar sample complexity bound, albeit one where the cost difference does not tend to zero as the number of experiments tends to infinity. Assume that we are in the setting of Theorem 4, except let be a length- (where ) FIR estimate of obtained using the prescribed coarse-grained identification. Furthermore, assume that . Then, if
with probability at least the controller stabilizes the true system and has a suboptimal cost bounded by
To prove the above results, we first derive a robust variant of the SLS framework presented in Section 3, and then show how it can be used to pose a robust synthesis problem that admits suboptimality guarantees. In particular, these guarantees characterize the degradation in performance of the synthesized controller as a function of the size of the uncertainty on the transfer function coefficients . We then combine this characterization of performance degradation with high-probability bounds on the estimation error produced by the coarse-grained identification procedure to provide an end-to-end analysis of the Coarse-ID control procedure.
4.1 Robust SLS
We call equations (3.2a), (4.1), and (3.2c) the robust SLS constraints. We now have the ingredients needed to connect the main and robust SLS constraints. The proof is mostly algebraic and is thus deferred to the Appendix. [Robust Equivalence] Consider system reponses and , where the latter is given by
where by assumption exists and is in . Let be a given plant, and consider the following statements.
satisfies the robust SLS constraints for .
satisfies the SLS constraints for .
Under the assumptions, . Furthermore, let , and let
Then, is equivalent to a third statement : satisfies the SLS constraints for .
A chain of corollaries follow from Lemma 4.1 that will be useful in quantifying the performance achieved on the true system of a controller designed using an approximate system model. Unless otherwise noted, let be defined as in Lemma 4.1.
First note that the robust SLS constraints imply that . Next, assume . By Lemma 4.1, satisfies the SLS constraints for (2.2). Thus, by Theorem 3.2, is stabilizing and achieves the closed-loop response . Moreover, is precisely equal to .
Conversely, assume exists but is not in (if it does not exist the system response (4.2) is obviously not well-defined). It then follows that is not in as is square and invertible. ∎
This immediately gives us a sufficient condition for robustness of the SLS procedure. Suppose that satisfies the robust SLS constraints for the system (2.2). A sufficient condition for the controller to stabilize the system (2.2) and achieve closed-loop response is that , for any induced norm .
We now specialize our results to the case where the plant , as defined in (2.2), is FIR. In this case, the modeling error arises only in the coefficient vector defining the impulse response. To that end, we define the the estimated plant with the realization and note that the resulting error arises only in the and terms of the corresponding estimated plant , where these state-space parameters are defined as in (2.4). To that end, we define the estimation error vector allowing us to further specialize Corollary 4.1. Suppose satisfies the SLS constraints for the estimated system . If for any induced norm , then the controller stabilizes the true system and achieves the closed-loop response as specified in (4.2). Additionally, if the induced norm is either the or norm, the response simplifies to
By Lemma 4.1, satsisfies the robust SLS constraints for . The sufficient condition then follows by applying Corollary 4.1.Furthermore, if the induced norm used in Corollary 4.1 and Corollary 4.1 is either the or norm, it follows from Hölder’s inequality that implies that on . Hence, we can use the Sherman-Morrison identity,
We now use this robust parameterization to formulate a robust SLS problem that yields a controller with stability and performance guarantees.
We will use these two facts without fanfare in the following sections. Let . Then . for all .
Define to be the performance (i.e. the objective in (3.3)) of the controller induced by when placed in closed-loop with the FIR plant specified by impulse response coefficients . Now, assume we design a response , with corresponding controller , that satisfies the SLS constraints specified by the estimate system . We saw in the previous section that under suitable conditions, the response on the true system is given by , as specified in Corollary 4.1. By the triangle inequality, Corollary 4.1, and our parametric assumption (2.4)-(2.5), we can then bound the difference between expectation and reality as follows:
where we assume for the bound to be valid.
For any estimated response satisfying , it then follows that
for any satisfying
noting that , which implies , is equivalent to . We denote the right-hand side of this bound as
The bound (4.4) then suggests the following robust controller synthesis procedure, which balances between solving for the optimal controller for the approximate system and controlling a perturbative term. We call this problem the robust SLS problem for .
Although this problem is not jointly convex in and the system responses , one can use a golden section search on in practice. Moreover, the sum of norms can be split into two norm constraints using an epigraph formulation (see , Ch. 3).
4.2 Sub-optimality guarantees for robust SLS
We now show a bound on the change in the optimal control cost when the controller is synthesized using the robust SLS problem (4.6).
Let and , as well as , be defined as in Theorem (4), and let . If , we have that
To prove this proposition, we require a technical lemma that ensures that the true controller stabilizes the estimate system specified by the FIR coefficients , i.e. that the optimal system response can be used to construct a feasible solution to the approximate SLS synthesis problem (4.6). Let and its induced controller be as defined in Theorem 4, and let , with . Then
is strictly positive, and the controller is stabilizing for the estimate system specified by and achieves the system response defined by
Furthermore, are feasible solutions to the approximate SLS synthesis problem (4.6).
Both of these points are conditional on existing in :
By Lemma 4.1, satisfies the SLS constraints for , and is thus part of a feasible point for the approximate SLS synthesis problem (4.6). Now, we need to check that the corresponding is also part of a feasible solution. Toward that end, by the Sherman-Morrison identity, we see that
the final feasibility condition of (4.6).
It therefore remains to verify that exists and is in . As we have seen, a sufficient condition is , and this condition is implied by the assumption that . ∎
Proof of Proposition 4.2.
We immediately invoke Lemma 4.2 by noting that our assumption on ensures , and we are assured that is a feasible point for the approximate SLS synthesis problem (4.6). From inequality (4.4), we then have that
where the second inequality follows from the optimality of , and the final inequality from the definitions of and . Now, we repeat the argument used to derive (4.4) with expectation and reality reversed: this time we assume our design expectation was but our reality is . This is a valid analogy as satisfies the SLS equations for . With the true and estimated parameters reversed, we can thus bound by
4.3 Coarse-grained ID and the proof of Theorem 4
First, to prove the sample complexity of synthesizing a stabilizing controller based on an approximate system, we require an intermediary lemma on how well coarse-grained identification can identify the true system. The proof of the lemma (and the related change necessary for Corollary 4) is deferred to the Appendix.  Assume we estimate the system by a length- FIR system using coarse-grained identification (output noise only) on experiments, where the inputs are constrained to lie in a unit ball. Then, with probability at least ,
Taking large enough such that (implied by taking ), we have
Finally, we show that is stabilizing for the true system . Since is optimal for the approximate SLS synthesis problem for , it is feasible, and thus allows us to invoke Corollary 4.1, as we have that .
The robust SLS procedure analyzed in the previous section requires solving an infinite-dimensional optimization problem as the responses are not required to be FIR. However, as an approximation, we limit them to be FIR responses of a prescribed length . By making this restriction, the resulting optimization problem is then finite-dimensional and admits an efficient solution using off-the-shelf convex optimization solvers444Code for these computations can be found at https://github.com/rjboczar/OF-end-to-end-CDC.
Figure 4 shows a quantification of this approximation. In this experiment, for each
, we chose random FIR plants with impulse response coefficients uniformly distributed in. We then computed the smallest such that the robust performance returned by the SLS program was within 2% relative error of the performance calculated by MATLAB’s hinfsyn with relative tolerance . Figure 4 also shows this calculation when each plant was normalized to have unit -norm.
5.1 Optimization Model
Let , , and be the (static) identity transfer function. Furthermore, appealing to the SDP characterization of -bounded FIR systems ( Thm. 5.8), define
Then, under the approximate assumption that are FIR of length , and using the notation for the -th block of , we can write the full optimization problem of solving (4.6) for a fixed :