Adaptive control is a well established sub-field of control which compensates for parametric uncertainties that occur online so as to lead to regulation and tracking [1, 2, 3, 4, 5]. This is accomplished by constructing estimates of the uncertainties in real-time and ensuring that the closed loop system is well behaved even when these uncertainties are learned imperfectly. Both in the adaptive control and system identification literature, numerous tools for ensuring that the parameter estimates converge to their true values have been derived over the past four decades [6, 7, 8, 9]. While most of the current literature in these two topics makes an assumption that the unknown parameters are constants, the desired problem statement involves plants where the unknown parameters are varying with time. This paper proposes a new algorithm for such plants.
Parameter convergence in adaptive systems requires a necessary and sufficient condition, denoted as persistent excitation, which ensures that the convergence is uniform in time [10, 11, 12]. If instead, a weaker condition is enforced where the excitation holds only over a finite interval, then parameter errors decrease only over a finite time. It is therefore of interest to achieve a fast rate of convergence by leveraging any excitation that may be available so that the parameter estimation error remains as small as possible, even in the presence of time-variations. The algorithm proposed in this paper will be shown to lead to such a fast convergence under a range of excitation conditions.
The underlying structure in many of the adaptive identification and control problems consists of a linear regression relation between two dominant errors in the system[4, 13, 14]. Examples include adaptive observers [15, 16, 17, 18, 19] and certain classes of adaptive controllers . The underlying algebraic relation is often leveraged in order to lead to a fast convergence through the introduction of a time-varying learning rate in the parameter estimation algorithm, which leads to the well-known recursive least squares algorithm . Together with the use of an outer product of the underlying regressor, a matrix of time-varying learning rates is often adjusted to enable fast convergence [21, 22, 23, 15, 18, 24, 25]. In many cases, however, additional dynamics are present in the underlying error model that relates the two dominant errors, which prevents the derivation of the corresponding algorithm and therefore a fast convergence of the parameter estimates. To overcome this roadblock, filtering has been proposed in the literature [21, 22, 23, 26, 27, 28, 29, 30]. This in turn leads to an algebraic regression, using which corresponding adaptive algorithms are derived in [21, 22, 23, 15, 18, 24, 25] with time-varying learning rates become applicable. In [26, 27], it is shown that parameter convergence can occur even with finite excitation for a class of adaptive control architectures considered in [28, 29, 30]. In all of these papers, the underlying unknown parameters are assumed to be constants. The disadvantage of such a filtering approach is that the convergence properties cannot be easily extended to the case when the unknown parameters are time-varying, as the filtering renders the problem intractable. The algorithm that we propose in this paper introduces no filtering of system dynamics, and is directly applied to the original error model with the dynamics intact. As such, we are able to establish fast decreases of errors, even in the presence of time-varying parameters under varied properties of excitation.
In [31, 32], the problem of parameter estimation has been tackled in the presence of time-varying parameters using a concurrent learning approach. It is assumed in these papers however, that state derivatives from previous time instances are available. In , this assumption is removed and an integration based method is used together with the same filtering approach mentioned above. The underlying parameters are assumed to be constant, and thus the approach in  becomes intractable when these parameters vary.
This paper focuses on the ultimate goal of all adaptive control and identification problems, which is to provide a tractable parameter estimation algorithm for problems where the unknown parameters are time-varying. We will derive such an algorithm that guarantees, in the presence of time-varying parameters, 1) exponentially fast tending of parameter errors and tracking errors to a compact set for a range excitation conditions, 2) introduces no filtering of system dynamics, and 3) is applicable to a large class of adaptive systems. An error model approach as in [1, 34, 35] is adopted to describe the underlying class. The algorithm consists of time-varying learning rates in order to guarantee fast parameter convergence. Rather than use a forgetting factor, continuous projection algorithms are employed in order to ensure that the learning rates are bounded. Additionally, the resulting computational requirements are significantly smaller than those in existing literature.
This paper proceeds as follows: Section 2 presents mathematical preliminaries regarding continuous projection-based operators and definitions of persistent and finite excitation. The underlying problem is introduced in Section 3. The main algorithm with time-varying learning rates is presented in Section 4. Stability and convergence properties of this algorithm are established for a range of excitation conditions in Section 5. Computational comparisons with existing adaptive controllers are provided alongside numerical simulations in Section 6. Concluding remarks follow in Section 7.
In this paper we use to represent the 2-norm. Definitions, key lemmas, and properties of the Projection Operator (c.f. [36, 37, 38, 4, 39, 5]) are all presented in this section. Proofs of all lemmas can be found in the appendix, and omitted where it is straightforward.
We begin with a few definitions and properties of convex sets and convex, coercive functions.
Definition 1 ()
A set is convex if for all , , and .
Definition 2 ()
A function is convex if
for all .
Definition 3 ()
A function is said to be coercive if for all sequences , with then .
Lemma 1 ()
For a convex function and any constant , the subset is convex.
For a coercive function and any constant , any nonempty subset is bounded.
For a coercive, convex function and a constant , any nonempty subset is convex and bounded.
Lemma 3 ()
For a continuously differentiable convex function and any constant , let be an interior point of the subset , i.e. , and let a boundary point be such that . Then .
We now present properties related to projection operators which include a symmetric positive definite function. While some of these properties have been previously reported (c.f. [37, 38, 4, 39, 5]), they are derived to help discuss the main result of this paper.
Definition 4 ()
The -projection operator for general matrices is defined as,
where , , , are convex continuously differentiable functions, is a symmetric positive definite matrix and ,
The projection operator for positive definite matrices is defined as,
where , and is a convex continuously differentiable function.
An example of a coercive, continuously differentiable convex function commonly used in projection for adaptive control is given by 
where and are positive scalars. It is easy to see that when and when . This function is commonly used in a projection-based parameter update law to result in a bounded parameter estimate (proven in this paper in Lemma 8). It should be noted that numerous choices other than the one in (5) exist for .
Let , , , , where are convex continuously differentiable functions, is a symmetric positive definite matrix, and , then
The following lemma lists two key properties related to matrix inversion in the presence of time-variations.
For a matrix , the following identities hold:
A central component of this paper is with regards to excitation of a regressor for which two definitions are provided.
Definition 6 ()
A bounded function is persistently exciting (PE) if there exists and such that
A bounded function is finitely exciting (FE) on an interval if there exists , , and such that
3 Adaptive Control of a Class of Plants with Time-Varying Parameters
|State Feedback MRAC |
|Output Feedback MRAC|
|A.S, SPR |
|Output Feedback MRAC|
|A.S., not SPR |
Large classes of problems in adaptive identification and control can be represented in the form of differential equations containing two errors, and . The first is an error that represents an identification error or tracking error. The second is the underlying parameter error, either in estimation of the plant parameter or the control parameter. The parameter error is commonly expressed as the difference between a parameter estimate and the true unknown value as . The differential equations which govern the evolution of with are referred to as error models [1, 34, 35], and provide insight into how stable adaptive laws for adjusting the parameter error can be designed for a large class of adaptive systems. The class of error models we focus on in this paper is of the form
where the regressor and is a measurable error at each . The corresponding adaptive law for adjusting is assumed to be of the form
where is a known function that is implementable at each and is a symmetric positive definite matrix referred to as the learning rate. In addition, for a given and , is chosen so that , is an equilibrium point of the system. Assuming that is a constant, the law in (7) can be written as
We consider all classes of adaptive systems that can be expressed in the form of (6) and (8) where , , , and are such that all solutions are bounded, and where . In particular, we assume that , , and are such that a quadratic Lyapunov function candidate
yields a derivative for the case of constant as
where and are symmetric positive definite matrices. Due to the choice of the adaptive law in (8), it follows therefore . Further conditions on , , and guarantee that as . We formalize this assumption below:
Assumption 1 (Class of adaptive systems)
Several adaptive systems that have been discussed in the literature satisfy Assumption 1, some examples of which are shown in Table 1. They include plants where state feedback is possible and certain matching conditions are satisfied, and where only outputs are accessible and a strictly positive real transfer function can be shown to exist. For a SISO plant that is minimum phase, the same assumption can be shown to hold as well. Finally, for a class of nonlinear plants, where the underlying relative degree does not exceed two, Assumption 1 once again can be shown to be satisfied.
3.1 Problem Formulation
The class of error models we consider is of the form (6), where , the time-varying unknown parameter, is such that if , then the solutions of (6) are globally bounded, with remaining an equilibrium point. This is formalized in the following assumption:
Assumption 2 (Uncertainty variation)
The problem that we address in this paper is the determination of an adaptive law similar to (8) for all error models of the form (6) where Assumptions 1 and 2 hold. Our goal is to ensure global boundedness of solutions of (6) and exponentially fast tending of both and to a compact set with finite excitation.
4 Adaptive Law with a Time-Varying Learning Rate
The adaptive law that we propose is a modification of (8) with a time-varying learning rate as . To ensure a bounded , we include a projection operator in this adaptive law which is stated compactly as
where is defined as in Definition 4. The -projection operator in (11) uses , where are coercive, continuously differentiable convex functions. Define the subsets , , and . Via Assumption 2, each are chosen such that and corresponds to , , .
The time-varying learning rate is adjusted using the projection operator for positive definite matrices (see Definition 5) as
where , are positive scalars and . is a symmetric positive definite constant matrix chosen so that , where is a coercive, continuously differentiable convex function. Lemma 2 implies there exists a constant such that for all . We assume that is chosen so that for all . It should be noted that a large contributes to a decrease in
Finally the matrix is adjusted as
where is a symmetric positive semi-definite matrix with and denotes a filtered normalized regressor matrix. and are arbitrary positive scalars and is chosen so that . These scalars represent various weights of the proposed algorithm. The main contribution of this paper is the adaptive law in (11), (12), and (13), which will be shown to result in bounded solutions in Section 5 which tend exponentially fast to a compact set if is finitely exciting. If in addition, is persistently exciting, exponentially fast convergence to a compact set will occur .
While the time-varying forgetting factor, , also achieves a bounded , it is more conservative than the projection operator in (12) as it is always active. In comparison, the projection operator as in (12) only provides limiting action if is in a specified boundary region and the direction of evolution of causes to increase. An equivalence between time-varying forgetting factors and projection operators may be drawn using the square root of the function in (5) with , , and the limiting action always remaining active.
From a stability standpoint, the filtering in (13) for linear time-invariant error dynamics is optional, i.e. may be used in place of in (12). The inclusion of (13) however provides for a more smooth adjustment of in the presence of sharp changes in and enhances finite excitation properties by restricting all directions of increase of after a finite excitation and no additional excitation.
5 Stability and Convergence Analysis
We now state and prove the main result. The following assumption is needed for discussion of a finite excitation. We define an excitation level on an interval as
where , , and .
Assumption 3 (Finite excitation)
There exists a time and a time such that the regressor in (6) is finitely exciting over , with excitation level .
5.1 Propagation of Excitation and Boundedness of Information Matrix, Time-Varying Learning Rate
We first prove a few important properties of and under different excitation conditions.
, , ,
, , ,
where . If in addition is finitely exciting as in Assumption 3, then there exists a such that
, , ,
where , , , and . If in addition is persistently exciting (see Definition 6), with interval and level , then there exists a , , and such that
, , ,
The properties of and for a persistently exciting are relatively well known. For a finitely exciting , it should be noted that after a certain time elapses, the lower bound for is realized. This propagation is illustrated in Table 2.
The choice of the finite excitation level in Assumption 3 enables a fast convergence rate as follows: The denominator in ensures that the update in (12) pushes away from , provides for a bound for away from a minimum value, and accounts for excitation propagation through (13). The numerator scaling accounts for the normalization in (13), and provides for a bound away from a minimum excitation level.
5.2 Stability and Convergence Analysis
With the properties of the learning rate and filtered regressor above, we now proceed to the main theorem. The following lemma and corollary state important properties of the parameter estimate .
The update for in (11) guarantees that there exists a such that , .
The following definitions are useful for stating the main result in Theorem 1. Define scalars and as
It is easy to see that and , where
where . Define a compact set as
alongside a corresponding set , defined as
We now state the main theorem of stability and convergence.
boundedness of the trajectories of and , .
If in addition is finitely exciting as in Assumption 3, then
the trajectories of , tend exponentially fast towards a compact set , .
If in addition is persistently exciting as in Definition 6 with level and interval , then
exponential convergence of the trajectories follows, of , towards a compact set , .
[Proof] Let , , . Consider a candidate Lyapunov function of the form
It follows that
From the first case of (24) it can be seen that for . From Lemmas 6, 7, 8, and Corollary 2, each of , , , and are bounded. Thus the trajectories of the closed loop system remain bounded. This proves Theorem 1-A).
Furthermore given that and , it can be noted that , . Therefore it can be seen that over the interval of time , the state error and parameter error tend exponentially fast towards the bounded set . This proves Theorem 1-B).
denotes the convergence rate of . This in turn follows if , i.e. if is bounded away from . The latter follows from Lemma 6-3) and 6-4) if is either finitely exciting or persistently exciting, with an exponentially fast trajectory of towards a compact set occurring over a finite interval or for all , respectively. This convergence rate however is upper bounded by .
Theorem 1-C) guarantees convergence of to a compact set , while Theorem 1-B) guarantees that approaches . This set scales with the signal in (16), which contains contributions both from and . For static parameters () and low excitation (i.e., and ), from (23) it can be shown that the trajectories of , tend towards the origin, i.e. the set .
Since we did not introduce any filtering of the underlying signals, the bound on the uncertain parameters is explicit in the compact set . It can be seen that directly scales with from (18). Such an explicit bound cannot be derived using existing approaches in the literature which filter dynamics.
The dependence of on is reasonable. As the time-variations in the uncertain parameters grow, it should be expected that the residue will increase as well. The dependence of on the filtered regressor is introduced due to the structure of our algorithm in (11), (12), and (13). As a result, even with persistent excitation, we can only conclude convergence of to a compact set as opposed to convergence to the origin. This compact set will be present even in the absence of time-variations in . This disadvantage, however, is offset by the property of exponential convergence to the compact set, which is virtue of the fact that we have a time-varying .
A closer examination of the convergence properties of the proposed algorithm is worth carrying out for the case of constant parameters. It is clear from (23) that the negative contributions to come from the first term, while any positive contribution comes if . That is, if there is a large enough excitation, then the third term can be positive. This in turn is conservatively reflected in the magnitude of . It should however be noted that a large with persistent excitation, leads to a large , which implies that as the third term in (23) becomes positive, it leads to a first term that is proportionately large and negative as well, thereby resulting in a net contribution that is negative. An analytical demonstration of this effect, however, is difficult to obtain. For this reason, the nature of our main result is convergence to a bounded set rather than to zero, in the presence of persistent excitation. Finally, we note that in our simulation studies, remained negative for almost the entire period of interest, resulting in a steady convergence of the parameter estimation error to zero.
6 Algorithm Discussion and Simulations
In this section we analyze the memory and computation requirements of our proposed algorithm and provide numerical simulations to demonstrate the algorithm in an illustrative application.
6.1 Memory Requirement and Computation
The standard parameter update in (8) requires integrations to adjust the parameters . Given that the updates for both and result in symmetric matrices, an additional integrations are required for each update for a total increase of integrations.
In comparison, the composite approach with finite excitation analysis presented in  results in an additional integrations to filter the error dynamics, integrations to filter the regressor, integrations to compute a symmetric information matrix, and integrations to compute an auxiliary matrix; for a total increase of
integrations. In order to avoid the knowledge of state derivatives in the approach used in the concurrent learning approach, estimates of past state derivatives is proposed using smoothing techniques with a forward and backward Kalman filter[31, 32]. This however significantly increases the memory and computational requirements compared to the proposed algorithm.
6.2 Numerical Simulations
In this section we present numerical simulation results for linearized F-16 longitudinal dynamics trimmed at a straight and level flying condition with a velocity of ft/s and an altitude of ft. We present results for the case of a large constant unknown parameter in order to demonstrate the exponential convergence properties towards the origin with finite excitations and significant uncertainties. We include integral tracking of commands thus resulting in an extended state plant model given by