Parameter Estimation in Adaptive Control of Time-Varying Systems Under a Range of Excitation Conditions

11/10/2019 ∙ by Joseph E. Gaudio, et al. ∙ MIT Boeing 0

This paper presents a new parameter estimation algorithm for the adaptive control of a class of time-varying plants. The main feature of this algorithm is a matrix of time-varying learning rates, which enables parameter estimation error trajectories to tend exponentially fast towards a compact set whenever excitation conditions are satisfied. This algorithm is employed in a large class of problems where unknown parameters are present and are time-varying. It is shown that this algorithm guarantees global boundedness of the state and parameter errors of the system, and avoids an often used filtering approach for constructing key regressor signals. In addition, intervals of time over which these errors tend exponentially fast toward a compact set are provided, both in the presence of finite and persistent excitation. A projection operator is used to ensure the boundedness of the learning rate matrix, as compared to a time-varying forgetting factor. Numerical simulations are provided to complement the theoretical analysis.



There are no comments yet.


page 1

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Adaptive control is a well established sub-field of control which compensates for parametric uncertainties that occur online so as to lead to regulation and tracking [1, 2, 3, 4, 5]. This is accomplished by constructing estimates of the uncertainties in real-time and ensuring that the closed loop system is well behaved even when these uncertainties are learned imperfectly. Both in the adaptive control and system identification literature, numerous tools for ensuring that the parameter estimates converge to their true values have been derived over the past four decades [6, 7, 8, 9]. While most of the current literature in these two topics makes an assumption that the unknown parameters are constants, the desired problem statement involves plants where the unknown parameters are varying with time. This paper proposes a new algorithm for such plants.

Parameter convergence in adaptive systems requires a necessary and sufficient condition, denoted as persistent excitation, which ensures that the convergence is uniform in time [10, 11, 12]. If instead, a weaker condition is enforced where the excitation holds only over a finite interval, then parameter errors decrease only over a finite time. It is therefore of interest to achieve a fast rate of convergence by leveraging any excitation that may be available so that the parameter estimation error remains as small as possible, even in the presence of time-variations. The algorithm proposed in this paper will be shown to lead to such a fast convergence under a range of excitation conditions.

The underlying structure in many of the adaptive identification and control problems consists of a linear regression relation between two dominant errors in the system

[4, 13, 14]. Examples include adaptive observers [15, 16, 17, 18, 19] and certain classes of adaptive controllers [1]. The underlying algebraic relation is often leveraged in order to lead to a fast convergence through the introduction of a time-varying learning rate in the parameter estimation algorithm, which leads to the well-known recursive least squares algorithm [20]. Together with the use of an outer product of the underlying regressor, a matrix of time-varying learning rates is often adjusted to enable fast convergence [21, 22, 23, 15, 18, 24, 25]. In many cases, however, additional dynamics are present in the underlying error model that relates the two dominant errors, which prevents the derivation of the corresponding algorithm and therefore a fast convergence of the parameter estimates. To overcome this roadblock, filtering has been proposed in the literature [21, 22, 23, 26, 27, 28, 29, 30]. This in turn leads to an algebraic regression, using which corresponding adaptive algorithms are derived in [21, 22, 23, 15, 18, 24, 25] with time-varying learning rates become applicable. In [26, 27], it is shown that parameter convergence can occur even with finite excitation for a class of adaptive control architectures considered in [28, 29, 30]. In all of these papers, the underlying unknown parameters are assumed to be constants. The disadvantage of such a filtering approach is that the convergence properties cannot be easily extended to the case when the unknown parameters are time-varying, as the filtering renders the problem intractable. The algorithm that we propose in this paper introduces no filtering of system dynamics, and is directly applied to the original error model with the dynamics intact. As such, we are able to establish fast decreases of errors, even in the presence of time-varying parameters under varied properties of excitation.

In [31, 32], the problem of parameter estimation has been tackled in the presence of time-varying parameters using a concurrent learning approach. It is assumed in these papers however, that state derivatives from previous time instances are available. In [33], this assumption is removed and an integration based method is used together with the same filtering approach mentioned above. The underlying parameters are assumed to be constant, and thus the approach in [33] becomes intractable when these parameters vary.

This paper focuses on the ultimate goal of all adaptive control and identification problems, which is to provide a tractable parameter estimation algorithm for problems where the unknown parameters are time-varying. We will derive such an algorithm that guarantees, in the presence of time-varying parameters, 1) exponentially fast tending of parameter errors and tracking errors to a compact set for a range excitation conditions, 2) introduces no filtering of system dynamics, and 3) is applicable to a large class of adaptive systems. An error model approach as in [1, 34, 35] is adopted to describe the underlying class. The algorithm consists of time-varying learning rates in order to guarantee fast parameter convergence. Rather than use a forgetting factor, continuous projection algorithms are employed in order to ensure that the learning rates are bounded. Additionally, the resulting computational requirements are significantly smaller than those in existing literature.

This paper proceeds as follows: Section 2 presents mathematical preliminaries regarding continuous projection-based operators and definitions of persistent and finite excitation. The underlying problem is introduced in Section 3. The main algorithm with time-varying learning rates is presented in Section 4. Stability and convergence properties of this algorithm are established for a range of excitation conditions in Section 5. Computational comparisons with existing adaptive controllers are provided alongside numerical simulations in Section 6. Concluding remarks follow in Section 7.

2 Preliminaries

In this paper we use to represent the 2-norm. Definitions, key lemmas, and properties of the Projection Operator (c.f. [36, 37, 38, 4, 39, 5]) are all presented in this section. Proofs of all lemmas can be found in the appendix, and omitted where it is straightforward.

We begin with a few definitions and properties of convex sets and convex, coercive functions.

Definition 1 ([36])

A set is convex if for all , , and .

Definition 2 ([36])

A function is convex if
for all .

Definition 3 ([36])

A function is said to be coercive if for all sequences , with then .

Lemma 1 ([39])

For a convex function and any constant , the subset is convex.

Lemma 2

For a coercive function and any constant , any nonempty subset is bounded.

Corollary 1

For a coercive, convex function and a constant , any nonempty subset is convex and bounded.

Remark 1

Definitions 1, 2, 3, Lemmas 1, 2, and Corollary 1 hold by simple extension to functions of matrices i.e., .

Lemma 3 ([39])

For a continuously differentiable convex function and any constant , let be an interior point of the subset , i.e. , and let a boundary point be such that . Then .

We now present properties related to projection operators which include a symmetric positive definite function. While some of these properties have been previously reported (c.f. [37, 38, 4, 39, 5]), they are derived to help discuss the main result of this paper.

Definition 4 ([39])

The -projection operator for general matrices is defined as,


where , , , are convex continuously differentiable functions, is a symmetric positive definite matrix and ,

Definition 5

The projection operator for positive definite matrices is defined as,


where , and is a convex continuously differentiable function.

Remark 2

The projection operator in (3) may be expressed more compactly as where


From (3), (4), and as displayed in Figure 1, it can be seen that on the inside of the projection boundary and on the outside edge of the boundary if .

Remark 3

An example of a coercive, continuously differentiable convex function commonly used in projection for adaptive control is given by [39]


where and are positive scalars. It is easy to see that when and when . This function is commonly used in a projection-based parameter update law to result in a bounded parameter estimate (proven in this paper in Lemma 8). It should be noted that numerous choices other than the one in (5) exist for .

Lemma 4

Let , , , , where are convex continuously differentiable functions, is a symmetric positive definite matrix, and , then

Figure 1: (adapted from [39]) -Projection operator in with . Uncertain parameter .

The following lemma lists two key properties related to matrix inversion in the presence of time-variations.

Lemma 5

For a matrix , the following identities hold:

A central component of this paper is with regards to excitation of a regressor for which two definitions are provided.

Definition 6 ([1])

A bounded function is persistently exciting (PE) if there exists and such that

Definition 7 (adapted from [26, 27])

A bounded function is finitely exciting (FE) on an interval if there exists , , and such that

In both Definitions 6 and 7, the degree of excitation is given by . It can be noted that the PE condition in Definition 6 pertains to a property over a moving window for all , whereas the FE condition in Definition 7 pertains to a single interval .

3 Adaptive Control of a Class of Plants with Time-Varying Parameters

Name Error Model
State Feedback MRAC [1]   
Output Feedback MRAC   
A.S, SPR [1]
Output Feedback MRAC   
A.S., not SPR [1]
Nonlinear Adaptive   
Backstepping [3]   
Relative Degree
Table 1: Adaptive Control Systems with a Common Structure

Large classes of problems in adaptive identification and control can be represented in the form of differential equations containing two errors, and . The first is an error that represents an identification error or tracking error. The second is the underlying parameter error, either in estimation of the plant parameter or the control parameter. The parameter error is commonly expressed as the difference between a parameter estimate and the true unknown value as . The differential equations which govern the evolution of with are referred to as error models [1, 34, 35], and provide insight into how stable adaptive laws for adjusting the parameter error can be designed for a large class of adaptive systems. The class of error models we focus on in this paper is of the form


where the regressor and is a measurable error at each . The corresponding adaptive law for adjusting is assumed to be of the form


where is a known function that is implementable at each and is a symmetric positive definite matrix referred to as the learning rate. In addition, for a given and , is chosen so that , is an equilibrium point of the system. Assuming that is a constant, the law in (7) can be written as


We consider all classes of adaptive systems that can be expressed in the form of (6) and (8) where , , , and are such that all solutions are bounded, and where . In particular, we assume that , , and are such that a quadratic Lyapunov function candidate


yields a derivative for the case of constant as


where and are symmetric positive definite matrices. Due to the choice of the adaptive law in (8), it follows therefore . Further conditions on , , and guarantee that as . We formalize this assumption below:

Assumption 1 (Class of adaptive systems)

For the case of a constant unknown parameter (), the error model in (6) and the adaptive law in (8) are such that they admit a Lyapunov function as in (9) which satisfies the inequality in (10).

Several adaptive systems that have been discussed in the literature satisfy Assumption 1, some examples of which are shown in Table 1. They include plants where state feedback is possible and certain matching conditions are satisfied, and where only outputs are accessible and a strictly positive real transfer function can be shown to exist. For a SISO plant that is minimum phase, the same assumption can be shown to hold as well. Finally, for a class of nonlinear plants, where the underlying relative degree does not exceed two, Assumption 1 once again can be shown to be satisfied.

3.1 Problem Formulation

The class of error models we consider is of the form (6), where , the time-varying unknown parameter, is such that if , then the solutions of (6) are globally bounded, with remaining an equilibrium point. This is formalized in the following assumption:

Assumption 2 (Uncertainty variation)

The uncertainty, , in (6) is such that , . In addition, its time derivative, , is assumed to be bounded, i.e. , . Furthermore, if , then the solutions of (6) are globally bounded, with an equilibrium point at .

The problem that we address in this paper is the determination of an adaptive law similar to (8) for all error models of the form (6) where Assumptions 1 and 2 hold. Our goal is to ensure global boundedness of solutions of (6) and exponentially fast tending of both and to a compact set with finite excitation.

4 Adaptive Law with a Time-Varying Learning Rate

The adaptive law that we propose is a modification of (8) with a time-varying learning rate as . To ensure a bounded , we include a projection operator in this adaptive law which is stated compactly as


where is defined as in Definition 4. The -projection operator in (11) uses , where are coercive, continuously differentiable convex functions. Define the subsets , , and . Via Assumption 2, each are chosen such that and corresponds to , , .

The time-varying learning rate is adjusted using the projection operator for positive definite matrices (see Definition 5) as


where , are positive scalars and . is a symmetric positive definite constant matrix chosen so that , where is a coercive, continuously differentiable convex function. Lemma 2 implies there exists a constant such that for all . We assume that is chosen so that for all . It should be noted that a large contributes to a decrease in

Finally the matrix is adjusted as


where is a symmetric positive semi-definite matrix with and denotes a filtered normalized regressor matrix. and are arbitrary positive scalars and is chosen so that . These scalars represent various weights of the proposed algorithm. The main contribution of this paper is the adaptive law in (11), (12), and (13), which will be shown to result in bounded solutions in Section 5 which tend exponentially fast to a compact set if is finitely exciting. If in addition, is persistently exciting, exponentially fast convergence to a compact set will occur .

Remark 4

The projection operator employed in (12) is one method to bound the time-varying learning rate. Instead of (12), one can also use a time-varying forgetting factor to provide for of the form


While the time-varying forgetting factor, , also achieves a bounded , it is more conservative than the projection operator in (12) as it is always active. In comparison, the projection operator as in (12) only provides limiting action if is in a specified boundary region and the direction of evolution of causes to increase. An equivalence between time-varying forgetting factors and projection operators may be drawn using the square root of the function in (5) with , , and the limiting action always remaining active.

Remark 5

It can be noted that while the regressor normalization in (13) is optional for linear regression systems [21, 22], it is required for general adaptive control problems in the presence of system dynamics as the regressor cannot be assumed to be bounded.

Remark 6

From a stability standpoint, the filtering in (13) for linear time-invariant error dynamics is optional, i.e. may be used in place of in (12). The inclusion of (13) however provides for a more smooth adjustment of in the presence of sharp changes in and enhances finite excitation properties by restricting all directions of increase of after a finite excitation and no additional excitation.

5 Stability and Convergence Analysis

Table 2: Norm of Signals in Phases of Excitation Propagation

We now state and prove the main result. The following assumption is needed for discussion of a finite excitation. We define an excitation level on an interval as


where , , and .

Assumption 3 (Finite excitation)

There exists a time and a time such that the regressor in (6) is finitely exciting over , with excitation level .

5.1 Propagation of Excitation and Boundedness of Information Matrix, Time-Varying Learning Rate

We first prove a few important properties of and under different excitation conditions.

Lemma 6

For the algorithm in (13), it follows that for any ,

  1. , ,

  2. , .

If in addition is finitely exciting as in Assumption 3, then

  1. , ,

where and . If in addition is persistently exciting (see Definition 6), with interval and level , , then

  1. , ,

, .

Lemma 7

The solutions of (12) and (4) satisfy the following:

  1. , , ,

  2. , ,

  3. , , ,

where . If in addition is finitely exciting as in Assumption 3, then there exists a such that

  1. , , ,

  2. , ,

where , , , and . If in addition is persistently exciting (see Definition 6), with interval and level , then there exists a , , and such that

  1. , , ,

  2. , .

The properties of and for a persistently exciting are relatively well known. For a finitely exciting , it should be noted that after a certain time elapses, the lower bound for is realized. This propagation is illustrated in Table 2.

The choice of the finite excitation level in Assumption 3 enables a fast convergence rate as follows: The denominator in ensures that the update in (12) pushes away from , provides for a bound for away from a minimum value, and accounts for excitation propagation through (13). The numerator scaling accounts for the normalization in (13), and provides for a bound away from a minimum excitation level.

5.2 Stability and Convergence Analysis

With the properties of the learning rate and filtered regressor above, we now proceed to the main theorem. The following lemma and corollary state important properties of the parameter estimate .

Lemma 8

The update for in (11) guarantees that there exists a such that , .

Corollary 2

Under Assumption 2, the update for in (11) provides for a constant such that , .

The following definitions are useful for stating the main result in Theorem 1. Define scalars and as


It is easy to see that and , where


Define as


where . Define a compact set as


alongside a corresponding set , defined as


We now state the main theorem of stability and convergence.

Theorem 1

Under Assumptions 1 and 2, the update laws in (11), (12), and (13) for the error model in (6) guarantee for any ,

  1. boundedness of the trajectories of and , .

If in addition is finitely exciting as in Assumption 3, then

  1. the trajectories of , tend exponentially fast towards a compact set , .

If in addition is persistently exciting as in Definition 6 with level and interval , then

  1. exponential convergence of the trajectories follows, of , towards a compact set , .

[Proof] Let , , . Consider a candidate Lyapunov function of the form


It follows that

Using (11), (12), and Lemma 5, may be simplified as

Using Lemma 4 and in (4), we obtain that


Using (16), (17), Corollary 2, and Assumption 2, the inequality becomes


From the first case of (24) it can be seen that for . From Lemmas 6, 7, 8, and Corollary 2, each of , , , and are bounded. Thus the trajectories of the closed loop system remain bounded. This proves Theorem 1-A).

From (22) and (24), it can be noted that in , where the compact set is defined in (20). Applying the Comparison Lemma (see [40], Lemma 3.4) for the second case of (24), we obtain that


with transition function . It can be noted that from Lemma 7 it was shown that , , and thus , , which follows from (17), (19). Thus (25) is simplified using (18) as


Furthermore given that and , it can be noted that , . Therefore it can be seen that over the interval of time , the state error and parameter error tend exponentially fast towards the bounded set . This proves Theorem 1-B).

If is persistently exciting with level , and interval , then it follows from Lemma 7-7) that (25) and (26) hold for all , which proves Theorem 1-C).

Remark 7

denotes the convergence rate of . This in turn follows if , i.e. if is bounded away from . The latter follows from Lemma 6-3) and 6-4) if is either finitely exciting or persistently exciting, with an exponentially fast trajectory of towards a compact set occurring over a finite interval or for all , respectively. This convergence rate however is upper bounded by .

Remark 8

Theorem 1-C) guarantees convergence of to a compact set , while Theorem 1-B) guarantees that approaches . This set scales with the signal in (16), which contains contributions both from and . For static parameters () and low excitation (i.e., and ), from (23) it can be shown that the trajectories of , tend towards the origin, i.e. the set .

Remark 9

Since we did not introduce any filtering of the underlying signals, the bound on the uncertain parameters is explicit in the compact set . It can be seen that directly scales with from (18). Such an explicit bound cannot be derived using existing approaches in the literature which filter dynamics.

Remark 10

The dependence of on is reasonable. As the time-variations in the uncertain parameters grow, it should be expected that the residue will increase as well. The dependence of on the filtered regressor is introduced due to the structure of our algorithm in (11), (12), and (13). As a result, even with persistent excitation, we can only conclude convergence of to a compact set as opposed to convergence to the origin. This compact set will be present even in the absence of time-variations in . This disadvantage, however, is offset by the property of exponential convergence to the compact set, which is virtue of the fact that we have a time-varying .

A closer examination of the convergence properties of the proposed algorithm is worth carrying out for the case of constant parameters. It is clear from (23) that the negative contributions to come from the first term, while any positive contribution comes if . That is, if there is a large enough excitation, then the third term can be positive. This in turn is conservatively reflected in the magnitude of . It should however be noted that a large with persistent excitation, leads to a large , which implies that as the third term in (23) becomes positive, it leads to a first term that is proportionately large and negative as well, thereby resulting in a net contribution that is negative. An analytical demonstration of this effect, however, is difficult to obtain. For this reason, the nature of our main result is convergence to a bounded set rather than to zero, in the presence of persistent excitation. Finally, we note that in our simulation studies, remained negative for almost the entire period of interest, resulting in a steady convergence of the parameter estimation error to zero.

Remark 11

The bounds for and derived in Lemmas 6 and 7 were crucial in establishing the lower bound for . That this occurs over an interval of time results in exponential and not just asymptotic properties. This has obvious implications of robustness (c.f. [40] §9.1).

6 Algorithm Discussion and Simulations

In this section we analyze the memory and computation requirements of our proposed algorithm and provide numerical simulations to demonstrate the algorithm in an illustrative application.

6.1 Memory Requirement and Computation

The standard parameter update in (8) requires integrations to adjust the parameters . Given that the updates for both and result in symmetric matrices, an additional integrations are required for each update for a total increase of integrations.

In comparison, the composite approach with finite excitation analysis presented in [27] results in an additional integrations to filter the error dynamics, integrations to filter the regressor, integrations to compute a symmetric information matrix, and integrations to compute an auxiliary matrix; for a total increase of

integrations. In order to avoid the knowledge of state derivatives in the approach used in the concurrent learning approach, estimates of past state derivatives is proposed using smoothing techniques with a forward and backward Kalman filter

[31, 32]. This however significantly increases the memory and computational requirements compared to the proposed algorithm.

6.2 Numerical Simulations

(a) State and control time histories.

(b) Lyapunov function, tracking error norm, parameter error norm and time-varying learning rate time histories.
Figure 2: Time histories of numerical simulation.

In this section we present numerical simulation results for linearized F-16 longitudinal dynamics trimmed at a straight and level flying condition with a velocity of ft/s and an altitude of ft. We present results for the case of a large constant unknown parameter in order to demonstrate the exponential convergence properties towards the origin with finite excitations and significant uncertainties. We include integral tracking of commands thus resulting in an extended state plant model given by