Higher-order algorithms for nonlinearly parameterized adaptive control

A set of new adaptive control algorithms is presented. The algorithms are applicable to linearly parameterized systems and to nonlinearly parameterized systems satisfying a certain monotonicity requirement. A variational perspective based on the Bregman Lagrangian (Wibisono, Wilson, Jordan, 2016) is adopted to define a general framework that systematically generates higher-order in-time speed gradient algorithms. These general algorithms reduce to a recently developed method as a special case, and naturally extend to composite methods that combine perspectives from both adaptive control and system identification. A provocative connection between adaptive control algorithms for nonlinearly parameterized systems and algorithms for isotonic regression and provable neural network learning is utilized to extend these higher-order algorithms to the realm of nonlinearly parameterized systems. Modifications to all presented algorithms inspired by recent progress in distributed stochastic gradient descent algorithms are developed. Time-varying learning rate matrices based on exponential forgetting/bounded gain forgetting least squares can be stably employed in the higher-order context, and conditions for their applicability in the nonlinearly parameterized setting are provided. A consistent theme of our contribution is the exploitation of strong connections between classical nonlinear adaptive control techniques and recent progress in optimization and machine learning.

Authors

• 6 publications
• 15 publications
• Accelerated Learning in the Presence of Time Varying Features with Applications to Machine Learning and Adaptive Control

Features in machine learning problems are often time varying and may be ...
03/12/2019 ∙ by Joseph E. Gaudio, et al. ∙ 0

• On Higher-order Moments in Adam

In this paper, we investigate the popular deep learning optimization rou...
10/15/2019 ∙ by Zhanhong Jiang, et al. ∙ 17

• A High-order Tuner for Accelerated Learning and Control

03/23/2021 ∙ by Spencer McDonald, et al. ∙ 0

• Connections Between Adaptive Control and Optimization in Machine Learning

This paper demonstrates many immediate connections between adaptive cont...
04/11/2019 ∙ by Joseph E. Gaudio, et al. ∙ 0

• The variational discretizaton of the constrained higher-order Lagrange-Poincaré equations

In this paper we investigate a variational discretization for the class ...
01/02/2018 ∙ by Anthony Bloch, et al. ∙ 0

• On Decidability of the Bisimilarity on Higher-order Processes with Parameterization

Higher-order processes with parameterization are capable of abstraction ...
08/24/2021 ∙ by Xian Xu, et al. ∙ 0

• Scalable Nonlinear Learning with Adaptive Polynomial Expansions

Can we effectively learn a nonlinear representation in time comparable t...
10/02/2014 ∙ by Alekh Agarwal, et al. ∙ 0

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Adaptive control is an online learning problem concerned with controlling an unknown dynamical system. This task is accomplished by constructing an approximation to the true dynamics through the adjustment of a set of parameters . The fundamental goal of adaptive control is stable concurrent learning and control of dynamical systems.

Adaptive control theory is a mature field, and many results exist tailored to specific system structures (Ioannou  Sun, 2012; Narendra  Annaswamy, 2005; JJ. Slotine  Li, 1991)

. An adaptive control algorithm typically consists of a parameter estimator coupled in feedback to the controlled system. While often strongly inspired by gradient-based optimization algorithms, an added complexity is that the estimator must not only be convergent, but must be stable when connected with the system in feedback.

Despite the difficulty of the problem, significant success has been achieved even for nonlinear systems in the linearly parameterized setting, where the dynamics approximation is of the form for some known regressor . Prominent examples include the adaptive robot trajectory controller of JJE. Slotine  Li (1987) and the neural network-based controller of Sanner  Slotine (1992), which employs a mathematical expansion in physical basis functions to uniformly approximate the unknown dynamics.

Unlike its linear counterpart, solutions to the adaptive control problem in the general nonlinearly parameterized setting

have remained elusive. Intuitively, this is unsurprising because gradient-based algorithms generally have guarantees only for convex loss functions; in the linearly parameterized setting, this requirement will be satisfied, but when the parameters appear nonlinearly, the problem is immediately in the difficult realm of non-convex optimization. Nevertheless, progress has been made in specific cases, such as with a convex or concave parameterization

(Annaswamy ., 1998; Ai-Poh Loh ., 1999; Kojić  Annaswamy, 2002), with a monotonicity condition (Tyukin ., 2007; I. Tyukin, 2011), through the Immersion and Invariance (I&I) approach (Astolfi  Ortega, 2003; Liu ., 2010), and through the speed gradient methodology (Fradkov, 1979; A. L. Fradkov, 1999; Ortega, 1996).

Our work continues in a recent tradition that utilizes a continuous-time view to analyze optimization algorithms; see, for example, Wibisono . (2016); Betancourt . (2018); Boffi  Slotine (2020); Muehlebach  Jordan (2019); Diakonikolas  Jordan (2019); Maddison . (2018). While the continuous-time view of optimization has seen a resurgence after it was used by Su . (2016) to provide an intuitive justification for Nesterov’s accelerated gradient method (Nesterov, 1983), continuous-time differential equations were used as early as 1964 by Polyak to derive the classical momentum or “heavy ball” optimization method (B. Polyak, 1964). Given the gradient-based nature of many adaptive control algorithms, the continuous-time view of optimization provides a natural bridge from modern optimization to modern adaptive control.

Continuous-time often affords simpler proofs, and it enables the application of physical intuition when reasoning about optimization algorithms, but finding the limiting differential equations may still be a daunting task. In a significant advance, Wibisono . (2016) showed that many accelerated methods in optimization can all be derived via a variational point of view from a single mathematical object known as the Bregman Lagrangian.

In this paper, we contribute to the linearly and nonlinearly parameterized (under the monotonicity assumptions of (Tyukin ., 2007)) problems. We utilize the Bregman Lagrangian in tandem with the speed gradient formalism (Fradkov, 1979; A. L. Fradkov, 1999) to define a general methodology to generate higher-order in-time (Morse, 1992) speed gradient algorithms. This contribution generalizes and extends a recently developed algorithm (Gaudio ., 2019), which can be seen as a special case of one of our higher-order speed gradient laws. Based on the first-order speed gradient methodology, these higher-order laws lead naturally to the development of composite higher-order adaptation algorithms for linearly parameterized systems (JJ. Slotine  Li, 1991). By use of a proportional-integral (PI) form, these composite laws are driven directly by the function approximation error itself, and do not require any explicit filtering of the system dynamics. Much like the well-known reduced-order Luenberger observer, the PI form enables obtaining in the adaptation law despite the fact that this signal is not explicitly measured (Luenberger, 1979).

By analogy between the nonlinearly parameterized law presented by Tyukin . (2007) and recent results in isotonic regression (Kakade ., 2011; Goel  Klivans, 2017; Goel ., 2018), we extend these higher-order algorithms to the nonlinearly parameterized setting. In a similar vein, we draw an orthogonal connecting thread to machine learning, and demonstrate a stable modification to our algorithms inspired by the Elastic Averaging Stochastic Gradient Descent (EASGD) algorithm (Zhang ., 2014). We then show how to combine all of our algorithms with time-dependent learning rates through the bounded gain forgetting formalism (JJ. Slotine  Li, 1991).

The paper is organized as follows. In Sec. 2, we present some required mathematical background. This includes a basic review of direct adaptive control (Sec. 2.1), the speed gradient formalism (Sec. 2.2), the Bregman Lagrangian and a higher-order adaptive control algorithm (Sec. 2.3), and the reduced-order Luenberger observer (Sec. 2.4). Sec. 3 presents our main contributions, with our higher-order speed gradient laws in Sec. 3.1, our first-order non-filtered composite laws in Sec. 3.2, our higher-order composite laws and higher-order laws for nonlinearly parameterized systems in Sec. 3.3, the elastic modification in Sec. 3.4, and our extension to time-dependent learning rates in Sec. 3.5. We conclude with some closing remarks and future directions in Sec. 4.

2 Preliminaries

We begin with an introduction to the formalism of direct adaptive control, and describe the systems to which our results apply. For simplicity, we restrict ourselves to the class of -order nonlinear systems

 x(n)+f(x,a,t)=u (1)

where denotes the derivative of , is the overall system state,

is a vector of unknown parameters,

is of known functional form but is unknown through its dependence on , and is the control input. We seek to design a feedback control law that depends on a set of estimated parameters and ensures that where is a known desired trajectory. We require that, along the way, the system remains stable and all system signals remain bounded. The estimated parameters are updated according to a learning rule or adaptation law

 ˙^a=g(a,^a,x) (2)

where must be implementable solely in terms of known system signals despite its potential dependence on . Hence, adaptive control is fundamentally an online learning problem where the data-generating process is a nonlinear dynamical system coupled in feedback to the learning process. For order systems as considered in (1), a common approach is to define the sliding variable (JJ. Slotine  Li, 1991)

 s=(ddt+λ)n−1~x=~x(n−1)−~x(n−1)r (3)

where we have defined and as the remainder. According to the definition (3), obeys the differential equation

 ˙s=u−f(x,a,t)−~x(n−1)r. (4)

Hence, from (4), we may choose to obtain the stable first-order linear filter

 ˙s=−ηs+f(x,^a,t)−f(x,a,t). (5)

For future convenience, we define and we will omit its arguments when clear from the context. From the definition of in (3), defines the dynamics

 (ddt+λ)(n−1)~x=0. (6)

Equation (6) is a stable -order filter which ensures that exponentially. For systems of the form (1), it is thus sufficient to consider the two first-order dynamics (2) and (5), and the adaptive control problem has been reduced to finding a learning algorithm that ensures .

Remark 2.1.

Systems in the matched uncertainty form

 ˙x=Ax+b(u−f(x,a,t))

where the constant pair is controllable and the constant parameter vector in the nonlinear function is unknown, can always be put in the form (1) by using a state transformation to the second controllability canonical form see Luenberger (1979), Chapter 8.8. After such a transformation, the new state variables satisfy for and . Defining as in (3) and correctly computing leads to (5). Hence, all results in this paper extend immediately to such systems.

Remark 2.2.

The fundamental utility of defining the variable is its conversion of the adaptive control problem for the -order system (1) to an adaptive control problem for the first-order system (5). Our results may be simply extended to other error models (Narendra  Annaswamy, 2005; Ai-Poh Loh ., 1999) of the form (5), or error models with similar input-output guarantees, as summarized by Lemma A.1.

Remark 2.3.

We will use to denote the equivalent first-order system to (1), , where and .

A classic setting for adaptive control is when the unknown nonlinear dynamics depends linearly on the set of unknown parameters, that is

 f(x,a,t)=Y(x,t)a

with a known function. In this setting, a well-known algorithm is the adaptive controller of JJ. Slotine  Coetsee (1986), given by

 ˙^a=−sPYT (7)

and its extension to multi-input adaptive robot control (JJE. Slotine  Li, 1987), where is a constant positive definite matrix of learning rates. Consideration of the Lyapunov-like function shows stability of the feedback interconnection of (5) and (7) via an application of Barbalat’s Lemma (JJ. Slotine  Li, 1991). We will refer to (7) as the Slotine and Li controller.

In this work, we make a mild additional assumption that simplifies some of the proofs. It requires the following definition.

Definition 2.1.

A function is said to be locally bounded in and uniformly in if it is bounded whenever and are finite.

Following Definition 2.1, we make the following assumption.

Assumption 2.1.

The dynamics is locally bounded in and uniformly in .

We now provide a brief introduction to a class of adaptive control methods known as speed gradient algorithms (A. L. Fradkov, 1999; Fradkov, 1979; Ortega, 1996). Speed gradient algorithms are applicable to nonlinearly parameterized systems that satisfy a convexity requirement described in Assumption 8. In their most basic form, these algorithms are specified by a “local” goal functional . is required to satisfy three main assumptions.

Assumption 2.2.

is non-negative and radially unbounded: for all , and when . is also uniformly continuous in whenever is bounded.

Assumption 2.3.

There exists an ideal set of control parameters such that the origin of the system (1) is globally asymptotically stable when the control is evaluated at . Furthermore, is a Lyapunov function for the system when the control is evaluated at . That is, there exists a strictly increasing function such that with .

Assumption 2.4.

The time-derivative of is convex in the control parameters . The first-order condition for convexity

 ˙Q(x,a1,t)≥˙Q(x,a2,t)+(a1−a2)T∇a2˙Q(x,a2,t) (8)

must be satisfied for all .

When Assumptions 2.2-8 are satisfied, the speed gradient adaptive control law is defined as

 ˙^a=−P∇^a˙Q(x,^a,t). (9)

where is a positive definite matrix of learning rates. Its properties are summarized in the following proposition (A. L. Fradkov, 1999).

Proposition 2.1.

Consider the speed gradient algorithm (9) with local goal functional satisfying Assumptions 2.2-8. Then, all solutions given by (1) and (9) remain bounded, and

 limt→∞Q(x(t),t)=0

for all .

The proof follows by consideration of the Lyapunov function . Intuitively, while the goal functional may only depend on the control parameters indirectly through , its time derivative will depend explicitly on through . The adaptation law (9) ensures that moves in a direction to decrease . Under the conditions specified by Assumptions 2.2-8, this causes to be negative for long enough to accomplish the desired goal (A. L. Fradkov, 1999).

Remark 2.4.

If is chosen so that depends on only through and is linearly parameterized, then Assumption 8 will immediately be satisfied by convexity of affine functions. Indeed, consider defining the goal functional for system (1). It is clear that this proposed goal functional satisfies Assumptions 2.2 and 2.3111Strictly speaking, does not have to be uniformly continuous in for bounded . This is a technical condition useful for the proof, but in many cases stability may still be shown via other means.. Then , and (9) exactly recovers the Slotine and Li controller (7), originally derived based on Lyapunov considerations222Note that is still convex in despite the fact that may change sign because is linear in by assumption.. In this sense, speed gradient algorithms represent a flexible class of methods that contain as particular cases some pre-existing approaches.

Rather than a local functional, one may instead specify an integral goal functional of the form . In this case, (9) takes the form

 ˙^a=−P∇^aR(x,^a,t). (10)

Equation (10) is a gradient flow algorithm on the loss function . Replacing Assumptions 2.2 and 2.3 by the slightly modified setting:

Assumption 2.5.

is a non-negative function: for all .

Assumption 2.6.

There exists an ideal set of controller parameters and a scalar function such that , , and for all .

The properties of algorithm (10) are summarized in the following proposition (A. L. Fradkov, 1999).

Proposition 2.2.

Consider the speed gradient algorithm (10) where the goal functional satisfies Assumptions 8-2.6. Then where

 α=12~a(0)TP−1~a(0)+∫∞0μ(t′)dt′

and for any bounded solution .

Integral functionals allow the specification of a control goal that depends on all past data and that may have an explicit dependence on . is chosen so that it does not depend on the structure of the dynamics. Local functionals, on the other hand, result in adaptation laws that do have an explicit dependence on the dynamics through the appearance of the term in .

Integral functionals can be particularly useful if implies the desired control goal, and in this work, we will generally focus on the setting where is the function approximation error. Goal functionals can also be written as a sum of local and integral functionals with similar guarantees, and these approaches will lead to composite algorithms in the subsequent sections; the interested reader is referred to A. L. Fradkov (1999), Chapter 3 for more details.

2.3 The Bregman Lagrangian and accelerated optimization algorithms

Beginning with the seminal paper of Su . (2016)

, there has been a recent revival of interest in the analysis of optimization algorithms as discretizations of continuous-time ordinary differential equations. From this angle, the analysis of an optimization algorithm can be broken into two steps: first, an understanding of the quantitative convergence rates of the continuous-time differential equation, and second, a search over numerical discretization techniques that preserve these convergence rates. In

Wibisono . (2016), the Bregman Lagrangian was shown to generate a suite of accelerated optimization algorithms by appealing to the Euler Lagrange equations through the principle of least action. In its original form, the Bregman Lagrangian is given by

 L(x,˙x,t)=e¯¯¯α+¯¯¯γ(Dh((x+e−¯¯¯α˙x,x)−e−¯¯¯βf(x)) (11)

where is a distance generating function that may be taken to be the Euclidean norm in . is the loss function, and is the Bregman Divergence

 Dh(y,x)=h(y)−h(x)−(y−x)T∇h(x).

The quantities , and are arbitrary time-dependent functions that ultimately set the damping and learning rates for the resulting algorithms. The Bregman Divergence may be understand as the discrepancy between evaluated at two points and the value predicted by a first-order Taylor expansion. It is guaranteed to be non-negative for convex functions by the gradient inequality (8).

To generate accelerated optimization algorithms, Wibisono, Wilson, and Jordan required two ideal scaling conditions: and . These conditions come out of the Euler Lagrange equations, where they are used to eliminate an unwanted term, and a Lyapunov argument, where they are used to ensure negativity of a chosen Lyapunov function.

Gaudio . (2019) recently utilized the Bregman Lagrangian to derive a momentum-like adaptive control algorithm. To do so, they defined , , and 333Note that these conditions validate the second ideal scaling condition but not the first. As mentioned above, the second ideal scaling condition is required only by the choice of Lyapunov function in the original work, which was used to derive convergence rates for optimization algorithms (Wibisono ., 2016). In this sense, they are not required for adaptive control.. Here, and

are non-negative scalar hyperparameters and

is a context-dependent normalizing signal. With these definitions, and in the setting presented in Sec. 2.1444The authors in Gaudio . (2019) consider a linear dynamical system, where the form of (12) differs slightly but identical considerations apply., (11) reduces to

 (12)

Comparing (11) and (12), it is clear that the loss function in (11) has been replaced by in (12). Following Remark 2.4, this is precisely the speed gradient functional that gives rise to the Slotine and Li controller. For (12), the Euler-Lagrange equations lead to the adaptation law

 ¨^a+˙^a(βN−˙NN)=−γβNsYT. (13)

(13) may be understood as a higher-order version of the Slotine and Li adaptive controller. It may also be re-written as two first-order systems

 ˙^v =−γsYT, (14) ˙^a =βN(^v−^a). (15)

(14) & (15) are useful for proving stability. The properties of (13) are summarized in the following proposition.

Proposition 2.3.

Consider the higher-order adaptation algorithm (13) or its equivalent representation (14) & (15) with and . Then, all trajectories remain bounded, , , and .

The proof follows by consideration of the Lyapunov function  (Gaudio ., 2019).

Remark 2.5.

While this transformation to a system of two first-order systems may seem somewhat ad-hoc, it is readily apparent by use of the Bregman Hamiltonian

 H(^a,p)=12βNe−¯¯¯γ∥p∥2+γe¯¯¯γ[ddt12s2], (16)

which, via Hamilton’s equations, leads to

 ˙p =−∂H∂^a=−γe¯¯¯γsYT, ˙^a =∂H∂p=βNe−¯¯¯γp.

Defining leads immediately to (14) & (15). As is typical in classical mechanics, the Bregman Hamiltonian may be obtained from a Legendre transform of the Bregman Lagrangian. The Hamiltonian equations may be useful for discrete-time algorithm development through application of symplectic discretization techniques (Betancourt ., 2018; França ., 2019; Shi ., 2019).

2.4 Reduced-order observers and proportional-integral adaptation laws

The reduced-order Luenberger observer (Luenberger, 1979) is a key tool in linear systems theory. Consider an -dimensional completely observable dynamical system

 ˙x1 =A11x1+A12x2+B1u ˙x2 =A21x1+A22x2+B2u y =x2

where is the state, , , is the system output, and the and matrices are known. Define an auxiliary variable

 z=x1−Lx2, (17)

where in (17) is an arbitrary design matrix. Then obeys the dynamics

 (18)

where it is clear that all quantities in (18) are known except for itself. Constructing an observer of identical form

ensures that the error obeys the dynamics

Because the original system is completely observable, can be selected by choice of

to have arbitrary eigenvalues by the pole placement theorem. The original state variables can then be reconstructed as

.

While is unknown through its dependence on , itself is known. Intuitively, unknown quantities contained in can thus be obtained in the observer dynamics through a proportional term containing . Similar concepts can be extended to nonlinear observers; see Lohmiller  Slotine (1998), Sec. 4.1. This idea of gaining a “free” derivative has also been used in adaptive control, with particular success when applied to nonlinear parameterizations. Proportional-integral adaptive laws of this type have been known as algorithms in finite form (A. L. Fradkov, 1999; IY. Tyukin, 2003) and appear in the well-known I&I framework (Astolfi  Ortega, 2003; Liu ., 2010). This approach will be the basis for our algorithms for nonlinearly parameterized systems.

3 Main contributions

In this section, we present the main contributions of this work. We begin by noting that the Bregman Lagrangian generates higher-order speed gradient algorithms, of which the adaptation law (13) is a special case. We prove some general conditions under which these higher-order algorithms will achieve tracking. By analogy with integral speed gradient functionals, we derive a proportional-integral scheme to implement a first-order composite adaptation law (JJ. Slotine  Li, 1991) driven directly by the function approximation error rather than its filtered version. We subsequently fuse the generating functional for the composite law with the Bregman Lagrangian to construct a higher-order composite algorithm.

Combining a connection between the techniques of isotonic regression (Kakade ., 2011; Goel  Klivans, 2017; Goel ., 2018) and algorithms for monotone nonlinear parameterizations (Tyukin ., 2007; I. Tyukin, 2011), we demonstrate how to modify our higher-order speed gradient framework to derive higher-order algorithms for nonlinearly parameterized systems. We follow this development by discussing a new form of high-order algorithm inspired by the Elastic Averaging Stochastic Gradient Descent (EASGD) algorithm (Zhang ., 2014; Boffi  Slotine, 2020) and extensions to distributed adaptation (Wensing  Slotine, 2017). We conclude by demonstrating how to use time-varying learning rates based on the bounded gain forgetting technique with our presented algorithms (JJ. Slotine  Li, 1991).

As noted in Sec. 2.3, the Bregman Lagrangian (12) that generates the higher-order algorithm (13) contains the local speed gradient functional that gives rise to the Slotine and Li controller. Based on this observation, we define local and integral higher-order speed gradient algorithms via the Bregman Lagrangian. We begin with the local functional

 L(^a,˙^a,t)=e∫t0βN(t)dt1βN(t)(12˙^aT˙^a−γβN(t)ddtQ(x,t)),

which generates the higher-order law

 ¨^a+˙^a(βN−˙NN)=−γβN∂˙Q(x,^a,t)∂^a. (19)

Algorithm (19) can be re-written as two first-order systems

 ˙^v =−γ∂˙Q(x,^a,t)∂^a (20) ˙^a =βN(^v−^a). (21)

To achieve the control goal, we require the following technical assumption in addition to Assumptions 2.2 & 8. This assumption replaces Assumption 2.3 for first-order speed gradient algorithms.

Assumption 3.1.

There exists a time-varying normalizing signal and non-negative scalar values such that the time-derivative of the goal functional evaluated at the true parameters, , satisfies the following inequality.

 ˙Q(x,a,t)−βμγN(t)∥^a−^v∥2+2(^a−^v)T∂˙Q∂^a≤−ρ(Q) (22)

where is positive definite, uniformly continuous in , and satisfies .

With Assumption 3.1 in hand, we can state the following proposition.

Proposition 3.1.

Consider the algorithm (19) or its equivalent form (20) & (21), and assume satisfies Assumptions 2.2, 8, and 3.1. Then, all solutions remain bounded, , and .

Proof.

Consider the Lyapunov-like function

 V=Q(x,t)+12γ~vT~v+12γ(^a−^v)T(^a−^v). (23)

Equation (23) implies that, with ,

 ˙V =˙Q(x,^a,t)−~aT∂˙Q∂^a−βγ∥^a−^v∥2−βμγN(t)∥^a−^v∥2+2(^a−^v)T∂˙Q∂^a ≤˙Q(x,a,t)−βγ∥^a−^v∥2−βμγN(t)∥^a−^v∥2+2(^a−^v)T∂˙Q∂^a ≤−ρ(Q)−βγ∥^a−^v∥2. (24)

By radial unboundedness of in , (23) & (24) show that remains bounded. Similarly, radial unboundedness of in and show that and remain bounded. Integrating (24) shows that , so that . An identical argument shows that . Now, because and are bounded, and because is locally bounded in and uniformly in by assumption, writing shows that is uniformly continuous in . Because is uniformly continuous when is bounded, and because is uniformly continuous in , is uniformly continuous in and by Barbalat’s lemma. This shows that . ∎

Remark 3.1.

The uniform continuity assumptions on and used in the general setting handled by the proof of Prop. 3.1 are not strictly necessary. Without them, we can conclude existence of the integral but not that . In many cases, signal chasing arguments based on the finiteness of this integral are sufficient, as will be shown in the coming sections.

By taking in Prop. 3.1, we immediately recover Prop. 2.3555Following Remark 3.1, we conclude that , and by local boundedness of in and uniformly in , showing by Barbalat’s lemma.. In this sense, Prop. 3.1 elucidates the underlying structure exploited by the Bregman Lagrangian to generate higher-order algorithms.

We now consider the integral functional

 L(^a,˙^a,t)=e∫t0βN(t)dt1βN(t)(12˙^aT˙^a−γβN(t)ddt∫t0R(x(t′),^a(t′),t′)dt′),

which generates the higher-order law

 ¨^a+˙^a(βN−˙NN)=−γβN∂R(x,^a,t)∂^a. (25)

We again re-write (25) as two first-order systems

 ˙^v =−γ∂R(x,^a,t)∂^a, (26) ˙^a =βN(^v−^a). (27)

We now require a modified version of Assumption  3.1.

Assumption 3.2.

is a non-negative function: for all , and . Furthermore, there exists a time-dependent normalizing signal and non-negative scalar values such that

 R(x,a,t)−R(x,^a,t)−βμγN(t)∥^a−^v∥2+2(^a−^v)T∇^aR(x,^a,t)≤−kR(x,^a,t)

for some constant .

With Assumption 3.2, we can state the following proposition.

Proposition 3.2.

Consider algorithm (25) or its equivalent form (26) & (27) along with Assumptions  8 & 3.2. Then, for any bounded trajectory of (1), and remain bounded, , and .

Proof.

Consider the Lyapunov-like function

 V=12γ~vT~v+12γ(^a−^v)T(^a−^v). (28)

Equation (28) implies that, with ,

 ˙V =−~aT∂R(x,^a,t)∂^a−βγ∥^a−^v∥2−βμγN(t)∥^a−^v∥2+2(^a−^v)T∂R(x,^a,t)∂^a ≤R(x,a,t)−R(x,^a,t)−βγ∥^a−^v∥2−βμγN(t)∥^a−^v∥2+2(^a−^v)T∂R(x,^a,t)∂^a ≤−kR(x,^a,t)−βγ∥^a−^v∥2 (29)

(28) & (29) show that, for any bounded solution , and remain bounded. Furthermore, integrating (29) shows that and . ∎

As mentioned in Sec. 2.2, we will be particularly interested in Prop. 3.2 when , which will generate composite adaptation algorithms.

Remark 3.2.

Classically, Lyapunov functions used in adaptive control consist of a sum of tracking and parameter estimation error terms, with chosen to cancel a term of unknown sign. Several Lyapunov functions in this work consist only of parameter estimation error terms, such as (28). From a mathematical point of view, all that matters is that is negative semi-definite and contains signals related to the tracking error. Integrating allows the application of tools from functional analysis to ensure that the control goal is accomplished.

Here we consider the linearly parameterized setting , and derive new first- and second-order composite adaptation laws. Composite adaptation laws are driven by two sources of error: the tracking error itself, as summarized by in the Slotine and Li controller, and a prediction error generally obtained from an algebraic relation which is itself constructed by filtering the dynamics (JJ. Slotine  Li, 1991). A starting point for our first proposed algorithm is to consider a hybrid local and integral speed gradient functional

 Q(x,t)=γ2s2+κ2∫t0~f2(x(t′),^a(t′),t′)dt′ (30)

where are positive design parameters weighting the contributions of each term. As discussed in Sec. 2.2, the first term leads to the Slotine and Li controller. The second can be clearly seen to satisfy Assumptions 2.5 and 2.6 with . It also satisfies Assumption 8, as is a quadratic function of for linear . Following the speed gradient formalism, the resulting adaptation law is given by

 ˙^a=−P(γs+κY~a)YT (31)

which is a composite adaptation law simultaneously driven by and the instantaneous function approximation error . Equation (31) depends on the function approximation error , which is not measured and hence cannot be used directly in an adaptation law. Nevertheless, it can be obtained through a PI law as discussed in Sec. 2.4. To do so, we define

 ξ(x,xd,t) =−κPs(x,xd)YT(x,t) (32) ρ(x,xd,t) =κP∫xn(t)xn(t0)s(x,xd)∂Y(x,t)T∂xndxn (33) ^a =¯¯¯a+ξ(x,xd,t)+ρ(x,xd,t) (34) ˙¯¯¯a =−(κη+γ)sPYT+κsn−1∑i=1P∂Y∂xi˙xi−n−1∑i=1(∂ρ∂xi)T˙xi =             −(∂ρ∂xd)T˙xd−∂ξ∂t−∂ρ∂t (35)

Computing demonstrates that (31) is obtained despite its dependence on through only the known signals contained in (32)-(3.2). A few remarks concerning the algorithm (31)-(3.2) are in order.

Remark 3.3.

The term may also be obtained by following the I&I formalism (Astolfi  Ortega, 2003; Liu ., 2010). To our knowledge, this discussion is the first that demonstrates the possibility of using a PI law in combination with a standard Lyapunov-stability motivated adaptation law to obtain a composite law.

Remark 3.4.

More error signals may be used for additional terms in the adaptation law. For example, a prediction error obtained by filtering the dynamics may also be employed, leading to a three-term composite algorithm.

Remark 3.5.

Much like the standard composite law obtained by filtering the dynamics, rearranging (31) shows that , so that the additional term can be seen to add a damping term that smooths adaptation (JJ. Slotine  Li, 1991).

Remark 3.6.

As mentioned in Sec. 2.1, for clarity of presentation we have restricted our discussion to the -order system (1). In general, the PI form (34) leads to unwanted unknown terms contained in in addition to the desired unknown term. In this case, the desired unknown term is while the undesired unknown term is . Indeed, the purpose of introducing the additional proportional term in (32) is to cancel the undesired unknown term. In general, cancellation of the undesired terms can be obtained by choosing to solve a PDE, and solutions to this PDE will only exist if the undesired term is the gradient of an auxiliary function. is then set to this auxiliary function. In some cases, the PDE can be avoided, such as through dynamic scaling techniques (Karagiannis ., 2009) or the similar embedding technique of Tyukin (I. Tyukin, 2011).

The properties of the adaptive law (31) may be summarized with the following proposition.

Proposition 3.3.

Consider the adaptation algorithm (31) with a linearly parameterized unknown, . Then all trajectories remain bounded, , , , and .

Proof.

Consider the Lyapunov-like function

 V=12s2+12γ~aTP−1~a,

which has time derivative

 ˙V=−ηs2−κγ~f2.

This immediately shows and . Because , by definition of the sliding variable (JJ. Slotine  Li, 1991). Integrating shows that and . The result follows by application of Lemma A.1 or directly by Barbalat’s Lemma. ∎

Following the accelerated speed gradient approach of Sec. 3.1, we now obtain a higher-order composite algorithm, and give a PI implementation. We again consider a hybrid local and integral speed gradient functional, so that (11) takes the form

 L(^a,˙^a,t)=e∫t0βN(t)dt1βN(t)(12˙^aT˙^a−βN(t)ddt[γ2s2+κ2∫t0~f2(x(t′),^a(t′),t′)dt′]) (36)

where and are positive constants weighting the two error terms. The Euler-Lagrange equations then lead to the higher-order composite system

 ¨^a+(βN−˙NN)˙^a=−βN(γs+κY~a)YT. (37)

As in Sec. 2.3, (37) may be implemented as two first-order systems

 ˙^v (38) ˙^a =βN(^v−^a) (39)

where now (38) is obtained through the PI form with , , and given by (32), (33), and (3.2) respectively with . The properties of the higher-order composite adaptation law (37) are stated in the following proposition.

Proposition 3.4.

Consider the higher-order composite adaptation algorithm (37) or its equivalent (38) & (39) for a linearly parameterized unknown, . Set and . Then all trajectories remain bounded, , ,