Adaptive Systems: History, Techniques, Problems, and Perspectives

Abstract

We survey some of the rich history of control over the past century with a focus on the major milestones in adaptive systems. We review classic methods and examples in adaptive linear systems for both control and observation/identification. The focus is on linear plants to facilitate understanding, but we also provide the tools necessary for many classes of nonlinear systems. We discuss practical issues encountered in making these systems stable and robust with respect to additive and multiplicative uncertainties. We discuss various perspectives on adaptive systems and their role in various fields. Finally, we present some of the ongoing research and expose problems in the field of adaptive control.

1. Introduction

Any system—engineering, natural, biological, or social—is considered adaptive if it can maintain its performance, or survive in spite of large changes in its environment or in its own components. In contrast, small changes or small ranges of change in system structure or parameters can be treated as system uncertainty, which can be remedied in dynamic operation either by the static process of design or the design of feedback and feed-forward control systems. By systems, we mean those in the sense of classical mechanics. The knowledge of initial conditions and governing equations determines, in principle, the evolution of the system state or degrees of freedom (a rigid body for example has twelve states–three components each of position, velocity, orientation and angular velocity). All system performance, including survival or stability, is in principle expressible as functions or functionals of system state. The maintenance of such performance functions in the presence of large changes to either the system or its environment is termed adaptation in the control systems literature. Adaptation of a system, as in biological evolution, can be of two kinds–adapting the environment to maintain performance, and adapting itself to environmental changes. In all cases, adaptive systems are inherently nonlinear, as they possess parameters that are functions of their states. Thus, adaptive systems are simply a special class of nonlinear systems that measure their own performance, operating environment, and operating condition of components, and adapt their dynamics, or those of their operating environments to ensure that measured performance is close to targeted performance or specifications.

The organization of the paper is as follows: Section 2 surveys some of the rich history of adaptive systems over the last century, followed by Section 3 with provides a tutorial on some of the more popular and common methods used in the field: Model Reference Adaptive Control, Adaptive Pole Placement, Adaptive Sliding Mode Control, and Extremum Seeking. Section 4 provides a tutorial for the early adaptive identification methods of Kudva, Luders, and Narendra. A brief introductory discussion is provided for the non-minimal realizations used by Luders, Narendra, Kreisselmeier, Marino, and Tomei. Section 5 discusses some of the weak points of control and identification methods such as nonlinear behavior, observability and controllability for nonlinear systems, stability, and robustness. This section also includes some of the solutions for handling these problems. Section 6 discusses some of the interesting perspectives related to control, observation, and adaptation. Section 7 presents some of the open problems and future work related to control and adaptation such as nonlinear regression, partial stability, non-autonomous systems, and averaging.

2. History of Adaptive Control and Identification

The first notable and widespread use of ‘adaptive control’ was in the aerospace industry during the 1950s in an attempt to further the design of autopilots [1]. After the successful implementation of jet engines into aircraft, flight envelopes increased by large amounts and resulted in a wide range of operating conditions for a single aircraft. Flight envelopes grew even more with developing interest in hypersonic vehicles from the community. The existing autopilots at the time left much to be desired in the performance across the flight envelope, and engineers began experimenting with methods that would eventually lead to Model Reference Adaptive Control (MRAC). One of the earliest MRAC designs, developed by Whitaker [2,3], was used for flight control. During this time however, the notion of stability in the feedback loop and in adaptation was not well understood or as mature as today. Parks was one of the first to implement Lyapunov based adaptation into MRAC [4]. An immature theory coupled with bad and/or incomplete hardware configurations led to significant doubts and concerns in the adaptive control community, especially after the crash of the X-15. This caused a major, albeit necessary, detour from the problem of adaptation to focus on stability.

The late 1950s and early 1960s saw the formulation of the state-space system representation as well as the use of Lyapunov stability for general control systems, by both Kalman and Bertram [5,6]. Aleksandr Lyapunov first published his book on stability in 1892, but the work went relatively unnoticed (at least outside of Russia) until this time. It has since been the main tool used for general system stability and adaptation law design. The first MRAC adaptation law based on Lyapunov design was published by Parks in 1966 [1]. During this time Filippov, Dubrovskii and Emelyanov were working on the adaptation of variable structure systems, more commonly known as sliding mode control [7]. Similar to Lyapunov’s method, sliding mode control had received little attention outside of Russia until researchers such as Utkin published translations as well as novel work on the subject [8]. Adaptive Pole Placement, often referred to as Self-Tuning Regulators, were also developed in the 1970s by Astrom and Egardt with many successful applications [9,10], with the added benefit of application to non-minimum phase systems. Adaptive identifiers/observers for LTI systems were another main focal point during this decade with numerous publications relating to model reference designs as well as additional stabilization problems associated with not having full state measurement [11,12,13,14,15,16]. However, Egardt [17] showed instability in adaptive control laws due to small disturbances which, along with other concerns such as instabilities due to: high gains, high frequencies, fast adaptation, and time-varying parameters, led to a focus on making adaptive control (and observation) robust in the 1980s. This led to the creation of Robust Adaptive Control law modifications such as: σ-modification [18, ϵ-modification [19], Parameter Projection [20], and Deadzone [21]. As an alternative for making systems more robust with relatively fast transients a resurgence in Sliding Mode Control and its adaptive counterpart was seen, particularly in the field of robotics [22,23,24]. The ideas of persistent excitation and sufficient richness were also formulated in response to the stability movement by Boyd, Sastry, Bai, and Shimkin [25,26,27,28].

These three decades were also a fertile time for nonlinear systems theory. Kalman published his work on controllability and observability for linear systems in the early 1960s, and it took about 10 years to extend these ideas to nonlinear systems through the use of Lie theory [29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44]. Feedback Linearization was formulated in the early to mid-1980s as a natural extension from applying Lie theory to control problems [45,46,47,48,49,50,51,52]. Significant improvements on our understanding of nonlinear systems and adaptation in the early 1990s was facilitated by the work on Backstepping and its adaptive counterpart by Kokotovic, Tsinias, Krstic, and Kanellakopoulos [53]. While Backstepping was being developed for matched and mismatched uncertainties, Yao and Tomizuka created a novel control method Adaptive Robust Control [54]. Rather than design an adaptive controller and include robustness later, Yao and Tomizuka proposed designing a robust controller first to guarantee transient performance to some error bound, and include parameter adaptation later using some of the methods developed in the 1980s. The previous work on nonlinear controller design also led to the first adaptive nonlinear observers during this time [55].

Another side of the story is related to Extremum Seeking control and Neural Networks, whose inception came earlier but development and widespread use as non-model and non-Lyapunov based adaptation methods took much longer. The first known appearance of Extremum Seeking (ES) in the literature was published by LeBlanc in 1922 [56]; well before the controls community was focused on adaptation. However, after the first few publications, work on ES slowed to a crawl with only a handful of papers being published over the next 78 years [57]. In 2000, Krstic and Wang provided the first rigorous stability proof [58], which rekindled excitement and interest in the subject. Choi, Ariyur, Lee, and Krstic then extended ES to discrete-time systems in 2002 [59]. Extremum Seeking was also extended to slope seeking by Ariyur and Krstic in 2004 [60], and Tan et al. discussed global properties of Extremum Seeking in [61,62]. This sudden resurgence of interest, has also led to the discovery of many interesting applications of Extremum Seeking such as antiskid braking [63], antilock braking systems [64], combustion instabilities [65], formation flight [66], bioreactor kinetics [67], particle accelerator beam matching [68], and PID tuning [69].

The idea of Neural Networks as a mathematical logic system was developed during the 1940s by McCulloch and Pitts [70]. The first presentation of a learning rule for synaptic modification came from Hebb in 1949 [71]. While many papers and books were published on subjects related to neural networks over the next two decades, perhaps the most important accomplishment was the introduction of the Perceptron and its convergence theorem by Rosenblatt in 1958 [72]. Widrow and Hoff then proposed the trainable Multi-Layered Perceptron in 1962 using the Least Mean Square Algorithm [73], but Minsky and Papert then showed the fundamental limitations of single Perceptrons, and also proposed the ‘credit assignment problem’ for Multi-Layer Perceptron structures [74]. After a period of diminished funding and interest, these problems were finally solved in the early 1980s. Shortly after this, Hopfield [75] showed that information could be stored in these networks which led to a revival in the field. He was also able to prove stability, but convergence only to a local minimum not necessarily to the expected/desired minimum. This period also saw the re-introduction of the back-propagation algorithm [76], which has become extremely relevant to neural networks in control. Radial Basis Functions (RBFs) were created in the late 80s by Broomhead and Lowe [77] and were shortly followed by Support Vector Machines (SVMs) in the early 90s [78]. Support Vector Machines dominated the field until the new millennium, after which previous methods came back into popularity due to significant technological improvements as well as the popularization of deep learning for fast ANN training [79].

In terms of the most recent developments (2006–Present) in adaptive control, the situation is a little complicated. The period from 2006 to 2011 saw the creation of the L₁-AC method [80,81,82,83,84,85] which garnered a lot of excitement and widespread implementation for several years. Some of the claimed advantages of the method included: decoupling adaptation and robustness, guaranteed fast adaptation, guaranteed transient response (without persistent excitation), and guaranteed time-delay margin. However, in 2014 two high profile papers [86,87] brought many of the method’s proofs and claimed advantages into question. The creators of the method were invited to write rebuttal papers in response to these criticisms, but ultimately declined these opportunities and opted instead to post non-peer-reviewed comments on their websites [88]. Other supporters of the method also posted non-peer-reviewed rebuttals on their website [89]. Many in the controls community are uncertain about the future of the method, especially since all of the main papers were reviewed and published in very reputable journals. In order to sort out the truths with respect to the proofs and claims of the method, more work needs to be done.

3. Adaptive Control Techniques

The goal of this section is to provide a survey of the more popular methods in adaptive control through analysis and examples. The following analyses and examples assume a familiarity with Lyapunov stability theory, but an in-depth treatise on the subject may be found in [90] if more background is needed.

3.1. Model Reference Adaptive Control

Model Reference Adaptive Control, or MRAC, is a control system structure in which the desired performance of a minimum phase (stable zeros) system is expressed in terms of a reference model that gives a desired response to a command signal. The command signal is fed to both the model as well as the actual system, and the controller adapts the gains such that the output errors are minimized and the actual system responds like the model (desired) system. We show the block diagram for this structure in Figure 1.

Figure 1. Model reference adaptive control structure.

Figure 1. Model reference adaptive control structure.

To show a simple example of MRAC, consider a simple first order LTI system

$$\dot{x}=-ax+bu$$

(1)

where a and b are unknown but constant parameters. We now define a stable reference model that represents the performance we want our unknown system to have for some trajectory r

$${\dot{x}}_m=-a_m{x}_m+b_{m}r$$

(2)

Now we match the equations with x in place of $x_m$

$$-ax+bu=-a_{m}x+b_{m}r$$

(3)

and define our control law as

$$u=\frac{1}{b}\left[(a-a_m)x+b_{m}r\right]$$

(4)

At this point, we may choose to make our adaptive controller ‘direct’ or ‘indirect’. The direct method adapts controller parameters directly, so $(a-a_m)/b$ becomes one parameter, even though $a_m$ is already known. The indirect method adapts estimates of the plant parameters, and then uses these values to update static controller relations. In other words a and b are estimated and then used in $(a-a_m)/b$ and $b_m/b$ to calculate the controller parameters. Direct methods typically rely on a gradient method such as the MIT rule, or are calculated based on Lyapunov stability theory. Indirect methods include algorithms such as Recursive Least Squares. For this example, we will choose the direct method and leave an indirect example for the next section.

Defining the parameters $p_1$ and $p_2$, we replace them with their estimates in the control law

$$u={\widehat{p}}_{1}x+{\widehat{p}}_{2}r$$

(5)

We the define the output error as the difference between the system and the reference model, that is

$$e\triangleq x-x_m$$

(6)

We substitute the control law into the error dynamics equation, and attempt to find a solution such that the error will be driven to zero, and parameter errors will go to zero as well. The error dynamics are written as

$$\dot{e}=-ax+b({\widehat{p}}_{1}x+{\widehat{p}}_{2}r)+a_m{x}_m-b_{m}r$$

(7)

Using the relation $\widehat{p}=p-\tilde{p}$, we cancel out all of the terms with the exact parameter values that we do not know to get

$$\dot{e}=-ax+b(p_1-{\tilde{p}}_1)x+b(p_2-{\tilde{p}}_2)r+a_m{x}_m-b_{m}r$$

(8)

Finally we get a representation that only relies on the parameter errors,

$$\dot{e}=-a_{m}e+b{\Phi }^T\tilde{p}$$

(9)

We construct a Lyapunov candidate

$$V=\frac{1}{2}{e}^2+\frac{1}{2}{\tilde{p}}^T{\Gamma }^{-1}\tilde{p}$$

(10)

and take the derivative

$$\dot{V}=e(-a_{m}e+b{\Phi }^T\tilde{p})+{\tilde{p}}^T{\Gamma }^{-1}\dot{\tilde{p}}$$

(11)

to prove its stability by showing each term is negative definite.

We can see that the first error term will be stable, and the entire system will be stable if we can force the other terms to be zero. We then simplify the expression and attempt to solve for the parameter adaptation. It is important to note here that since b is a constant and Γ is a gain matrix that we design, b can easily be ‘absorbed’ by Γ. The final representation of the Lyapunov analysis is shown as

$$\dot{V}=-a_m{e}^2+{\tilde{p}}^T({\Gamma }^{-1}\dot{\tilde{p}}+\Phi e)$$

(12)

The parameter adaptation law is, as we might think, a negative gradient descent relation that is a function of the output error of the system,

$$\dot{\tilde{p}}=-\Gamma \Phi e$$

(13)

It should be clear that in the case of a system of dimension larger than one, the preceding analysis will require linear algebra as well as the need to solve the Lyapunov equation $A^{T}P+PA=-Q$, but the results are more or less the same. Using the parameters $a=b=0.75$, $a_m=b_m=2$, ${\widehat{p}}_1\left(0\right)=0.8$, ${\widehat{p}}_2\left(0\right)=0.5$, ${\gamma }_1={\gamma }_2=5000$ we get the following simulation results. Figure 2 compares the output response of the plant with the reference model and reference signal. The figure clearly shows that the plant tracks the reference model will little to no error. Figure 3 shows the control input that forces the plant to follow the reference model. The initial oscillations come from the parameters being adapted to force the error to zero. The control parameter estimates are shown in Figure 4. We may note that the convergence of parameters for this system is quite fast, but also that their convergence to incorrect values has no effect on the output response of the system.

Figure 2. Output response for MRAC.

Figure 2. Output response for MRAC.

Figure 3. Control input for MRAC.

Figure 3. Control input for MRAC.

Figure 4. Parameter estimates for MRAC.

Figure 4. Parameter estimates for MRAC.

Remark 1 (Simulation). Simulations were performed using MATLAB Simulink. While simulations may be performed using coded loops, Simulink provides a convenient graphical environment that allows the user to use the control block diagrams to construct their system and controller. Each function block then contains the necessary equations to solve for the closed loop response at each time step. Transfer function blocks may be used in place of function blocks in many cases. This methodology may be applied to all control block diagrams. Keep in mind that the nonlinear nature of adaptive controllers will require a small time-step in simulation (typically on the order of ${10}^{-3}$). We show an example Simulink structure in Figure 5 for clarity.

Figure 5. MRAC simulink structure.

Figure 5. MRAC simulink structure.

3.2. Adaptive Pole-Placement

Adaptive Pole Placement Control (APPC) methods represent the largest class of adaptive control methods [91], and may be applied to minimum and non-minimum phase (NMP) systems. The idea behind APPC is to use the feedback loop to place the closed loop poles in locations that give us dynamics we desire. Figure 6 shows the basic control structure for APPC. If we also choose to design the zeros of the system, this adds an assumption of a minimum phase system, and leads to the Model Reference Adaptive Control (MRAC) method from the previous section. Consider the system and feedback control law

$$\begin{array}{cc}\hfill Ax& =Bu\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill Ru& =Tr-Sx\hfill \end{array}$$

(15)

where A, B, R, T, and S are differential operator polynomials with $\text{deg}\left(A\right)\ge \text{deg}\left(B\right)$. We also assume without loss of generality that A and B are coprime (no common factors), and R, T, and S are to be determined. The closed loop system becomes

$$x=\frac{BT}{AR+BS}r$$

(16)

If we want our system to follow a specified model we construct the relation

$$\frac{BT}{AR+BS}=\frac{BT}{A_c}=\frac{B_m}{A_m}$$

(17)

Figure 6. Indirect adaptive pole placement structure.

Figure 6. Indirect adaptive pole placement structure.

So far we have not made any assumptions about the stability of the system, only that A and B do not have any common factors. First we factor B into two parts, the stable and the unstable $B=B^{+}{B}^{-}$ as suggested in [1]. A cancellation must exist in $BT/A_c$ in order to achieve $B_m/A_m$ and we know that we cannot cancel $B^{-}$ with the controller, so $B^{+}$ must be a factor of $A_c$. We also know that $A_m$ must be a factor of $A_c$, so we may separate $A_c$ into three parts

$$A_c=A_0{A}_m{B}^{+}$$

(18)

Since $B^{+}$ is a factor of B and $A_c$ it must also be a factor of R, giving $R=R^{\prime }{B}^{+}$. This is due to the fact that it cannot be a factor of A since A and B are coprime. The closed loop characteristic equation finally reduces to

$$A{R}^{\prime }+B^{-}S=A_0{A}_m=A_c^{\prime }$$

(19)

Going back to the numerator, since we cannot cancel $B^{-}$ it must be a factor of $B_m$, giving $B_m=B^{-}{B}_m^{\prime }$. Finally, using the closed loop relation

$$\frac{BT}{A_0{A}_m{B}^{+}}=\frac{B^{-}T}{A_0{A}_m}$$

(20)

we can easily see that in order to have the system follow $B_m/A_m$ we must have

$$T=A_0{B}_m^{\prime }$$

(21)

We did not need the assumption of a minimum phase system as with MRAC because we did not cancel $B^{-}$, which is a big advantage over that method. Finally we introduce some causality conditions for the polynomials $A_0$, T, R, and S [1]:

$$\begin{array}{cc}\hfill \text{deg}\left(A_c\right)& =2\text{deg}\left(A\right)-1\hfill \\ \hfill \text{deg}\left(A_0\right)& =\text{deg}\left(A\right)-\text{deg}{\left(B\right)}^{+}-1\hfill \\ \hfill \text{deg}\left(R\right)& =\text{deg}\left(A_c\right)-\text{deg}\left(A\right)\hfill \\ \hfill \text{deg}\left(S\right)& \le \text{deg}\left(R\right)\hfill \\ \hfill \text{deg}\left(T\right)& \le \text{deg}\left(R\right)\hfill \end{array}$$

Consider a second order system of relative degree one and its desired reference model

$$\begin{array}{cc}\hfill G& =\frac{b_{0}s+b_1}{s^2+a_{1}s+a_2}\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill G_m& =\frac{b_m}{s^2+a_{m_1}s+a_{m_2}}\hfill \end{array}$$

(23)

Choosing not to cancel the zero, and using the minimal design and causality conditions $\text{deg}\left(A_c\right)=3$, $\text{deg}\left(S\right)=1$, $\text{deg}\left(R\right)=1$, $\text{deg}\left(A_0\right)=1$, $\text{deg}\left(T\right)=1$ we may solve for the controller parameter values

$$\begin{array}{cc}\hfill r& =\frac{b_1^2(a_{m_1}+a_0-a_1)-(a_1{b}_1-a_2{b}_0)(a_{m_2}+a_0{a}_{m_1})-a_2{b}_1{a}_0{a}_{m_2}}{b_1^2+a_2{b}_0^2-a_1{b}_0{b}_1}\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill s_0& =\frac{b_1(a_{m_2}+a_0{a}_{m_1}-a_2)+a_2{b}_0{a}_0{a}_{m_2}-b_0{b}_1(a_{m_1}+a_0-a_1)}{b_1^2+a_2{b}_0^2-a_1{b}_0{b}_1}\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill s_1& =\frac{b_0^2(a_{m_1}+a_0-a_1)-b_0(a_{m_2}+a_0{a}_{m_1}-a_2)+a_0{a}_{m_2}(b_1-a_1{b}_0)}{b_1^2+a_2{b}_0^2-a_1{b}_0{b}_1}\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill t_0& =b_m^{\prime }\hfill \end{array}$$

(27)

While both direct and indirect adaptation methods may be used with Self-Tuning Regulators, the indirect option is typically chosen. This is because solving for adaptation laws in the direct sense can become intractable since we may choose not to cancel zeros, which may lead to parametric nonlinearities depending on the system. Performing system identification through something like Recursive Least Squares (RLS) and having static control relationships is a much easier alternative that works provided signals are persistently exciting enough and parameters values are not ill-conditioned. We consider the popular example of a DC motor [1] whose transfer function and desired model are

$$\begin{array}{cc}\hfill G& =\frac{b}{s(s+a)}\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill G_m& =\frac{b_m}{s^2+a_{m_1}s+a_{m_2}}\hfill \end{array}$$

(29)

and $b_m$ is chosen to be the same as $a_{m_2}$ for simplicity. Following the procedure above the relations for $r_1$, $s_0$, and $s_1$ reduce to

$$\begin{array}{cc}\hfill r_1& =a_{m_1}+a_0-a\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill s_0& =\frac{a_{m_2}+a_{m_1}{a}_0-a{r}_1}{b}\hfill \end{array}$$

(31)

$$\begin{array}{cc}\hfill s_1& =\frac{a_0{a}_{m_2}}{b}\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill t_0& =\frac{a_{m_2}}{b}\hfill \end{array}$$

(33)

On-line system identification is done with the RLS algorithm

$$\begin{array}{cc}\hfill e_{\theta }& =y_f^{\left(n\right)}-{\varphi }^T\widehat{\theta }\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill \dot{\widehat{\theta }}& =P\varphi e_{\theta }\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill \dot{P}& =\alpha P-P\varphi {\varphi }^{T}P\hfill \end{array}$$

(36)

which uses signals generated by passing the input and outputs through stable filters. Choosing $a=b=1$, $a_{m_1}=1.4$, $a_{m_2}=1$, $a_0=2$, $\alpha =0$, $P(1,1)=100$, $P(2,2)=100$, $\widehat{a}\left(0\right)=2$ and $\widehat{b}\left(0\right)=0.2$ along with zero initial conditions as in [1], we get the following simulation results. Figure 7 shows the output response for the Self-Tuning Regulator (Adaptive Pole Placement) system. We achieve the goal of model-following within ten seconds after some initial overshoot. Figure 8 shows the quick and accurate convergence of the parameter estimates given that the reference signal is persistently exciting, and Figure 9 shows the control input to the system.

Figure 7. Output response for indirect APPC.

Figure 7. Output response for indirect APPC.

Figure 8. Plant parameter estimates for indirect APPC.

Figure 8. Plant parameter estimates for indirect APPC.

Figure 9. Control input for indirect APPC.

Figure 9. Control input for indirect APPC.

Remark 2 (Implementation Issue). It turns out that there can be some implementation issues specifically related to adapting transfer function blocks in certain graphical control simulation software, so the control needs to be converted to state space but cannot depend on derivative signals of inputs. Consider an example input

$$u=\frac{t_0(s+a_0)}{s+r_1}r-\frac{s_{0}s+s_1}{s+r_1}y$$

(37)

whose differential equation is obviously

$$\dot{u}+r_{1}u=t_0\dot{r}+a_0{t}_{0}r-s_0\dot{y}-s_{1}y$$

(38)

We start by defining a new intermediate variable equal to the integral of the derivative terms in the control input

$$q=u-t_{0}r+s_{0}y$$

(39)

whose update equation becomes

$$\dot{q}=-r_1(q+t_{0}r-s_{0}y)+t_0{a}_{0}r-s_{1}y$$

(40)

Finally the control input may be realized through

$$u=q+t_{0}r-s_{0}y$$

(41)

which may easily be updated in function blocks.

Remark 3 (Stochastic and Predictive Methods). Stochastic and predictive methods, as mentioned above, have seen the most widespread implementation out of all of the adaptive control methods, especially in the oil and chemical industries. These methods consist of optimizing the current time step and predicting future ones, according to models obtained from system identification and specified cost functions. In the case of adaptive stochastic and predictive methods, the identification is done on-line, and since sampling often creates non-minimum phase properties most of these methods are naturally based on the Self-Tuning Regulator structure. Some examples of these methods are: Minimum-Variance, Moving-Average, Linear Quadratic Gaussian, and the so called ‘Shooting’ methods for nonlinear systems.

3.3. Adaptive Sliding Mode Control

Adaptive Sliding Mode Control, or ASMC, is a variable structure control method that specifies a manifold or surface along with the system will operate or ‘slide’. When the performance deviates from the manifold, the controller provides an input in the direction back towards the manifold to force the system back to the desired output. We show the control structure in Figure 10. ASMC has been shown to be much more robust to noise, uncertainty, and disturbances than MRAC, but requires larger input signals.

Figure 10. Adaptive sliding mode control structure.

Figure 10. Adaptive sliding mode control structure.

We start by guaranteeing the sliding dynamics are always driven towards a specified sliding surface, that is

$$\frac{1}{2}\frac{d}{dt}{s}^2\le -k\parallel s\parallel$$

(42)

The above simplifies to

$$s\dot{s}=-k\parallel s\parallel$$

(43)

and finally gives the non-equivalent portion of the controller, which is

$$\dot{s}=-k\text{sgn}\left(s\right)$$

(44)

However, we can see that if s is close to zero the controller will chatter, which can damage the system. A fix for this is to introduce a ‘boundary layer’ around the sliding surface in the form of a saturation function, to smooth out the controller response and remove the chatter. The saturation function is defined as

$$\text{sat}\left(\frac{s}{\varphi }\right)=\left\{\begin{array}{cc}\frac{s}{\varphi },\hfill & s\le \varphi \hfill \\ \text{sgn}\left(s\right),\hfill & s>\varphi \hfill \end{array}\right.$$

(45)

where $\varphi$ is the ‘boundary layer’. We now start in the same way as before by defining our system and the performance we would like our system to have with:

$$\dot{x}=-ax+bu$$

(46)

and

$${\dot{x}}_m=-a_m{x}_m+b_{m}r$$

(47)

Then we define some stable manifold we want our system to slide along and determine its dynamics, in this case:

$$s\triangleq {\left(\frac{d}{dt}+\lambda \right)}^r{\int }_0^{t}ed\tau$$

(48)

and

$$\dot{s}=\dot{e}+\lambda e$$

(49)

We replace the surface dynamics with the discrete term that represents the trajectory motion towards the manifold which is

$$-k\text{sat}(s/\varphi )=-ax+bu+a_m{x}_m-b_{m}r+\lambda e$$

(50)

Solving for u we obtain the control law

$$u=\frac{1}{b}\left[-k\text{sat}(s/\varphi )+ax-a_m{x}_m+b_{m}r-\lambda e\right]$$

(51)

Then replacing the parameters with their estimates, we attempt to substitute u into $\dot{s}$ and obtain an equation for $\dot{s}$ that is stable and includes $\tilde{p}$ which is shown by:

$$u={\widehat{p}}_1\left[-k\text{sat}(s/\varphi )-a_m{x}_m+b_{m}r-\lambda e\right]+{\widehat{p}}_{2}x$$

(52)

$$\dot{s}=-ax+b(p_1-{\tilde{p}}_1)\left[-k\text{sat}(s/\varphi )-a_m{x}_m+b_{m}r-\lambda e\right]+b(p_2-{\tilde{p}}_2)x+a_m{x}_m-b_{m}r+\lambda e$$

(53)

$$\dot{s}=-b{\tilde{p}}_1\left[-k\text{sat}(s/\varphi )-a_m{x}_m+b_{m}r-\lambda e\right]-b{\tilde{p}}_{2}x-k\text{sat}(s/\varphi )$$

(54)

and

$$\dot{s}=b{\Phi }^T\tilde{p}-k\text{sat}(s/\varphi )$$

(55)

In order to solve for an adaptation law that forces the error to go to zero, we consider the Lyapunov candidate

$$V=\frac{1}{2}{s}^2+\frac{1}{2}{\tilde{p}}^T{\Gamma }^{-1}\tilde{p}$$

(56)

and take its derivative

$$\dot{V}=s(b{\Phi }^T\tilde{p}-k\text{sat}(s/\varphi ))+{\tilde{p}}^T{\Gamma }^{-1}\dot{\tilde{p}}$$

(57)

to determine the stability by showing each term is negative definite. We isolate the $\tilde{p}$ terms in

$$\dot{V}=-sk\text{sat}(s/\varphi )+{\tilde{p}}^T({\Gamma }^{-1}\dot{\tilde{p}}+b\Phi s)$$

(58)

and solve for the adaptation law

$$\dot{\tilde{p}}=-\Gamma \Phi s$$

(59)

which will guarantee the sliding surface is stable. Notice that we still get a gradient descent relationship, which is dependent upon s instead of e. Using the parameters $a=b=0.75$, $a_m=b_m=2$, ${\widehat{p}}_1\left(0\right)=0.8$, ${\widehat{p}}_2\left(0\right)=0.5$, ${\gamma }_1={\gamma }_2=5000$, $\varphi =0.1$, $k=10$, and $\lambda =5$ we get the following simulation results. Figure 11 shows the output response for the Adaptive Sliding Mode System. The response is very similar to the MRAC case, and converges to the reference model quite fast. Figure 12 shows the control input for the system. One of the advantages of sliding mode is the attenuation of oscillations, because the sliding mode term adds some robustness to the system. Figure 13 shows the convergence of the parameter estimates, and the oscillations are also absent from the parameter adaptation.

Figure 11. Output response for ASMC.

Figure 11. Output response for ASMC.

Figure 12. Control input for ASMC.

Figure 12. Control input for ASMC.

Figure 13. Parameter estimates for ASMC.

Figure 13. Parameter estimates for ASMC.

3.4. Extremum Seeking

Extremum Seeking is a strong optimization tool that is widely used in industry and more often categorized as a method of adaptive control. The reason is that Extremum Seeking is capable of dealing with unknown plants whose input to output maps possess an extremum (a minimum or a maximum), and this extremum depends on some parameter. The way Extremum Seeking works is by measuring the gradient of the output through adding (sinusoidal) perturbations to the system. This makes Extremum Seeking a gradient estimate method, with the additional advantage that the estimate happens in real-time. This has led to many industrial applications. The problem is formulated as follows. Suppose we have an unknown map $f\left(\theta \right)$. All we know about this map is that it has a minimum, but the value of this minimum and the $\theta ={\theta }^{*}$ at which it occurs are both unknown to us. We would like to find the value of ${\theta }^{*}$ that minimizes this map. Figure 14 shows the basic Extremum Seeking loop. The output of this map is fed to a washout filter. The purpose of this filter is to remove the bias of the map from the origin. The signal is then demodulated and modulated by a sinusoidal perturbation and integrated to estimate ${\theta }^{*}$ and the result is fed back to $f\left(\theta \right)$, which is also referred to as the cost function. Running this loop several times will lead to the exponential convergence of θ to ${\theta }^{*}$. For simplicity, we will not explain the details of how Extremum Seeking works. The reader can refer to [92] and references therein for more information. Instead, we provide the following simple example.

Figure 14. Extremum seeking structure.

Figure 14. Extremum seeking structure.

Suppose that the map is described by $f\left(\theta \right)={(\theta +5)}^2+2$. This means that the optimal value for the parameter is $\theta ={\theta }^{*}=-5$. Assuming $f\left(\theta \right)$ is unknown to us, we run Extremum Seeking as shown in Figure 14 with an initial guess of $\theta \left(0\right)=50$ for the parameter. Furthermore, we set the perturbation frequency and amplitude to $\omega =5$ rad/s and $a=0.2$, respectively. The integral gain is set to $k=1$, and finally, the washout filter is designed with $h=5$. Simulation results are shown in Figure 15 and Figure 16. We see that despite our poor initial guess, the algorithm manages to detect the true value of ${\theta }^{*}$ exponentially fast.

Figure 15. Output response for ES.

Figure 15. Output response for ES.

Figure 16. Parameter convergence for ES.

Figure 16. Parameter convergence for ES.

4. Adaptive Observer Techniques

Thus far we have only considered instances where full state measurement is available, a rather unlikely scenario for controlling real systems. Work on the adaptive observer problem has been on-going since the early seventies, but typically does not receive as much attention as the control problem. Adaptive identification methods not only solve the output feedback problem for adaptive systems, but they may also be used in the field of health monitoring. The various observers take advantage of a variety of canonical forms, each having their own advantages and disadvantages for implementation. Two realizations of an adaptive Luenberger-type observer will be discussed in detail, while the others briefly mentioned are non-minimal extensions of these two basic forms. The adaptive observer/identifier to be discussed can be visualized as an MRAS (Model Reference Adaptive System) structure shown in Figure 17. Exchanging the locations of the plant and model is the key to creating the MRAS structure identifier [12]. In the control case we were modifying the plant to behave like the reference model, where-as in this case we are modifying the observer/model to behave like the unknown plant.

Figure 17. MRAS observer structure.

Figure 17. MRAS observer structure.

4.1. Adaptive Luenberger Observer Type I

The following introduces one of the first adaptive observer designs, originally published by Kudva and Narenda [14]. It is well known throughout the dynamics and control literature that any observable LTI system may be transformed into the so-called ‘Observable Canonical Form’, and this will be our natural starting point:

$$\dot{x}=[-\mathbf{\text{a}}|\frac{I}{0\cdots 0}]x+\mathbf{\text{b}}u$$

(60)

We may then equivalently express this system (for reasons seen later) as

$$\begin{array}{cc}\hfill \dot{x}& =Kx+(\mathbf{\text{k}}-\mathbf{\text{a}})x_1+\mathbf{\text{b}}u\hfill \end{array}$$

(61)

$$\begin{array}{cc}\hfill y& =h^{T}x=x_1\hfill \end{array}$$

(62)

where K is defined

$$K=[-\mathbf{\text{k}}|\frac{I}{0\cdots 0}]$$

(63)

Define the state observer to have a similar structure with state and parameter estimates $\widehat{x}$, $\widehat{a}$, and $\widehat{b}$

$$\begin{array}{cc}\hfill \dot{\widehat{x}}& =K\widehat{x}+(k-\widehat{\mathbf{\text{a}}})x_1+\widehat{\mathbf{\text{b}}}u+w+r\hfill \end{array}$$

(64)

$$\begin{array}{cc}\hfill \widehat{y}& =h^T\widehat{x}={\widehat{x}}_1\hfill \end{array}$$

(65)

The purpose of the two additional signals w and r may not be clear at first, especially because we did not need additional signals in the control problem. In the control problem, we assumed that we had access to all states which also meant that we had access to each error term, but this is not the case here. In the observer problem, these two additional signals are used to maintain stability for the system, since everything must now be based only on the signals that we do have access to. Next we define the observer tracking and parameter errors as $e=\widehat{x}-x$, $\tilde{a}=a-\widehat{a}$, and $\tilde{b}=\widehat{b}-b$. The error dynamics become

$$\dot{e}=Ke+\tilde{a}{x}_1+\tilde{b}u+w+r$$

(66)

In order to design the governing equations for our additional signals w and r, we consider the scalar representation of the system

$$e_1^{\left(n\right)}+\sum _{i=1}^n{k}_i{e}_1^{(n-i)}=\sum _{i=1}^n{p}^{(n-i)}\left({\tilde{a}}_i{x}_1+{\tilde{b}}_{i}u+w_i+r_i\right)=p^{(n-1)}\sum _{i=1}^n\frac{{\tilde{a}}_i{x}_1+{\tilde{b}}_{i}u+w_i+r_i}{p^{i-1}}$$

(67)

The following equality

$$\sum _{i=1}^n\frac{{\tilde{a}}_i{x}_1+{\tilde{b}}_{i}u+w_i+r_i}{p^{i-1}}=\sum _{i=1}^n\frac{d_i}{p^{i-1}}({\tilde{a}}^{T}v+{\tilde{b}}^{T}q)$$

(68)

and subsequent definition of the auxiliary signals

$$\begin{array}{cc}\hfill w_m& =-\sum _{i=1}^{m-1}{\dot{\tilde{a}}}_i\sum _{k=m}^n{d}_k{v}_{k-m+i+1}+\sum _{i=m}^n{\dot{\tilde{a}}}_i\sum _{k=1}^{m-1}{d}_k{v}_{k-m+i+1}\hfill \end{array}$$

(69)

$$\begin{array}{cc}\hfill r_m& =-\sum _{i=1}^{m-1}{\dot{\tilde{b}}}_i\sum _{k=m}^n{d}_k{q}_{k-m+i+1}+\sum _{i=m}^n{\dot{\tilde{b}}}_i\sum _{k=1}^{m-1}{d}_k{q}_{k-m+i+1}\hfill \end{array}$$

(70)

where $w_1=r_1=0$, and for $m=2,\cdots ,n$ by Kudva and Narendra [14] are the key to the analysis. The error dynamics then become

$$\dot{ϵ}=Kϵ+d({\tilde{a}}^{T}v+{\tilde{b}}^{T}q)$$

(71)

where $ϵ=e$, and v and q are generated by applying stable filters to the accessible signals $x_1$ and u

$$\begin{array}{cc}\hfill v_i& =\frac{s^{n-i}}{s^{n-1}+d_2{s}^{n-2}+\cdots +d_n}{x}_1\hfill \end{array}$$

(72)

$$\begin{array}{cc}\hfill q_i& =\frac{s^{n-i}}{s^{n-1}+d_2{s}^{n-2}+\cdots +d_n}u\hfill \end{array}$$

(73)

The Lyapunov candidate is chosen to be

$$V=\frac{1}{2}{ϵ}^{T}Pϵ+\frac{1}{2}{\tilde{a}}^T{\Gamma }_1^{-1}\tilde{a}+\frac{1}{2}{\tilde{b}}^T{\Gamma }_2^{-1}\tilde{b}$$

(74)

which makes the derivative

$$\dot{V}=\frac{1}{2}{ϵ}^T(K^{T}P+PK)ϵ+{ϵ}^{T}Pd({\tilde{a}}^{T}v+{\tilde{b}}^{T}q)+{\tilde{a}}^T{\Gamma }_1^{-1}\dot{\tilde{a}}+{\tilde{b}}^T{\Gamma }_2^{-1}\dot{\tilde{b}}$$

(75)

We are still unsure about the first term, but we may choose the adaptation laws as

$$\begin{array}{cc}\hfill \dot{\tilde{a}}& =-{\Gamma }_1{ϵ}^{T}Pdv\hfill \end{array}$$

(76)

$$\begin{array}{cc}\hfill \dot{\tilde{b}}& =-{\Gamma }_2{ϵ}^{T}Pdq\hfill \end{array}$$

(77)

The first term is proven to be stable by choosing K and d to satisfy the Kalman-Yakubovich (Strictly-Positive-Real) Lemma

$$\begin{array}{cc}\hfill K^{T}P+PK& =-Q\hfill \end{array}$$

(78)

$$\begin{array}{cc}\hfill Pd& =h\hfill \end{array}$$

(79)

The adaptive laws then change to $\dot{\tilde{a}}=-{\Gamma }_1{e}_{1}v$ and $\dot{\tilde{b}}=-{\Gamma }_2{e}_{1}q$. Thus the system is proven to be stable in the Lyapunov sense. Now consider the system given in [14] already in observable canonical form

$$\dot{x}=\left[\begin{array}{cc}-5& 1\\ -10& 0\end{array}\right]x+\left[\begin{array}{c}1\\ 2\end{array}\right]u$$

(80)

with initial conditions $x_1=x_2=0$ and square wave input with amplitude 5 and 10 second period. The observer is chosen to be

$$\dot{\widehat{x}}=\left[\begin{array}{cc}-6& 1\\ -8& 0\end{array}\right]\widehat{x}+(k-\widehat{a})x_1+\widehat{b}u+w+r$$

(81)

with initial conditions $z_1=z_2=0$. We choose $d={[1,3]}^T$ and see that the K-Y lemma is satisfied with

$$\begin{array}{cc}\hfill P& =\left[\begin{array}{cc}2.5& -1/2\\ -1/2& 1/6\end{array}\right]\hfill \end{array}$$

(82)

$$\begin{array}{cc}\hfill Q& =\left[\begin{array}{cc}22& -25/6\\ -25/6& 1\end{array}\right]\hfill \end{array}$$

(83)

The gains are chosen to be

$$\begin{array}{cc}\hfill {\Gamma }_1& =\left[\begin{array}{cc}23& 0\\ 0& 3.8\end{array}\right]\hfill \end{array}$$

(84)

$$\begin{array}{cc}\hfill {\Gamma }_2& =\left[\begin{array}{cc}2& 0\\ 0& 0.4\end{array}\right]\hfill \end{array}$$

(85)

with the intial parameter estimates at ${\widehat{a}}_0={[3,12]}^T$ and ${\widehat{b}}_0={[0.5,1]}^T$. Figure 18 shows the actual plant output along with the observer estimate for the observed state $x_1$. Naturally we see quick convergence to the actual observed signal. The true power of the adaptive observer is seen in Figure 19 which shows the estimate for the unobserved state in the presence of parametric uncertainty. It is really quite remarkable that we can achieve correct state estimates in the presence of uncertain parameters. Indeed, we are also able construct control laws for adaptive systems that are based on both observed and unobserved states, a natural concern for real systems where full state measurement is uncommon. Figure 20 shows the estimates for parameters $a_1$ and $a_2$. Figure 21 shows the estimates for parameters $b_1$ and $b_2$. In both parameter estimate figures the parameters converge to their true values because the square-wave input is persistently exciting. Note that we would still achieve asymptotic convergence for the state estimate in the absence of persistent excitation because we were able to design the adaptation laws using Lyapunov stability theory.

Figure 18. Observed state estimate for adaptive observer type I.

Figure 18. Observed state estimate for adaptive observer type I.

Figure 19. Unobserved state estimate for adaptive observer type I.

Figure 19. Unobserved state estimate for adaptive observer type I.

Figure 20. State coefficient estimates for adaptive observer type I.

Figure 20. State coefficient estimates for adaptive observer type I.

Figure 21. Control coefficient estimates for adaptive observer type I.

Figure 21. Control coefficient estimates for adaptive observer type I.

4.2. Adaptive Luenberger Observer Type II

The next adaptive observer we will design is based on systems of the form

$$\dot{x}=\left[\mathbf{\text{a}}\right|\frac{1\cdots 1}{\Lambda }]x+\mathbf{\text{b}}u$$

(86)

It was shown in [15], that any observable LTI system may be transformed into the structure above. The purpose of converting systems to this form is to combat implementation issues with the previous observer as will be seen. The transformation is defined assuming the system is already in the OCF structure. First we design the diagonal matrix

$$\Lambda =\left[\begin{array}{ccc}-{\lambda }_2& \cdots & 0\\ 0& \ddots & 0\\ 0& \cdots & -{\lambda }_n\end{array}\right]$$

(87)

where the values ${\lambda }_i$ result in a Hurwitz characteristic polynomial. Luders and Narendra then define the vector $c(\Lambda )$

$$\text{det}(sI-\Lambda )=c{(\Lambda )}^T\left[\begin{array}{c}{s}^{n-1}\\ ⋮\\ 1\end{array}\right]$$

(88)

and $n-1$ vectors $c(\Lambda /{\lambda }_i)$

$$\frac{\text{det}(sI-\Lambda )}{s-{\lambda }_i}=c{(\Lambda /{\lambda }_i)}^T\left[\begin{array}{c}{s}^{n-2}\\ ⋮\\ 1\end{array}\right]$$

(89)

The transformation is finally defined as

$$T=\left[\begin{array}{cc}c(\Lambda )& \begin{array}{ccc}0& \cdots & 0\\ t_2& \cdots & t_n\end{array}\end{array}\right]$$

(90)

where $t_i=c(\Lambda /{\lambda }_i)$. The observer is then given as

$$\dot{\widehat{x}}=[\widehat{\mathbf{\text{a}}}|\frac{1\cdots 1}{\Lambda }][\begin{array}{c}{x}_1\\ {\widehat{x}}_2\\ ⋮\\ {\widehat{x}}_n\end{array}]+\widehat{\mathbf{\text{b}}}u+\left[\begin{array}{c}-k_1({\widehat{x}}_1-x_1)\\ \mathbf{\text{w}}\end{array}\right]$$

(91)

where w is chosen to make the observer dynamics stable. The error dynamics of the system then become

$$\dot{e}=\left[\begin{array}{ccc}\begin{array}{c}-k_1\\ \mathbf{0}\end{array}& |& \begin{array}{c}\begin{array}{ccc}1& \cdots & 1\end{array}\\ \Lambda \end{array}\end{array}\right]e+\tilde{a}{x}_1+\tilde{b}u+\mathbf{\text{w}}$$

(92)

where $e=\widehat{x}-x$. We now focus on one of the difficulties of adaptive observers as compared to adaptive controllers. Since we are not measuring states $x_2$ through $x_n$, we may not access $e_2$ through $e_n$. Since these values are used in the dynamics of $e_1$, we need to find a way to be able to analytically integrate $e_2$ through $e_n$. Consider just the error dynamics for $i=2,\cdots ,n$:

$$\dot{\overline{e}}=\Lambda \overline{e}+\tilde{\overline{a}}{x}_1+\tilde{\overline{b}}u+w$$

(93)

Choose w to be

$$w={(sI-\Lambda )}^{-1}(x_1\dot{\varphi }+u\dot{\psi })$$

(94)

so that we may use the convenient relation

$$\begin{array}{cc}\hfill (sI-\Lambda )\overline{e}& =\varphi x_1+\psi u+{(sI-\Lambda )}^{-1}(\dot{\varphi }{x}_1+\dot{\psi }u)\hfill \end{array}$$

(95)

$$\begin{array}{cc}& =(sI-\Lambda )\left({(sI-\Lambda )}^{-1}(\varphi x_1+\psi u)\right)\hfill \end{array}$$

(96)

It follows that

$$\overline{e}={(sI-\Lambda )}^{-1}(\varphi x_1+\psi u)+e^{\Lambda t}\overline{e}\left(t_0\right)$$

(97)

leading to the error dynamics

$${\dot{e}}_1=-k_1{e}_1+{\varphi }_1{x}_1+{\psi }_{1}u+h_1^T{(sI-\Lambda )}^{-1}(\varphi x_1+\psi u)+h_1^T{e}^{\Lambda t}\overline{e}\left(t_0\right)$$

(98)

The standard Lyapunov candidate will then lead to adaptive laws similar to the previous section. Notice that we were able to make it to this point without the need for complicated stabilizing signals, without needing to design K and d as they are both taken care of through Λ, and most importantly without the need to check the K-Y Lemma which can be tedious. By using a slightly more complex transformation (a one time event), Luders and Narendra were able to significantly simplify the observer.

4.3. Non-Minimal Adaptive Observers

The previous sections focused on minimal adaptive observer forms and detailed analysis. We now provide a brief exposure to some of the concepts of non-minimal observer form. Shortly after the progress in creating the minimal observers from the previous sections, Luders and Narendra developed a non-minimal representation [11,16]

$$\begin{array}{cc}\hfill {\dot{\widehat{x}}}_1& =-\lambda {\widehat{x}}_1+{\widehat{\theta }}^T\widehat{\omega }\hfill \end{array}$$

(99)

$$\begin{array}{cc}\hfill {\dot{\widehat{\omega }}}_1& =\Lambda {\widehat{\omega }}_1+lu\hfill \end{array}$$

(100)

$$\begin{array}{cc}\hfill {\dot{\widehat{\omega }}}_2& =\Lambda {\widehat{\omega }}_2+ly\hfill \end{array}$$

(101)

$$\begin{array}{cc}\hfill \widehat{y}& ={\widehat{x}}_1\hfill \end{array}$$

(102)

where ${\theta }^T=\left[\begin{array}{cccc}{c}_0& {\overline{c}}^T& d_0& {\overline{d}}^T\end{array}\right]$, ${\omega }^T=\left[\begin{array}{cccc}u& {\omega }_1& y& {\omega }_2\end{array}\right]$ and the pair $(\Lambda ,l)$ is chosen to be controllable. The representation gives the transfer function

$$\frac{Y\left(s\right)}{U\left(s\right)}=\frac{{\overline{c}}^T{(sI-\Lambda )}^{-1}l+c_0}{(s+\lambda )-{\overline{d}}^T{(sI-\Lambda )}^{-1}l-d_0}$$

(103)

and the following error dynamics

$${\dot{e}}_1=-\lambda e_1+{\tilde{\theta }}^T\widehat{w}+{\theta }^T\tilde{w}$$

(104)

Choosing the standard Lyapunov candidate

$$V=\frac{1}{2}{e}_1^2+\frac{1}{2}{\tilde{\theta }}^T{\Gamma }^{-1}\tilde{\theta }+\frac{\beta }{2}{\tilde{w}}^{T}P\tilde{w}$$

(105)

where ${\overline{\Lambda }}^{T}P+P\overline{\Lambda }=-Q$ and $\overline{\Lambda }=\text{diag}(\Lambda ,\Lambda )$ the adaptation law is given as

$$\dot{\tilde{\theta }}=-\Gamma \widehat{w}{e}_1$$

(106)

Two other non-minimal observers that came from this design were developed by Kreisselmeier [13], and Marino and Tomei [55]. The K-Filter [13] was originally developed for LTI systems as a reduced order ‘Series-Parallel’ observer, but was quickly extended to cases where nonlinearities may be expressed as functions of the measurements [53]. The K-Filter representation is given by

$$\begin{array}{cc}\hfill \dot{x}& =Ax+F{(y,u)}^T\theta \hfill \end{array}$$

(107)

$$\begin{array}{cc}\hfill y& =c^{T}x\hfill \end{array}$$

(108)

where $F(y,u)$ is akin to the ω terms in the original non-minimal representation. The K-Filter gives the state estimate as

$$\widehat{x}=\xi +{\Omega }^T\widehat{\theta }$$

(109)

which uses the property of superposition for the contributions to the system from u and y by separating them into two separate filters (like each ${\omega }_i$)

$$\begin{array}{cc}\hfill \dot{\xi }& =A_0\xi +ky\hfill \end{array}$$

(110)

$$\begin{array}{cc}\hfill {\dot{\Omega }}^T& =A_0{\Omega }^T+F{(y,u)}^T\hfill \end{array}$$

(111)

where $A_0=A-k{c}^T$ and satisfies the Lyapunov equation $P{A}_0+A_0^{T}P=-Q$. The reduced observer form is given as

$$\begin{array}{cc}\hfill \dot{\xi }& =A\xi +ky+\varphi \left(y\right)\hfill \end{array}$$

(112)

$$\begin{array}{cc}\hfill \dot{\Xi }& =A\Xi +\Phi \left(y\right)\hfill \end{array}$$

(113)

$$\begin{array}{cc}\hfill \dot{\lambda }& =A\lambda +e_n\sigma \left(y\right)u\hfill \end{array}$$

(114)

$$\begin{array}{cc}\hfill v_j& =A_0^j\lambda \hfill \end{array}$$

(115)

$$\begin{array}{cc}\hfill {\Omega }^T& =[v_m,\cdots ,v_1,v_0,\Xi ]\hfill \end{array}$$

(116)

where $e_n$ is the unit vector with the non-zero component at location n. The Marino-Tomei or MT-Filter is very similar to the K-Filter as shown below

$$\begin{array}{cc}\hfill \dot{\xi }& =A\xi +B\varphi \left(y\right)\hfill \end{array}$$

(117)

$$\begin{array}{cc}\hfill \dot{\Xi }& =A\Xi +B\Phi \left(y\right)\hfill \end{array}$$

(118)

$$\begin{array}{cc}\hfill \dot{\lambda }& =A\lambda +e_n\sigma \left(y\right)u\hfill \end{array}$$

(119)

$$\begin{array}{cc}\hfill v_j& =A_0^j\lambda \hfill \end{array}$$

(120)

$$\begin{array}{cc}\hfill {\Omega }^T& =[v_m,\cdots ,v_1,v_0,\Xi ]\hfill \end{array}$$

(121)

We have briefly introduced some of the non-minimal realizations of adaptive observers for LTI systems, but a much more detailed presentation and analysis of these methods may be found in [11,53].

5. Problems in Control and Adaptation

5.1. Nonlinear Systems

Nonlinear behavior is one of the most (if not the most) difficult aspects of adaptive control. Unfortunately there cannot be a general nonlinear theory. We must settle for using specific tools and methods that apply to the sets of system structures that we do understand. There are many types of nonlinear behaviors: limit cycles, bifurcations, chaos, deadzone, saturation, backlash, hysteresis, nonlinear friction, stiction, etc. Figure 22 shows some example plots of common nonlinearities.

Nonlinear behaviors are sometimes divided into two classes: ‘hard’ and ‘soft’. Soft nonlinearities are those which may be linearly approximated, such as $x^2$ or special types of hysteresis. Typically this means that as long as we do not stray too far from our operating point, we may use linear control methods since we can linearize the system. Hard nonlinearities are those which may not be linearly approximated, such as: Coulomb friction, saturation, deadzones, backlash, and most forms of hysteresis. Hard nonlinearities may easily lead to instability and/or limit cycles, and they unfortunately appear in many real systems. Moreover, since we cannot linearize we are forced to use nonlinear control methods in addition to adaptation. Fortunately for us there are methods for handling nonlinear control design: Feedback Linearization and Backstepping.

Figure 22. Examples of non-lipschitz nonlinearities. (a) Relay; (b) Deadzone; (c) Saturation; (d) Quantization; (e) Backlash; (f) Hysteresis-Relay.

Figure 22. Examples of non-lipschitz nonlinearities. (a) Relay; (b) Deadzone; (c) Saturation; (d) Quantization; (e) Backlash; (f) Hysteresis-Relay.

Feedback Linearization is a method in which a nonlinear coordinate transformation between the input and output is found such that the transformed system is linear along all trajectories. The first r-derivatives of the output are the coordinate transformations, and the coordinate transformation is required to be a diffeomorphism (invertible and smooth). We then design the input such that the $r^{\text{th}}$ output derivative is equivalent to some desired dynamics, ν, and all nonlinearities are canceled. Consider the following example of system and output dynamics [93]:

$$\begin{array}{cc}\hfill {\dot{x}}_1& =x_3-x_2^2\hfill \end{array}$$

(122)

$$\begin{array}{cc}\hfill {\dot{x}}_2& =-x_2-u\hfill \end{array}$$

(123)

$$\begin{array}{cc}\hfill {\dot{x}}_3& =x_1^2-x_3+u\hfill \end{array}$$

(124)

$$\begin{array}{cc}\hfill y& =x_1\hfill \end{array}$$

(125)

For input-output linearization we essentially take derivatives of y until the control input shows up in our equations

$$\begin{array}{cc}\hfill \dot{y}& =x_3-x_2^2\hfill \end{array}$$

(126)

$$\begin{array}{cc}\hfill \ddot{y}& =x_1^2-x_3+2x_2^2+(2x_2+1)u\hfill \end{array}$$

(127)

Our goal is to replace the $\ddot{y}$ with some desired dynamics ν, so we choose our control input to be

$$u=\frac{\nu -x_1^2-2x_2^2+x_3}{2x_2+1}$$

(128)

This transforms the system dynamics into

$$\dot{z}=Az+B\nu$$

(129)

where $z=\varphi \left(x\right)$ and $\varphi$ is some nonlinear coordinate transformation. In order to form a nonlinear coordinate transformation, we need to find a global diffeomorphism for the system in consideration. We know that the Lie derivative is defined on all manifolds, and the inverse function theorem will allow us to form a transformation using the output and its $n-1$ Lie derivatives. We consider the transformations

$$\begin{array}{cc}\hfill z_1& =x_1\hfill \end{array}$$

(130)

and

$$\begin{array}{cc}\hfill z_2& =x_3-x_2^2\hfill \end{array}$$

(131)

to construct our diffeomorphism for the example system. However, we still do not have enough functions for a coordinate transformation because we only needed to take two derivatives. The remaining transformation is often referred to as the ‘zero dynamics’ or the ‘internal dynamics’ of the system. In order to find a global diffeomorphism, we need to determine the final coordinate transformation $z_3$ such that the Lie derivative with respect to g is zero (as in the first two coordinate changes), shown as

$$\frac{\partial z_3}{\partial x}g\left(x\right)=\frac{\partial z_3}{\partial x_1}\left(0\right)+\frac{\partial z_3}{\partial x_2}(-1)+\frac{\partial z_3}{\partial x_3}\left(1\right)=0$$

(132)

We can see that an easy solution is $z_3=x_2+x_3$. Now we check the Jacobian to see if it is regular (no critical points) for all x with

$$\frac{\partial \varphi }{\partial x}=\left[\begin{array}{ccc}1& 0& 0\\ 0& -3x_2^2& 0\\ 0& 1& 1\end{array}\right]$$

(133)

The Jacobian is regular for all x and invertible, thus it is a global diffeomorphism. The last step one would normally take is to try and determine whether the zero dynamics are stable or not through functional analysis methods. However, we should beware of the complications that arise when we require all nonlinearities to be canceled. This means that we need to know our system exactly in order to truly have linear dynamics; any unmodeled dynamics can have disastrous effects. The other downside to this method is that the control signal may be unnecessarily large because we also cancel helpful nonlinearities (like $\dot{x}=-x^3$) in the process. It seems as though Feedback Linearization will perform well on systems containing soft nonlinearities, but maybe not on systems with hard nonlinearities unless we use neural networks.

Backstepping was created shortly after Feedback Linearization to address some of the aforementioned issues. It is often called a ‘Lyapunov-Synthesis’ method, because it recursively uses Lyapunov’s second method to design virtual inputs all the way back to the original control input. The approach removes the restrictions of having to know the system exactly and remove all nonlinearities because we use Lyapunov’s method at each step to guarantee stability. Backstepping is typically applied to systems of the triangular form:

$$\begin{array}{cc}\hfill {\dot{x}}_1& =f_1\left(x_1\right)+g_1\left(x_1\right)x_2\hfill \end{array}$$

$$\begin{array}{cc}\hfill {\dot{x}}_2& =f_2(x_1,x_2)+g_2(x_1,x_2)x_3\hfill \end{array}$$

$$\begin{array}{cc}& ⋮\hfill \end{array}$$

(134)

$$\begin{array}{cc}\hfill {\dot{x}}_{n-1}& =f_{n-1}(x_1,\cdots ,x_{n-1})+g_{n-1}(x_1,\cdots ,x_{n-1})x_n\hfill \end{array}$$

$$\begin{array}{cc}\hfill {\dot{x}}_n& =f_n(x_1,\cdots ,x_n)+g_n(x_1,\cdots ,x_n)u\hfill \end{array}$$

We view each state equation in this structure as its own subsystem, where the term coupled to the next state equation is viewed as a virtual control signal. An ideal value for signal is constructed, and the difference between the ideal and actual values is constructed such that the error is exponentially stable by Lyapunov. We may the error of the system to be the state or the difference between the system and a model in the cases of regulation and model following respectively. Assuming a model following problem, we consider a Lyapunov candidate for the first subsystem

$$V_1=\frac{1}{2}{e}_1^2$$

(135)

Following Lyapunov’s method, we would take the derivative which results in

$${\dot{V}}_1=e_1\left(f_1\left(x_1\right)+g_1\left(x_1\right)x_2-{\dot{x}}_{m_1}\right)$$

(136)

The $x_2$ term that connects the first subsystem to the next, is treated as the virtual control input. We choose ${\alpha }_1$ as the ideal value for virtual control $x_2$ such that $e_2$ becomes $e_2=x_2-{\alpha }_1$. We will be able to prove that the term containing $e_1$ is negative definite (guaranteeing stability), but we will be left with an $e_2$ from substituting in $x_2=e_2+\alpha$. This leads us to design a virtual control for the next subsystem in the same fashion, and then combining the Lyapunov candidates to get

$$V_2=V_1+\frac{1}{2}{e}_2^2$$

(137)

This continues on with ${\alpha }_{i-1}$ as the ideal value for $x_i$ and their corresponding errors, until we reach the real control input u. The final Lyapunov candidate function is

$$V_n=V_1+\cdots +V_{n-1}+\frac{1}{2}{e}_n^2$$

(138)

One of the main advantages to Backstepping is that we may leave helpful nonlinear terms in the equations. In Feedback Linearization, we have to cancel out all of the nonlinearities using the control input and various integrators. This makes Backstepping much more robust than Feedback Linearization, and also allows us to use nonlinear damping for control augmentation as well as extended matching for adaptation with tuning functions [53].

5.2. Observability

First consider the LTI system:

$$\begin{array}{cc}\hfill \dot{x}& =Ax+Bu\hfill \end{array}$$

(139)

$$\begin{array}{cc}\hfill y& =Cx+Du\hfill \end{array}$$

(140)

For observability, we want to see if we find the initial conditions based on our outputs which is represented by

$$\left[\begin{array}{c}y\\ \dot{y}\\ \ddot{y}\\ ⋮\\ y^{(n-1)}\end{array}\right]=O{x}_0+T\left[\begin{array}{c}u\\ \dot{u}\\ \ddot{u}\\ ⋮\\ u^{(n-1)}\end{array}\right]$$

(141)

where O is the observability matrix we are solving for, and T is the lower triangular matrix

$$T=\left[\begin{array}{cccc}D& 0& \cdots & 0\\ CB& D& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ C{A}^{(n-2)}B& \cdots & CB& D\end{array}\right]$$

(142)

The observability matrix for this system is then

$$\begin{array}{cc}\hfill O& =\left[\begin{array}{c}C\\ CA\\ ⋮\\ C{A}^{n-1}\end{array}\right]\hfill \end{array}$$

(143)

The observability matrix must be full rank for the system to be fully observable, that is

$$\text{rank}\left(O\right)=n$$

(144)

This is a corollary to saying the system dynamics are injective, or a one-to-one mapping. This states that if the function f is injective for all a and b in its domain, then $f\left(a\right)=f\left(b\right)$ implies $a=b$. More intuitively, for linear systems this means that if the rows are linearly independent, each state is observable through linear combinations of the output. We can also notice that we can get each column vector in O by taking the first $n-1$ derivatives of the output y. So in order to extend this to nonlinear systems, we can just use the Lie derivative. For nonlinear systems, the observation space $O_s$ is defined as the space of all repeated Lie derivatives of the covector $h\left(x\right)$, written as

$$\begin{array}{cc}\hfill O& =\left[\begin{array}{ccc}{L}_f^0{h}_1& \cdots & L_f^0{h}_p\\ \cdots & \cdots & \cdots \\ L_f^{n-1}{h}_1& \cdots & L_f^{n-1}{h}_p\end{array}\right]\hfill \end{array}$$

The system is said to be observable if

$$dim\left(d{O}_s\right)=n$$

(145)

We can see that this is true if we substitute $Cx$ for $h\left(x\right)$, and $Ax$ for $f\left(x\right)$. The collection of Lie derivatives will then produce the observability matrix for the LTI system. Finally, an often over-looked but very important part of observability is to check the condition number for the observability matrix of the actual system once sensors and their locations have been chosen. The condition number of the observability matrix can give an indication of how well the sensor choices and locations are for the system at hand. It may also be possible to optimize this process through the use of LMIs. This is especially important for adaptive systems considering their nonlinear behavior, which requires reliable observations to facilitate the adaptation process as well as provide good feedback for the controller.

5.3. Controllability

Consider the same LTI system dynamics

$$\dot{x}=Ax+Bu$$

(146)

The reachable subspace for the LTI system by the Cayley-Hamilton theorem is

$$R={\int }_0^t{e}^{A(t-\tau )}Bu\left(\tau \right)d\tau$$

(147)

Expanding out the matrix exponential we get the controllability matrix

$$C=\left[B\mid AB\mid \cdots \mid A^{n-1}B\right]$$

(148)

As with the observability matrix we say the system is controllable if this matrix is full rank, shown as

$$\text{rank}\left(C\right)=n$$

(149)

This is a corollary to saying the system dynamics are surjective, a mapping that is onto. The function f is said to be surjective if for every y in its range there exists at least one x in its domain such that $f\left(x\right)=y$. For linear systems this implies that the columns are linearly independent. It is also important to note that in linear systems, controllability and observability are related through transposition.

We want to find the controllability or reachability for the nonlinear system, so we look at the reachability equation above. After expanding the matrix exponential, we will get a collection of matrix multiplications of A and B. For nonlinear systems, we can multiply vector fields by using the Lie bracket. Thus the controllability matrix is the collection of Lie brackets on the vector fields of the nonlinear triangular system, shown as

$$C=\left[g_1,\cdots ,g_m,[g_i,g_j],\cdots ,[a{d}_{g_i}^k,g_j],\cdots ,[f,g_i],\cdots ,[a{d}_f^k,g_i],\cdots \right]$$

(150)

Like the observability problem, controllability is obtained if the matrix is full rank, and we can easily check this formulation and see that it even works for linear systems by substituting $Ax$ for $f\left(x\right)$ and B for $g\left(x\right)$. Similarly, checking the condition number of the actual controllability matrix after actuators have been chosen and placed is important. An ill-conditioned matrix indicates that the control authority could be improved, which is important for adaptive systems because of the need for high bandwidth response due to the nonlinear nature of the control law.

5.4. Stability & Robustness

Thus far we have not imposed any conditions on the external signals for stability, convergence, or robustness. In doing this we will explore another difference between the direct and indirect methods, as well as the overall robustness of adaptive controllers to external disturbances and unmodeled dynamics. First consider the problem of finding a set of parameters that best fit a data set. We have the actual and estimated outputs y and $\widehat{y}$, the set of inputs ${\varphi }^T$, and the actual and estimated parameters θ and $\widehat{\theta }$.

$$\begin{array}{cc}\hfill y& ={\varphi }^T\theta \hfill \end{array}$$

(151)

$$\begin{array}{cc}\hfill \widehat{y}& ={\varphi }^T\widehat{\theta }\hfill \end{array}$$

(152)

The goal is to minimize the error e between the actual and estimated outputs y and $\widehat{y}$, which is done by considering the squared error below.

$$\begin{array}{cc}\hfill J& =\frac{1}{2}{e}^{T}e\hfill \end{array}$$

(153)

$$\begin{array}{cc}& =\frac{1}{2}{(y-{\varphi }^T\widehat{\theta })}^T(y-{\varphi }^T\widehat{\theta })\hfill \end{array}$$

(154)

We minimize the squared error by differentiating with respect to the parameter estimates

$$\frac{\partial J}{\partial \widehat{\theta }}={(y-{\varphi }^T\widehat{\theta })}^T(-{\varphi }^T)$$

(155)

and then rearrange to solve for these estimates

$$\widehat{\theta }={\left(\varphi {\varphi }^T\right)}^{-1}\varphi y$$

(156)

In order for the estimates to be valid, the matrix $\left(\varphi {\varphi }^T\right)$ must be full rank (non-singular). Extending this to a recursive estimation, we want to minimize the integral of the squared error

$$J=\frac{1}{2}{\int }_0^t{e}^2\left(\tau \right)d\tau$$

(157)

We follow the same approach as before by differentiating with respect to the parameter estimate and equating with zero,

$$\frac{\partial }{\partial \theta }J={\int }_0^t(y-{\varphi }^T\widehat{\theta })(-{\varphi }^T)d\tau =0$$

(158)

The parameter estimate may then be expressed using

$$\widehat{\theta }={\left({\int }_0^t\varphi {\varphi }^{T}d\tau \right)}^{-1}{\int }_0^t\varphi yd\tau$$

(159)

where we once again require a non-singular condition on the matrix $\varphi {\varphi }^T$. The inverse term is often redefined as a covariance-like variable, which is shown in the RLS algorithm from the Self-Tuning Regulator section. We see that in order to minimize the integral squared error by obtaining accurate parameter estimates, we need the matrix $\varphi {\varphi }^T$ to be full rank. This condition is often rewritten as

$${\rho }_{1}I\ge {\int }_0^t\varphi {\varphi }^{T}d\tau \ge {\rho }_{2}I$$

(160)

for some non-zero positive constants ${\rho }_1$ and ${\rho }_2$, and is called the ‘persistent excitation’ condition. The persistent excitation condition was given in the early 1980s by Boyd and Sastry. In the case of the indirect self-tuning regulator, we designed our controller from the perspective that our recursive estimator will give us the correct estimates. If we do not have a persistently exciting reference signal, then our error will not converge to zero, because our parameter estimates will not converge to their true values. In many cases this may be accomplished by adding various types of dither signals to the overall reference signal that we want to track. This shows the key difference between indirect and direct adaptive control methods. Indirect methods rely on the reference signal(s) to be persistently exciting in order to get parameter error convergence which directly affects the tracking error. Direct methods will still achieve tracking error convergence without a persistently exciting reference signal, but if the signal is PE the parameters will also converge to their true values. It can also be shown that given a persistently exciting reference signal, the system will have exponential convergence rather than asymptotic convergence [94].

Apart from the mathematical condition itself, signals are often determined to be persistently exciting enough based on the number of fundamental frequencies contained within the signal. From the frequency space perspective signals may be approximated by sums of sines and cosines of these frequencies, and n frequencies may identify up to $2n$ independent parameters. When the input signal is persistently exciting, both simulations and analysis indicate that adaptive control systems have some robustness with respect to non-parametric uncertainties. This makes sense because the parameter estimates will converge to their true values, thus the equivalent control principle is truly satisfied and we get a ‘perfect’ controller. However, when the signals are not persistently exciting, even small uncertainties may lead to severe problems for adaptive controllers. Consider Rohrs’ examples [95], where the system and model are assumed to be

$$H_0\left(s\right)=\frac{2}{s+1}$$

(161)

and

$$M\left(s\right)=\frac{3}{s+3}$$

(162)

However, the real plant is

$$Y\left(s\right)=\frac{2}{s+1}\frac{229}{s^2+30s+229}U\left(s\right)$$

(163)

A direct MRAC controller is constructed assuming the plant is of first order, and the initial parameter estimates are ${\widehat{p}}_1\left(0\right)=1.14$ and ${\widehat{p}}_2\left(0\right)=-0.65$. The first example we look at, is when the reference signal is a constant $r=4.3$ and the gains we have chosen are ${\gamma }_1={\gamma }_2=2$.

Figure 23 and Figure 24 show that having unmodeled dynamics in the system can have disastrous effects even for small adaptation gains. The second example considers the reference signal $r=0.3+1.85sin(16.1t)$ (gains the same), which we would naturally assume to be persistently exciting without the unmodeled dynamics.

Figure 25 and Figure 26 show that the system starts to converge to the constant input, but becomes unstable as the oscillations grow and the parameters drift. The final example considers the case in which we have noise instead of a disturbance, with the reference signal and noise being $r=2$ and $n=0.5sin(16.1t)$ respectively. We also increase the gains to ${\gamma }_i=8.7075$.

Figure 27 and Figure 28 show a particularly troubling result. The system initial converges in both tracking error and parameter estimates, but after some time it becomes wildly unstable and the parameters drift by drastic amounts. Astrom correctly pointed out in a commentary (later added to [95]) that Rohrs’ examples did not completely characterize the robustness problem, but were never-the-less important to present to the community. While they may not fully characterize the issues, they should successfully motivate the results of the next section.

Figure 23. Output response for Rohrs’ first example.

Figure 23. Output response for Rohrs’ first example.

Figure 24. Parameter estimates for Rohrs’ first example.

Figure 24. Parameter estimates for Rohrs’ first example.

Figure 25. Output response for Rohrs’ second example.

Figure 25. Output response for Rohrs’ second example.

Figure 26. Parameter estimates for Rohrs’ second example.

Figure 26. Parameter estimates for Rohrs’ second example.

Figure 27. Output response for Rohrs’ third example.

Figure 27. Output response for Rohrs’ third example.

Figure 28. Parameter estimates for Rohrs’ third example.

Figure 28. Parameter estimates for Rohrs’ third example.

5.5. Robust Adaptive Control

Now that we’ve discussed some of the issues related to adaptation in the presence of noise and disturbances, we present some of the adaptation law modifications that were developed to handle these issues. Control law modifications are mentioned in the next section. First consider the reference model and system with uncertainty:

$$\begin{array}{c}\hfill \dot{x}=-ax+bu+\Delta \end{array}$$

(164)

$$\begin{array}{c}\hfill {\dot{x}}_m=-a_m{x}_m+b_{m}r\end{array}$$

(165)

We construct the control law in the same fashion as MRAC

$$u=\frac{(a-a_m)}{b}x+\frac{b_m}{b}r={\varphi }^T\widehat{\theta }$$

(166)

which gives us the error dynamics that now depend on the uncertainty

$$\dot{e}=-a_{m}e+b{\varphi }^T\tilde{\theta }+\Delta$$

(167)

We next consider the derivative of the standard quadratic Lyapunov candidate

$$\dot{V}=e(-a_{m}e+b{\varphi }^T\tilde{\theta }+\Delta )+\tilde{\theta }{\Gamma }^{-1}\dot{\tilde{\theta }}$$

(168)

After plugging in the traditional Lyapunov adaptation law, we may convert this to the following inequality

$$\parallel \dot{V}\parallel \le -a_m{\parallel e\parallel }^2+\parallel e\parallel \parallel \delta \parallel$$

(169)

where $\parallel \delta \parallel$ is the bound on the uncertainty. Since we require $\dot{V}$ to be negative definite the limiting case is when $\dot{V}=0$, and we may solve for the error norm in this case

$$\parallel e\parallel \le \frac{\parallel \delta \parallel }{a_m}=d$$

(170)

Anytime the error norm is within this bound, $\dot{V}$ can become positive and stability may be lost. The parameter error growth will become unstable regardless of the disturbance size, and is known as ‘parameter drift.’ One of the first adaptation law modifications created to handle this problem was the Deadzone method [21]

$$\dot{\widehat{\theta }}=\left\{\begin{array}{cc}-\Gamma \varphi e\hfill & if\parallel e\parallel >d\hfill \\ 0\hfill & if\parallel e\parallel \le d\hfill \end{array}\right.$$

(171)

which stops adaptation once the tracking error enters the bound establish in the preceding section. While the disturbance issue is solved, not only does the adaptive controller lose its asymptotic error convergence, but the control may chatter as the tracking error hovers around the Deadzone limit. The natural extension to this problem was to try and remove the a-priori bound on the disturbance. The σ-Modification method [18] removes the need for an a-priori bound on the disturbance by adding in a correction term as shown below

$$\dot{\widehat{\theta }}=-\Gamma \left(\varphi be+\sigma \widehat{\theta }\right)$$

(172)

After plugging this into the derivative of the Lyapunov function we can construct the inequality

$$\dot{V}\le -a_m{\parallel e\parallel }^2+\Delta \left(t\right)\parallel e\parallel -\sigma \parallel \theta \tilde{\theta }\parallel +\sigma \parallel \tilde{\theta }{\parallel }^2$$

(173)

which after applying the Holder inequality reduces to

$$\dot{V}\le \parallel e\parallel (\Delta \left(t\right)-a_m\parallel e\parallel )-\sigma \parallel \tilde{\theta }\parallel (\parallel \theta \parallel -\parallel \tilde{\theta }\parallel )$$

(174)

As long as the tracking and parameter errors are outside of these limits the system will remain stable. This method helped to remove the requirement on a-priori disturbance bounds, but presented another problem. If we look at the adaptation law itself, when the tracking error becomes small the trajectory of parameter estimates becomes

$$\widehat{\theta }\left(t\right)\approx e^{-\Gamma \sigma t}\widehat{\theta }\left(0\right)$$

(175)

and will decay back to the initial estimate. To prevent further modification of the estimates when the tracking error becomes small, the ϵ-Modification method [19] replaced the σ term with the tracking error

$$\dot{\widehat{\theta }}=-\Gamma \left(\varphi be-\parallel be\parallel \widehat{\theta }\right)$$

(176)

The other main drawback for the previous methods is that they can slow down the adaptation, which is the opposite of what we want. The Projection Method [20] is robust to disturbances and does not slow down the adaptation. Projection does, however, require bounds on the parameter values, but this is often acceptable for real systems. The Discontinuous Projection Algorithm is defined as [54]

$$\dot{\widehat{\theta }}=-\text{Proj}(\widehat{\theta },\Gamma \varphi e)$$

(177)

where the projection operator is

$$\text{Proj}(\widehat{\theta },•)=\left\{\begin{array}{cc}0\hfill & \text{if} \widehat{\theta }={\widehat{\theta }}_{max} \text{and}•>0\hfill \\ 0\hfill & \text{if} \widehat{\theta }={\widehat{\theta }}_{min} \text{and}•<0\hfill \\ •\hfill & \text{otherwise}\hfill \end{array}\right.$$

(178)

The discontinuous projection operator has similar chattering problems to adaptation with a deadzone, which led to the creation of smoothing terms similar to the saturation term in sliding mode control [22]. Stability analyses for these methods are excluded here to retain some form of brevity, but can be found in their respective references.

5.6. Adaptive Robust Control

An alternative to just modifying adaptation laws, is to also experiment with various non-equivalent control structures. Yao [54] approached the robustness problem from this perspective to create the Adaptive Robust Control (ARC) method whose structure is shown in Figure 29.

Figure 29. Adaptive robust control structure.

Figure 29. Adaptive robust control structure.

Assuming that the parameters lie in a compact set and the uncertainty is bounded, we start with a similar system as the previous section (input parameter removed for convenience)

$$\dot{x}={\varphi }^T\theta +\Delta +u$$

(179)

where $u=u_m+u_r$. The first input $u_m$ is the equivalent control input which compensates for the model ($u_m={\dot{x}}_d-{\varphi }^T\widehat{\theta }$), and the second input $u_r$ is a robust input. If we define the error as $e=x-x_d$, we get the error dynamics

$$\dot{e}=-{\varphi }^T\tilde{\theta }+\Delta +u_r$$

(180)

We typically separate the robust input into two parts $u_{r_1}$ and $u_{r_2}$. The first part is typically chosen as $u_{r_1}=-ke$. Thus we get

$$\dot{e}+ke=-{\varphi }^T\tilde{\theta }+\Delta +u_{r_2}$$

(181)

We want to design the robust input $u_{r_2}$ such that it stabilizes the system with respect to the terms ${\varphi }^T\tilde{\theta }$ and Δ. We assume that the components of $\varphi$ are measurable or observable, and we already know that the parameters θ and uncertainty Δ are bounded. With this information, we may construct a bounding function $h(x,t)$ that we may use for feedback control such that

$$|{\varphi }^T\tilde{\theta }-\Delta |\le h(x,t)$$

(182)

An example of a function $h(x,t)$ that satisfies this inequality is

$$h(x,t)=\parallel {\theta }_{max}-{\theta }_{min}{\parallel }^2{\parallel \varphi \parallel }^2+\parallel \Delta \parallel$$

(183)

where the norms of the parameters θ and regressor $\varphi$ are interpreted component-wise. An example of a robust control law that uses this bounding function is

$$u_{r_2}=-(\parallel {\theta }_{max}-{\theta }_{min}{\parallel }^2{\parallel \varphi \parallel }^2+\parallel \Delta \parallel )\text{sgn}\left(e\right)$$

(184)

As before in the sliding mode section, we will want to replace the $\text{sgn}\left(e\right)$ term with a smooth function such as $\text{sat}(e/b)$ (other options can be found in [54]). Consider the Lyapunov equation (no adaptation yet) $V=e^2/2$ along with its derivative

$$\dot{V}=-2kV+e\left(-{\varphi }^T\tilde{\theta }+\Delta +u_{r_2}\right)$$

(185)

Our goal here is to reduce the derivative of the Lyapunov stability equation to

$$\dot{V}=-2kV+ϵ$$

(186)

Using the final value theorem we get an error bound for the robust portion of control

$$\underset{s\to 0}{lim}sV\left(s\right)=\underset{s\to 0}{lim}\frac{ϵ}{s+2k}=\frac{ϵ}{2k}$$

(187)

and the transient response of the error norm is given by

$$V\left(t\right)=e^{-2kt}\left(V\left(0\right)-\frac{ϵ}{2k}\right)+\frac{ϵ}{2k}$$

(188)

In order to satisfy the robust tracking error bound as well as its transient, we require the robust portion of the control law to satisfy two requirements

$$\begin{array}{cc}\hfill e{u}_{r_2}& \le 0\hfill \end{array}$$

(189)

$$\begin{array}{cc}\hfill e(u_{r_2}-{\varphi }^T\tilde{\theta }+\Delta )& \le ϵ\hfill \end{array}$$

(190)

where ϵ is a positive design parameter. Using the expression for $u_{r_2}$ above (with sgn replaced with sat) the first inequality is obviously true, but the second requirement is not so trivial. When $e\ge b$

$$\begin{array}{cc}\hfill e(-h(x,t)\text{sgn}\left(e\right)-({\varphi }^T\tilde{\theta }-\Delta ))& \le \left|e\right|(-h(x,t)+|{\varphi }^T\tilde{\theta }-\Delta \left|\right)\hfill \end{array}$$

(191)

$$\begin{array}{cc}& \le 0\hfill \end{array}$$

(192)

but we must design b such that the second requirement is satisfied when $e<b$. In this case we get the inequality

$$e\left(-h(x,t)\frac{e}{b}-({\varphi }^T\tilde{\theta }-\Delta )\right)\le ϵ$$

(193)

If we choose $b=4ϵ/h(x,t)$, then we may rewrite the inequality as

$$ϵ-{\left(\frac{1}{2\sqrt{ϵ}}h(x,t)\left|e\right|-\sqrt{ϵ}\right)}^2\le ϵ$$

(194)

Using the Discontinuous Projection Method for on-line adaptation ($\dot{\tilde{\theta }}=\text{Proj}(\widehat{\theta },\Gamma \varphi e)$), we may decouple the robust and adaptive control designs, and finally analyze closed loop stability for the overall system. Consider the derivative of the standard quadratic Lyapunov candidate

$$\dot{V}=e\dot{e}+{\tilde{\theta }}^T{\Gamma }^{-1}\dot{\tilde{\theta }}$$

(195)

Plugging in the error dynamics and adaptation law we get

$$\begin{array}{cc}\hfill \dot{V}& =-2kV+e(-{\varphi }^{T}e+\Delta +u_{r_2})+{\tilde{\theta }}^T{\Gamma }^{-1}\text{Proj}(\Gamma \varphi e)\hfill \end{array}$$

(196)

$$\begin{array}{cc}& =-2kV+e(\Delta +u_{r_2})+{\tilde{\theta }}^T\left({\Gamma }^{-1}\text{Proj}(\Gamma \varphi e)-\varphi e\right)\hfill \end{array}$$

(197)

and using the inequalities from the robust control law design we have

$$\begin{array}{c}\hfill \dot{V}=-2kV+e{u}_{r_2}+{\tilde{\theta }}^T\left({\Gamma }^{-1}\text{Proj}(\Gamma \varphi e)-\varphi e\right)\end{array}$$

(198)

It is clear that the first term is negative definite, the second term is negative definite from our robust control law design conditions, and the third term is also negative definite due to the properties of the Discontinuous Projection law. Using this method in conjunction with Backstepping for nonlinear systems makes it extremely powerful especially due to bounded transient performance, but at the expense of a more complicated analysis. The interested reader may refer to [54] for details and simulations.

6. Perspectives in Control and Adaptation

6.1. Discrete Systems

Adaptive systems are inherently nonlinear, as must be evident by now, and hence their behavior cannot be captured without sufficiently large sampling frequencies. Hence, implementations of adaptation in engineering systems are based on high frequency sampling of system performance and environment, and therefore based on continuous system models and control designs. In those cases where the system is inherently discrete event, e.g., an assembly line, adaptation occurs from batch to batch as in Iterative Learning Control. The most widely used adaptive control is based on a combination of online system identification and Model Predictive Control. This method is ideal for process plants where system dynamics are slow compared to the sampling frequencies, speed of computation, and that of actuation. The actuators in these plants do not have weight or space constraints, and hence are made extremely powerful to handle a wide range of outputs and therefore of uncertainty and transient behavior. In this case a plant performance index such as time of reaction, or operating cost is minimized subject to the constraints of plant state, actuator and sensor constraints, and the discretized dynamics of the plant. Thus, this method explicitly permits incorporation of real world constraints into adaptation. The online system identification is performed using several tiny sinusoidal or square wave perturbations to actuator commands, and measuring resultant changes of performance. It is ideal for the process industry because of the great variability of the physical and chemical properties of feedstock, such as coal, iron ore, or crude oil.

6.2. Machine Learning and Artificial Intelligence

In general there are two types of models: ‘grey box’ and ‘black box’. Black box modeling indicates that hardly anything is known about the system except perhaps the output signal given some input. We have no idea what is inside the black box, but we may be able to figure it out based on the input/output relation. Grey box modeling is a situation where some a-priori information about the system is known and can be used. For example, knowing the design of the system and being able to use Newton’s laws of motions to understand the dominating dynamics would be considered grey box modeling. Even though we may know a lot about the system, there is no way we can know everything about it, hence the ‘grey’ designation. Uncertainty in our models and parameters are fundamental problems that we must unfortunately deal with in the real world.

The presence of uncertainty and modeling difficulties often leads to the field of machine learning, the process of machines ‘learning’ from data. Put simply, we approximate a relationship (function) between data points, and then hopefully achieving correct predictions based on its function approximation if it is to be used in control. There are many methods associated with machine learning (e.g., Neural Networks, Bayesian Networks, etc.), but we should be very careful in how and where we implement these algorithms. There are situations where using machine learning algorithms provide a significant advantage to the user and situations where it would be disadvantageous.

An example of a system where machine learning would be advantageous is a highly articulated robot such as a humanoid robot. A system of this type has too many states to handle in practice, not to mention complexities due to nonlinearities and coupling, so a machine learning algorithm would be very advantageous here. However, since the algorithm can only improve the function approximation by analyzing data, this approach obviously requires a lot of ‘learning’ time as well as computational resources. Therefore the system (or its data) has to be available to be tested in a controlled environment which is not always possible (consider a hypersonic vehicle). The other disadvantage to certain types of machine learning algorithms is illustrated by Anscombe’s Quartet, shown in Figure 30.

Figure 30. Anscombe’s quartet.

Figure 30. Anscombe’s quartet.

Anscombe’s Quartet was a famous data set given by Francis Anscombe [96] to show the limitations of regression analysis without graphing the data. Each of the significantly different data sets has the exact same mean, variance, correlation, and linear regression line. This shows that there will always be outliers that manipulate the system, but also that we should attempt to have a universal function identifier if we cannot narrow down the distributions of data we will see. There are machine learning algorithms that are universal function identifiers (see the Cybenko Theorem), but these theorems rest on the assumption that design parameters are correctly chosen, which is far from easy. This further illustrates the importance of the ‘learning’ period and training data, as well as using as much a-priori information as possible when it is available. That being said, machine learning algorithms have been successfully applied to a wide variety of very complex problems and constitute a fertile research topic. Some open problems related to Neural Networks are presented later.

6.3. Abstract Viewpoints

The fields of differential topology, differential geometry, and Lie theory are considerably more abstract than many are used to, but may offer a tremendous advantage to visualizing and thinking about more complex spaces. Topology is the study of the properties of a mathematical space under continuous deformations, and its differential counterpart is the study of these with respect to differentiable spaces and functions/vector fields. Differential geometry is quite similar to differential topology, but typically considers cases in which the spaces considered are equipped with metrics that define various local properties such as curvature. Lie theory is the mathematical theory that encompasses Lie algebras and Lie groups. Lie groups are groups that are differential manifolds, and Lie algebras define the structures of these groups. The combination of these fields allows us to classify and perform calculus on these spaces abstractly, regardless of their dimension. An example of the advantage of even understanding some of the most basic concepts is finding the curvature of a higher dimensional space. For problems in 3-dimensional space the cross-product can be used to find curvature, but the cross-product itself is only defined up to this dimension and thus cannot be used for n-dimensional problems. Differential geometry allows one to define curvature in an abstract sense for systems of any order, which will also be equivalent to the cross-product in 3-dimensional problems. We have already used some methods from these fields earlier in this paper, such as finding diffeomorphisms for Feedback Linearization and the Lie bracket for observability/controllability in nonlinear systems. For brevity we did not include the backgrounds of all of these fields, but the interested reader may refer to for more complete treatments of the subjects.

7. Open Problems and Future Work

Despite the extensive decades of research in adaptive control and adaptive systems, there are still many unsolved problems to be addressed. Some of these problems may not be ‘low hanging fruit’; however, their solution could lead to important applications. We discuss a few of these problems in this section. First, we discuss nonlinear regression problems, followed by transient performance issues. Finally qualify developing analysis tools that can ease the proofs of stability and boundedness for complex systems, as well as developing novel paradigms for adaptive control.

7.1. Nonlinear Regression

As with systems that are nonlinear in their states, there is no general nonlinear regression approach, it consists of a toolbox of methods that depend on the problem. Parameters appear linearly for many systems, but there are some systems (especially biological) where they appear nonlinearly. This happens to be one of the largest obstacles in the implementation of neural networks. In many neural network configurations, some parameters show up nonlinearly, and are typically chosen to be constant (lack of confidence guarantees), or an MIT rule approach is used (lack of stability guarantees). Even if network input and output weights show up linearly and adaptation laws are chosen using Lyapunov, it is all for naught if the nonlinear parameters are incorrect which can require extensive trial and error tuning by the designer or large amounts of training. An example of how control and identification may be used in an MRAC-like system is given in Figure 31.

The Radial Basis Function Neural Network design is a good example of when we might encounter difficulties related to nonlinear parameters. A basic RBFNN controller uses Gaussian activation functions that are weighted to form the control input u as shown in

$$\begin{array}{cc}\hfill h_j& =g\left(\frac{\parallel x-c_{ij}{\parallel }^2}{b_j^2}\right)\hfill \end{array}$$

(199)

$$\begin{array}{cc}\hfill u& =h^T\left(x\right)\widehat{w}\hfill \end{array}$$

(200)

Figure 31. Neural networks for MRAC.

Figure 31. Neural networks for MRAC.

The weights w appear linearly, and if we had ideal values for the centers c and biases b then we could easily construct a Lyapunov function to adapt the weights. However, finding ideal values for c and b can be quite difficult, so it would be advantageous to also find Lyapunov stable update laws for them as well. Logarithmic transformations exist for regression problems of the form $y=a{e}^{bx}$, but performing these transformations simultaneously with Lyapunov analysis can become quite overwhelming and even produce further problems related to the parameters that do not appear in the exponent. If more progress is made in finding transformations for turning nonlinear regression problems into linear ones, the field of neural networks and artificial intelligence can grow significantly. Training time for many systems will be significantly reduced, proving Lyapunov stability may become more tractable, and the problem illustrated by Anscombe’s quartet may be reduced.

7.2. Transient Performance

Adaptive controllers have the advantage that the error will converge to zero asymptotically, but for controlling real systems we often care more about the transient performance of the controlled system. It is generally not possible to give an a-priori transient performance bound for an adaptive controller because of the adaptation itself. When we use Lyapunov’s second method to derive an adaptation law, we are guaranteeing error convergence at the expense of parameter error convergence. Higher adaptation gains typically lead to faster convergence rates, but this is not always true given the instabilities that arise from high adaptation gains. Consider the standard scalar Lyapunov function

$$V=\frac{1}{2}{e}^2+\frac{1}{2{\gamma }_1}{\tilde{a}}^2+\frac{1}{2{\gamma }_2}{\tilde{b}}^2$$

(201)

After the adaptation law is chosen $\dot{V}=-a_m{e}^2$ (assuming no disturbances or modeling uncertainty). Setting up an inequality we may say that $\dot{V}\le -a_m{e}^2$. Now let’s analyze the ${\mathcal{L}}_2$ norm

$$\begin{array}{cc}\hfill {\parallel e\parallel }_2^2& ={\int }_0^{\infty }{\left|e\left(\tau \right)\right|}^{2}d\tau \le -\frac{1}{a_m}{\int }_0^{\infty }\dot{V}\left(\tau \right)d\tau \hfill \end{array}$$

(202)

$$\begin{array}{cc}& \le \frac{1}{a_m}V\left(0\right)-\frac{1}{a_m}V\left(\infty \right)\le \frac{1}{a_m}V\left(0\right)\hfill \end{array}$$

(203)

$$\begin{array}{cc}& \le \frac{1}{a_m}\left(\frac{1}{2}e{\left(0\right)}^2+\frac{1}{2{\gamma }_1}\tilde{a}{\left(0\right)}^2+\frac{1}{2{\gamma }_2}\tilde{b}{\left(0\right)}^2\right)\hfill \end{array}$$

(204)

The main problem here is that we do not have a good idea what the initial parameter errors are, so it is difficult to give a-priori predictions of transient performance. There are four common ways to help improve the transient performance: increase $a_m$, increase ${\gamma }_i$, use correct trajectory initialization to minimize $e\left(0\right)$, and perform system identification to minimize $\tilde{a}\left(0\right)$ and $\tilde{b}\left(0\right)$. Increasing $a_m$ is obviously a good way to improve performance, but this may not be an option depending on the control authority in the system (e.g., bandwidth, power, etc.). Increasing ${\gamma }_i$ is an option for ideal systems but as we discovered before, high adaptation gains in real systems can lead to instability and this will depend on the specific system at hand.

The use of robust adaptive control methods helps mitigate the stability problem, but at the expense of slowed transient performance or convergence to an error bound rather than zero. The projection method is reported to maintain fast adaptation and stability, but requires the designer to have bounds on parameters. This is not an unreasonable assumption, but there may be cases where it does not apply. Carefully initializing the trajectory to set $e\left(0\right)=0$ is an option that can be applied to most systems and does not pose any additional problems. Lastly, good system identification can provide accurate parameter estimates to initialize with in order to minimize $\tilde{a}$ and $\tilde{b}$, but this also assumes that the system may easily/cheaply be identified and that the correct identification method is used. There are many systems in existence that have highly nonlinear parametric models that are difficult (or impossible) to accurately identify with existing methods, or may be too costly to identify frequently. Improving transient performance is an on-going research problem in adaptive systems.

7.3. Developing Analysis Tools

The complexity of the tools needed to analyze the stability of a system grows with the complexity of the system. In this context, system has a broad meaning and is defined as a set of ordinary or partial differential equations. With this point of view, a control problem also reduces to a stability problem. For example, a control problem in which the objective is the perfect tracking of state variables, turns into a stability problem when we define tracking error and look at the governing error dynamics as our “system”. The stability of systems have been widely studied for more than a century, from the work of Lyapunov in 1900s. Hence, we will not discuss those results; however, we will mention a few areas where improvements could be made. The benefit of such research is twofold. Any new tool developed will have applications to any branch of science that deals with “dynamics”, i.e., any system that changes with time.

In order to provide more motivation, consider the following scenario. We have derived an adaptive control law for a nonlinear non-autonomous system under external disturbances. We would like to prove that the state tracking error converges to zero despite disturbances and the parameter tracking errors are at least bounded. If we fail this task, we would like to at least prove that the states remain bounded under disturbances, and that the error remains within a finite bound the whole time.

Non-autonomous systems are explicitly time-dependent and are generally described as

$$\dot{x}=f(t,x)$$

(205)

where $f:[t_0,t_1]\times D\to R^n$ is a function over some domain $D\subset R^n$. Simply put, a non-autonomous system is not invariant to shifts in the time origin. That is, changing t to $\tau =t-t_0$ will change the right hand side of the differential equation. The stability of such systems is most commonly studied via one of the following two approaches: (1) averaging theorems and (2) non-autonomous Lyapunov theorems.

7.3.1. Averaging Theorems

The first method is known as the averaging method, and applies to systems of the form

$$\dot{x}=\epsilon f(t,x,\epsilon )$$

(206)

where ε is a small number. Note that any system of form Equation (205) can be transformed to Equation (206) by the transformation $\tau =t/\epsilon$. General averaging theorems only require that f be bounded at all times on a certain domain. However, a simpler class of averaging theorems further require that f be T-periodic in t. The idea of averaging method is to integrate the system over one period. This process yields the average system as

$${\dot{x}}_{av}=\epsilon f_{av}\left(x_{av}\right)$$

(207)

where $f_{av}\left(x\right)=\frac{1}{T}{\int }_0^{T}f(\tau ,x,0)d\tau$. This removes the explicit time dependence and enables the use of vast stability theorems applicable to autonomous systems. However, the main question is whether the response of the new system is the same as the original system. The following theorem from [90] addresses this issue.

Theorem 1 

([90]). Let $f(t,x,\epsilon )$ and its partial derivatives with respect to $(x,\epsilon )$ up to the second order be continuous and bounded for $(t,x,\epsilon )\in [0,\infty )\times D_0\times [0,{\epsilon }_0]$, for every compact set $D_0\subset D$, where $D\subset R^n$ is a domain. Suppose f is T-periodic in t for some $T>0$ and ε is a positive parameter. Let $x(t,\epsilon )$ and $x_{av}\left(\epsilon t\right)$ denote the solutions of (206) and (207), respectively.

• If $x_{av}\left(\epsilon t\right)\in D\forall t\in [0,b/\epsilon ]$ and $x(0,\epsilon )-x_{av}\left(0\right)=O\left(\epsilon \right)$, then there exists ${\epsilon }^{*}>0$ such that for all $0<\epsilon <{\epsilon }^{*}$, $x(t,\epsilon )$ is defined and

  $$x(t,\epsilon )-x_{av}\left(\epsilon t\right)=O\left(\epsilon \right)on[0,b/\epsilon ]$$

  (208)
• If the origin $x=0\in D$ is an exponentially stable equilibrium point of the average system Equation (207), $\Omega \subset D$ is a compact subset of its region of attraction, $x_{av}\left(0\right)\in \Omega$, and $x(0,\epsilon )-x_{av}\left(0\right)=O\left(\epsilon \right)$, then there exists ${\epsilon }^{*}>0$ such that for all $0<\epsilon <{\epsilon }^{*}$, $x(t,\epsilon )$ is defined and

  $$x(t,\epsilon )-x_{av}\left(\epsilon t\right)=O\left(\epsilon \right)for allt\in [0,\infty )$$

  (209)
• If the origin $x=0\in D$ is an exponentially stable equilibrium point of the average system, then there exists positive constants ${\epsilon }^{*}$ and k such that, for all $0<\epsilon <{\epsilon }^{*}$, Equation (206) has a unique, exponentially stable, T-periodic solution $\overline{x}(t,\epsilon )$ with the property $\left|\right|\overline{x}(t,\epsilon )\left|\right|\le k\epsilon$.

Now suppose we want to apply this theorem to our hypothetical scenario and prove the stability of our adaptive controller. A reader familiar with adaptive control already knows that unless PE conditions are satisfied, the stability is only asymptotic. This means that parts 2 and 3 of this theorem will not be applicable, leaving us only with part 1. However, this part is also a weak result which is only valid for finite times. Therefore, averaging theorem in this form is not helpful in proving stability of adaptive control applied to non-autonomous systems.

Teel et al. [97,98] have shown that if the origin of the system is asymptotically stable with some additional conditions, then we can deduce practical asymptotic stability of the actual system. However, their theorem cannot be used in this scenario either. This is due to the fact that when we have asymptotic stability in adaptive control, the parameter errors generally do not converge to the origin. So even though the state tracking error converges to the origin, the parameters in general will converge to an unknown equilibrium manifold. This is sometimes referred to as “partial stability” and we will discuss it further. Therefore, one area of improving tools is to devise averaging theorems that not only work for asymptotically stable system, but also account for systems with partial stability where only state errors converge to the origin.

7.3.2. Non-Autonomous Lyapunov Stability

In search for a proper tool to analyze our hypothetical scenario, we next move on to Lyapunov theorems for non-autonomous systems. Such theorems, directly deal with systems that are explicitly time-dependent and are much less developed than theorems regarding autonomous systems. Khalil [90], Chapter 4, provides three such theorems one of which is mentioned here.

Theorem 2 

([90]). Let $x=0$ be an equilibrium point for Equation (205) and $D\subset R^n$ be a domain containing $x=0$. Let $V:[0,\infty )\times D\to R$ be a continuously differentiable function such that

$$\begin{array}{c}\hfill k_1{\left|\right|x\left|\right|}^a\le V(t,x)\le k_2{\left|\right|x\left|\right|}^a\end{array}$$

(210)

$$\begin{array}{c}\hfill \frac{\partial V}{\partial t}+\frac{\partial V}{\partial x}f(t,x)\le -k_3{\left|\right|x\left|\right|}^a\end{array}$$

(211)

$\forall t\ge 0$ and $\forall x\in D$, where $k_1$, $k_2$, $k_3$, and a are positive constants. Then, $x=0$ is exponentially stable. If the assumptions hold globally, then $x=0$ is globally exponentially stable.

In adaptive control, it is extremely difficult to satisfy condition (211). The derivative of the Lyapunov function is usually only semi-negative definite at best. This means that the right hand side of Equation (211) will only have some of the states and the inequality will not hold. We believe that creating new Lyapunov tools with less restricting conditions is an area that needs more attention.

7.3.3. Boundedness Theorems

When we cannot study the stability of the origin using known tools, or when we do not expect convergence to the origin due to the disturbances, the least we hope for is that x will be bounded in a small region. Boundedness theorems consider such cases and several of them are addressed in Chapter 9 of [90] who categorizes perturbations into vanishing perturbations and non-vanishing perturbations. The general approach is to separate the perturbation terms from the system. Therefore, the system is described as

$$\dot{x}=f(t,x)+g(t,x)$$

(212)

where $g:[0,\infty )\times D\to R^n$ is the perturbation term, and $D\subset R^n$ is a domain that contains the origin $x=0$. Note that the perturbation term could result from modeling errors, disturbances, uncertainties, etc. Therefore, boundedness theorems have wide applications in realistic problems.

Vanishing perturbations refer to the case where $g(t,0)=0$. Therefore, if $x=0$ is an equilibrium of the nominal system $\dot{x}=f(t,x)$, then it also becomes an equilibrium point of the perturbed system. Non-vanishing perturbations refer to cases where we cannot determine whether $g(t,0)=0$ . Therefore, the origin may not be an equilibrium point of the perturbed system.

Such theorems, although very useful in many cases, still need further development before they can be applied to our scenario and be useful for adaptive control. Most boundedness theorems require exponential stability at the origin. We know that in adaptive control, exponential stability is only possible when PE conditions are satisfied (in which case our first approach to use averaging theorems would have worked already!). Furthermore, in the absence of PE, the origin is not a unique equilibrium for the adaptive controller: some parameter estimates may converge to an unknown equilibrium manifold and we cannot transform the equilibrium of the adaptive controller to the origin. Since the objective of the adaptive controller is not the identification of parameters, but rather the convergence of tracking error, one wonders whether it is possible to study the stability of only parts of the states (i.e., the stability of the state tracking error). This is sometimes referred to as partial stability.

7.3.4. Partial Stability and Control

The first person to ever formulate partial stability was Lyapunov himself – the founder of modern stability theory. During the cold war, with the resurgence of interest in stability theories, this problem was pursued and Rumyantsev [99,100] published the first results. Much research has been done on partial stability all over the world, but mostly in Russia and the former USSR. For example, see [101,102,103,104,105,106]. Since this topic might be unfamiliar to many readers, we explain it a little further and refer the enthusiastic reader to the papers cited. In particular [103] provides a comprehensive survey of problems in partial stability.

Partial stability deals with systems for which the origin is an equilibrium point, however, only some of the states approach the origin. Such systems commonly occur in practice, and there have been a fair amount of research on their stability analysis using invariant sets and Lyapunov-like lemmas such as Barbalat’s Lemma or LaSalle’s Principle. Some of the motives for studying partial stability are [103]: systems with superfluous variables, sufficiency of partial stability for normal operations of system, estimation of system performance in “emergency” situations where regular stability is impossible, and the difficulties in rigorous proofs of global stability.

The problem of partial stability is formulated as follows. Consider the system

$$\dot{x}=f(t,x)$$

(213)

where $f:[0,\infty )\times D\to R^n$ is piecewise continuous in t, locally Lipschitz in x on $[0,\infty )\times D$, and $D\subset R^n$ contains the origin $x=0$. We break the state space into two sets of variables by writing

$$x={\left[\begin{array}{cc}{y}^T& z^T\end{array}\right]}^T$$

(214)

where y represents the variables converging to the origin, and z represents that variables that may or may not converge to origin. Thus, we write Equation (213) as

$$\begin{array}{c}\hfill \dot{y}=g(t,y,z)\end{array}$$

(215a)

$$\begin{array}{c}\hfill \dot{z}=h(t,y,z)\end{array}$$

(215b)

We say $x=0$ is an equilibrium point for Equation (213), if and only if $f(t,0)=0,\forall t\ge 0$. This translates to $g(t,0,0)=h(t,0,0)=0,\forall t\ge 0$. Partial stability is defined as follows

Definition 1. An equilibrium point $x=0$ of Equation (215) is

• y-stable, if for any $\epsilon >0$, and $t_0\ge 0$, there exists a $\delta =\delta (\epsilon ,t_0)>0$ such that

  $$\left|\right|x\left(t_0\right)\left|\right|<\delta \Rightarrow \left|\right|y\left(t\right)\left|\right|<\epsilon ,\forall t\ge t_0\ge 0$$

  (216)
• uniformly y-stable, if it is y-stable, and for each ε, $\delta =\delta \left(\epsilon \right)$ is independent of $t_0$.
• asymptotically y-stable, if it is y-stable, and for any $t_0$, there exists a positive constant $c=c\left(t_0\right)>0$ such that every solution of Equation (215) for which $\left|\right|x\left(t_0\right)\left|\right|<c$, satisfies $lim\left|\right|y\left(t\right)\left|\right|\to 0$ as $t\to \infty$.
• uniformly asymptotically y-stable, if it is uniformly y-stable, and there is a positive constant c, independent of $t_0$, such that for all $\left|\right|x\left(t_0\right)\left|\right|<c$, $lim\left|\right|y\left(t\right)\left|\right|\to 0$ as $t\to \infty$, uniformly in t. Meaning that for each $\eta >0$, there exists $T=T\left(\eta \right)>0$ such that

  $$\left|\right|y\left(t\right)\left|\right|<\eta ,\forall t\ge t_0+T\left(\eta \right),\forall \left|\right|x\left(t_0\right)\left|\right|<c$$

  (217)

There’s a myriad of theorems regarding partial stability of systems. However, in most these works, the conditions on the Lyapunov function are too restrictive, rendering them ineffective for the adaptive control problem of our interest. Furthermore, the behavior of systems under perturbations is not abundantly studied when the best we can do is partial stability. However, we believe that adaptive control could in general benefit from this tool due to the nature of its stability. Further development of partial stability tools and its application to adaptive control is an interesting problem that can be addressed.

7.4. Underactuated Systems

Systems that have fewer actuators than states to be controlled are referred to as underactuated. These systems are of interest from several viewpoints. First, in some applications it may not be possible to have actuators for all the desired states. Secondly, if an actuator failure happens the system descends into an underactuated mode. A successful control design for such situations can greatly enhance the safety and performance of the systems. Thirdly, a deliberate reduction in the number of actuators and reduce the manufacturing costs.

Underactuated systems have been studied for more than two decades now. Energy and passivity based control [107,108,109], energy shaping [110], and Controlled Lagrangians and Hamiltonians [111,112,113,114,115,116] are just a few methods among others proposed [117,118,119,120].

A survey on the methods and problems of underactuated systems requires a separate full length paper. However, we only look at them from the adaptive control perspective. Most of these proposed methods do not deal with uncertainties. Very few papers have been published that address the uncertainty issue in underactuated systems [121]. Addition of adaptation and adaptive control laws to methods that deal with underactuated systems is a subject that has been left mostly untouched. Research in these areas can greatly enhance the toolbox that we currently have for dealing with uncertain systems.

7.5. Possible New Methods

Although very difficult, it is still possible to create novel paradigms for adaptive control. Recently, [122,123,124,125] attempted at a new paradigm of adaptive control by employing Extremum Seeking as a means of adaptation (rather than a means of optimization). Their method augments the Model Reference Adaptive Controller with an adaptation law using Extremum Seeking loops. The main difference between this approach and the mainstream adaptive methods is that the adaptation occurs in real-time and no mathematical adaptive laws need be derived. This makes the implementation simple; however, it also brings several downsides that have not been addressed. Extremum Seeking perturbs the system. Therefore, the system requires some inherent robustness to perturbations, or if this is not the case, the controller must provide such robustness. Due to the addition of deliberate perturbations to the system, one expects that PE conditions would be automatically satisfied, making the real-time identification of parameters possible. However, this does not seem to be the case. Further study needs to be done, before this new paradigm becomes an acceptable method.

8. Conclusions

We have reviewed the major history of adaptive control over the last century as well as several of the popular methods in adaptive control including: Model Reference Adaptive Control, Adaptive Pole Placement (Self-Tuning Regulators), Adaptive Sliding Mode Control, and Extremum Seeking. We presented the Model Reference Identification approaches through detailed analysis and examples, and briefly discussed their non-minimal realizations. It has been made clear that the application of adaptive systems can solve many interesting problems. The necessary tools for extending these methods to nonlinear systems were also discussed. Stability and robustness issues related to adaptive control methods were shown through analysis and example, followed by possible solutions using Robust Adaptive Control and Adaptive Robust Control methods. We also provided various perspectives in control, observation, and adaptive systems as well as some of the important open problems and the direction of future work. Despite the length of the open problems section, we have only covered a few problems where improvements can be made in adaptive control, showing that the field is still open contrary to common belief. There are still plenty of unsolved problems to be addressed in adaptive control and we hope to see more researchers address them in years to come.