Asynchronously H_∞ Tracking Control and Optimization for Switched Flight Vehicles with Time-Varying Delay

Abstract

The current paper verifies the asynchronous $H_{\infty }$ control and optimization problem for flight vehicles with a time-varying delay. The nonlinear dynamic model and Jacobian linearization establish the flight vehicle’s switched model. An asynchronous $H_{\infty }$ tracking controller is designed, considering the existing asynchronous switching between the controllers and corresponding subsystems. In order to promote transient efficiency, the tracking controller comprises the model-based part and the learning-based part. The model-based part guarantees stability and prescribed efficiency, and the learning-based part compensates for undesirable uncertainties. The multiple Lyapunov function (MLF) and mode-dependent average dwell time (MDADT) methods are utilized to guarantee stability and the specified attenuation efficiency. The existing conditions and the solutions of model-based sub-controllers are represented by linear matrix inequalities (LMIs). The deep Q learning (DQL) algorithm provides the learning-based part. Different from the conventional method, the controller parameters are scheduled online. Therefore, robustness, stability, and dynamic efficiency can be met simultaneously. A numerical example illustrates the efficiency and advantage of the presented approach.

1. Introduction

As an efficient way to access space, flight vehicles have attracted considerable attention owing to their high civilian and military value [1,2]. The rapid development of various flight vehicles provides efficient and convenient tools for multiple missions, such as remote attacks, autonomous detection, and material transportation. Due to their potential applications in industry, agricultural military, and other fields, fruitful research results have emerged in relation to flight vehicles. The problems of stability analysis, control strategy design, and the formation control of flight vehicles have been studied by scholars at home and abroad [3,4]. For instance, tracking control in the event-triggered case was proposed in [5]; the observer-based backstepping control strategy was presented in [6] and an extended state observer was proposed in [7]. However, the characteristics of high nonlinearity, complex dynamics, strong couplings, and model uncertainties seriously influence the performance of flight vehicles, putting forward higher requirements for reliability and control performance.

Nonlinear systems have created many concerns due to the nonlinear nature of most practical systems [8,9]. The most popular approaches to analyzing nonlinear systems can be composed of two parts: the nonlinear design method and the linear design method [10]. The nonlinear properties of nonlinear systems are considered in the nonlinear design method, such as the backstepping and adaptive control methods. The linear design method approximates the nonlinear systems with linear systems around their operation points based on Jacobian linearization. Therefore, the controller design and analysis for complex nonlinear systems can be converted into designing controllers for linear systems, such as the gain-scheduled method, the proportional-integral-derivative (PID) method, and robust control. The linear design method can provide solvable results and stability guarantees for complex nonlinear systems. As an effective tool with which to address the design of complex nonlinear systems, switched systems theory has been investigated by researchers, and there is much literature on modeling, stability analysis, switching strategy design, fault detection, and fault-tolerance [11,12]. The switched systems establish the connection between complicated nonlinear and simplified linear systems [13], attracting a great deal of attention. In recent literature, various interesting results have been presented for various problems of switched systems. As a fundamental controller design problem, stability analysis has been fully studied, and several methods have been proposed. For example, the common Lyapunov function (CLF) method was presented for the switched systems with arbitrary switching, meaning a CLF was shared for all the subsystems. Therefore, the switching among subsystems cannot increase the energy. However, in most practical situations, finding a CLF for all the subsystems is a challenge, limiting the CLF method’s applications. Moreover, the switching logic depends on the time, states, or their combination in many practical applications, motivating the investigation of average dwell time (ADT) and MDADT methods. This means that the subsystems should dwell long enough in the subsystems with poor performance, which is mainly applied to restricted switching. Compared with the existing CLF approach, the ADT and MDADT methods lead to less conservative results. In [14], the quantized $H_{\infty }$ filtering problem was investigated for switched T-S fuzzy systems, and the ADT method was utilized to guarantee the exponential stability of the error system with a given $H_{\infty }$ performance. The quantization phenomenon and parameter perturbations were considered, and the fuzzy-based Lyapunov function approach was provided. In [1], the stability of highly nonlinear switched stochastic systems containing the time-varying delay was analyzed. Lyapunov function and ADT were employed to derive sufficient conditions to ensure the $H_{\infty }$ stability to avoid the inappropriate response induced by the time delay. The proposed method extends the stability results to the environment with a time delay, which is more applicable in a practical environment. Moreover, it can be deduced that the common dwell time can be employed for all the subsystems, indicating the worst situation, which results in conservativeness. The MDADT approach was presented in [15] to derive narrower bounds on dwell time. The features of every subsystem are considered, which has its own dwell time. This means that the dwell time depends on the system mods, releasing the ADT method restrictions [16]. Therefore, the MDADT method has been widely applied for stability analysis and stabilization. In [17], the fault estimation observer was proposed. The MDADT method was presented to realize the augmented system’s stability and $H_{\infty }$ performance. Compared with the conventional ADT approach, fewer conservative results were realized. In [18], the stability and robust control for switched systems were verified. The multiple discontinuous Lyapunov function and MDADT were integrated to guarantee the stability and prescribed weighted performance. It turns out that the proposed method realizes small bounds on dwell time. In [19], the event-triggered exponential $H_{\infty }$ filter design was investigated, and the MDADT approach was adopted to derive the exponential stability conditions. In this study, the event-triggered communication scheme is applied to promote the resource limitation through the network. The time-varying delay and bounded disturbance are considered, and the LMI technique is employed to derive sufficient conditions to attain the desired efficiency. Compared with the conventional results, the presented approach is more applicable and less conservative. One of the primary purposes of the switched systems is to obtain fewer conservative results and narrower dwell time bounds, which has motivated researchers in recent years.

In most practical systems, a time-varying delay is inevitable due to the transmission limitation in the flight vehicles network, degrading the performance and causing asynchronous switching. The asynchronous switching indicates that the controller switching lags behind the system mode switching. Therefore, we can see unmatched and matched periods in all subsystems, which will increase the Lyapunov energy in the unmatched periods. Thus, the energy function increases in the unmatched periods and decreases in the matched ones. There are many results of time delay and asynchronous switching. In [12], the stability and stabilization problems were studied for switched systems with impulsive switching signals under asynchronous switching. A novel Lyapunov-like function was established, and the conditions to ensure the system’s exponential stability were given by the edge-dependent switching signals. In [20], finite-time stabilization and finite-time bounded stabilization were investigated for switched systems. The asynchronous environmental switching was considered, and the sufficient stabilization conditions were presented as nonlinear differential matrix inequalities. The results in the paper validate that asynchronous switching can affect the system’s dynamic performance, and it is essential to avoid the undesirable response induced by asynchronous switching. Moreover, the multiple event-triggered strategies for switched systems with asynchronous switching were investigated in [21]. The controller-mode-dependent Lyapunov function was established using an asynchronous switching strategy. The multiple event-triggered schemes were applied, and the stability criteria were provided based on the ADT method. The state feedback controller was proposed to avoid the Zeno behavior. Additionally, the conventional method for time-delay systems lies in the Lyapunov–Krasovskii function. The time-varying delay bounds were considered, and the robust stability was realized for the worst case of unknown delay. In [22], the formation-containment control problem was verified for multi-agent systems (MAS) containing time-varying delays and switching topologies. It is assumed that the leaders can communicate through switching topologies with time-varying delays. The Lyapunov–Krasovskii function method was applied to ensure the convergence of the formation-containment error. An edge-based state observer was developed to evaluate the MAS states involving the time delay. The results demonstrate that the Lyapunov–Krasovskii function method can successfully solve the stabilization problem for time-delay systems. In [8], the adaptive fuzzy control problem was investigated for switched nonlinear systems containing the input delay. A nonlinear disturbance observer for arbitrary switching systems was presented, and a piecewise switched adaptive law was developed. Padé approximation and dynamic surface control methods were proposed to address the input delay problem. Unlike the traditional method, the proposed method realizes fewer conservative results and more relevant results. From the statement mentioned in the paper, it is evident that developing more applicable and less conservative methods for switched systems is interesting and necessary.

It is noticed that the model-dependent methods are proposed assuming that the structure or bounds of model uncertainties are known in prior [23,24]. However, in many situations in a realistic environment, the information of uncertainties cannot be obtained, which motivates the studies on model-free methods. With the development of computational ability, intelligent methods have been applied to the control issues of flight vehicles [25]. Among the artificial methods, deep learning and reinforcement learning have been extensively employed in many military and economic fields. The deep neural networks and reinforcement learning were utilized in deep learning to realize online fitting and achieve better performance by trial and error. Therefore, we can see that data-based algorithms like deep learning and reinforcement learning can improve the controller efficiency in the presence of system uncertainties. Deep reinforcement learning (DRL) integrates deep learning and reinforcement learning benefits. This was widely studied, and fruitful results emerged. In [26], the DRL algorithm was employed to design the missile’s guidance law, formulating a Markovian decision process in which the reward function was designed to realize the trade-off between accuracy, energy consumption, and interception time. The deep deterministic policy gradient algorithm was adopted, and the guidance gain was scheduled online. In [27], the PID controller was combined with the DRL algorithm to improve the control performance. The DRL algorithm was adopted to compensate for the system uncertainties. However, in the existing literature on DRL, we can see that ensuring the algorithm’s convergence is a challenge. Accordingly, an algorithm should be developed to simultaneously ensure stability and dynamic performance.

According to the above discussion, it can be deduced that the performance enhancement and stability preservation problems should be verified simultaneously. The controller design problem for switched systems in more applicable environments has not entirely been verified. The design flexibility and control performance can be enhanced by deriving the narrower dwell time bounds. Moreover, it is essential to incorporate the benefits of conventional robust control with an intelligent algorithm. Therefore, the current study investigates the asynchronous $H_{\infty }$ tracking control problem and optimization for switched flight vehicles with time-varying delays. The nonlinear dynamic model and Jacobian linearization can establish the flight vehicles’ switched model. The proposed controller comprises a dynamic-based sub-controller and a learning-based sub-controller. The nominal tracking controller is proposed considering the asynchronous switching induced by the time-varying delay. The MLF and MDADT approaches are integrated to ensure stability and attenuation performance. In order to ensure stability and transient performance simultaneously, a learning-based sub-controller is proposed to compensate for the system uncertainties. The DQL algorithm is provided to achieve better convergence. Therefore, the essential novelties of the current study are summarized as follows: (1) a more applicable and less conservative asynchronous $H_{\infty }$ tracking controller is proposed for switched flight vehicles with time-varying delays by utilizing MLF and MDADT approaches. The LMI approach is adopted to extract sufficient conditions ensuring the stability and prescribed attenuation index; (2) the presented tracking controller consists of a dynamic-based sub-controller and a learning-based sub-controller. The former is designed for the nominal case, and the latter is provided to compensate for the model uncertainties. The advantages of conventional $H_{\infty }$ tracking control and an intelligent algorithm are combined; (3) the DQL is adopted, and the online scheduling is described with a Markovian process. The controller parameters are defined as the output action to simultaneously realize stability, robustness, and dynamic performance.

The remainder of the current paper is arranged as follows. The problem formulation is presented in Section 2. In Section 3, the intelligent $H_{\infty }$ tracking controller is given. The simulation results are given in Section 4 to evaluate the efficiency of the presented approach. Finally, the paper concludes in Section 5.

2. Preliminaries and Problem Formulation

The current study considers the discrete-time switched systems with the time-varying delay as:

$$\left\{\begin{array}{c}\mathit{x}(k+1)={\mathit{A}}_i\mathit{x}(k)+{\mathit{A}}_{hi}\mathit{x}(k-h(k))+{\mathit{B}}_i\mathit{u}(k)+{\mathit{D}}_i\mathit{\omega }(k)\hfill \\ \mathit{y}(k)={\mathit{C}}_i\mathit{x}(k),\hfill \end{array}\right.$$

(1)

where $\mathit{x}(k)\in {\mathbb{R}}^x$ stands for the state vector; $\mathit{u}(k)\in {\mathbb{R}}^u$ describes the input signal; $\mathit{y}(k)\in {\mathbb{R}}^y$ stands for the output signal; $\mathit{\omega }(k)\in {\mathbb{R}}^{\omega }$ describes the external disturbance with $\mathit{\omega }(k)\in L_2[0,\infty )$; $h(k)\in \left[h_m,h_M\right]$ describes the time-varying delay in the systems caused by the network’s transformation limitation. We define ${\mathit{A}}_i$, ${\mathit{A}}_{hi}$, ${\mathit{B}}_i$, ${\mathit{C}}_i$ and ${\mathit{D}}_i$ as the state-space matrices with proper dimensions; $i=\sigma (k):[0,\infty )\to \Omega =\left\{1,2,...,n\right\}$ represents the piecewise continuous switching signal.

We define the command signal as $\mathit{r}(k)$. Therefore, the tracking error is described as:

$$\mathit{e}(k)=\mathit{r}(k)-\mathit{y}(k).$$

(2)

Remark 1.

The command signal is bounded with $-30deg\le r(k)\le 30deg$.

The tracking control problem of switched systems in (1) can be described as the controller design problem, such that

$$\underset{k\to \infty }{lim}\mathit{e}(k)=0.$$

(3)

The tracking error integral is defined in (4).

$$\mathit{g}(k)={\displaystyle \sum _{s=0}^{k-1}}\mathit{e}(s)={\displaystyle \sum _{s=0}^{k-1}}(\mathit{r}(s)-\mathit{y}(s)).$$

(4)

Then, the tracking controller for (1) can be proposed as:

$$\mathit{u}(k)={\mathit{K}}_{1i}\mathit{x}(k)+{\mathit{K}}_{2i}\mathit{g}(k),$$

(5)

where ${\mathit{K}}_{1i}$ and ${\mathit{K}}_{2i}$ are unknown parameter matrices to be determined.

Due to the time-varying delay in the system, the tracking controller switching always lags behind the system mode switching. All subsystems have unmatched and matched periods. The activated time instant of the ith subsystem is defined as $k_i$, and the activated time instant of the corresponding controller is defined as $k_i+{\Delta }_i$, where ${\Delta }_i$ denotes the unmatched period length in the ith subsystem.

Suppose

$$\tilde{\mathit{x}}(k)=[\begin{array}{cc}{\mathit{x}}^{\mathrm{T}}\left(k\right)& {\mathit{g}}^{\mathrm{T}}(k){]}^{\mathrm{T}}\end{array},\mathit{d}(k)=[\begin{array}{c}{\mathit{\omega }}^{\mathrm{T}}(k),{\mathit{r}}^{\mathrm{T}}(k){]}^{\mathrm{T}}\end{array}.$$

Now, the following closed-loop switched systems can be derived:

$$\begin{array}{c}\left\{\begin{array}{c}\tilde{\mathit{x}}(k+1)={\tilde{\mathit{A}}}_{ii}\tilde{\mathit{x}}(k)+{\tilde{\mathit{A}}}_{hi}\tilde{\mathit{x}}(k-\mathit{h}(k))+{\tilde{\mathit{D}}}_i\mathit{d}(k)\hfill \\ \mathit{e}\left(k\right)={\tilde{\mathit{C}}}_i\tilde{\mathit{x}}(k)+{\tilde{\mathit{E}}}_i\mathit{d}(k)\hfill \end{array}\right.,\forall k\in \left[k_i+{\Delta }_i,k_{i+1}\right]\hfill \end{array}$$

(6)

$$\begin{array}{c}\left\{\begin{array}{c}\tilde{\mathit{x}}\left(k+1\right)={\tilde{\mathit{A}}}_{ij}\tilde{\mathit{x}}\left(k\right)+{\tilde{\mathit{A}}}_{hi}\tilde{\mathit{x}}\left(k-\mathit{h}\left(k\right)\right)+{\tilde{\mathit{D}}}_i\mathit{d}\left(k\right)\hfill \\ \mathit{e}\left(k\right)={\tilde{\mathit{C}}}_i\tilde{\mathit{x}}\left(k\right)+{\tilde{\mathit{E}}}_i\mathit{d}\left(k\right)\hfill \end{array}\right.,\forall k\in \left[k_i,k_i+{\Delta }_i\right],\hfill \end{array}$$

(7)

where

$${\tilde{\mathit{A}}}_{ii}=\left[\begin{array}{cc}{\mathit{A}}_i+{\mathit{B}}_i{\mathit{K}}_{1i}& {\mathit{B}}_i{\mathit{K}}_{2i}\\ -{\mathit{C}}_i& \mathit{I}\end{array}\right],{\tilde{\mathit{A}}}_{ij}=\left[\begin{array}{cc}{\mathit{A}}_i+{\mathit{B}}_i{\mathit{K}}_{1j}& {\mathit{B}}_i{\mathit{K}}_{2j}\\ -{\mathit{C}}_i& \mathit{I}\end{array}\right]$$

$${\tilde{\mathit{A}}}_{hi}=\left[\begin{array}{cc}{\mathit{A}}_{hi}& 0\\ 0& 0\end{array}\right],{\tilde{\mathit{D}}}_i=\left[\begin{array}{cc}{\mathit{D}}_i& 0\\ 0& \mathit{I}\end{array}\right],{\tilde{\mathit{C}}}_i=\left[\begin{array}{cc}-{\mathit{C}}_i& 0\end{array}\right],{\tilde{\mathit{E}}}_i=\left[\begin{array}{cc}0& \mathit{I}\end{array}\right].$$

3. Main Results

3.1. Nominal Tracking Controller Design

The following definitions are provided for the convenience of the tracking controller design.

Definition 1

([15]). Consider the switching signal $\sigma (k)$ and $k_1>k_0\ge 0$, let $N_{\sigma i}(k_0,k_1)$ denote the switching number of the i-th subsystem occurring on the $[k_0,k_1]$; $T_i(k_0,k_1)$ describes the running time during time interval $[k_0,k_1]$. The positive constant ${\tau }_{ai}$ is defined as the MDADT of the switching signal $\sigma (k)$ such that

$$N_{\sigma i}\left(k_0,k_1\right)\le N_{0i}+\frac{T_i\left(k_0,k_1\right)}{{\tau }_{ai}},$$

(8)

where $N_{0i}\le 0$ are mode-dependent chatter bounds.

Definition 2.

If there are constants $\kappa >0$ and $\epsilon >0$ for a given switching signal $\sigma (k)$, the switched systems in (6) and (7) are globally uniformly exponentially stable (GUES) when $\mathit{d}(k)=0$ such that

$$∥\mathit{e}\left(k\right)∥\le \kappa e^{-\epsilon \left(k-k_0\right)}∥\mathit{e}\left(k_0\right)∥.$$

(9)

Definition 3.

For a given switching signal $\sigma (k)$, and constants $\gamma >0$ and $0<\lambda <1$, if the switched systems in (6) and (7) are GUES with the prescribed $H_{\infty }$ attenuation performance, such that

$${\displaystyle \sum _{s=k_0}^{\infty }}{\left(1-\lambda \right)}^s{\mathit{e}}^{\mathrm{T}}\left(s\right)\mathit{e}\left(s\right)\le {\gamma }^2{\displaystyle \sum _{s=k_0}^{\infty }}{\mathit{d}}^{\mathrm{T}}\left(s\right)\mathit{d}\left(s\right);$$

(10)

Accordingly, the tracking controller design problem can be divided into two parts: (1) the systems (6) and (7) are GUES when $\mathit{d}(k)=0$; (2) the closed-loop systems satisfy the prescribed attenuation index in (10).

Theorem 1 gives sufficient conditions to guarantee the systems (6) and (7) GUES.

Theorem 1.

For given constants ${\mu }_{1i}>1$, ${\mu }_{2i}>1$, $0<{\alpha }_i<1$, and ${\beta }_i>0$, if there are positive-definite matrices ${\mathit{P}}_i$, ${\mathit{Q}}_{1i}$, ${\mathit{Q}}_{2i}$, ${\mathit{Q}}_{3i}$, ${\mathit{P}}_{ij}$, ${\mathit{Q}}_{1ij}$, ${\mathit{Q}}_{2ij}$, and ${\mathit{Q}}_{3ij}$, the closed-loop systems in (6) and (7), with MDADT satisfying (15), are GUES if (11)–(14) hold.

$$\left\{\begin{array}{c}{\mathit{P}}_i\le {\mu }_{1i}{\mathit{P}}_{ij}\hfill \\ {\mathit{Q}}_{1i}\le {\mu }_{1i}{\mathit{Q}}_{1ij}\hfill \\ {\mathit{Q}}_{2i}\le {\mu }_{1i}{\mathit{Q}}_{2ij}\hfill \\ {\mathit{Q}}_{3i}\le {\mu }_{1i}{\mathit{Q}}_{3ij}\hfill \end{array}\right.$$

(11)

$$\left\{\begin{array}{c}{\mathit{P}}_{ij}\le {\mu }_{2i}{\mathit{P}}_j\hfill \\ {\mathit{Q}}_{1ij}\le {\mu }_{2i}{\mathit{Q}}_{1j}\hfill \\ {\mathit{Q}}_{2ij}\le {\mu }_{2i}{\mathit{Q}}_{2j}\hfill \\ {\mathit{Q}}_{3ij}\le {\mu }_{2i}{\mathit{Q}}_{3j}\hfill \end{array}\right.$$

(12)

$${\mathbf{\Phi }}_i=\left[\begin{array}{cc}-{\mathit{P}}_i^{-1}& {\mathbf{\Phi }}_{12i}\\ \ast & {\mathbf{\Phi }}_{22i}\end{array}\right]<0$$

(13)

$${\mathbf{\Phi }}_{ij}=\left[\begin{array}{cc}-{\mathit{P}}_{ij}^{-1}& {\mathbf{\Phi }}_{12ij}\\ \ast & {\mathbf{\Phi }}_{22ij}\end{array}\right]<0$$

(14)

$${\tau }_{ai}\ge {\tau }_{ai}^{\ast }=\frac{-\left(ln{\mu }_{1i}{\mu }_{2i}+{\Delta }_{i}ln{\theta }_i\right)}{ln{\tau }_{ai}},$$

(15)

where

$${\tilde{\alpha }}_i=1-{\alpha }_i,{\tilde{\beta }}_i=1+{\beta }_i,{\theta }_i={\tilde{\beta }}_i/{\tilde{\alpha }}_i,$$

$${\mathbf{\Phi }}_{12i}=\left[\begin{array}{cccc}{\tilde{\mathit{A}}}_{ii}& {\tilde{\mathit{A}}}_{hi}& 0& 0\end{array}\right],{\mathbf{\Phi }}_{12ij}=\left[\begin{array}{cccc}{\tilde{\mathit{A}}}_{ij}& {\tilde{\mathit{A}}}_{hi}& 0& 0\end{array}\right],$$

$${\mathbf{\Phi }}_{22i}=\mathrm{diag}\left(-\left(1-{\alpha }_i\right){\mathit{P}}_i+{\mathit{Q}}_{1i}+\left(h_M-h_m+1\right){\mathit{Q}}_{2i}+Q_{3i},-{\left(1-{\alpha }_i\right)}^{h_m}{\mathit{Q}}_{2i},-{\left(1-{\alpha }_i\right)}^{h_m}{\mathit{Q}}_{1i},-{\left(1-{\alpha }_i\right)}^{h_M}{\mathit{Q}}_{3i}\right),$$

$${\mathbf{\Phi }}_{22ij}=\mathrm{diag}\left(-\left(1+{\beta }_i\right){\mathit{P}}_{ij}+{\mathit{Q}}_{1ij}+\left(h_M-h_m+1\right){\mathit{Q}}_{2ij}+{\mathit{Q}}_{3ij},-{\left(1+{\beta }_i\right)}^{h_m}{\mathit{Q}}_{2ij},-{\left(1+{\beta }_i\right)}^{h_m}{\mathit{Q}}_{1ij},-{\left(1+{\beta }_i\right)}^{h_M}{\mathit{Q}}_{3ij}\right).$$

Proof. 

The Lyapunov-like function of the ith subsystem is described as:

$$V_i\left(k\right)={\displaystyle \sum _{l=1}^3}{V}_{li}\left(k\right)$$

(16)

where

$$V_{1i}(k)={\tilde{\mathit{x}}}^{\mathrm{T}}(k){\mathit{P}}_i\tilde{\mathit{x}}\left(k\right)$$

(17)

$$\begin{array}{c}{V}_{2i}\left(k\right)={\displaystyle \sum _{s=k-h_m}^{k-1}}{(1-\alpha )}^{k-s-1}{\tilde{\mathit{x}}}^{\mathrm{T}}(s){\mathit{Q}}_{1i}\tilde{\mathit{x}}(s)+{\displaystyle \sum _{s=k-h(k)}^{k-1}}{(1-\alpha )}^{k-s-1}{\tilde{\mathit{x}}}^{\mathrm{T}}(s){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(s\right)+{\displaystyle \sum _{s=k-h_M}^{k-1}}{\left(1-\alpha \right)}^{k-s-1}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{3i}\tilde{\mathit{x}}\left(s\right)\hfill \end{array}$$

(18)

$$\begin{array}{c}{V}_{3i}\left(k\right)={\displaystyle \sum _{j=k-h_M+1}^{k-h_m}}{\displaystyle \sum _{s=j}^{k-1}}{\left(1-\alpha \right)}^{k-s-1}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(s\right).\hfill \end{array}$$

(19)

The weighted difference of $V_i(k)$ is defined as:

$$\Delta V_{1i}\left(k\right)+\alpha V_{1i}\left(k\right)=V_{1i}\left(k+1\right)-\left(1-\alpha \right)V_{1i}\left(k\right)={\tilde{\mathit{x}}}^{\mathrm{T}}(k+1){\mathit{P}}_i\tilde{\mathit{x}}(k+1)-\left(1-\alpha \right){\tilde{\mathit{x}}}^{\mathrm{T}}(k){\mathit{P}}_i\tilde{\mathit{x}}(k)$$

$$\begin{array}{c}\Delta V_{2i}\left(k\right)+\alpha V_{2i}\left(k\right)=V_{2i}\left(k+1\right)-\left(1-\alpha \right)V_{2i}\left(k\right)\hfill \\ ={\displaystyle \sum _{s=k-h_m+1}^k}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{1i}\tilde{\mathit{x}}\left(s\right)+{\displaystyle \sum _{s=k-h(k)+1}^k}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(s\right)+{\displaystyle \sum _{s=k-h_M+1}^k}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{3i}\tilde{\mathit{x}}\left(s\right)\hfill \\ -{\displaystyle \sum _{s=k-h_m}^{k-1}}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{1i}\tilde{\mathit{x}}\left(s\right)-{\displaystyle \sum _{s=k-h(k)}^{k-1}}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(s\right)-{\displaystyle \sum _{s=k-h_M}^{k-1}}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{3i}\tilde{\mathit{x}}\left(s\right)\hfill \\ \le {\tilde{\mathit{x}}}^{\mathrm{T}}\left(k\right){\mathit{Q}}_{1i}\tilde{\mathit{x}}\left(k\right)-{\left(1-\alpha \right)}^{h_m}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h_m\right){\mathit{Q}}_{1i}\tilde{\mathit{x}}\left(k-h_m\right)+{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(k\right)-{\left(1-\alpha \right)}^{h_m}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h(k)\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(k-h(k)\right)\hfill \\ +{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k\right){\mathit{Q}}_{3i}\tilde{\mathit{x}}\left(k\right)-{\left(1-\alpha \right)}^{h_M}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h_M\right){\mathit{Q}}_{3i}\tilde{\mathit{x}}(k-h_M)+{\displaystyle \sum _{s=k-h_M+1}^{k-h_m}}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(s\right)\hfill \end{array}$$

$$\begin{array}{c}\Delta V_{3i}\left(k\right)+\alpha V_{3i}\left(k\right)=V_{3i}\left(k+1\right)-\left(1-\alpha \right)V_{3i}\left(k\right)\hfill \\ ={\displaystyle \sum _{j=k-h_M+2}^{k-h_m+1}}{\displaystyle \sum _{s=j}^k}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(s\right)-{\displaystyle \sum _{j=k-h_M+1}^{k-h_m}}{\displaystyle \sum _{s=j}^{k-1}}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(s\right)\hfill \\ =\left(h_M-h_m\right){\tilde{\mathit{x}}}^{\mathrm{T}}\left(k\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(k\right)-{\displaystyle \sum _{s=k-h_M+1}^{k-h_m}}{\left(1-\alpha \right)}^{k-s}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{2i}\tilde{\mathit{x}}\left(s\right).\hfill \end{array}$$

We define the augmented vector as:

$$\begin{array}{c}\mathit{\xi }\left(k\right)={\left[\begin{array}{cccc}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k\right)& {\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h(k)\right)& {\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h_m\right)& {\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h_M\right)\end{array}\right]}^{\mathrm{T}}.\hfill \end{array}$$

(20)

The following equations can be derived under zero-initial conditions when $\mathit{d}(k)=0$.

$$\begin{array}{c}\Delta V_i\left(k\right)+\alpha V_i\left(k\right)\le {\tilde{\mathit{x}}}^{\mathrm{T}}(k)\left({{\tilde{\mathit{A}}}_i}^{\mathrm{T}}{\mathit{P}}_i{\tilde{\mathit{A}}}_i\right)\tilde{\mathit{x}}(k)+{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h(k)\right)\left({\tilde{\mathit{A}}}_{hi}^{\mathrm{T}}{\mathit{P}}_i{\tilde{\mathit{A}}}_i\right)\tilde{\mathit{x}}\left(k-h(k)\right)+{\tilde{\mathit{x}}}^{\mathrm{T}}(k)\left(-\left(1-\alpha \right){\mathit{P}}_i+{\mathit{Q}}_{1i}\right.\hfill \\ \left.+\left(h_M-h_m+1\right){\mathit{Q}}_{2i}+{\mathit{Q}}_{3i}\right)\tilde{\mathit{x}}(k)+{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h(k)\right)\left(-{\left(1-\alpha \right)}^{h_m}{\mathit{Q}}_{2i}\right)\tilde{\mathit{x}}\left(k-h(k)\right)\hfill \\ +{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h_m\right)\left(-{\left(1-\alpha \right)}^{h_m}{\mathit{Q}}_{1i}\right)\tilde{\mathit{x}}\left(k-h_m\right)+{\tilde{\mathit{x}}}^{\mathrm{T}}\left(k-h_M\right)\left(-{\left(1-\alpha \right)}^{h_M}{\mathit{Q}}_{3i}\right)\tilde{\mathit{x}}\left(k-h_M\right)\hfill \\ ={\mathit{\xi }}^{\mathrm{T}}(k)\left({\mathbf{\Phi }}_{12i}^{\mathrm{T}}{\mathit{P}}_i{\mathbf{\Phi }}_{12i}+{\mathbf{\Phi }}_{22i}\right)\mathit{\xi }(k).\hfill \end{array}$$

(21)

Together with the Schur complement, we have

$$\begin{array}{c}\Delta V_i(k)+\alpha V_i(k)\le {\mathit{\xi }}^{\mathrm{T}}(k)\left({\mathbf{\Phi }}_{12i}^{\mathrm{T}}{\mathit{P}}_i{\mathbf{\Phi }}_{12i}+{\mathbf{\Phi }}_{22i}\right)\mathit{\xi }(k)\le {\mathit{\xi }}^{\mathrm{T}}(k){\mathbf{\Phi }}_i\mathit{\xi }(k)\le 0.\hfill \end{array}$$

(22)

Similarly, the Lyapunov-like function in unmatched periods is defined as:

$$V_{ij}\left(k\right)={\displaystyle \sum _{l=1}^3}{V}_{lij}\left(k\right),$$

(23)

where

$$V_{1ij}\left(k\right)={\tilde{\mathit{x}}}^{\mathrm{T}}\left(k\right){\mathit{P}}_{ij}\tilde{\mathit{x}}\left(k\right)$$

(24)

$$\begin{array}{c}{V}_{2ij}\left(k\right)={\displaystyle \sum _{s=k-h_m}^{k-1}}{\left(1+\beta \right)}^{k-s-1}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{1ij}\tilde{\mathit{x}}\left(s\right)+{\displaystyle \sum _{s=k-h(k)}^{k-1}}{\left(1+\beta \right)}^{k-s-1}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{2ij}\tilde{\mathit{x}}\left(s\right)+{\displaystyle \sum _{s=k-h_M}^{k-1}}{\left(1+\beta \right)}^{k-s-1}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{3ij}\tilde{\mathit{x}}\left(s\right)\hfill \end{array}$$

(25)

$$V_{3ij}\left(k\right)={\displaystyle \sum _{j=k-h_M+1}^{k-h_m}}{\displaystyle \sum _{s=j}^{k-1}}{\left(1+\beta \right)}^{k-s-1}{\tilde{\mathit{x}}}^{\mathrm{T}}\left(s\right){\mathit{Q}}_{2ij}\tilde{\mathit{x}}\left(s\right).$$

(26)

The following equation can be obtained by the difference method:

$$\begin{array}{c}\Delta V_i\left(k\right)-\beta V_i\left(k\right)\le {\mathit{\xi }}^{\mathrm{T}}(k)\left({\mathbf{\Phi }}_{12ij}^{\mathrm{T}}{\mathit{P}}_{ij}{\mathbf{\Phi }}_{12ij}+{\mathbf{\Phi }}_{22ij}\right)\mathit{\xi }(k)\le {\mathit{\xi }}^{\mathrm{T}}(k){\mathbf{\Phi }}_{ij}\mathit{\xi }(k)\le 0.\hfill \end{array}$$

(27)

Combining (22) with (27) gives:

$$\begin{array}{c}{V}_{\sigma \left(k_p\right)}\left(k_{p+1}^{-}\right)\le {\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-k_p-{\Delta }_p}{V}_{\sigma \left(k_p+{\Delta }_p\right)}\left(k_p+{\Delta }_p\right)\hfill \\ \le {\mu }_{1\sigma \left(k_p\right)}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-k_p-{\Delta }_p}{V}_{\sigma \left(k_p\right)}\left(k_p+{\Delta }_p\right)\hfill \\ \le {\mu }_{1\sigma \left(k_p\right)}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-k_p-{\Delta }_p}{\tilde{\beta }}_{\sigma \left(k_p\right)}^{{\Delta }_p}{V}_{\sigma \left(k_p\right)}\left(k_p\right)\hfill \\ \le {\mu }_{1\sigma \left(k_p\right)}{\mu }_{2\sigma \left(k_p\right)}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-k_p-{\Delta }_p}{\tilde{\beta }}_{\sigma \left(k_p\right)}^{{\Delta }_p}{V}_{\sigma \left(k_{p-1}\right)}\left(k_p^{-}\right)\hfill \\ ={\mu }_{1\sigma \left(k_p\right)}{\mu }_{2\sigma \left(k_p\right)}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-k_p}{\theta }_{\sigma \left(k_p\right)}^{{\Delta }_p}{V}_{\sigma \left(k_{p-1}\right)}\left(k_p^{-}\right).\hfill \end{array}$$

(28)

The switching instants in the interval $[k_0,k]$ are $k_1$, $k_2$, ⋯, and $k_{N_{\sigma }}$. Accordingly, the following equation can be derived by iteration:

$$\begin{array}{c}{V}_{\sigma \left(k_{N_{\sigma }}\right)}\left(k\right)\le {\mu }_{1\sigma \left(k_{N_{\sigma }}\right)}{\mu }_{2\sigma \left(k_{N_{\sigma }}\right)}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-k_{N_{\sigma }}}{\theta }_{\sigma \left(k_{N_{\sigma }}\right)}^{{\Delta }_{N_{\sigma }}}{V}_{\sigma \left(k_{N_{\sigma }-1}\right)}\left(k_{N_{\sigma }}\right)\hfill \\ \le {\mu }_{1\sigma \left(k_{N_{\sigma }}\right)}{\mu }_{1\sigma \left(k_{N_{\sigma }-1}\right)}{\mu }_{2\sigma \left(k_{N_{\sigma }}\right)}{\mu }_{2\sigma \left(k_{N_{\sigma }}-1\right)}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-k_{N_{\sigma }}}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }-1}\right)}^{k_{N_{\sigma }}-k_{N_{\sigma }-1}}{\theta }_{\sigma \left(k_{N_{\sigma }}\right)}^{{\Delta }_{N_{\sigma }}}{\theta }_{\sigma \left(k_{N_{\sigma }-1}\right)}^{{\Delta }_{N_{\sigma }-1}}{V}_{\sigma \left(k_{p-2}\right)}\left(k_{N_{\sigma }-1}\right)\hfill \\ ···\hfill \\ \le {\displaystyle \prod _{s=0}^{N_{\sigma }}}{\mu }_{1\sigma \left(k_s\right)}{\mu }_{2\sigma \left(k_s\right)}{\tilde{\alpha }}_{\sigma \left(k_s\right)}^{k_{s+1}-k_s}{\theta }_{\sigma \left(k_s\right)}^{{\Delta }_s}{V}_0\left(k_0\right)\hfill \\ =exp\left({\displaystyle \sum _{s=0}^{N_{\sigma }}}\left(ln{\mu }_{1\sigma \left(k_s\right)}{\mu }_{2\sigma \left(k_s\right)}+\left(k_{s+1}-k_s\right)ln{\tilde{\alpha }}_{\sigma \left(k_s\right)}+{\Delta }_{s}ln{\theta }_{\sigma \left(k_s\right)}\right)\right)V_0\left(k_0\right)\hfill \\ =exp\left({\displaystyle \sum _{i=1}^n}\left(N_{\sigma i}\left(k_0,k_1\right)\left(ln{\mu }_{1i}{\mu }_{2i}+{\Delta }_{i}ln{\theta }_i\right)+T_i\left(k_0,k_1\right)ln{\tilde{\alpha }}_i\right)\right)V_0\left(k_0\right).\hfill \end{array}$$

Together with (8), (22), (24), and (28), we can obtain that:

$$\begin{array}{c}{V}_{\sigma (k_{N_{\sigma }})}\left(k\right)\le exp\left\{{\displaystyle \sum _{i=1}^n}\left(\frac{T_i\left(k_0,k_1\right)}{{\tau }_{ai}}\left(ln{\mu }_{1i}{\mu }_{2i}+{\Delta }_{i}ln{\theta }_i\right)\right)+T_i\left(k_0,k_1\right)ln{\tilde{\alpha }}_i\right\}{V}_0\left(k_0\right)\hfill \\ \le exp\left\{{\displaystyle \sum _{i=1}^n}{T}_i\left(k_0,k_1\right)\times \left(\frac{ln{\mu }_{1i}{\mu }_{2i}+{\Delta }_{i}ln{\theta }_i}{{\tau }_{ai}}+ln{\tilde{\alpha }}_i\right)\right\}{V}_0\left(k_0\right).\hfill \end{array}$$

(29)

According to (15), we can obtain

$$\frac{ln{\mu }_{1i}{\mu }_{2i}+{\Delta }_{i}ln{\theta }_i}{{\tau }_{ai}}+ln{\tilde{\alpha }}_i<0.$$

(30)

Therefore, the switched systems in (6) and (7) are GUES if the MDADT satisfies (15). Now, the proof is finished.    □

Remark 2.

The (22) gives the variation of the Lyapunov function in the matched periods and (27) provides the variation of the Lyapunov function in the unmatched periods. According to (22) and (27), we can obtain the relationship between the Lyapunov function at time instant k and the initial state by iteration.

Theorem 1 provides a stability analysis. Accordingly, sufficient conditions to guarantee the prescribed attenuation index can be derived.

Theorem 2.

For given constants ${\mu }_{1i}>1$, ${\mu }_{2i}>1$, $0<{\alpha }_i<1$, ${\beta }_i>0$, if there are positive-definite matrices ${\mathit{P}}_i$, ${\mathit{Q}}_{1i}$, ${\mathit{Q}}_{2i}$, ${\mathit{Q}}_{3i}$, ${\mathit{P}}_{ij}$, ${\mathit{Q}}_{1ij}$, ${\mathit{Q}}_{2ij}$, and ${\mathit{Q}}_{3ij}$, and (31) and (32) hold, $\forall i,j\in N,i\ne j$, the closed-loop systems in (6) and (7), with MDADT meeting (15), are GUES with the prescribed attenuation index γ.

$${\tilde{\mathbf{\Phi }}}_i=\left[\begin{array}{ccc}-{\mathit{P}}_i^{-1}& \mathbf{0}& {\tilde{\mathbf{\Phi }}}_{13i}\\ \ast & -\mathit{I}& {\tilde{\mathbf{\Phi }}}_{23i}\\ \ast & \ast & {\tilde{\mathbf{\Phi }}}_{33i}\end{array}\right]<0$$

(31)

$${\tilde{\mathbf{\Phi }}}_{ij}=\left[\begin{array}{ccc}-{\mathit{P}}_{ij}^{-1}& \mathbf{0}& {\tilde{\mathbf{\Phi }}}_{13ij}\\ \ast & -\mathit{I}& {\tilde{\mathbf{\Phi }}}_{23i}\\ \ast & \ast & {\tilde{\mathbf{\Phi }}}_{33ij}\end{array}\right]<0,$$

(32)

where

$${\tilde{\mathbf{\Phi }}}_{13i}=\left[\begin{array}{ccccc}{\tilde{\mathit{A}}}_{ii}& {\tilde{\mathit{A}}}_{hi}& \mathbf{0}& \mathbf{0}& {\tilde{\mathit{D}}}_i\end{array}\right],{\tilde{\mathbf{\Phi }}}_{13ij}=\left[\begin{array}{ccccc}{\tilde{\mathit{A}}}_{ij}& {\tilde{\mathit{A}}}_{hi}& \mathbf{0}& \mathbf{0}& {\tilde{\mathit{D}}}_i\end{array}\right],{\tilde{\mathbf{\Phi }}}_{23i}=\left[\begin{array}{ccccc}{\tilde{\mathit{C}}}_i& \mathbf{0}& \mathbf{0}& \mathbf{0}& {\tilde{\mathit{E}}}_i\end{array}\right],$$

$$\begin{array}{c}{\tilde{\mathbf{\Phi }}}_{33i}=\mathrm{diag}\left(-\left(1-{\alpha }_i\right){\mathit{P}}_i+{\mathit{Q}}_{1i}+\left(h_M-h_m+1\right){\mathit{Q}}_{2i}+{\mathit{Q}}_{3i},-{\left(1-{\alpha }_i\right)}^{h_m}{\mathit{Q}}_{2i},-{\left(1-{\alpha }_i\right)}^{h_m}{\mathit{Q}}_{1i},-{\left(1-{\alpha }_i\right)}^{h_M}{\mathit{Q}}_{3i},-{\gamma }^2\mathit{I}\right),\hfill \end{array}$$

$$\begin{array}{c}{\tilde{\mathbf{\Phi }}}_{33ij}=\mathrm{diag}(-\left(1+{\beta }_i\right){\mathit{P}}_{ij}+{\mathit{Q}}_{1ij}+\left(h_M-h_m+1\right){\mathit{Q}}_{2ij}+{\mathit{Q}}_{3ij},\hfill \\ -{\left(1+{\beta }_i\right)}^{h_m}{\mathit{Q}}_{2ij},-{\left(1+{\beta }_i\right)}^{h_m}{\mathit{Q}}_{1ij},-{\left(1+{\beta }_i\right)}^{h_M}{\mathit{Q}}_{3ij},-{\gamma }^2\mathit{I}).\hfill \end{array}$$

Proof. 

By combining Theorem 1 and (31) and (32), it is evident that the closed-loop systems in (6) and (7) are GUES. Suppose

$$\begin{array}{c}\tilde{\mathit{\xi }}(k)={\left[\begin{array}{ccccc}{\tilde{\mathit{x}}}^{\mathrm{T}}(k)& {\tilde{\mathit{x}}}^{\mathrm{T}}(k-h(k))& {\tilde{\mathit{x}}}^{\mathrm{T}}(k-h_m)& {\tilde{\mathit{x}}}^{\mathrm{T}}(k-h_M)& {\mathit{d}}^{\mathrm{T}}(k);\end{array}\right]}^{\mathrm{T}},\hfill \end{array}$$

according to the Schur complement, we have

$$\begin{array}{c}\Delta V_i\left(k\right)+{\alpha }_i{V}_i\left(k\right)+W\left(k\right)\le {\tilde{\mathit{\xi }}}^{\mathrm{T}}(k)\left({\tilde{\mathbf{\Phi }}}_{13i}^{\mathrm{T}}{\mathit{P}}_i{\tilde{\mathbf{\Phi }}}_{13i}+{\tilde{\mathbf{\Phi }}}_{23i}^{\mathrm{T}}{\tilde{\mathbf{\Phi }}}_{23i}+{\tilde{\mathbf{\Phi }}}_{33i}\right)\tilde{\xi }(k)\le {\tilde{\mathit{\xi }}}^{\mathrm{T}}(k){\tilde{\mathbf{\Phi }}}_i\tilde{\xi }(k)<0\hfill \end{array}$$

(33)

$$\begin{array}{c}\Delta V_{ij}\left(k\right)-{\beta }_i{V}_{ij}\left(k\right)+W\left(k\right)\le {\tilde{\mathit{\xi }}}^{\mathrm{T}}(k)\left({\tilde{\mathbf{\Phi }}}_{13ij}^{\mathrm{T}}{\mathit{P}}_{ij}{\tilde{\mathbf{\Phi }}}_{13ij}+{\tilde{\mathbf{\Phi }}}_{23ij}^{\mathrm{T}}{\tilde{\mathbf{\Phi }}}_{23ij}+{\tilde{\mathbf{\Phi }}}_{33ij}\right)\tilde{\xi }(k)\le {\tilde{\mathit{\xi }}}^{\mathrm{T}}(k){\tilde{\mathbf{\Phi }}}_{ij}\tilde{\xi }(k)<0\hfill \end{array}$$

(34)

where $W\left(k\right)={\mathit{e}}^{\mathrm{T}}(k)\mathit{e}(k)-{\gamma }^2{\mathit{d}}^{\mathrm{T}}(k)\mathit{d}(k)$.

$$\begin{array}{c}{V}_{\sigma \left(k_p\right)}\left(k_{p+1}^{-}\right)\le {\mu }_{1\sigma \left(k_p\right)}{\mu }_{2\sigma \left(k_p\right)}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-k_p-{\Delta }_p}{\tilde{\beta }}_{\sigma \left(k_p\right)}^{{\Delta }_p}{V}_{\sigma \left(k_{p-1}\right)}\left(k_p^{-}\right)\hfill \\ -{\displaystyle \sum _{s=k_p+{\Delta }_p}^{k_{p+1}-1}}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-1-s}W\left(s\right)-{\displaystyle \sum _{s=k_p}^{k_p+{\Delta }_p-1}}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-k_p-{\Delta }_p-1}{\tilde{\beta }}_{\sigma \left(k_p\right)}^{k_p+{\Delta }_p-s}W\left(s\right)\hfill \\ ={\mu }_{1\sigma \left(k_p\right)}{\mu }_{2\sigma \left(k_p\right)}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-k_p}{\theta }_{\sigma \left(k_p\right)}^{{\Delta }_p}{V}_{\sigma \left(k_{p-1}\right)}\left(k_p^{-}\right)-{\displaystyle \sum _{s=k_p+{\Delta }_p}^{k_{p+1}-1}}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-1-s}W\left(s\right)-{\displaystyle \sum _{s=k_p}^{k_p+{\Delta }_p-1}}{\tilde{\alpha }}_{\sigma \left(k_p\right)}^{k_{p+1}-s-1}{\theta }_{\sigma \left(k_p\right)}^{k_p+{\Delta }_p-s}W\left(s\right).\hfill \end{array}$$

(35)

The following equation can be obtained by iteration on the interval $[k_0,k_1]$.

$$\begin{array}{c}{V}_{\sigma \left(k_{N_{\sigma }}\right)}\left(k\right)\hfill \\ \le {\mu }_{1\sigma \left(k_{N_{\sigma }}\right)}{\mu }_{2\sigma \left(k_{N_{\sigma }}\right)}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-k_{N_{\sigma }}}{\theta }_{\sigma \left(k_{N_{\sigma }}\right)}^{{\Delta }_{N_{\sigma }}}{V}_{\sigma \left(k_{N_{\sigma }-1}\right)}\left(k_{N_{\sigma }}\right)\hfill \\ -{\displaystyle \sum _{s=k_{N_{\sigma }}+{\Delta }_{N_{\sigma }}}^{k-1}}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-1-s}W\left(s\right)-{\displaystyle \sum _{s=k_{N_{\sigma }}}^{k_{N_{\sigma }}+{\Delta }_{N_{\sigma }}-1}}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-s-1}{\theta }_{\sigma \left(k_{N_{\sigma }}\right)}^{k_{N_{\sigma }}+{\Delta }_{N_{\sigma }}-s}W\left(s\right)\hfill \\ \le {\mu }_{1\sigma \left(k_{N_{\sigma }}\right)}{\mu }_{2\sigma \left(k_{N_{\sigma }}\right)}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-k_{N_{\sigma }}}{\theta }_{\sigma \left(k_{N_{\sigma }}\right)}^{{\Delta }_{N_{\sigma }}}({\mu }_{1\sigma \left(k_{N_{\sigma }-1}\right)}{\mu }_{2\sigma \left(k_{N_{\sigma }-1}\right)}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }-1}\right)}^{k_{N_{\sigma }}-k_{N_{\sigma }-1}}{\theta }_{\sigma \left(k_{N_{\sigma }-1}\right)}^{{\Delta }_{N_{\sigma }-1}}{V}_{\sigma \left(k_{N_{\sigma }-2}\right)}\left(k_{N_{\sigma }-1}\right)\hfill \\ -{\displaystyle \sum _{s=k_{N_{\sigma }-1}+{\Delta }_{N_{\sigma }-1}}^{k_{N_{\sigma }}-1}}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }-1}\right)}^{k_{N_{\sigma }}-1-s}W\left(s\right)-{\displaystyle \sum _{s=k_{N_{\sigma }-1}}^{k_{N_{\sigma }-1}+{\Delta }_{N_{\sigma }-1}-1}}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }-1}\right)}^{k_{N_{\sigma }}-s-1}{\theta }_{\sigma (k_{N_{\sigma }}-1)}^{k_{N_{\sigma }-1}+{\Delta }_{N_{\sigma -1}-s}}W(s))\hfill \\ -{\displaystyle \sum _{s=k_{N_{\sigma }}+{\Delta }_{N_{\sigma }}}^{k-1}}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-1-s}W\left(s\right)-{\displaystyle \sum _{s=k_{N_{\sigma }}}^{k_{N_{\sigma }}+{\Delta }_{N_{\sigma }}-1}}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-s-1}{\theta }_{\sigma \left(k_{N_{\sigma }}\right)}^{k_{N_{\sigma }}+{\Delta }_{N_{\sigma }}-s}W\left(s\right)\hfill \\ \cdots \hfill \\ \le {\displaystyle \prod _{s=0}^{N_{\sigma }}}{\mu }_{1\sigma \left(k_s\right)}{\mu }_{2\sigma \left(k_s\right)}{\tilde{\alpha }}_{\sigma \left(k_s\right)}^{k_{s+1}-k_s}{\theta }_{\sigma \left(k_s\right)}^{{\Delta }_s}{V}_0\left(k_0\right)\hfill \\ -{\displaystyle \prod _{r=0}^{N_{\sigma }}}{\mu }_{1\sigma \left(k_r\right)}{\mu }_{2\sigma \left(k_r\right)}{\tilde{\alpha }}_{\sigma \left(k_r\right)}^{k_{r+1}-k_r}{\theta }_{\sigma \left(k_r\right)}^{{\Delta }_r}\left({\displaystyle \sum _{s=k_0+{\Delta }_0}^{k_1-1}}{\tilde{\alpha }}_{\sigma \left(k_0\right)}^{k_1-1-s}W\left(s\right)+{\displaystyle \sum _{s=k_0}^{k_0+{\Delta }_0-1}}{\tilde{\alpha }}_{\sigma \left(k_0\right)}^{k_1-s-1}{\theta }_{\sigma \left(k_0\right)}^{k_0+{\Delta }_0-s}W\left(s\right)\right)\hfill \end{array}$$

$$\begin{array}{c}\cdots \hfill \\ -{\displaystyle \sum _{s=k_{N_{\sigma }}+{\Delta }_{N_{\sigma }}}^{k-1}}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-1-s}W\left(s\right)-{\displaystyle \sum _{s=k_{N_{\sigma }}}^{k_{N_{\sigma }}+{\Delta }_{N_{\sigma }}-1}}{\tilde{\alpha }}_{\sigma \left(k_{N_{\sigma }}\right)}^{k-s-1}{\theta }_{\sigma \left(k_{N_{\sigma }}\right)}^{k_{N_{\sigma }}+{\Delta }_{N_{\sigma }}-s}W\left(s\right)\hfill \\ \le {\displaystyle \prod _{i=1}^n}{\mu }_{1i}^{N_{\sigma i}\left(k_0,k\right)}{\mu }_{2i}^{N_{\sigma i}\left(k_0,k\right)}{\tilde{\alpha }}_i^{T_i\left(k_0,k_1\right)}{\theta }_i^{{\Delta }_i{N}_{\sigma i}\left(k_0,k\right)}{V}_0\left(k_0\right)\hfill \\ -\sum _{s=k_0}^{k-1}({\mu }_{1j}{\displaystyle \prod _{\begin{array}{c}m=1\\ m\ne j\end{array}}^n}{\mu }_{1m}^{N_{\sigma m}\left(s,k\right)}{\mu }_{2m}^{N_{\sigma m}\left(s,k\right)}{\mu }_{1j}^{N_{\sigma j}\left(s,k\right)}{\mu }_{2j}^{N_{\sigma j}\left(s,k\right)}\times {\tilde{\alpha }}_m^{T_m\left(s,k\right)}{\theta }_m^{{\Delta }_m{N}_{\sigma m}\left(s,k\right)}{\tilde{\alpha }}_j^{T_j\left(s,k\right)-1}{\theta }_j^{{\Delta }_j{N}_{\sigma j}\left(s,k\right)})W\left(s\right)\hfill \\ -\sum _{s=k_0}^{k-1}({\mu }_{1j}{\displaystyle \prod _{\begin{array}{c}m=1\\ m\ne j\end{array}}^n}{\mu }_{1m}^{N_{\sigma m}\left(s,k\right)}{\mu }_{2m}^{N_{\sigma m}\left(s,k\right)}{\mu }_{1j}^{N_{\sigma j}\left(s,k\right)}{\mu }_{2j}^{N_{\sigma j}\left(s,k\right)}\times {\tilde{\alpha }}_m^{T_m\left(s,k\right)}{\theta }_m^{{\Delta }_m{N}_{\sigma m}\left(s,k\right)}{\tilde{\alpha }}_j^{T_j\left(s,k\right)-1}{\theta }_j^{{\Delta }_j\left(N_{\sigma j}\left(s,k\right)+1\right)-1})W\left(s\right),\hfill \end{array}$$

where m and j are the number of inactivated and activated subsystems in the s-th step, respectively.

Considering the initial condition and $V_{\sigma (k)}(k)\ge 0$, we have

$$\begin{array}{c}{\displaystyle \sum _{s=k_0}^{k-1}}({\mu }_{1j}{\displaystyle \prod _{\begin{array}{c}m=1\\ m\ne j\end{array}}^n}{\mu }_{1m}^{N_{\sigma m}\left(s,k\right)}{\mu }_{2m}^{N_{\sigma m}\left(s,k\right)}{\mu }_{1j}^{N_{\sigma j}\left(s,k\right)}{\mu }_{2j}^{N_{\sigma j}\left(s,k\right)}\hfill \\ \times {\tilde{\alpha }}_m^{T_m\left(s,k\right)}{\theta }_m^{{\Delta }_m{N}_{\sigma m}\left(s,k\right)}{\tilde{\alpha }}_j^{T_j\left(s,k\right)-1}{\theta }_j^{{\Delta }_j{N}_{\sigma j}\left(s,k\right)})W\left(s\right)\hfill \\ +{\displaystyle \sum _{s=k_0}^{k-1}}({\mu }_{1j}{\displaystyle \prod _{\begin{array}{c}m=1\\ m\ne j\end{array}}^n}{\mu }_{1m}^{N_{\sigma m}\left(s,k\right)}{\mu }_{2m}^{N_{\sigma m}\left(s,k\right)}{\mu }_{1j}^{N_{\sigma j}\left(s,k\right)}{\mu }_{2j}^{N_{\sigma j}\left(s,k\right)}\hfill \\ \times {\tilde{\alpha }}_m^{T_m\left(s,k\right)}{\theta }_m^{{\Delta }_m{N}_{\sigma m}\left(s,k\right)}{\tilde{\alpha }}_j^{T_j\left(s,k\right)-1}{\theta }_j^{{\Delta }_j\left(N_{\sigma j}\left(s,k\right)+1\right)-1})W\left(s\right)\le 0.\hfill \end{array}$$

(36)

Multiplying both sides of (36) by

$${\displaystyle \prod _{\begin{array}{c}m=1\end{array}}^n}{\mu }_{1m}^{-N_{\sigma m}\left(k_0,k\right)}{\mu }_{2m}^{-N_{\sigma m}\left(k_0,s\right)}{\mu }_{1j}^{-N_{\sigma j}\left(k_0,k\right)}{\theta }_m^{-{\Delta }_m{N}_{\sigma m}\left(k_0,k\right)},$$

we have

$$\begin{array}{c}{\displaystyle \sum _{s=k_0}^{k-1}}({\displaystyle \prod _{\begin{array}{c}m=1\\ m\ne j\end{array}}^n}{\mu }_{1m}^{-N_{\sigma m}\left(k_0,s\right)}{\mu }_{2m}^{-N_{\sigma m}\left(k_0,s\right)}{\mu }_{1j}^{-N_{\sigma j}\left(k_0,s\right)}{\mu }_{2j}^{-N_{\sigma j}\left(k_0,s\right)}\hfill \\ \times {\tilde{\alpha }}_m^{T_m\left(s,k\right)}{\theta }_m^{-{\Delta }_m{N}_{\sigma m}\left(k_0,s\right)}{\tilde{\alpha }}_j^{T_j\left(s,k\right)-1}{\theta }_j^{-{\Delta }_j{N}_{\sigma j}\left(k_0,s\right)}){\mathit{e}}^{\mathrm{T}}\left(s\right)\mathit{e}\left(s\right)\hfill \\ +{\displaystyle \sum _{s=k_0}^{k-1}}({\mu }_{1j}{\displaystyle \prod _{\begin{array}{c}m=1\\ m\ne j\end{array}}^n}{\mu }_{1m}^{-N_{\sigma m}\left(k_0,s\right)}{\mu }_{2m}^{-N_{\sigma m}\left(k_0,s\right)}{\mu }_{1j}^{-N_{\sigma j}\left(k_0,s\right)}\hfill \\ \times {\mu }_{2j}^{-N_{\sigma j}\left(k_0,s\right)}{\tilde{\alpha }}_m^{T_m\left(s,k\right)}{\theta }_m^{-{\Delta }_m{N}_{\sigma m}\left(k_0,s\right)}{\tilde{\alpha }}_j^{T_j\left(s,k\right)-1}{\theta }_j^{{\Delta }_j\left(-N_{\sigma j}\left(k_0,s\right)+1\right)-1}){\mathit{e}}^{\mathrm{T}}\left(s\right)\mathit{e}\left(s\right)\hfill \\ \le {\displaystyle \sum _{s=k_0}^{k-1}}({\displaystyle \prod _{\begin{array}{c}m=1\\ m\ne j\end{array}}^n}{\mu }_{1m}^{-N_{\sigma m}\left(k_0,s\right)}{\mu }_{2m}^{-N_{\sigma m}\left(k_0,s\right)}{\mu }_{1j}^{-N_{\sigma j}\left(k_0,s\right)}\hfill \\ \times {\mu }_{2j}^{-N_{\sigma j}\left(k_0,s\right)}{\tilde{\alpha }}_m^{T_m\left(s,k\right)}{\theta }_m^{-{\Delta }_m{N}_{\sigma m}\left(k_0,s\right)}{\tilde{\alpha }}_j^{T_j\left(s,k\right)-1}{\theta }_j^{-{\Delta }_j{N}_{\sigma j}\left(k_0,s\right)}){\gamma }^2{\mathit{d}}^{\mathrm{T}}\left(s\right)\mathit{d}\left(s\right)\hfill \\ +{\displaystyle \sum _{s=k_0}^{k-1}}({\mu }_{1m}{\displaystyle \prod _{\begin{array}{c}m=1\\ m\ne j\end{array}}^n}{\mu }_{1m}^{-N_{\sigma m}\left(k_0,s\right)}{\mu }_{2m}^{-N_{\sigma m}\left(k_0,s\right)}{\mu }_{1j}^{-N_{\sigma j}\left(k_0,s\right)}\hfill \\ \times {\mu }_{2j}^{-N_{\sigma j}\left(k_0,s\right)}{\tilde{\alpha }}_m^{T_m\left(s,k\right)}{\theta }_m^{-{\Delta }_m{N}_{\sigma m}\left(k_0,s\right)}{\tilde{\alpha }}_j^{T_j\left(s,k\right)-1}{\theta }_j^{{\Delta }_j\left(-N_{\sigma j}\left(k_0,s\right)+1\right)-1}){\gamma }^2{\mathit{d}}^{\mathrm{T}}\left(s\right)\mathit{d}\left(s\right).\hfill \end{array}$$

Together with (8) and (15), we can obtain that

$$0\le N_{\sigma m}(k_0,s)\le \frac{T_m(k_0,s)}{{\tau }_{am}}\le -\frac{T_m(k_0,s)ln{\tilde{\alpha }}_m}{ln{\mu }_{1m}{\mu }_{2m}+{\Delta }_{m}ln{\theta }_m}.$$

(37)

Combining (36) and (37) with ${\mu }_{1m}>1$, ${\mu }_{2m}>1$, $0<{\tilde{\alpha }}_m<1$, ${\theta }_m>1$, and ${\Delta }_m>1$, it can be inferred that:

$$\begin{array}{c}{\displaystyle \sum _{s=k_0}^{k-1}}\left({\displaystyle \prod _{m=1}^n}{\left({\mu }_{1m}{\mu }_{2m}{\theta }_m^{{\Delta }_m}\right)}^{\frac{T_m\left(k_0,s\right)ln{\tilde{\alpha }}_m}{ln{\mu }_{1m}{\mu }_{2m}+{\Delta }_{m}ln{\theta }_m}}{\tilde{\alpha }}_m^{T_m\left(s,k\right)-1}\right){\mathit{e}}^{\mathrm{T}}\left(s\right)\mathit{e}\left(s\right)\hfill \\ +{\displaystyle \sum _{s=k_0}^{k-1}}\left({\mu }_{1j}{\theta }_j^{{\Delta }_j-1}{\displaystyle \prod _{\begin{array}{c}m=1\\ m\ne j\end{array}}^n}{\left({\mu }_{1m}{\mu }_{2m}{\theta }_m^{{\Delta }_m}\right)}^{\frac{T_m\left(k_0,s\right)ln{\tilde{\alpha }}_m}{ln{\mu }_{1m}{\mu }_{2m}+{\Delta }_{m}ln{\theta }_m}}{\tilde{\alpha }}_m^{T_m\left(s,k\right)-1}\right){\mathit{e}}^{\mathrm{T}}\left(s\right)\mathit{e}\left(s\right)\hfill \\ \le {\displaystyle \sum _{s=k_0}^{k-1}}\left({\displaystyle \prod _{m=1}^n}{\tilde{\alpha }}_m^{T_m\left(s,k\right)-1}\right){\gamma }^2{\mathit{d}}^{\mathrm{T}}\left(s\right)\mathit{d}\left(s\right)+{\displaystyle \sum _{s=k_0}^{k-1}}\left({\mu }_{1j}{\theta }_j^{{\Delta }_j-1}{\displaystyle \prod _{m=1}^n}{\tilde{\alpha }}_m^{T_m\left(s,k\right)-1}\right){\gamma }^2{\mathit{d}}^{\mathrm{T}}\left(s\right)\mathit{d}\left(s\right).\hfill \end{array}$$

(38)

Thus, we have

$$\begin{array}{c}{\displaystyle \sum _{s=k_0}^{k-1}}\left({\displaystyle \prod _{m=1}^n}{\tilde{\alpha }}_m^{T_m(k_0,k)-1}\right){\mathit{e}}^{\mathrm{T}}(s)\mathit{e}(s)+{\displaystyle \sum _{s=k_0}^{k-1}}\left({\mu }_{1j}{\theta }_j^{{\Delta }_j-1}{\displaystyle \prod _{m=1}^n}{\tilde{\alpha }}_m^{T_m\left(k_0,k\right)-1}\right){\mathit{e}}^{\mathrm{T}}\left(s\right)\mathit{e}\left(s\right)\hfill \\ \le {\displaystyle \sum _{s=k_0}^{k-1}}\left({\displaystyle \prod _{m=1}^n}{\tilde{\alpha }}_m^{T_m\left(s,k\right)-1}\right){\gamma }^2{\mathit{d}}^{\mathrm{T}}\left(s\right)\mathit{d}\left(s\right)+{\displaystyle \sum _{s=k_0}^{k-1}}\left({\mu }_{1j}{\theta }_j^{{\Delta }_j-1}{\displaystyle \prod _{m=1}^n}{\tilde{\alpha }}_m^{T_m\left(s,k\right)-1}\right){\gamma }^2{\mathit{d}}^{\mathrm{T}}\left(s\right)\mathit{d}\left(s\right).\hfill \end{array}$$

(39)

When $k\to \infty$, we have

$${\displaystyle \sum _{s=k_0}^{\infty }}{\left(1-{\alpha }_{max}\right)}^{s-k_0}{\mathit{e}}^{\mathrm{T}}\left(s\right)\mathit{e}\left(s\right)\le {\gamma }^2{\displaystyle \sum _{s=k_0}^{\infty }}{\mathit{d}}^{\mathrm{T}}\left(s\right)\mathit{d}\left(s\right),$$

(40)

where ${\alpha }_{max}=\max \left\{{\mathrm{ff}}_{\mathrm{m}}\right\}$.

According to Definition 3, the switched systems in (6) and (7) are GUES with the prescribed attenuation index $\gamma$. This completes the proof.    □

Accordingly, the LMI problem in the theorem can be solved. Theorem 3 is given to obtain the solution of the tracking controller.

Theorem 3.

For given constants ${\mu }_{1i}>1$, ${\mu }_{2i}>1$, $0<{\alpha }_i<1$, ${\beta }_i>0$, if there are positive-definite matrices ${\mathit{P}}_i$, ${\mathit{S}}_i$, ${\mathit{Q}}_{1i}$, $Q_{2i}$, ${\mathit{Q}}_{3i}$, ${\mathit{P}}_{ij}$, $Q_{1ij}$, ${\mathit{Q}}_{2ij}$, and ${\mathit{Q}}_{3ij}$, $\forall i,j\in N,i\ne j$, the solution of the nominal tracking controller can be realized by the following equations:

$$\min \mathrm{tr}({\mathit{S}}_i,{\mathit{P}}_i)s.t.$$

$$\left[\begin{array}{cc}{\mathit{S}}_i& \mathit{I}\\ \mathit{I}& {\mathit{P}}_i\end{array}\right]\ge 0$$

(41)

$${\tilde{\mathbf{\Phi }}}_i=\left[\begin{array}{ccc}-{\mathit{S}}_i& \mathbf{0}& {\tilde{\mathbf{\Phi }}}_{13i}\\ \ast & -\mathit{I}& {\tilde{\mathbf{\Phi }}}_{23i}\\ \ast & \ast & {\tilde{\mathbf{\Phi }}}_{33i}\end{array}\right]<0$$

(42)

$${\tilde{\mathbf{\Phi }}}_{ij}=\left[\begin{array}{ccc}-{\mathit{S}}_{ij}& \mathbf{0}& {\tilde{\mathbf{\Phi }}}_{13ij}\\ \ast & -\mathit{I}& {\tilde{\mathbf{\Phi }}}_{23i}\\ \ast & \ast & {\tilde{\mathbf{\Phi }}}_{33ij}\end{array}\right]<0$$

(43)

$${\mathit{S}}_i{\mathit{P}}_i=\mathit{I}.$$

(44)

Then, the closed systems in (6) and (7) are GUES with the prescribed attenuation performance $\gamma$ with the MDADT satisfying (15).

3.2. Learning-Based Tracking Controller Design

The nominal tracking controller is proposed according to the $H_{\infty }$ control theory. The stability and prescribed attenuation performance of switched systems are guaranteed. However, as we all know, attaining the optimal compromise of robustness and transient efficiency is challenging. Therefore, the online scheduling algorithm is presented using the DQL approach. The learning-based tracking controller compensates for the system uncertainties to realize performance improvement.

The DQL is an important DRL algorithm. The agent learns an action from an unknown environment according to the reward function in the reinforcement learning framework. In this paper, the controller parameters’ online scheduling is viewed as a Markovian decision process. The action is described with the learning-based tracking controller parameters, which are given to compensate for the undesirable response induced by system uncertainties. The action is provided to maximize the expected discounted reward function during the predefined interval. The action and the reward function are given as follows:

$$A_k=\left[\begin{array}{cc}\Delta K_{1i}& \Delta K_{2i}\end{array}\right]$$

(45)

$$\begin{array}{c}{R}_k={\displaystyle \sum _{i=k}^{K_f}}{\gamma }_d^{i-k}{r}_{c,i}=r_{c,k}+{\gamma }_d{r}_{c,k+1}+{\gamma }_d^2{r}_{c,k+2}+\cdots +{\gamma }_d^{K_f-k}{r}_{c,K_f}=r_{c,k}+{\gamma }_d{R}_{k+1},\hfill \end{array}$$

(46)

where ${\gamma }_d\in [0,1]$ denotes the discount factor; $K_f$ is the terminal step.

The state vector of the DRL algorithm is defined as:

$$S_k=\left[\begin{array}{ccc}\mathit{x}\left(k\right)& \mathit{e}\left(k\right)& \mathit{g}\left(k\right).\end{array}\right]$$

(47)

The reward function is defined as:

$$r_{c,k}={\epsilon }_1{\left|\mathit{e}\left(k\right)\right|}^2+{\epsilon }_2{r}_{em}+{\epsilon }_3{r}_s$$

(48)

$$r_{em}=\left\{\begin{array}{c}{k}_{em},\left|e\right|\ge e_m\hfill \\ 0,\left|e\right|<e_m\hfill \end{array}\right.$$

(49)

$$r_s=\left\{\begin{array}{c}{k}_{um},\left|u\right|\ge u_m\hfill \\ 0,\left|u\right|<u_m,\hfill \end{array}\right.$$

(50)

where ${\epsilon }_1$, ${\epsilon }_2$, and ${\epsilon }_3$ denote the weights of the reward function, and $k_{em}$ and $k_{um}$ are given constants.

In the DQL algorithm, a deep neural network is introduced to approximate the action–state value function $Q_e^{\ast }\left(S_k,A_k\right)$, and the action is developed according to the maximum Q value:

$$a^{\ast }=arg\underset{a}{max}{Q}_e\left(S_k,A_k\right).$$

(51)

It is supposed that the algorithm has two neural networks, including the critic and target neural networks. The weights of the critic neural network are changed according to the TD error, and the weights of the target critic neural network are updated according to (52).

$$\begin{array}{c}{Q}_e\left(S_k,A_k,{\omega }^{-}\right)=L_r\left[R+{\gamma }_d{\max }_{a^{\prime }}(Q_e\left(S^{\prime },A^{\prime },{\omega }^{-}\right))\right]+(1-L_r)Q_e(S_k,A_k,\omega ),\hfill \end{array}$$

(52)

where $Q_e\left(s_k,a_k,\omega \right)$ and $Q_e\left(s_k,a_k,{\omega }^{-}\right)$ denote the outputs of the critic and target critic neural networks, respectively; $L_r$ denotes the learning rate; R is the value of the reward function from the state $S_k$ to $S^{\prime }$; ${max}_{a^{\prime }}\left(Q\left(s^{\prime },a^{\prime },{\omega }^{-}\right)\right)$ represents the target critic network’s maximum Q value. Therefore, the design algorithm of the learning-based tracking controller can be given as shown in Algorithm 1.

Algorithm 1 Learning-based tracking controller design.

1: Calculate the nominal tracking controller based on Theorem 3. 2: Initialize the parameters of the Q value network and the target Q value network. 3: Initialize the Replay buffer R, $episode$ = 0. 4: for $episode$ = 1 to M do 5:    Initialize the state S1 with a random value and take the initial observation. 6:    for t = 1 to K do 7:      Calculate the action $A_k$ according to the state and the reward function. 8:      Calculate the reward function and the state of the next time instant. 9:      Store the transition pair $(S_k,A_k,R_k,S_{k+1})$ in the Replay 10:      Sample a random minibatch transition pair from the Replay buffer. 11:      Update the weights of the Q value function. 12:      Update the weights of the target Q value network. 13:    end for 14: end for

Remark 3.

The model-based method and learning-based method are combined in this paper. The model-based method is provided based on the robust control theory and the switched control theory. To improve the transient performance, the deep Q learning algorithm is applied in the learning-based method. The output of action is supposed to be the compensation of controllers, and the reward function is defined as the weighted function of tracking error and constraints. By using this method, the parameters of controllers can be trained, and the stability, robustness, and transient performance can be guaranteed simultaneously.

4. Numerical Example

The flight vehicles studied in this paper are the HiMAT vehicles, which can be modeled as switched systems in the flight envelope. This paper considers the switching signals between subsystems 1, 2, 8, 12, and 18. The flight conditions of the operating points are given in

Table 1. Flight conditions of the vehicle.

Point Number	Ma	Height/m	Attack Angle
1	0.29	762.5	3.18
2	0.4	762.5	1.49
8	0.6	6100	1.48
12	0.9	7625	1.19
18	1.2	12,200	2.23

:

Therefore, the sampling time is chosen as $T_s=0.02$. The switched model of longitudinal motion dynamics can be described as:

$${\mathit{A}}_1=\left[\begin{array}{cc}0.9804& 0.0188\\ 0.1768& 0.9720\end{array}\right],{\mathit{B}}_1=\left[\begin{array}{ccc}-0.0049& -0.0034& 0.0007\\ -0.1579& -0.0979& 0.0993\end{array}\right]$$

$${\mathit{A}}_2=\left[\begin{array}{cc}0.9728& 0.0188\\ 0.3773& 0.9622\end{array}\right],{\mathit{B}}_2=\left[\begin{array}{ccc}-0.0075& -0.0050& 0.0014\\ -0.2941& -0.1765& 0.1831\end{array}\right]$$

$${\mathit{A}}_8=\left[\begin{array}{cc}0.9766& 0.0190\\ 0.3312& 0.9668\end{array}\right],{\mathit{B}}_8=\left[\begin{array}{ccc}-0.0077& -0.0054& 0.0018\\ -0.3759& -0.2798& 0.2113\end{array}\right]$$

$${\mathit{A}}_{12}=\left[\begin{array}{cc}0.9649& 0.0188\\ 0.2242& 0.9509\end{array}\right],{\mathit{B}}_{12}=\left[\begin{array}{ccc}-0.0136& -0.0094& 0.0042\\ -0.9015& -0.6166& 0.4367\end{array}\right]$$

$${\mathit{A}}_{18}=\left[\begin{array}{cc}0.9657& 0.0191\\ -0.9772& 0.9523\end{array}\right],{\mathit{B}}_{18}=\left[\begin{array}{ccc}-0.0061& -0.0033& 0.0023\\ -0.4595& -0.2426& 0.2576\end{array}\right]$$

$$\left\{\begin{array}{c}\mathit{v}\left(k+1\right)=\left[\begin{array}{cc}0.9922& 0.1247\\ -0.1247& 0.9922\end{array}\right]\mathit{v}\left(k\right)\hfill \\ \mathit{d}\left(k\right)=\left[\begin{array}{cc}1& 0\end{array}\right]\mathit{v}\left(k\right),\hfill \end{array}\right.$$

(53)

where $\mathit{v}(k)$ stands for the state vector, and the initial value of $\mathit{v}(k)$ is chosen as ${[0.01,0]}^{\mathrm{T}}$.

The current paper considers the switched systems parameters as: $a_1=0.25$, $a_2=0.23$, $a_8=0.26$, $a_{12}=0.23$, $a_{18}=0.27$, $b_1=0.03$, $b_2=0.04$, $b_8=0.02$, $b_{12}=0.05$, $b_{18}=0.04$, ${\Delta }_1=0.1$, ${\Delta }_2=0.12$, ${\Delta }_8=0.16$, ${\Delta }_{12}=0.14$, ${\Delta }_{18}=0.12$, ${\mu }_{11}=1.22$, ${\mu }_{12}=1.15$, ${\mu }_{18}=1.16$, ${\mu }_{112}=1.18$, ${\mu }_{118}=1.17$, ${\mu }_{21}=1.21$, ${\mu }_{22}=1.17$, ${\mu }_{28}=1.18$, ${\mu }_{212}=1.16$, and ${\mu }_{218}=1.15$. Therefore, the MDADT in (16) can be calculated as follows:

$${\tau }_{a1}^{\ast }=1.4641,{\tau }_{a2}^{\ast }=1.2735,{\tau }_{a3}^{\ast }=1.2131,{\tau }_{a4}^{\ast }=1.3673,{\tau }_{a5}^{\ast }=1.0779.$$

The ADT can be obtained as ${\tau }_a^{\ast }=1.6800$.

It is evident that the dwell time obtained by the MDADT approach is less than that obtained by the ADT method. Therefore, it leaves more room for controller design. The system can stay in the modes with a better performance for a longer time.

We set the attenuation performance $\gamma$ as 0.7. According to the conditions mentioned above, the nominal tracking controller can be derived using Theorem 3. The time delay and switching logic are presented in Figure 1 and Figure 2, respectively.

Firstly, we give a comparison between the ADT and MDADT methods, as described in Figure 3, Figure 4, Figure 5 and Figure 6. It is evident that the MDADT method can realize a better performance compared with the conventional ADT method. The attack angle response based on the MDADT method can avoid the undesirable response induced by the external disturbance. The generated controller can improve the robustness to the environment. Moreover, the actuator response is admissible.

Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11 compare the MDADT approach with the presented method. It can be seen that the transient performance and robustness are enhanced using the DQL algorithm. The controller can compensate for the adverse effect induced by the system uncertainties. A better response of attack angle can be obtained based on the compensation of the uncertainties. Moreover, the actuator response of the proposed controller is admissible.

Accordingly, the proposed method can realize a better transient performance and robustness than the conventional method. The fewer conservative results can be derived through the MDADT approach. The presented method can eliminate the undesirable response induced by time-varying delays and asynchronous switching. The learning-based tracking controller compensates for the system uncertainties. Therefore, we can see that the presented method can simultaneously ensure stability, robustness, and transient performance.

5. Conclusions

This paper presents a novel asynchronously $H_{\infty }$ tracking controller design and an online scheduling method for flight vehicles containing time-varying delays. The feedback controller is constructed using the flight vehicle’s switched model, derived based on Jacobian linearization. Due to the limitation of network transmission, the asynchronous switching induced by the time-varying delay is considered. The nominal tracking controller is proposed to guarantee robustness and stability. The learning-based tracking controller is established to avoid the undesirable response generated by system uncertainties. A combination of the MLF and MDADT methods is considered to verify the stability and attenuation efficiency. The sufficient existing conditions are derived based on the LMI technique. Then, the learning-based tracking controller is designed using the DQL to enhance the tracking controller’s transient efficiency. The proposed approach ensures stability, robustness, and transient efficiency, simultaneously.