Feedforward–Feedback Controller Based on a Trained Quaternion Neural Network Using a Generalised $\mathbb{HR}$ Calculus with Application to Trajectory Control of a Three-Link Robot Manipulator ^†

Abstract

This study derives a learning algorithm for a quaternion neural network using the steepest descent method extended to quaternion numbers. This applies the generalised Hamiltonian–Real calculus to obtain derivatives of a real–valued cost function concerning quaternion variables and designs a feedback–feedforward controller as a control system application using such a network. The quaternion neural network is trained in real-time by introducing a feedback error learning framework to the controller. Thus, the quaternion neural network-based controller functions as an adaptive-type controller. The designed controller is applied to the control problem of a three-link robot manipulator, with the control task of making the robot manipulator’s end effector follow a desired trajectory in the Cartesian space. Computational experiments are conducted to investigate the learning capability and the characteristics of the quaternion neural network used in the controller. The experimental results confirm the feasibility of using the derived learning algorithm based on the generalised Hamiltonian–Real calculus to train the quaternion neural network and the availability of such a network for a control systems application.

1. Introduction

Recently, artificial intelligence, including machine learning, has been frequently used to find solutions for intractable problems in science and engineering and other fields, such as social science, economics and finance. Their ability to learn by example makes them capable of applicable solutions in real-world problems [1]. For instance, artificial intelligence could offer approaches for resolving robotics issues in control engineering, especially for autonomous robotics [2]. Neural networks (NNs) have been applied to solve numerous engineering problems over the last half-century due to their flexibility, learning capability and highly complex mappings [3]. Additionally, NNs incorporating deep-learning techniques and reinforcement learning integrating deep NNs are currently being used to handle challenging problems in the real world. They have demonstrated successful applications in many engineering fields [4,5]. Although deep learning is commonly utilised to deal with such problems, high-dimensional NNs, where all the network parameters and signals are based on complex and hypercomplex numbers including quaternion and octonion, have become an attractive alternative for handling such difficulties more efficiently and easily. Since the high-dimensional representation using such numbers can easily address multi-dimensional correlated data as a single entity and has many advantages, including parameter and computational efficiency [6], high-dimensional NNs have been explored to solve multi-dimensional and complicated problems [7,8,9,10,11,12].

A servo-level controller using a high-dimensional NN based on quaternion numbers was proposed in our previous work. Its characteristics for controlling nonlinear dynamical systems were investigated as a control system application of quaternion NNs (QNNs) [13,14,15]. Moreover, we found that the QNN outperformed the other networks in the learning and control of a three-link robot manipulator [16], such as real- and hypercomplex-valued NNs. It applies the steepest descent against the real-valued cost function defined by the error between the desired and system output to train the QNN. Since the cost function’s derivative to the quaternion variables representing the network parameters was required to derive the QNN’s learning algorithm, the derivative was calculated using the pseudo-derivative that calculates component-wise real derivatives concerning the real variables of the quaternion number. The pseudo-derivative is often used to derive the learning algorithm in many QNN applications, e.g., time-series signal processing, image processing, speech processing and pattern classification [10,17]. However, the formal derivatives of a real-valued function for the quaternion variables, including their involution, have been proposed using Hamiltonian–Real ($\mathbb{HR}$) calculus, which is similar to the Wirtinger calculus relating complex numbers to quaternion numbers, to develop learning algorithms for signal processing using quaternion numbers [18]. Furthermore, a generalised form of the $\mathbb{HR}$ calculus called generalised $\mathbb{HR}$ (G$\mathbb{HR}$) calculus has been proposed to develop a more suitable learning algorithm [19]. Despite an increasing interest in learning algorithms based on the G$\mathbb{HR}$ calculus [20,21,22,23], its applicability on QNNs has not been adequately investigated. Moreover, practical, real-world applications using QNNs trained by such algorithms have not been fully explored, although such a QNN has been applied to control of a robot manipulator in a previous study [24].

Considering the aforementioned research gap, the main aim of this study is to determine how the specific properties of the learning algorithm based on the G$\mathbb{HR}$ calculus affect the QNN in control systems. In this study, the learning algorithm of a multi-layer QNN based on the G$\mathbb{HR}$ calculus is derived, and the possibility of using such a QNN in control system applications is explored. A servo controller based on the QNN is presented, where the input to the QNN consists of several input–output sets of the plant to handle plant dynamics and the output from the QNN is used to synthesise the control input to the plant. A feedback error learning (FEL) [25] framework extended to the quaternion numbers is employed to train the QNN-based controller. We used the QNN-based controller to perform computational experiments on the trajectory-tracking control of a three-link robot manipulator. We evaluated the feasibility and characteristics of the QNN trained by the learning algorithm based on the G$\mathbb{HR}$ calculus.

2. Feedforward–Feedback Controller Based on a Quaternion Neural Network

2.1. Generalised $\mathbb{HR}$ Calculus

Suppose the algebra of the quaternion numbers is $\mathbb{H}=\{q\mid q=q_0+q_1ı+q_{2}j+q_3\kappa ,q_0,q_1,q_2,q_3\in \mathbb{R}\}$ using four real numbers and the basis $\{1,ı,j,\kappa \}$, where the units ı, j and $\kappa$ satisfy Hamilton’s rules: ${ı}^2=j^2={\kappa }^2=ıj\kappa =-1,ıj=-jı=\kappa ,j\kappa =-\kappa j=ı$ and $\kappa ı=-ı\kappa =j$. It forms a four-dimensional noncommutative, associative normed division algebra over real numbers.

Some basics of the $\mathbb{HR}$ calculus [18] are summarised first. Let the transformation for any quaternion number q be denoted as $q^{\mu }=\mu q{\mu }^{-1}$ where $\mu$ is any nonzero quaternion number. The real variables of the quaternion number $q_i$ ($i=0,1,2,3$) can be expressed using the quaternion involutions: ${\displaystyle q_0=\frac{1}{4}(q+q^{ı}+q^j+q^{\kappa })}$, ${\displaystyle q_1=\frac{1}{4ı}(q+q^{ı}-q^j-q^{\kappa })}$, ${\displaystyle q_2=\frac{1}{4j}(q-q^{ı}+q^j-q^{\kappa })}$ and ${\displaystyle q_3=\frac{1}{4\kappa }(q-q^{ı}-q^j+q^{\kappa })}$. Let $f:\mathbb{H}\to \mathbb{H}$ be the quaternion-valued function $f\left(q\right)=f_0(q_0,q_1,q_2,q_3)+f_1(q_0,q_1,q_2,q_3)ı+f_1(q_0,q_1,q_2,q_3)j+f_3(q_0,q_1,q_2,q_3)\kappa$, the function f is real differentiable when the components $f_i$ ($i=0,1,2,3$) are differentiable as functions of real variables $q_i$ ($i=0,1,2,3$). If the function f is real differentiable, the $\mathbb{HR}$ derivatives of the function f to the involution are defined as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\displaystyle \frac{\partial f}{\partial q}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}-\frac{\partial f}{\partial q_1}ı-\frac{\partial f}{\partial q_2}j-\frac{\partial f}{\partial q_3}\kappa \right)}\hfill \\ {\displaystyle \frac{\partial f}{\partial q^{ı}}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}-\frac{\partial f}{\partial q_1}ı+\frac{\partial f}{\partial q_2}j+\frac{\partial f}{\partial q_3}\kappa \right)}\hfill \\ {\displaystyle \frac{\partial f}{\partial q^j}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}+\frac{\partial f}{\partial q_1}ı-\frac{\partial f}{\partial q_2}j+\frac{\partial f}{\partial q_3}\kappa \right)}\hfill \\ {\displaystyle \frac{\partial f}{\partial q^{\kappa }}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}+\frac{\partial f}{\partial q_1}ı+\frac{\partial f}{\partial q_2}j-\frac{\partial f}{\partial q_3}\kappa \right)}\hfill \end{array}\right.,\hfill \end{array}$$

(1)

and its conjugates are as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\displaystyle \frac{\partial f}{\partial q^{*}}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}+\frac{\partial f}{\partial q_1}ı+\frac{\partial f}{\partial q_2}j+\frac{\partial f}{\partial q_3}\kappa \right)}\hfill \\ {\displaystyle \frac{\partial f}{\partial q^{ı*}}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}+\frac{\partial f}{\partial q_1}ı-\frac{\partial f}{\partial q_2}j-\frac{\partial f}{\partial q_3}\kappa \right)}\hfill \\ {\displaystyle \frac{\partial f}{\partial q^{j*}}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}-\frac{\partial f}{\partial q_1}ı+\frac{\partial f}{\partial q_2}j-\frac{\partial f}{\partial q_3}\kappa \right)}\hfill \\ {\displaystyle \frac{\partial f}{\partial q^{\kappa *}}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}-\frac{\partial f}{\partial q_1}ı-\frac{\partial f}{\partial q_2}j+\frac{\partial f}{\partial q_3}\kappa \right)}\hfill \end{array}\right..\hfill \end{array}$$

(2)

Consequently, the G$\mathbb{HR}$ derivatives of the function f with respect to $q^{\mu }$ and $q^{\mu *}$ ($\mu \ne 0$, $\mu \in \mathbb{H}$) are defined as follows [19]:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\displaystyle \frac{\partial f}{\partial q^{\mu }}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}-\frac{\partial f}{\partial q_1}{ı}^{\mu }-\frac{\partial f}{\partial q_2}{j}^{\mu }-\frac{\partial f}{\partial q_3}{\kappa }^{\mu }\right)}\hfill \\ {\displaystyle \frac{\partial f}{\partial q^{\mu *}}=\frac{1}{4}\left(\frac{\partial f}{\partial q_0}+\frac{\partial f}{\partial q_1}{ı}^{\mu }+\frac{\partial f}{\partial q_2}{j}^{\mu }+\frac{\partial f}{\partial q_3}{\kappa }^{\mu }\right)}\hfill \end{array}\right.,\hfill \end{array}$$

(3)

where $\{1,{ı}^{\mu },j^{\mu },{\kappa }^{\mu }\}$ is the the generalised basis. Some properties of the G$\mathbb{HR}$ derivatives are given as follows:

$$\begin{array}{ccc}\hfill \mathrm{p}\mathrm{r}\mathrm{o}\mathrm{d}\mathrm{u}\mathrm{c}\mathrm{t} \mathrm{r}\mathrm{u}\mathrm{l}\mathrm{e}:\hfill & & {\displaystyle \frac{\partial \left(fg\right)}{\partial q}=\frac{\partial f}{\partial q^g}g+f\frac{\partial g}{\partial q},\frac{\partial \left(fg\right)}{\partial q^{*}}=\frac{\partial f}{\partial q^{g*}}g+f\frac{\partial g}{\partial q^{*}},}\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill \mathrm{r}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n} \mathrm{r}\mathrm{u}\mathrm{l}\mathrm{e}:\hfill & & {\displaystyle {\left({\displaystyle \frac{\partial f}{\partial q}}\right)}^{\mu }=\frac{\partial f^{\mu }}{\partial q^{\mu }},{\left({\displaystyle \frac{\partial f}{\partial q^{*}}}\right)}^{\mu }=\frac{\partial f^{\mu }}{\partial q^{\mu *}},}\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill \mathrm{c}\mathrm{h}\mathrm{a}\mathrm{i}\mathrm{n} \mathrm{r}\mathrm{u}\mathrm{l}\mathrm{e}:\hfill & & {\displaystyle \frac{\partial f\left(g\right(q\left)\right)}{\partial q}=\sum _{\mu \in \{1,ı,j,\kappa \}}\frac{\partial f}{\partial g^{\mu }}\frac{\partial g^{\mu }}{\partial q},\frac{\partial f\left(g\right(q\left)\right)}{\partial q^{*}}=\sum _{\mu \in \{1,ı,j,\kappa \}}\frac{\partial f}{\partial g^{\mu }}\frac{\partial g^{\mu }}{\partial q^{*}}.}\hfill \end{array}$$

(6)

where the functions $f:\mathbb{H}\to \mathbb{H}$ and $g:\mathbb{H}\to \mathbb{H}$ are real differentiable.

Using the G$\mathbb{HR}$ derivatives, Taylor’s theorem for quaternion-valued functions of quaternion variables can be formulated as a useful tool in calculus. The first-order Taylor series expansion of the function $f\left(q\right)\in \mathbb{H}$ is derived as follows:

$$\begin{array}{c}\hfill f(q+dq)=f\left(q\right)+\sum _{\mu \in \{1,ı,j,\kappa \}}\frac{\partial f\left(q\right)}{\partial q^{\mu }}d{q}^{\mu }+O\left(d{q}^2\right).\hfill \end{array}$$

(7)

If and only if the direction of the steepest descent $dq$ is set to ${\displaystyle dq=-\eta {\left({\displaystyle \frac{\partial f}{\partial q}}\right)}^{*}=-\eta \frac{\partial f}{\partial q^{*}}}$ with the learning factor $\eta \in {\mathbb{R}}^{+}$, the following relationship can be obtained:

$$\begin{array}{ccc}f(q+dq)-f\left(q\right)\hfill & \approx \hfill & {\displaystyle \sum _{\mu \in \{1,ı,j,\kappa \}}}\frac{\partial f\left(q\right)}{\partial q^{\mu }}d{q}^{\mu }\hfill \\ \hfill & =\hfill & \mathtt{Re}\left(4\frac{\partial f}{\partial q}\right)dq\le -4\eta {∥\frac{\partial f}{\partial q}∥}^2<0.\hfill \end{array}$$

(8)

This yields the steepest descent method for quaternion numbers, which could be used as a learning algorithm for the QNN. Since the learning factor $\eta$ defines a step size in the steepest descent, it functions to adjust the learning speed of the QNN.

2.2. Quaternion Neural Network

Considering a three-layer QNN with an input layer, a hidden layer and an output layer, the network’s input–output relationship can be given as:

$$\begin{array}{c}\hfill {\displaystyle {\sigma }_l=\sum _j{\omega }_{lj}\psi {\displaystyle (}\sum _i{w}_{ji}{s}_i{\displaystyle )},}\hfill \end{array}$$

(9)

where $s_i\in \mathbb{H}$ is an external input to the input layer’s i-th unit, $w_{ji}\in \mathbb{H}$ is the connection weight between the input layer’s i-th unit and the hidden layer’s j-th unit, ${\omega }_{lj}\in \mathbb{H}$ is the connection weight between the hidden layer’s j-th unit and the output layer’s l-th unit, ${\sigma }_l\in \mathbb{H}$ is the output layer’s l-th output and $\psi :\mathbb{H}\to \mathbb{H}$ is the unit’s activation function in the hidden layer. The unit’s threshold in the hidden and output layers can be treated as a connection weight against a constant input.

Assuming that ${ϵ}_l\in \mathbb{H}$ is the output error defined as ${ϵ}_l=t_l-{\sigma }_l$, where $t_l\in \mathbb{H}$ is the teaching signal of the output layer’s l-th unit, the QNN’s connection weights are trained using the steepest descent method that minimises the following cost function defined using the norm of the output error:

$$\begin{array}{c}\hfill E=\frac{1}{2}\sum _l{\parallel {ϵ}_l\parallel }^2.\hfill \end{array}$$

(10)

Although the cost function is a real-valued function since the norm of the quaternion number is a real number, it is also a function of the quaternion number and its conjugate due to the definition of the norm: $\parallel q\parallel =\sqrt{q{q}^{*}}$ where $q\in \mathbb{H}$. Calculating the cost function gradient to the connection weights using the G$\mathbb{HR}$ calculus updates the connection weights according to the steepest descent method extended to quaternion numbers derived as follows:

$$\begin{array}{ccc}{\omega }_{lj}\hfill & =\hfill & {\omega }_{lj}-\eta \frac{\partial E}{\partial {\omega }_{lj}^{*}}\hfill \\ \hfill & =\hfill & {\omega }_{lj}-\eta \left[-\frac{1}{2}\left\{\frac{\partial {\sigma }_l}{\partial {\omega }_{lj}^{{ϵ}_l^{*}*}}{ϵ}_l^{*}+{ϵ}_l\frac{\partial {\sigma }_l^{*}}{\partial {\omega }_{lj}^{*}}\right\}\right]\hfill \\ \hfill & =\hfill & {\omega }_{lj}+\frac{1}{4}\eta {ϵ}_l{\nu }_j^{*},\hfill \end{array}$$

(11)

$$\begin{array}{ccc}{w}_{ji}\hfill & =\hfill & w_{ji}-\eta \frac{\partial E}{\partial w_{ji}^{*}}\hfill \\ \hfill & =\hfill & w_{ji}-\eta \left[-\frac{1}{2}\sum _l\left\{\frac{\partial {\sigma }_l}{\partial w_{ji}^{{ϵ}_l^{*}*}}{ϵ}_l^{*}+{ϵ}_l\frac{\partial {\sigma }_l^{*}}{\partial w_{ji}^{*}}\right\}\right]\hfill \\ \hfill & =\hfill & w_{ji}+\frac{1}{4}\eta \sum _l\left\{\mathtt{Re}\left[{ϵ}_l{\left(\frac{\partial {\nu }_j}{\partial x_{j_0}}\right)}^{*}{\omega }_{lj}^{*}\right]+\mathtt{Re}\left[{ϵ}_l{\left(\frac{\partial {\nu }_j}{\partial x_{j_1}}\right)}^{*}{\omega }_{lj}^{*}\right]ı\right.\hfill \\ \hfill & \hfill & \left.+\mathtt{Re}\left[{ϵ}_l{\left(\frac{\partial {\nu }_j}{\partial x_{j_2}}\right)}^{*}{\omega }_{lj}^{*}\right]j+\mathtt{Re}\left[{ϵ}_l{\left(\frac{\partial {\nu }_j}{\partial x_{j_3}}\right)}^{*}{\omega }_{lj}^{*}\right]\kappa \right\}{s}_i^{*}.\hfill \end{array}$$

(12)

where ${\nu }_j=\psi \left(x_j\right)$ is the hidden layer’s j-th output, and ${\displaystyle x_j=\sum _i{w}_{ji}{s}_i}$ is an internal state of the hidden layer’s j-th unit. These update rules are the back–propagation algorithm extended to quaternion numbers.

2.3. Feedforward–Feedback Controller

In this section, the QNN is used to design a controller which makes a plant output follow a reference signal. The control input to the plant is composed of both the QNN output and a feedback controller output in this controller because a FEL framework is introduced to train the controller’s QNN. To simplify designing the controller, the following discrete-time linear plant is considered:

$$\begin{array}{c}\hfill y(k+d)=\sum _{i=0}^{n-1}{\alpha }_{i}y(k-i)+\sum _{i=0}^{m+d-1}{\beta }_{i}u(k-i),\hfill \end{array}$$

(13)

where $y\in \mathbb{H}$ is the plant output, $u\in \mathbb{H}$ is the control input, ${\alpha }_i\in \mathbb{H}$ and ${\beta }_i\in \mathbb{H}$ are the plant’s coefficients, n and m are the plant orders, d is the dead time and k is the sampling number. The control input consists of the QNN output $\sigma$ and the feedback controller output v, such as $u\left(k\right)=\sigma \left(k\right)+v\left(k\right)$. In defining the control error between the reference signal r and the plant output y: $e\left(k\right)=r\left(k\right)-y\left(k\right)$, it can be rewritten using Equation (6) as:

$$\begin{array}{ccc}e(k+d)\hfill & =\hfill & r(k+d)-y(k+d)\hfill \\ \hfill & =\hfill & {\beta }_0\{{\mathbf{\xi }}^{\mathtt{T}}s\left(k\right)-\sigma \left(k\right)\},\hfill \end{array}$$

(14)

where ${(·)}^{\mathtt{T}}$ is the transpose operator,

$$\begin{array}{c}\hfill \mathbf{\xi }={\beta }_0^{-1}{\left[\begin{array}{cccccccc}1& -{\alpha }_0& \cdots & -{\alpha }_{n-1}& -{\beta }_1& \cdots & {\beta }_{p-1}& -{\beta }_0\end{array}\right]}^{\mathtt{T}},\hfill \end{array}$$

(15)

$$\begin{array}{c}\hfill s\left(k\right)={\left[\begin{array}{cccccccc}r(k+d)& y\left(t\right)& \cdots & y(k-n+1)& u(k-1)& \cdots & u(k-p+1)& v\left(k\right)\end{array}\right]}^{\mathtt{T}},\hfill \end{array}$$

(16)

and $p=m+d$. Assuming the control error becomes zero, the three-layer QNN is designed using the vector $s\left(k\right)$ as the network’s input:

$$\begin{array}{c}\hfill \sigma \left(k\right)={\mathbf{\omega }}^{\mathtt{T}}\left(k\right)\mathbf{\psi }\left(W\left(k\right)s\left(k\right)\right),\hfill \end{array}$$

(17)

where the component of the matrix $W\in {\mathbb{H}}^{M\times L}$ comprises the connection weights between the input and hidden layers, and the component of the vector $\mathbf{\omega }\in {\mathbb{H}}^M$ consists of the connection weights between the hidden and output layers. Here L is the number of units in the input layer defined as $L=n+p+1$, and M is the number of units in the hidden layer.

The connection weights are updated with the learning algorithm using Equations (11) and (12) at every sampling number, trained in real-time. Thus, the QNN acts as an adaptive controller. When the generalised $\delta$-rule is applied to the QNN training by considering the plant’s dead time, the connection weight training in the controller is performed as follows:

$$\begin{array}{c}\hfill \mathbf{\omega }(k+1)=\mathbf{\omega }(k-d)-\eta \frac{\partial E\left(k\right)}{\partial {\mathbf{\omega }}^{*}\left(k\right)},\hfill \end{array}$$

(18)

$$\begin{array}{c}\hfill W(k+1)=W(k-d)-\eta \frac{\partial E\left(k\right)}{\partial W^{*}\left(k\right)}.\hfill \end{array}$$

(19)

where ${(·)}^{*}$ is the quaternion conjugate operator. The cost function, Equation (10), is defined using the feedback controller output instead of the output error; namely, the feedback controller output is minimised during the QNN training process according to the FEL. While the feedback controller is expected to keep the control system stable at the beginning of the QNN training process, the QNN comes to be a feedforward controller when the QNN training is ideally completed. Thus, such a controller is called a feedforward–feedback controller [26]. Figure 1 shows the resulting control system schematic.

2.4. Remarks on the Stability Condition of the Controller

The stability analysis of the QNN-based feedforward–feedback controller is important. However, it is generally difficult, owing to the nonlinearity of the controller and the plant, the noncommutative multiplication of quaternion numbers and the analyticity of the quaternion function used in the activation function. Therefore, the local stability condition of the controller is discussed by introducing the following assumptions:

(i)

The plant is represented by Equation (13), where the orders of the plant, the dead time and the sign of the high–frequency gain are known.

(ii)

The QNN’s activation function is a split-type function using a component-wise linear function even though it is not analytic in the field of quaternion number.

(iii)

The feedback controller is a P–controller, and the QNN’s connection weights allow the feedback controller output to be sufficiently small.

Using assumption (ii), the QNN, Equation (17) and the connection weight’s learning algorithm, Equations (18) and (19) can be rewritten as follows:

$$\begin{array}{c}\hfill \sigma \left(k\right)={\mathbf{\omega }}^{\mathtt{T}}\left(k\right)W\left(k\right)s\left(k\right),\hfill \end{array}$$

(20)

$$\begin{array}{c}\hfill W(k+1)=W(k-d)+\frac{1}{2}\eta {\mathbf{\omega }}^{*}\left(k\right)v\left(k\right)s^{\mathtt{H}}\left(k\right),\hfill \end{array}$$

(21)

$$\begin{array}{c}\hfill \mathbf{\omega }(k+1)=\mathbf{\omega }(k-d)+\frac{1}{4}v\left(k\right){\left\{s^{\mathtt{H}}\left(k\right)W^{\mathtt{H}}\left(k\right)\right\}}^{\mathtt{T}},\hfill \end{array}$$

(22)

where ${(·)}^{\mathtt{H}}$ is the Hermitian operator. Using assumption (i), the control error is represented by Equation (14). Substituting Equation (20) into Equation (14) yields:

$$\begin{array}{ccc}e(k+d)\hfill & =\hfill & {\beta }_0\{{\mathbf{\xi }}^{\mathtt{T}}-{\mathbf{\omega }}^{\mathtt{T}}\left(k\right)W\left(k\right)\}s\left(k\right)\hfill \\ \hfill & =\hfill & {\beta }_0{\mathbf{\zeta }}^{\mathtt{T}}\left(k\right)s\left(k\right),\hfill \end{array}$$

(23)

where ${\mathbf{\zeta }}^{\mathtt{T}}\left(k\right)={\mathbf{\xi }}^{\mathtt{T}}-{\mathbf{\omega }}^{\mathtt{T}}\left(k\right)W\left(k\right)$ denotes the parameter error vector. Substituting Equations (21) and (22) into the parameter error vector and using the condition $\parallel v\left(k\right)\parallel \ll 1$ obtained by assumption (iii) yields:

$$\begin{array}{ccc}{\mathbf{\zeta }}^{\mathtt{T}}(k+1)\hfill & =\hfill & {\mathbf{\xi }}^{\mathtt{T}}-{\mathbf{\omega }}^{\mathtt{T}}(k+1)W(k+1)\hfill \\ \hfill & \approx \hfill & {\mathbf{\zeta }}^{\mathtt{T}}(k-d)\left\{E-\frac{1}{4}h\eta \mathrm{\Omega }(k-d)\right\},\hfill \end{array}$$

(24)

where

$$\begin{array}{c}\hfill \mathrm{\Omega }(k-d)=s(k-d)\left\{s^{\mathtt{H}}\left(k\right)W^{\mathtt{H}}\left(k\right)W(k-d)+2v^{-1}\left(k\right){\mathbf{\omega }}^{\mathtt{T}}(k-d){\mathbf{\omega }}^{*}\left(k\right)v\left(k\right)s^{\mathtt{H}}\left(k\right)\right\},\hfill \end{array}$$

(25)

E is the identity matrix, and the feedback controller is assumed $v\left(k\right)=h{\beta }_0^{-1}e\left(k\right)$ where $h\in {\mathbb{R}}^{+}$ is the feedback gain. Assuming the function $V\left(k\right)={\parallel \mathbf{\zeta }\left(k\right)\parallel }^2\ge 0$ as the Lyapunov function candidate yields:

$$\begin{array}{ccc}\Delta V\left(k\right)\hfill & =\hfill & V(k+1)-V(k-d)\hfill \\ \hfill & =\hfill & -\frac{1}{4}h\eta {\mathbf{\zeta }}^{\mathtt{T}}(k-d)\mathrm{\Phi }(k-d){\mathbf{\zeta }}^{*}(k-d),\hfill \end{array}$$

(26)

where

$$\begin{array}{c}\hfill \mathrm{\Phi }(k-d)=\mathrm{\Omega }(k-d)+{\mathrm{\Omega }}^{\mathtt{H}}(k-d)-\frac{1}{4}h\eta \mathrm{\Omega }(k-d){\mathrm{\Omega }}^{\mathtt{H}}(k-d).\hfill \end{array}$$

(27)

Consequently, the stability of the parameter error vector $\mathbf{\zeta }\left(k\right)$ is guaranteed when $\Delta V\left(k\right)\le 0$ in Equation (26), namely, the matrix $\mathrm{\Phi }(k-d)$ is positive semi-definite. The matrix $\mathrm{\Phi }(k-d)$ is Hermitian because of its definition, so its eigenvalues are real numbers. This result clarifies the stability characteristics qualitatively, i.e., the stability depends on the learning factor, the feedback gain and the connection weight, which is strongly related to both the initial value and the convergence trajectory of the connection weight. Therefore, the brute force approach which changes the learning factor, the feedback gain and the connection weight’s initial values is necessary to find the stability condition quantitatively. Additionally, the convergence of the parameter error vector $\mathbf{\zeta }\left(k\right)$ would not always satisfy the relationship ${\mathbf{\xi }}^{\mathtt{T}}={\mathbf{\omega }}^{\mathtt{T}}\left(k\right)W\left(k\right)$ even though the feedback controller output becomes sufficiently small; namely, the control error approaches zero. The connection weight’s learning algorithm for the QNN using the steepest descent method primarily minimises the feedback controller output by matching the response between the plant output and the reference signal but does not estimate the true values of the plant parameter as in the adaptive control’s parameter identification. This shows that the QNN has a redundancy to realise the controller.

3. Computational Experiments

To investigate the feasibility and the characteristics of the QNN trained using the learning algorithm based on the G$\mathbb{HR}$ calculus in control systems application, computational experiments are performed to control a robot manipulator.

3.1. Robot Manipulator

The robot manipulator which has three joints, rotation–pivot–pivot type, shown in Figure 2, is the target plant controlled using the QNN-based feedforward–feedback controller. The motion equation of the robot manipulator is given as:

$$M\left(\mathbf{\theta }\right)\ddot{\mathbf{\theta }}+C\dot{\mathbf{\theta }}+K(\mathbf{\theta },\dot{\mathbf{\theta }})=\mathbf{\tau },$$

(28)

where $\mathbf{\theta }\in {\mathbb{R}}^3$ is the joint angle vector, $\mathbf{\tau }\in {\mathbb{R}}^3$ is the control torque vector, $M\left(\mathbf{\theta }\right)\in {\mathbb{R}}^{3\times 3}$ is the inertia matrix, $C=\mathtt{diag}(c_1,c_2,c_3)\in {\mathbb{R}}^{3\times 3}$ is the viscous matrix, and $K(\mathbf{\theta },\dot{\mathbf{\theta }})\in {\mathbb{R}}^3$ is the nonlinear force vector including the Coriolis force, centrifugal force, gravitational force and solid frictional force at the joints.

The control task is executed in the joint space, although the control objective for the robot manipulator is to ensure that the end-effector tracks the desired trajectory given in the workspace (Cartesian space). The desired position on the trajectory is transformed into the desired joint angles in advance through the robot manipulator’s inverse kinematics model. The movable range of each joint angle is restricted to avoid singular configurations and multiple solutions of the robot manipulator.

3.2. Controller Condition

In the QNN-based feedforward–feedback controller, the QNN has nine units in the input layer and one unit in the output layer. As described in Section 2.3, the input to the QNN $s_i\left(k\right)$ ($i=1,2,\cdots ,9$) consists of the tapped–delay–line input–output sets of the robot manipulator as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{s}_1\left(k\right)=r_1(k+1)ı+r_2(k+1)j+r_3(k+1)\kappa \hfill \\ s_2\left(k\right)={\theta }_1\left(k\right)ı+{\theta }_2\left(k\right)j+{\theta }_3\left(k\right)\kappa \hfill \\ s_3\left(k\right)={\theta }_1(k-1)ı+{\theta }_2(k-1)j+{\theta }_3(k-1)\kappa \hfill \\ s_4\left(k\right)={\theta }_1(k-2)ı+{\theta }_2(k-2)j+{\theta }_3(k-2)\kappa \hfill \\ s_5\left(k\right)={\theta }_1(k-3)ı+{\theta }_2(k-3)j+{\theta }_3(k-3)\kappa \hfill \\ s_6\left(k\right)=u_1(k-1)ı+u_2(k-1)j+u_3(k-1)\kappa \hfill \\ s_7\left(k\right)=u_1(k-2)ı+u_2(k-2)j+u_3(k-2)\kappa \hfill \\ s_8\left(k\right)=u_1(k-3)ı+u_2(k-3)j+u_3(k-3)\kappa \hfill \\ s_9\left(k\right)=v_1\left(k\right)ı+v_2\left(k\right)j+v_3\left(k\right)\kappa \hfill \end{array}\right.,\hfill \end{array}$$

(29)

where $r_i\left(k\right)$ is the reference signal to the joint angle ${\theta }_i\left(k\right)$, $u_i\left(k\right)$ is the control input, and $v_i\left(k\right)$ is the feedback controller output ($i=1,2,3$). Using the real variables of the imaginary part of the QNN output $\sigma \left(k\right)={\sigma }_0\left(k\right)+{\sigma }_1\left(k\right)ı+{\sigma }_2\left(k\right)j+{\sigma }_3\left(k\right)\kappa$, the control input is synthesised by $u_i\left(k\right)={\sigma }_i\left(k\right)+v_i\left(k\right)$ ($i=1,2,3$). The feedback controller output is given by $v_i\left(k\right)=\gamma h_i^{\mathtt{T}}{e}_i\left(k\right)$ where $\gamma$ is the scale factor, $h_i\in {\mathbb{R}}^2$ is the feedback gain vector, $e_i\left(k\right)={\left[\begin{array}{cc}{e}_i^{\theta }\left(k\right)& {\Delta }_i\left(k\right)\end{array}\right]}^{\mathtt{T}}$ is the error vector, $e_i^{\theta }\left(k\right)$ is the control error in the joint space, and ${\Delta }_i\left(k\right)=e_i^{\theta }\left(k\right)-e_i^{\theta }(k-1)$ is the difference of the control error ($i=1,2,3$). Consequently, the robot manipulator’s control torque is given by ${\tau }_i\left(k\right)=T_i{u}_i\left(k\right)$, where $T_i$ is the torque constant ($i=1,2,3$).

3.3. Numerical Simulations

The robot manipulator’s trajectory-tracking control experiments were conducted using the QNN-based feedforward–feedback controller. In the QNN, the unit’s activation function in the hidden layer is the split-type function using a component-wise sigmoid function. Each real variable of the quaternion numbers representing the connection weights was initialised randomly from the interval $[-1,1]$. The FEL of the QNN was conducted in real-time. It was terminated when the normalised cost function, defined as the mean of the cost function, Equation (10), within one iteration of the desired trajectory, was less than the threshold value 0.0005 after five continuous iterations. The network training was restarted using the new connection weight’s initial values if the normalised cost function did not satisfy the termination condition after 100 iterations or if it tended to exceed a threshold and diverge during the training process. When the number of resets exceeded 10,000, the network training was defined as a failure. In the training process, the end-effector’s desired trajectory was a circle of radius 0.2 in the vertical plane parallel to the O–YZ plane. The trajectory consisted of 1000 data points per round, and one round was defined as one iteration. The feedback gains were $h_1={\left[\begin{array}{cc}4.0& 0.04\end{array}\right]}^{\mathtt{T}}$, $h_2=h_3={\left[\begin{array}{cc}2.0& 0.02\end{array}\right]}^{\mathtt{T}}$, and the scale factor was $\gamma =1$. The torque constants was $T_i=10$ ($i=1,2,3$). The motion equation, Equation (28), was solved by the Runge–Kutta method with a 0.01 s step size in the numerical simulations. In the robot manipulator, the links 1, 2 and 3 were 0.05 m long and 0.4 kg mass, 0.25 m long and 0.2 kg mass, and 0.25 m long and 0.2 kg mass, respectively. The damping factor was $c_i=0.005$ ($i=1,2,3$), and the solid frictional forces were 0.2 Nm, 0.1 Nm and 0.1 Nm at the joints 1, 2 and 3, respectively. The robot manipulator’s initial condition was $\mathbf{\theta }\left(0\right)={\left[\begin{array}{ccc}0& -\frac{\pi }{12}& \frac{5\pi }{12}\end{array}\right]}^{\mathtt{T}}$.

First, the feasibility of training the QNN with the proposed learning algorithm derived using the G$\mathbb{HR}$ calculus and controlling the robot manipulator using the QNN-based feedforward–feedback controller was investigated through numerical simulations. Figure 3 shows the change of the normalised cost function where the number of units in the hidden layer is two and the learning factor is 0.012. The normalised cost function decreases rapidly, and the network training is completed when the training’s terminating condition is satisfied. The normalised cost function decreases as the number of iterations progress and converge, indicating that it is possible to train the QNN in the control system. Figure 4 shows the response of the robot manipulator’s end-effector in the workspace and the variation of the cost function at each sampling time, indicating the responses at the first and termination of the training process shown in Figure 3. At the beginning of the training process, the end-effector does not follow the desired trajectory due to a lack of enough adaptation using the QNN. However, after the training converges, the end-effector could follow the desired trajectory; namely, the QNN-based feedforward–feedback controller achieves the control task. When the network training is terminated, the cost function at each sampling time is sufficiently small. This means that the major role of controlling the robot manipulator has shifted from the feedback controller to the QNN-based feedforward–feedback controller. These results confirmed that the QNN learning algorithm derived from the G$\mathbb{HR}$ calculus is functioning well. Therefore, it is feasible to apply the QNN-based controller for controlling the robot manipulator.

Next, the QNN’s learning characteristics were evaluated using two indexes: the number of iterations required to satisfy the network training termination condition and the number of resets during the training process. Here, 50 experimental results with different initial network weights were used to calculate these indexes.

Table 1. Learning characteristics of the QNN concerning the number of units in the hidden layer.

Steel–Dwass Test						Iteration
		1	2	3	4	Median (IQR)
Number of units	1	–				42.5 (32.5–59.5)
	2	*	–			60.5 (38.5–81)
	3	$**$	ns	–		63 (44.75–82.25)
	4	$**$	ns	ns	–	75 (60.75–87)
Steel–Dwass Test						Reset
		1	2	3	4	Median (IQR)
Number of units	1	–				48.5 (20.75–83.75)
	2	ns	–			75 (33.75–143.25)
	3	$**$	$**$	–		256.5 (92.25–634.5)
	4	$**$	$**$	$**$	–	1006.5 (317.75–2378.75)

(ns: not significant, * p < 0.05,** p < 0.01).

summarises the number of iterations and resets when the number of units in the hidden layer is varied from one to four with a 0.012 learning factor. The index’s median and interquartile range are used instead of the mean and standard deviation because normal distribution is unobserved in these indexes according to the Shapiro–Wilk test with a 5% significant level. Additionally, multiple comparisons are conducted using the Steel–Dwass test. This result indicates that the number of units in the hidden layer has less effect on the learning speed if the network learning could progress, but the dependency on the connection weight’s initial values to proceed with the network training increases as the number of units increases.

Table 2. Learning characteristics of the QNN concerning the learning factor.

Steel–Dwass Test							Iteration
		0.006	0.012	0.018	0.024	0.030	Median (IQR)
Learning factor $\eta$	0.006	–					72 (55.25–87.25)
	0.012	ns	–				60.5 (38.5–81)
	0.018	$**$	ns	–			48.5 (37–67.5)
	0.024	$**$	*	ns	–		45 (29–66.25)
	0.030	$**$	†	ns	ns	–	43.5 (29.25–66)
Steel–Dwass Test							Reset
		0.006	0.012	0.018	0.024	0.030	Median (IQR)
Learning factor $\eta$	0.006	–					219.5 (91–681.5)
	0.012	$**$	–				75 (33.75–143.25)
	0.018	$**$	ns	–			75.5 (31.75–141.25)
	0.024	†	ns	†	–		129 (47.75–270.5)
	0.030	ns	*	$**$	ns	–	193.5 (64.5–392.25)

(ns: not significant, † p < 0.1, * p < 0.05, ** p < 0.01).

summarises the number of iterations and resets when the learning factor is changed, where the number of units in the hidden layer is two. Increasing the learning factor improves the learning speed; however, the number of reset increases, namely, the dependency on the connection weight’s initial values to proceed with the network training increases. Furthermore, using a small learning factor increases the number of resets. This is because the first-order approximation of the steepest descent method makes the network training unstable with a large learning factor, whereas it is easy to reach the upper limit of the number of iterations with a small learning factor.

Table 3. Learning characteristics of the QNN concerning the feedback gain.

Steel–Dwass Test						Iteration
		0.5	1	2	10	Median (IQR)
Scale factor $\gamma$	0.5	–				62.5 (45.25–78.5)
	1	ns	–			60.5 (38.5–81)
	2	ns	ns	–		71 (53–87.75)
	10	ns	ns	ns	–	71 (54.75–92)
Steel–Dwass Test						Reset
		0.5	1	2	10	Median (IQR)
Scale factor $\gamma$	0.5	–				287.5 (124.5–432)
	1	$**$	–			75 (33.75–143.25)
	2	$**$	ns	–		72 (27.75–119.25)
	10	*	$**$	$**$	–	483.5 (236.25–758.5)

(ns: not significant, * p < 0.05,** p < 0.01).

summarises the number of iterations and resets when the feedback gains are changed by the scale factor, where the number of units in the hidden layer is two and the learning factor is 0.012. Using smaller or larger feedback gains increases the number of resets although there is no statistical difference in the number of iterations. The network training did not converge when the scale factor was $\gamma =0.1$. Since the QNN output at the beginning of the network training affects the feedback controller as a disturbance, it is difficult to keep the control system stable with the smaller feedback gains. The control system becomes easily unstable with larger feedback gains due to the nonlinear characteristics of the robot manipulator because the feedback controller uses a simple error feedback. This result shows that an appropriate value of the feedback gain is required to proceed with the network training while ensuring the stability of the control system.

The control ability of the QNN-based feedforward–feedback controller was tested against untrained conditions after the QNN training process with the circular trajectory was completed. Figure 5 illustrates examples of the system response in the workspace where the desired trajectory and the initial condition of the joint angle were different from those set in the training process. Here, the number of units in the hidden layer was two, and the learning factor was 0.012. The FEL of the QNN was continued in real-time during the test. Although control errors can be observed partially around the curvature of the desired trajectory changes, the end-effector can follow the desired trajectory. The cost function sometimes increased with the control error but decreased soon as the sampling number progressed because the FEL can be performed stably in real-time after the pretraining with the circular trajectory. These results show that the QNN’s learning and generalisation capabilities are effective to compensate for the robustness of the control system against varying the robot manipulator’s experimental conditions.

Finally, the learning algorithm derived using the pseudo-derivative to calculate the cost function derivative to the network parameters was applied to train the QNN as a reference.

Table 4. Comparison of learning characteristics between the connection weight’s learning algorithms.

	Iteration	Reset	Success Rate [%]
	Median (IQR)	Median (IQR)
G$\mathbb{HR}$ calculus	69.5 (51.25–85.75)	874.5 (460.5–2038.5)	100
Pseudo–derivative	45.5 (29.75–75.25)	2106 (935–4656.5)	94
Mann–Whitney U test	$**$	$**$

(** p < 0.01).

compares the number of iterations and resets the learning algorithm obtained using the G$\mathbb{HR}$ calculus with those obtained by the learning algorithm using the pseudo-derivative. A total of 100 experimental results were used to calculate these indexes. Here, there were two units in the hidden layer, and the learning factor was 0.012. To compare the learning characteristics under more severe conditions, the threshold value for terminating the network training was 0.0002. The learning algorithm using the pseudo-derivative attains a little faster learning speed, and the Mann–Whitney U test shows a statistical difference in the number of iterations. However, the learning algorithm using the pseudo-derivative requires many resets to proceed with the network training. Additionally, the network training by the learning algorithm using the pseudo-derivative sometimes failed to converge with the same value of the learning factor used in the learning algorithm using the G$\mathbb{HR}$ calculus. These results indicate that the learning algorithm using the G$\mathbb{HR}$ calculus can perform the network training stably with relatively less dependence on the connection weight’s initial values.

4. Conclusions

This study investigated a feedforward–feedback controller based on the QNN trained by the learning algorithm derived using the G$\mathbb{HR}$ calculus. In the QNN-based feedforward–feedback controller, the tapped–delay–line input–output sets of a plant were input to the QNN, and the control input to the plant was synthesised using the QNN output. A FEL framework performed at every sampling time was introduced to train the controller’s QNN to ensure that the plant output followed the desired output in real-time. Additionally, the local stability condition of the controller was analysed under linear assumptions of the QNN and plant. The result clarified how the learning factor, the feedback gain and the QNN’s connection weights affect the stability of the parameter error. Computational experiments on the trajectory-tracking control of a three-link robot manipulator using the QNN-based feedforward–feedback controller were conducted. Simulation results confirmed the feasibility of the learning algorithm derived using the G$\mathbb{HR}$ calculus to train the QNN, and the effectiveness of the QNN’s learning and generalisation capabilities to maintain the robot manipulator’s control performance. The experimentation of the QNN-based feedforward–feedback controller on an actual robot manipulator in real-world scenarios will be discussed in future work.