Lyapunov Functions to Caputo Fractional Neural Networks with Time-Varying Delays

Abstract

One of the main properties of solutions of nonlinear Caputo fractional neural networks is stability and often the direct Lyapunov method is used to study stability properties (usually these Lyapunov functions do not depend on the time variable). In connection with the Lyapunov fractional method we present a brief overview of the most popular fractional order derivatives of Lyapunov functions among Caputo fractional delay differential equations. These derivatives are applied to various types of neural networks with variable coefficients and time-varying delays. We show that quadratic Lyapunov functions and their Caputo fractional derivatives are not applicable in some cases when one studies stability properties. Some sufficient conditions for stability of equilibrium of nonlinear Caputo fractional neural networks with time dependent transmission delays, time varying self-regulating parameters of all units and time varying functions of the connection between two neurons in the network are obtained. The cases of time varying Lipschitz coefficients as well as nonLipschitz activation functions are studied. We illustrate our theory on particular nonlinear Caputo fractional neural networks.

1. Introduction

Neural networks have a wide range of applications in pattern recognition, associative memory, combinatorial optimization, etc. Delays are incorporated into the model equations of the networks because of the finite speeds of the switching and transmission of signals in a network and this leads to time delays in a working network. It was observed both experimentally and numerically in [1] that time delay could induce instability, causing sustained oscillations which may be harmful to a system. Additionally, when one considers memory and hereditary properties of various materials and processes ([2]), it is natural to consider fractional derivatives in the model. Neural networks in biology, coupled lasers, wireless communication and power-grid networks in physics and engineering ([3,4,5,6]) are modeled by fractional order differential equations.

In controlling nonlinear systems, the Lyapunov second method provides a way to analyze the stability of the system without explicitly solving the differential equations. Stability results concerning integer-order neural networks can be found in [7,8,9] and recently Lyapunov stability theory for fractional order systems was discussed (see [10,11]). Fractional order Lyapunov stability theory was applied for various types of fractional neural networks using quadratic Lyapunov functions (see [2,12,13,14]) and stability analysis of fractional-order delay neural networks can be found in [15,16,17,18,19], for example. Finite-time stability of fractional-order neural networks with constant transmission delay and constant self-regulating parameters is studied in [20,21] and for distributed delays and constant self-regulating parameters see [22].

The main goal of this paper is to study stability properties of Caputo fractional order neural networks with delays. We study the general case of time varying transmission delays as well as time varying self-regulating parameters of all units and time varying functions of the connection between two neurons in the network. Note that usually in the literature the activation functions in the models are Lipschitz with constant Lipschitz coefficients. In this paper, the cases of time varying Lipschitz coefficients as well as nonLipschitz activation functions are studied. The investigation is based on the fractional Lyapunov method. It is worth pointing out that the Lyapunov functional method can not be easily generalized to fractional order systems. It is mainly connected with the derivative taken in the given fractional system. Taking these factors into consideration, we present a brief overview of the existing literature on the various definitions of fractional order derivatives of Lyapunov functions among Caputo fractional differential equations with variable delays and we compare them in several examples, discussing their advantages and disadvantages. Using the appropriate fractional derivative of Lyapunov functions, some stability sufficient criteria are provided and illustrated with examples.

2. Lyapunov Functions and Their Derivatives among the Delay Fractional Differential Equation

We first consider the derivative of Lyapunov functions among the delay fractional differential equation.

In this paper we consider Caputo fractional derivatives with order $q\in (0,1)$ defined by (see, for example [23])

$${}_{t_0}^C{D}_t^{q}m\left(t\right)=\frac{1}{\mathsf{\Gamma }\left(1-q\right)}\underset{t_0}{\overset{t}{\int }}{\left(t-s\right)}^{-q}{m}^{\prime }\left(s\right)ds,t\ge t_0,$$

where $\mathsf{\Gamma }(.)$ denotes the Gamma function.

Consider the initial value problem (IVP) for the nonlinear Caputo fractional delay differential equations (FrDDE)

$${}_{t_0}^C{D}_t^{q}x\left(t\right)=f(t,x_t)\mathrm{for}t\in [t_0,t_0+T),x(t_0+\mathsf{\Theta })={\psi }_0\left(\mathsf{\Theta }\right),\mathsf{\Theta }\in [-r,0],$$

(1)

where $x\in {\mathbb{R}}^n,f\in C[[t_0,t_0+T)\times {\mathbb{R}}^n,{\mathbb{R}}^n]$, $t_0\in {\mathbb{R}}_{+}$ is the initial time, ${\psi }_0\in C([-r,0],{\mathbb{R}}^n)$ is the initial function, $T\le \infty$ and for any $t\in [t_0,t_0+T)$ the notation $x_t\left(\mathsf{\Theta }\right)=x(t+\mathsf{\Theta }),\mathsf{\Theta }\in [-r,0]$ is used.

Let $x\left(t\right),t\in [t_0-r,t_0+T),$ be a solution of the IVP for the FrDDE (1) and let $V(t,x)$ be a Lyapunov function, i.e., $V(t,x):[t_0-r,t_0+T)\times \Delta \to {\mathbb{R}}_{+}$ is continuous and locally Lipschitzian with respect to its second argument, where $\Delta \subset {\mathbb{R}}^n,0\in \Delta$.

In the literature there are three types of derivatives of Lyapunov functions among solutions of fractional differential equations used to study stability properties:

-

first type–Let $x\left(t\right)$ be a solution of IVP for FrDDE (1). Then the Caputo fractional derivative of the Lyapunov function $V(t,x(t\left)\right)$ is defined by

$${}_{t_0}^C{D}_t^{q}V(t,x\left(t\right))=\frac{1}{\mathsf{\Gamma }\left(1-q\right)}\underset{t_0}{\overset{t}{\int }}{\left(t-s\right)}^{-q}\frac{d}{ds}\left(V(s,x(s\left)\right)\right)ds,t\in (t_0,t_0+T).$$

(2)

This type of derivative is applicable for continuously differentiable Lyapunov functions. It is used mainly for quadratic Lyapunov functions to study several stability properties of fractional differential equations (see, for example [10]).

-

second type–this type of derivative of $V(t,x)$ among FrDDE (1) was introduced by I. Stamova, G. Stamov (Definition 1.12 [24]):

$$D_{\left(1\right)}^{+}V(t,\varphi \left(0\right))=\underset{h\to 0}{lim\; sup}\frac{1}{h^q}\left[V(t,\varphi \left(0\right))-V(t-h,x-h^{q}f(t,{\varphi }_0))\right],t\in (t_0,t_0+T)$$

(3)

where $\varphi \in C([-r,0],{\mathbb{R}}^n)$ and the notation ${\varphi }_0\left(\mathsf{\Theta }\right)=\varphi (0+\mathsf{\Theta }),\mathsf{\Theta }\in [-r,0]$ is used.

Note that the operator defined by (3) has no memory (the memory is typical for fractional derivatives) (see Case 2.1 of Example 1 and Case 2.2 of Example 2).

Remark 1.

In general $D_{\left(1\right)}^{+}V(t,x\left(t\right))\ne {}_0^C{D}_t^{q}V(t,x\left(t\right))$ where $x\left(t\right)$ is a solution of (1).

Now, let us recall the remark in [25] concerning definition (3) where $V(t-h,x-h^{q}f(t,x))$ is defined by

$$V(t-h,x-h^{q}f(t,x))=\sum _{r=1}^{\left[\frac{t-t_0}{h}\right]}{(-1)}^{r+1}{}_q{C}_{r}V(t-rh,x-h^{q}f(t,x))$$

where ${}_q{C}_r=\frac{q(q-1)\cdots (q-r+1)}{r!}$.

Following this notation the fractional derivative of the Lyapunov function is defined by

$$\begin{array}{c}{\mathcal{D}}_{\left(1\right)}^{+}V(t,\varphi \left(0\right);t_0)\hfill \\ =\underset{h\to 0}{lim\; sup}\frac{1}{h^q}\left[V(t,\varphi \left(0\right))-\sum _{r=1}^{\left[\frac{t-t_0}{h}\right]}{(-1)}^{r+1}{}_q{C}_{r}V(t-rh,\varphi \left(0\right)-h^{q}f(t,{\varphi }_0))\right].\hfill \end{array}$$

(4)

The derivative (4) has memory and it depends on the initial time $t_0$. It is closer to both the Grunwald-Letnikov fractional derivative and the Riemann-Liouville fractional derivative, than to the Caputo fractional derivative of a function. We will call the derivative (4) the Dini fractional derivative of the Lyapunov function. The Dini fractional derivative is applicable for continuous Lyapunov functions.

Remark 2.

In the general case ${\mathcal{D}}_{\left(1\right)}^{+}V(t,\varphi \left(0\right))\ne D_{\left(1\right)}^{+}V(t,\varphi \left(0\right))$.

-

third type–for any $\varphi \in C([-r,0],{\mathbb{R}}^n)$ the derivative of the Lyapunov function $V(t,x)$ among IVP for FrDDE (1) with initial point $t_0$ and initial function ${\psi }_0\in C([-r,0],{\mathbb{R}}^n)$ is defined by:

$$\begin{array}{c}{}_{\left(1\right)}^c{D}_{+}^{q}V(t,\varphi \left(0\right);t_0,{\psi }_0)=\underset{h\to 0^{+}}{lim\; sup}\frac{1}{h^q}\{V(t,\varphi \left(0\right))-V(t_0,{\psi }_0\left(0\right))\hfill \\ -\sum _{r=1}^{\left[\frac{t-t_0}{h}\right]}{(-1)}^{r+1}{}_q{C}_r\left(V(t-rh,\varphi \left(0\right)-h^{q}f(t,{\varphi }_0))-V(t_0,{\psi }_0\left(0\right))\right)\}\hfill \\ \mathrm{for}t\in (t_0,t_0+T),\hfill \end{array}$$

(5)

or its equivalent

$$\begin{array}{c}{}_{\left(1\right)}^c{D}_{+}^{q}V(t,\varphi \left(0\right);t_0,{\psi }_0)\hfill \\ =\underset{h\to 0^{+}}{lim\; sup}\frac{1}{h^q}\left\{V(t,\varphi \left(0\right))+\sum _{r=1}^{\left[\frac{t-t_0}{h}\right]}{(-1)}^r{}_q{C}_{r}V(t-rh,\varphi \left(0\right)-h^{q}f(t,{\varphi }_0))\right\}\hfill \\ -\frac{V(t_0,{\psi }_0\left(0\right))}{{(t-t_0)}^q\mathsf{\Gamma }(1-q)},\mathrm{for}t\in (t_0,t_0+T).\hfill \end{array}$$

(6)

The derivative (6) depends significantly on both the fractional order q and the initial data $(t_0,{\psi }_0)$ of IVP for FrDDE (1) and this type of derivative is close to the idea of the Caputo fractional derivative of a function.

We call the derivative given by (5) or its equivalent (6) the Caputo fractional Dini derivative. This type of derivative is applicable for continuous Lyapunov functions (see, for example [26,27,28]).

Remark 3.

The equality

$${}_{\left(1\right)}^c{D}_{+}^{q}V(t,\varphi \left(0\right);t_0,{\psi }_0)={\mathcal{D}}_{\left(1\right)}^{+}V(t,\varphi \left(0\right);t_0)-\frac{V(t_0,{\psi }_0\left(0\right))}{{(t-t_0)}^q\mathsf{\Gamma }(1-q)}$$

(7)

holds for any $t\in (t_0,t_0+T),{\varphi }_0\in C([-r,0],{\mathbb{R}}^n)$ and for any initial data $(t_0,{\psi }_0)\in {\mathbb{R}}_{+}\times C([-r,0],{\mathbb{R}}^n)$ :

$${}_{\left(1\right)}^c{D}_{+}^{q}V(t,\varphi \left(0\right);t_0,{\psi }_0)={\mathcal{D}}_{\left(1\right)}^{+}V(t,\varphi \left(0\right);t_0),ifV(t_0,{\psi }_0\left(0\right))=0$$

(8)

$${}_{\left(1\right)}^c{D}_{+}^{q}V(t,\varphi \left(0\right);t_0,{\psi }_0)<{\mathcal{D}}_{\left(1\right)}^{+}V(t,\varphi \left(0\right);t_0),ifV(t_0,{\psi }_0\left(0\right))>0.$$

(9)

Remark 4.

Note that in the case of delays in the differential equations, the derivative of the Lyapunov function is considered for functions $\varphi \in C([-r,0],{\mathbb{R}}^n)$ and points t such that $V(t,\varphi \left(0\right))={sup}_{s\in [-r,0]}V(t+s,\varphi \left(s\right))$. This condition is called the Razumikhin condition.

We will illustrate the application of the above given derivatives on the quadratic Lyapunov function (used in stability of neural networks [2,12,13,14]).

Example 1.

Let $n=1$. Consider the quadratic Lyapunov function i.e., $V(t,x)=x^2$.

Case 1. First type of derivative. Let $x\left(t\right)$ be a solution of the IVP for FrDDE (1). According to [29] we get

$${}_{t_0}^C{D}_t^{q}V(t,x\left(t\right))=\frac{2}{\mathsf{\Gamma }\left(1-q\right)}\underset{t_0}{\overset{t}{\int }}{\left(t-s\right)}^{-q}x\left(s\right)x^{\prime }\left(s\right)ds\le 2x\left(t\right){}_{t_0}^C{D}_t^{q}x\left(t\right)=2x\left(t\right)f(t,x_t).$$

(10)

Case 2. Second type of derivative.

Case 2.1: Let the function $\varphi \in C([-r,0],{\mathbb{R}}^n)$. From Formula (3) we obtain

$$D_{\left(1\right)}^{+}V(t,\varphi \left(0\right))=\underset{h\to 0}{lim\; sup}\frac{1}{h^q}\left[{\left(\varphi \left(0\right)\right)}^2-{\left(\varphi \left(0\right)-h^{q}f(t,{\varphi }_0)\right)}^2\right]=2\varphi \left(0\right)f(t,{\varphi }_0).$$

(11)

It is seen that the derivative $D_{\left(1\right)}^{+}V(t,\varphi \left(0\right))$ does not depend on any initial data of the fractional equation.

Case 2.2: Dini fractional derivative. Let the function $\varphi \in C([-r,0],{\mathbb{R}}^n)$. From Formula (4) we obtain

$$\begin{array}{c}{\mathcal{D}}_{\left(1\right)}^{+}V(t,\varphi \left(0\right);t_0)\hfill \\ =\underset{h\to 0}{lim\; sup}\frac{1}{h^q}\left[{\left(\varphi \left(0\right)\right)}^2-\sum _{r=1}^{\left[\frac{t-t_0}{h}\right]}{(-1)}^{r+1}{}_q{C}_r{\left(\varphi \left(0\right)-h^{q}f(t,{\varphi }_0)\right)}^2\right]\hfill \\ =\underset{h\to 0}{lim\; sup}\frac{1}{h^q}\left[{\left(\varphi \left(0\right)\right)}^2+\sum _{r=1}^{\left[\frac{t-t_0}{h}\right]}{(-1)}^r{}_q{C}_r{\left(\varphi \left(0\right)-h^{q}f(t,{\varphi }_0)\right)}^2\right]\hfill \\ ={\left(\varphi \left(0\right)\right)}^2\underset{h\to 0}{lim\; sup}\frac{1}{h^q}\sum _{r=0}^{\left[\frac{t-t_0}{h}\right]}{(-1)}^r{}_q{C}_r\hfill \\ +\underset{h\to 0}{lim\; sup}\left(-2\varphi \left(0\right)f(t,{\varphi }_0)+h^q{f}^2(t,{\varphi }_0)\right)\sum _{r=1}^{\left[\frac{t-t_0}{h}\right]}{(-1)}^r{}_q{C}_r]\hfill \\ =2\varphi \left(0\right)f(t,{\varphi }_0)+\frac{{\left(\varphi \left(0\right)\right)}^2}{{(t-t_0)}^q\mathsf{\Gamma }(1-q)}\hfill \end{array}$$

(12)

The Dini fractional derivative depends on both the fractional order q and initial time.

Case 3. Third type of derivative. Caputo fractional Dini derivative. According to Remark 4 and Case 2.2 the inequality

$${}_{\left(1\right)}^c{D}_{+}^{q}V(t,\varphi \left(0\right);0,{\psi }_0)=2\varphi \left(0\right)f(t,{\varphi }_0)+\frac{{\left(\varphi \left(0\right)\right)}^2}{{(t-t_0)}^q\mathsf{\Gamma }(1-q)}-\frac{{\left({\psi }_0\left(0\right)\right)}^2}{{(t-t_0)}^q\mathsf{\Gamma }(1-q)}$$

(13)

holds.

The Caputo fractional Dini derivative depends on both the fractional order q and initial data $(t_0,{\psi }_0)$.

From the literature we note that one of the sufficient conditions for stability is connected with the sign of the derivative of the Lyapunov function of the equation.

Example 2.

Consider the IVP for the scalar linear FrDDE

$${}_0^C{D}_t^{q}x\left(t\right)=g\left(t\right)x\left(t\right)+0.1x(t-\pi )fort>0,x\left(s\right)=1fors\in [-\pi ,0],$$

(14)

where $q\in (0,1)$, $g\left(t\right)=-0.5\frac{{}_0^{RL}{D}^q\left({sin}^2\left(t\right)+0.1\right)}{si{n}^2\left(t\right)+0.1}-0.1,t>0$.

Case 1. Consider the quadratic Lyapunov function [12,13,14]) i.e., $V(t,x)=x^2$. Let $x\left(t\right)$ be a solution of IVP for FrDDE (14) and $t>0$ be such that $V(t,x\left(t\right))=x^2\left(t\right)={sup}_{s\in [-\pi ,t]}{x}^2\left(s\right)={sup}_{s\in [-\pi ,t]}V(s,x\left(s\right))$. Then according to [29] we get

$${}_0^C{D}_t^{q}V(t,x\left(t\right))\le 2x\left(t\right){}_0^C{D}_t^{q}x\left(t\right)=2{\left(x\left(t\right)\right)}^{2}g\left(t\right)+0.2x\left(t\right)x(t-\pi )\le 2x^2\left(t\right)(0.1+g\left(t\right)).$$

The sign of ${}_0^C{D}_t^{q}V(t,x\left(t\right))$ is changeable (the graph of the function $g\left(t\right)+0.1$ for various values of q is given on Figure 1).

Case 2. Consider the function $V(t,x)=({sin}^2\left(t\right)+0.1)x^2$.

Case 2.1: Caputo fractional derivative. Let $x\left(t\right)$ be a solution of IVP for FrDDE (14). From Equation (2) we obtain

$${}_0^C{D}_t^{q}V(t,x\left(t\right))=\frac{1}{\mathsf{\Gamma }\left(1-q\right)}\underset{0}{\overset{t}{\int }}{\left(t-s\right)}^{-q}\left(2({sin}^2\left(s\right)+0.1)x\left(s\right)x^{\prime }\left(s\right)+sin\left(2s\right)x^2\left(s\right)\right)ds.$$

The fractional derivative of this function V is difficult to obtain so it is difficult to discuss its sign.

Case 2.2: Use Formula (3) and for any function $\varphi \in C\left(\right[-\pi ,0],\mathbb{R})$ and any point $t>0$ such that $V(t,\varphi \left(0\right))=({sin}^2\left(t\right)+0.1){\varphi }^2\left(0\right)={sup}_{s\in [-\pi ,0]}({sin}^2(t+s)+0.1){\varphi }^2\left(s\right)={sup}_{s\in [-\pi ,0]}V(t+s,\varphi \left(s\right))$ we obtain

$$\begin{array}{c}{D}_{\left(14\right)}^{+}V(t,\varphi \left(0\right))\hfill \\ =\underset{h\to 0}{lim\; sup}\frac{1}{h^q}[\left({sin}^2\left(t\right)-{sin}^2(t-h)\right){\varphi }^2\left(0\right)\hfill \\ -2h^q({sin}^2(t-h)+0.1)\varphi \left(0\right)\left(\varphi \left(0\right)g\left(t\right)+0.1\varphi (-1)\right)\hfill \\ +h^{2q}({sin}^2(t-h)+0.1){\left(\varphi \left(0\right)g\left(t\right)+0.1\varphi (-\pi )\right)}^2]\hfill \\ =2\varphi \left(0\right)(si{n}^2\left(t\right)+0.1)(\varphi \left(0\right)g\left(t\right)+0.1\varphi (-\pi ))\hfill \\ \le 2{\varphi }^2\left(0\right)(si{n}^2\left(t\right)+0.1)g\left(t\right)+0.2\varphi \left(0\right)\varphi (-\pi )(si{n}^2\left(t\right)+0.1)\hfill \\ \le 2{\varphi }^2\left(0\right)(si{n}^2\left(t\right)+0.1)g\left(t\right)+0.1{\varphi }^2\left(0\right)(si{n}^2\left(t\right)+0.1)+0.1{\varphi }^2(-\pi )(si{n}^2(t-\pi )+0.1)\hfill \\ \le 2{\varphi }^2\left(0\right)(si{n}^2\left(t\right)+0.1)(g\left(t\right)+0.1)\hfill \end{array}$$

i.e., the sign of the derivative $D_{\left(14\right)}^{+}V(t,\varphi )$ is changeable.

Case 2.3: Dini fractional derivative. For any function $\varphi \in C\left(\right[-\pi ,0],\mathbb{R})$ and $t>0$ such that $V(t,\varphi \left(0\right))=({sin}^2\left(t\right)+0.1){\varphi }^2\left(0\right)={sup}_{s\in [-\pi ,0]}({sin}^2(t+s)+0.1){\varphi }^2\left(s\right)={sup}_{s\in [-\pi ,0]}V(t+s,\varphi \left(s\right))$ we apply Formula (4) and obtain

$$\begin{array}{c}{\mathcal{D}}_{\left(14\right)}^{+}V(t,\varphi \left(0\right);0)\hfill \\ =\underset{h\to 0}{lim\; sup}\frac{1}{h^q}[({sin}^2\left(t\right)+0.1){\varphi }^2\left(0\right)\hfill \\ -\sum _{r=1}^{\left[\frac{t}{h}\right]}{(-1)}^{r+1}{}_q{C}_r({sin}^2(t-rh)+0.1){\left(\varphi \left(0\right)-h^q\left(\varphi \left(0\right)g\left(t\right)+0.1\varphi (-\pi )\right)\right)}^2]\hfill \\ ={\varphi }^2\left(0\right)\underset{h\to 0}{lim\; sup}\frac{1}{h^q}\sum _{r=1}^{\left[\frac{t}{h}\right]}{(-1)}^r({sin}^2(t-rh)+0.1)\hfill \\ -2\underset{h\to 0}{lim\; sup}\sum _{r=1}^{\left[\frac{t}{h}\right]}{(-1)}^r{}_q{C}_r({sin}^2(t-rh)+0.1)\left({\varphi }^2\left(0\right)g\left(t\right)+0.1\varphi \left(0\right)\varphi (-\pi )\right)\hfill \\ +\underset{h\to 0}{lim\; sup}{h}^q\sum _{r=1}^{\left[\frac{t}{h}\right]}{(-1)}^{r+1}{}_q{C}_r({sin}^2(t-rh)+0.1){\left(\varphi \left(0\right)g\left(t\right)+0.1\varphi (-\pi )\right)}^2\hfill \\ \le 2{\varphi }^2\left(0\right)(g\left(t\right)+0.1)({sin}^2\left(t\right)+0.1)+{\varphi }^2\left(0\right){}_0^{RL}{D}^q\left({sin}^2\left(t\right)+0.1\right)=0.\hfill \end{array}$$

Case 2.4: Caputo fractional Dini derivative. According to Remark 4 and Case 2.3 the inequality

$${}_{\left(14\right)}^c{D}_{+}^{q}V(t,\varphi \left(0\right);0,{\psi }_0)={\mathcal{D}}_{\left(14\right)}^{+}V(t,\varphi \left(0\right);0)-\frac{0.1{\psi }_0^2\left(0\right)}{t^q\mathsf{\Gamma }(1-q)}=-\frac{0.1{\psi }_0^2\left(0\right)}{t^q\mathsf{\Gamma }(1-q)}\le 0$$

holds.

Therefore, for (14) both the Dini fractional derivative and the Caputo fractional Dini derivative seem to be more applicable than the Caputo fractional derivative of Lyapunov function.

Remark 5.

The above example notes that the quadratic function for studying stability properties of neural network might not be successful (especially when the right hand side depends directly on the time variable). Formula (3) is not appropriate for applications to fractional equations. The most general derivatives for non-homogenous fractional differential equations are Dini fractional derivatives and Caputo fractional Dini derivatives.

3. Remarks on Some Applications of Derivatives of Lyapunov Functions

Some authors use the derivative defined by (3) to study stability properties of delay fractional differential equations ([24,30]) and delayed reaction-diffusion cellular neural networks of fractional order ([31]). The proofs are based on the following result:

Lemma 1.

([24]) Assume that:

1.

The function $g:[t_0,\infty )\times {\mathbb{R}}_{+}\to \mathbb{R}$ is continuous in $[t_0,\infty )\times {\mathbb{R}}_{+}$ and is nondecreasing in u for each $t\ge t_0$.

2.

The maximal solution $u^{+}\left(t\right)$ of the IVP for scalar Caputo fractional differential equation ${}_{t_0}^c{D}_t^{q}u\left(t\right)=g(t,u\left(t\right)),t>t_0,u\left(t_0\right)=1$ exists on $[t_0,\infty )$.

3.

The Lipschitz function $V(t,x)$ is such that for $t\ge t_0,\varphi \in C([-r,0],\Delta )$ the inequality

$$D_{\left(1\right)}^{+}V(t,\varphi \left(0\right))\le g(t;V(t,\varphi \left(0\right)))$$

is valid whenever $V(t+\mathsf{\Theta },\varphi (\mathsf{\Theta }\left)\right)\le V(t,\varphi (0\left)\right)$ for $-r\le \mathsf{\Theta }\le 0$ where $\Delta \subset {\mathbb{R}}^n,0\in \Delta$.

Then ${max}_{-r\le \mathsf{\Theta }\le 0}V(t_0+\mathsf{\Theta })\le u_0$ implies $V(t,x(t;t_0,{\varphi }_0))\le u^{+}(t;t_0,u_0),t\in [t_0,\infty )$, where $x(t;t_0,{\varphi }_0)$ is the solution of (1) with initial data $(t_0,{\psi }_0)\in {\mathbb{R}}_{+}\times C[[-r,0],\Delta ]$.

The claim of Lemma 1 is not true. We will give a counterexample.

Example 3.

Consider the IVP for FrDDE (1) with $n=1$, $t_0=0$, $r=0$, $g(t,u)=h\left(t\right)$, $f(t,x_t)=0.5\frac{h\left(t\right)}{x\left(t\right)}$, ${\psi }_0\left(t\right)\equiv 1$, and $h\left(t\right)=-\frac{t^{2-q}{}_p{F}_q[\left\{1\right\},\{3/2-q/2,2-q/2\},-(t^2/4)]}{\mathsf{\Gamma }(1-q)(1-q)(2-q)}$ where ${}_p{F}_q[\left\{a\right\},\{b,c\},z]={\sum }_{n=0}^{\infty }\frac{{\left(a\right)}_n}{{\left(b\right)}_n{\left(c\right)}_n}\frac{z^n}{n!}$ with ${\left(a\right)}_n=a(a+1)(a+2)\cdots (a+n-1)$ is the general hypergeometric function or Aomoto-Gelfand hypergeometric function [32]. We denote its solution by $x\left(t\right)$.

Consider the Lyapunov function $V(t,x)=x^2$. According to Example 1, Case 2.1, Equation (11) we get $D_{\left(1\right)}^{+}V(t,\varphi \left(0\right))=2\varphi \left(0\right)f(t,{\varphi }_0)=h\left(t\right)$.

The function $u\left(t\right)=cos\left(t\right)$ is a solution of the IVP for the comparison scalar fractional differential equation

$${}_0^c{D}_t^{q}u\left(t\right)=h\left(t\right),t>0,u\left(0\right)=1$$

Then $V(0,x(0\left)\right)=1$ but the inequality

$$0\le V(t,x\left(t\right))={\left(x\left(t\right)\right)}^2\le cos\left(t\right)=u\left(t\right)$$

does not hold for $t\ge 0$.

The application of the derivative $D_{\left(1\right)}^{+}V(t,\varphi \left(0\right))$ defined by (3) is not appropriate in studying stability properties of neural networks of fractional order (as presented in [31]). We will consider fractional neural networks with delays and using Lyapunov functions and their Caputo fractional derivative, Dini fractional derivative or Caputo fractional Dini derivative (and without applying Lemma 1) we study stability.

4. Stability for Caputo Fractional Order Neural Networks with Time Varying Delays

4.1. System Description

Consider the general model of fractional order neural networks with time varying delay (FODNN)

$$\begin{array}{c}{}_0^C{D}_t^q{x}_i\left(t\right)=-c_i\left(t\right)x_i\left(t\right)+\sum _{j=1}^n{a}_{ij}\left(t\right)f_j\left(x_j\left(t\right)\right)+\sum _{j=1}^n{b}_{ij}\left(t\right)g_j\left(x_j(t-\tau \left(t\right))\right)+I_i\left(t\right)\hfill \\ \mathrm{for}t>0,i=1,2,\cdots n,\hfill \end{array}$$

(15)

where $q\in (0,1)$, n represents the number of units in the network, $x_i\left(t\right)$ is the pseudostate variable of the i-th unit of master system, $x\left(t\right)={[x_1\left(t\right),x_2\left(t\right),\cdots ,x_n\left(t\right)]}^T\in {\mathbb{R}}^n$, $c_i\left(t\right)>0,i=1,2,\cdots ,n,$ is the self-regulating parameters of the i-th unit, , $a_{ij}\left(t\right),b_{ij}\left(t\right),i,j=1,2,\cdots ,n,$ correspond to the connection of the i-th neuron to the j-th neuron at times t and $t-\tau \left(t\right)$ respectively, $f_j\left(x_j\left(t\right)\right)$ and $g_j\left(x_j(t-\tau \left(t\right))\right)$ denote the activation functions of the neurons at time t and $t-\tau \left(t\right)$, respectively, $f\left(x\right)={[f_1\left(x_1\right),f_2\left(x_2\right),\cdots ,f_n\left(x_n\right)]}^T$, $g\left(x\right)={[g_1\left(x_1\right),g_2\left(x_2\right),\cdots ,g_n\left(x_n\right)]}^T$ and $I={[I_1,I_2,\cdots ,I_n]}^T$ is an external bias vector, $\tau \left(t\right)\in C({\mathbb{R}}_{+},[0,r])$ is the transmission delay.

The initial conditions of the given system (15) are listed as

$$x_i\left(t\right)={\psi }_i\left(t\right)\mathrm{for}t\in [-r,0],$$

(16)

where ${\psi }_i\in C(-r,0],\mathbb{R})$, $i=1,2,\cdots ,n,$ with the norm $\left|\right|\psi \left|\right|={sup}_{t\in [-r,0]}{\sum }_{i=1}^n\left|{\psi }_i\left(t\right)\right|$.

Remark 6.

The problem of existence and uniqueness of equilibrium states of fractional-order neural networks was investigated by several authors; for sufficient conditions which guarantee the existence and uniqueness of solutions for FODNN with constant delays we refer the reader to [15].

Definition 1.

A vector $x^{*}\in {\mathbb{R}}^n,x^{*}=(x_1^{*},x_2^{*},\cdots ,x_n^{*})$ is an equilibrium point of Caputo FODNN (15), if the equalities $0=-c_i\left(t\right)x_i^{*}+{\sum }_{j=1}^n{a}_{ij}\left(t\right)f_j\left(x_j^{*}\right)+{\sum }_{j=1}^n{b}_{ij}\left(t\right)g_j\left(x_j^{*}\right)+I_i\left(t\right)$ hold for all $t>0,i=1,2,\cdots ,n$.

We will discuss the equilibrium points on some FODNN with various activation functions. It will be useful for stability analysis.

Example 4.

Consider the system of FODNN (15) with $n=3$, $c_i\left(t\right)\equiv c_i$, with the activation functions $f_j\left(s\right)=g_j\left(s\right)=0.5tanh\left(s\right)$, and $a_{ij}\left(t\right)\equiv a_{ij},b_{ij}\left(t\right)\equiv b_{ij}$ and $I_i\left(t\right)\equiv I_i$. Then the point $x^{*}=({\xi }_1,{\xi }_2,{\xi }_3)$ is an equilibrium point iff $0=2c_i{\xi }_i+{\sum }_{j=1}^3{a}_{i,j}tanh\left({\xi }_i\right)+{\sum }_{j=1}^3{b}_{i,j}tanh\left({\xi }_i\right)+I_i,i=1,2,3$.

Example 5.

Let $n=1$, c is a constant and consider the scalar FODNN

$${}_0^C{D}_t^{q}x\left(t\right)=-cx\left(t\right)+a\left(t\right)f\left(x\left(t\right)\right)+b\left(t\right)g\left(x(t-r)\right)+Ifort>0.$$

(17)

Case 1. Let $I=0.5\pi c$ and the activation function be the cosine function $f\left(u\right)=g\left(u\right)=cos\left(u\right)$ (see [33]).

The point $x^{*}=0.5\pi$ is an equilibrium point of FODNN(17) because $-c0.5\pi +a\left(t\right)cos\left(0.5\pi \right)+a\left(t\right)cos\left(0.5\pi \right)+0.5\pi c=0$ for all $t>0$.

Case 2. Let $I=0.5c$ and the activation function be the Probit function $f\left(u\right)=g\left(u\right)=\sqrt{2}{erf}^{-1}(2u-1)$ (see Figure 2) where $erf\left(u\right)=\frac{2}{\sqrt{\pi }}{\int }_0^u{e}^{-t^2}dt$ is the error function. Then ${erf}^{-1}\left(0\right)=0$.

The point $x^{*}=0.5$ is an equilibrium point of FODNN(17) because $-c0.5+a\left(t\right)f\left(0.5\right)+b\left(t\right)g\left(0.5\right)+0.5c=0$ for all $t>0$.

Case 3. Let $I=0.5c$ and the activation function be the Logit function $f\left(u\right)=g\left(u\right)=log\left(\frac{u}{1-u}\right)$ (see Figure 3). Then $f\left(0.5\right)=0$.

The point $x^{*}=0.5$ is an equilibrium point of Caputo FODNN (17) because $-c0.5+a\left(t\right)f\left(0.5\right)+b\left(t\right)g\left(0.5\right)+0.5c=0$ for all $t>0$.

Assumption 1.

Let the Caputo FODNN (15) have an equilibrium point $x^{*}$.

If Assumption 1 is satisfied then we can shift the equilibrium point $x^{*}$ of system (15) to the origin. The transformation $y\left(t\right)=x\left(t\right)-x^{*}$ is used to put system (15) in the following form:

$$\begin{array}{c}{}_0^C{D}_t^q{y}_i\left(t\right)=-c_i\left(t\right)y_i\left(t\right)+\sum _{j=1}^n{a}_{ij}\left(t\right)F_j\left(y_j\left(t\right)\right)+\sum _{j=1}^n{b}_{ij}\left(t\right)G_j\left(y_j(t-\tau \left(t\right))\right)\hfill \\ \mathrm{for}t>0,i=1,2,\cdots n,\hfill \end{array}$$

(18)

where $F_j\left(u\right)=f_j(u+x_j^{*})-f_j\left(x_j^{*}\right),G_j\left(u\right)=g_j(u+x_j^{*})-g_j\left(x_j^{*}\right),u\in \mathbb{R},j=1,2,\cdots ,n$.

We will give the Dini fractional derivative of a Lyapunov function depending directly on the time t among the FODNN (18) which will be used in the proofs of some of the main results.

Example 6.

Consider the IVP for FODNN (18).

Consider the function $V(t,x)=p\left(t\right){\sum }_{i=1}^n{x}_i^2$ where $p\in C([-r,\infty ),{\mathbb{R}}_{+}:\alpha \le p\left(t\right)\le \beta$ for $t\ge -r$ where $\alpha ,\beta >0$ are constants. Let $\varphi \in C([-r,0],{\mathbb{R}}^n)$ and $t>0$ be such that $V(t,\varphi \left(0\right))=p\left(t\right){\sum }_{i=1}^n{\varphi }_i^2\left(0\right)={sup}_{s\in [-r,0]}p(t+s){\sum }_{i=1}^n{\varphi }_i^2\left(s\right)={sup}_{s\in [-r,0]}V(t+s,\varphi \left(s\right))$. Then from Formula (4) we obtain for the Dini fractional derivative:

$$\begin{array}{c}{\mathcal{D}}_{\left(18\right)}^{+}V(t,\varphi \left(0\right);0)\hfill \\ =\sum _{i=1}^n{\varphi }_i^2\left(0\right)\underset{h\to 0}{lim\; sup}\frac{1}{h^q}\sum _{r=1}^{\left[\frac{t}{h}\right]}{(-1)}^{r}p(t-rh)\hfill \\ +2\sum _{i=1}^n{c}_i\left(t\right){\varphi }_i^2\left(0\right)\underset{h\to 0}{lim\; sup}\sum _{r=1}^{\left[\frac{t}{h}\right]}{(-1)}^r{}_q{C}_{r}p(t-rh)\hfill \\ -2\left(\sum _{i=1}^n{\varphi }_i\left(0\right)\sum _{j=1}^n{a}_{ij}\left(t\right)F_j\left({\varphi }_j\left(0\right)\right)\right)\underset{h\to 0}{lim\; sup}\sum _{r=1}^{\left[\frac{t}{h}\right]}{(-1)}^r{}_q{C}_{r}p(t-rh)\hfill \\ -2\left(\sum _{i=1}^n{\varphi }_i\left(0\right)\sum _{j=1}^n{b}_{ij}\left(t\right)G_j({\varphi }_j(-\tau \left(t\right))\right)\underset{h\to 0}{lim\; sup}\sum _{r=1}^{\left[\frac{t}{h}\right]}{(-1)}^r{}_q{C}_{r}p(t-rh)\hfill \\ +\underset{h\to 0}{lim\; sup}{h}^q\sum _{r=1}^{\left[\frac{t}{h}\right]}{(-1)}^{r+1}{}_q{C}_{r}p(t-rh)\sum _{i=1}^n(-c_i\left(t\right){\varphi }_i\left(0\right)+\sum _{j=1}^n{a}_{ij}\left(t\right)F_j\left({\varphi }_j\left(0\right)\right))\hfill \\ +\sum _{j=1}^n{b}_{ij}\left(t\right)G_j({\varphi }_j(-\tau \left(t\right)){)}^2\hfill \\ \le 2p\left(t\right)\left(-\sum _{i=1}^n{c}_i\left(t\right){\varphi }_i^2\left(0\right)+\sum _{i=1}^n{\varphi }_i\left(0\right)\sum _{j=1}^n\left(a_{ij}\left(t\right)F_j\left({\varphi }_j\left(0\right)\right)+b_{ij}\left(t\right)G_j({\varphi }_j(-\tau \left(t\right))\right)\right)\hfill \\ +\left(\sum _{i=1}^n{\varphi }_i^2\left(0\right)\right){}_0^{RL}{D}^{q}p\left(t\right).\hfill \end{array}$$

4.2. Some Results for Caputo Fractional Differential Equations

We will give some results for fractional derivatives of Lyapunov functions among Caputo fractional differential equations which will be used for our main results concerning the stability of FODNN.

Lemma 2.

([29]). Let $P\in {\mathbb{R}}^{n\times n}$ be constant, symmetric and positive definite matrix and $X\left(t\right):{\mathbb{R}}_{+}\to {\mathbb{R}}^n$ be a function with the Caputo fractional derivative existing.

Then $\frac{1}{2}{}_0^C{D}_t^q\left(x^T\left(t\right)Px\left(t\right)\right)\le x^T\left(t\right)P{}_0^C{D}_t^{q}x\left(t\right),t\ge 0.$

Lemma 3.

(Theorem 3.1 [16]). Let $w_1,w_2\in C(\mathbb{R},{\mathbb{R}}_{+})$ be nondecreasing functions, $w_i\left(s\right)>0,i=1,2$ for $s>0$, $w_i\left(0\right)=0,i=1,2$, $w_2\left(s\right)$ us strictly increasing and there exists a continuously differentiable Lyapunov function $V(t,x)\in {\mathbb{R}}_{+}\times {\mathbb{R}}^n\to {\mathbb{R}}_{+}$ such that

(i) 

$w_1\left(\right|\left|x\right|\left|\right)\le V(t,x)\le w_2\left(\right|\left|x\right|\left|\right),t\ge 0,x\in {\mathbb{R}}^n,$

(ii) 

for any $t>0$ such that ${sup}_{\mathsf{\Theta }\in [-r,t]}V(\mathsf{\Theta },x\left(\mathsf{\Theta }\right))=V(t,x\left(t\right))$ the inequality

$${}_0^C{D}_t^{\beta }V(t,x\left(t\right))\le 0$$

holds where $x\left(t\right)$ is a solution of (1) with $t_0=0$.

Then the equilibrium point of (1) is uniformly stable.

For the Caputo fractional Dini derivative we have:

Lemma 4.

(Theorem 6 [34]). Assume $x=0$ be an equilibrium point for (1) with $T=\infty$ and there exists a Lyapunov function $V(t,x):V(t,0)=0,t\ge 0,$ such that

(i) 

${\alpha }_1\left(\right|\left|x\right|\left|\right)\le V(t,x)\le {\alpha }_2\left(\right|\left|x\right|\left|\right)$ for $t\ge t_0,x\in \Delta ,$

where ${\alpha }_i\in C([0,\infty ),[0,\infty )),i=1,2$ are strictly increasing and ${\alpha }_i\left(0\right)=0$.

(ii) 

for any initial data $(t_0,{\psi }_0)\in {\mathbb{R}}_{+}\times C([-r,0],\Delta )$ and any function $\varphi \in C\left(\right[-r,0],\Delta )$ such that if for a point $t\ge t_0$ we have $V(t+\mathsf{\Theta },\varphi (\mathsf{\Theta }\left)\right)\le V(t,\varphi (0\left)\right)$ for $\mathsf{\Theta }\in [-r,0)$ then the inequality

$${}_{\left(1\right)}^c{D}_{+}^{q}V(t,\varphi \left(0\right);t_0,{\psi }_0)\le 0$$

holds.

Then the equilibrium point of (1) is uniformly stable.

4.3. Stability Analysis

We will study stability properties of several different types of FODNN (1) using different types of Lyapunov functions and their fractional derivatives given in Section 2.

4.3.1. Lipschitz Activation Functions and Quadratic Lyapunov Functions

The stability analysis of the fractional-order neural networks with constant delays and Lipschitz active functions was studied in [35] and the argument was based on topological degree theory, nonsmooth analysis and a nonlinear measure method. We will apply the Lyapunov method to derive some sufficient conditions for stability in the case of variable delays.

We assume the following:

Assumption 2.

The neuron activation functions are Lipschitz, i.e., there exist positive numbers $L_i,H_i,i=1,2,\cdots ,n$ such that $|f_i\left(u\right)-f_i\left(v\right)|\le L_i|u-v|$ and $|g_i\left(u\right)-g_i\left(v\right)|\le H_i|u-v|,i=1,2,\cdots ,n$ for $u,v\in \mathbb{R}$.

Assumption 3.

There exists positive numbers $M_{i,j},C_{i,j},i,j=1,2,\cdots ,n$ such that $|a_{i,j}\left(t\right)|\le M_{i,j}$, $|b_{i,j}\left(t\right)|\le C_{i,j}$ for $t>0$.

Assumption 4.

The inequality

$$2\underset{i=\overline{1,n}}{min}{c}_i>\sum _{i=1}^n(\underset{j}{max}{M}_{ij}{L}_j+\underset{j}{max}{C}_{ij}{H}_j)+\underset{i}{max}(\sum _{j=1}^n(M_{ij}{L}_j+C_{ij}{H}_j))$$

(19)

holds.

Remark 7.

If Assumption 2 is satisfied then the functions $F,G$ in FODNN (18) satisfy $|F_j\left(u\right)|\le L_j\left|u\right|,|G_j\left(u\right)|\le H_j\left|u\right|,j=1,2,\cdots ,n$ for any $u\in \mathbb{R}$.

The case of multiple time constant delays and the functions of the connection of the i-th neuron to the j-th neuron at time t in FODNN (15) are constants studied using the quadratic Lyapunov function in [14]. We will study the case of variable bounded functions of connection.

Theorem 1.

Let $c_j\left(t\right)\equiv c_j>0,j=1,2,\cdots ,n$ and Assumptions 1–4 are satisfied.

Then the equilibrium point $x^{*}$ of Caputo FODNN (15) is uniformly stable.

Proof. 

Consider the quadratic functions $V(t,x)=x^{T}x$. Let the point $t>0$ be such that ${sup}_{\mathsf{\Theta }\in [-r,t]}V(\mathsf{\Theta },x\left(\mathsf{\Theta }\right))={sup}_{\mathsf{\Theta }\in [-r,t]}{\sum }_{j=1}^n{x}_j^2\left(\mathsf{\Theta }\right)={\sum }_{j=1}^n{x}_j^2\left(t\right)=V(t,x\left(t\right))$ where $x\left(t\right)$ is a solution of the FODNN (18). Then since $\tau \left(t\right)\in [0,r],t\ge 0,$ we have ${\sum }_{j=1}^n{\left(x_j\left(t\right)\right)}^2\ge {\sum }_{j=1}^n{\left(x_j(t-\tau \left(t\right))\right)}^2,$ $i=1,2,\cdots ,n$. Applying Lemma 2 we get

$$\begin{array}{cc}\hfill {}_0^C{D}_t^{q}V(t,x\left(t\right))& ={}_0^C{D}_t^q{x}^T\left(t\right)x\left(t\right)\le 2x^T\left(t\right){}_0^C{D}_t^{q}x\left(t\right)=2\sum _{i=1}^n{x}_i\left(t\right){}_0^C{D}_t^q{x}_i\left(t\right)\hfill \\ \hfill & \le -2\sum _{i=1}^n{c}_i{x}_i^2\left(t\right)+2\sum _{i=1}^n\sum _{j=1}^n|a_{ij}\left(t\right)||F_j\left(x_j\left(t\right)\right)|x_i\left(t\right)|\hfill \\ \hfill & +2\sum _{i=1}^n\sum _{j=1}^n|b_{ij}\left(t\right)||G_j\left(x_j(t-\tau \left(t\right))\right)|x_i\left(t\right)|\hfill \\ \hfill & \le -2\underset{i=\overline{1,n}}{min}{c}_i\sum _{i=1}^n{x}_i^2\left(t\right)+\sum _{i=1}^n\sum _{j=1}^n{M}_{ij}{L}_j(x_j^2\left(t\right)+x_i^2\left(t\right))\hfill \\ \hfill & +2\sum _{i=1}^n\sum _{j=1}^n{C}_{ij}{H}_j|x_j(t-\tau \left(t\right))||x_i\left(t\right)|\hfill \\ \hfill & \le -2\underset{i=\overline{1,n}}{min}{c}_i\sum _{i=1}^n{x}_i^2\left(t\right)+\sum _{i=1}^n\sum _{j=1}^n{M}_{ij}{L}_j{x}_j^2\left(t\right)+\sum _{i=1}^n\sum _{j=1}^n{M}_{ij}{L}_j{x}_i^2\left(t\right)\hfill \\ \hfill & +\sum _{i=1}^n\sum _{j=1}^n{C}_{ij}{H}_j{x}_j^2(t-\tau \left(t\right))+\sum _{i=1}^n\sum _{j=1}^n{C}_{ij}{H}_j{x}_i^2\left(t\right)\hfill \\ \hfill & \le -2\underset{i=\overline{1,n}}{min}{c}_i\sum _{i=1}^n{x}_i^2\left(t\right)+\left(\sum _{i=1}^n(\underset{j}{max}{M}_{ij}{L}_j+\underset{j}{max}{C}_{ij}{H}_j)\right)\sum _{j=1}^n{x}_j^2\left(t\right)\hfill \\ \hfill & +\sum _{i=1}^n{x}_i^2\left(t\right)\left(\underset{i}{max}(\sum _{j=1}^n(M_{ij}{L}_j+C_{ij}{H}_j)\right)\le 0.\hfill \end{array}$$

(20)

Inequality (20) and Lemma 3 prove the claim. ☐

Example 7.

Consider the system of FODNN (15) with $n=3$, $c_i\left(t\right)\equiv c_i$, with the activation functions $f_j\left(s\right)=g_j\left(s\right)=0.5tanh\left(s\right)$, the delay $\tau \left(t\right)\equiv 1$ and $|a_{ij}\left(t\right)|\le M_{ij},|b_{ij}\left(t\right)|\le C_{ij},i,j=1,2,3,t\ge 0$ where $M=\left\{M_{ij}\right\},C=\left\{C_{ij}\right\}$ are given by

$$M=\left(\begin{array}{ccc}-0.1& 0.5& 0.3\\ -0.2& 0.3& 0.2\\ 0.4& -0.2& -0.1\end{array}\right)C=\left(\begin{array}{ccc}0.1& -0.1& -0.2\\ 0.3& 0.2& -0.1\\ -0.2& 0.5& 0.3\end{array}\right)$$

(21)

The point $x^{*}=(0,0,0)$ is an equilibrium point of Caputo FODNN (15) if $I_i\left(t\right)\equiv 0,i=1,2,3$ (see Example 4). Then $L_i=H_i=0.5$ and ${\sum }_{i=1}^3({max}_j{M}_{ij}{L}_j+{max}_j{C}_{ij}{H}_j)=(0.25+0.1)+(0.15+0.15)+(0.2+0.25)=1.1$, ${max}_i({\sum }_{j=1}^3(M_{ij}{L}_j+C_{ij}{H}_j)=0.85$.

Therefore, if $c_i>\frac{1.95}{2}=0.925,i=1,2,3$ then according to Theorem 1 the zero equilibrium is uniformly stable.

4.3.2. Non-Lipschitz Activation Functions and Quadratic Lyapunov Functions

There are many types of activation functions which are not Lipschitz (see Example 2, Cases 2 and 3). In this case we assume:

Assumption 5.

There exists a function $\xi \in C({\mathbb{R}}_{+},\mathbb{R})$ such that for any solution $x\left(t\right)$ of the FODNN (18) and any point $t>0:{\sum }_{j=1}^n{x}_j^2\left(t\right)={sup}_{s\in [-r,0]}{\sum }_{j=1}^n{x}_j^2\left(s\right)$ and $i=1,2,\cdots ,n$ the inequality

$$\begin{array}{c}\hfill x_i\left(t\right)\left(\sum _{j=1}^n{a}_{i,j}\left(t\right)(f_j(x_j\left(t\right)+x_j^{*})-f_j\left(x_j^{*}\right))+\sum _{j=1}^n{b}_{i,j}\left(t\right)(g_j(x_j(t-\tau \left(t\right))+x_j^{*})-g_j\left(x_j^{*}\right))\right)\hfill \\ \hfill \le \xi \left(t\right)x_i^2\left(t\right)\hfill \end{array}$$

(22)

holds.

Assumption 6.

There exists a function $\eta \in C({\mathbb{R}}_{+},(0,\infty ))$ such that

$$c_i\left(t\right)\ge \eta \left(t\right),i=1,2,\cdots ,n,t\ge 0.$$

(23)

Theorem 2.

Let Assumptions 1, 5 and 6 be satisfied with $\xi \left(t\right)\le \eta \left(t\right),t\ge 0$.

Then the equilibrium point $x^{*}$ of Caputo FODNN (15) is uniformly stable.

Proof. 

Consider the quadratic functions $V(t,x)=x^{T}x$ . Let $x\left(t\right)$ be a solution of the FODNN (18). Let $t>0$ be such that ${sup}_{\mathsf{\Theta }\in [-r,t]}V(\mathsf{\Theta },x\left(\mathsf{\Theta }\right))={sup}_{\mathsf{\Theta }\in [-r,t]}{\sum }_{j=1}^n{x}_j^2\left(\mathsf{\Theta }\right)={\sum }_{j=1}^n{x}_j^2\left(t\right)=V(t,x\left(t\right))$. Then applying Lemma 2, we get for the Caputo fractional derivative

$$\begin{array}{c}{}_0^C{D}_t^{q}V(t,x\left(t\right))\hfill \\ \le -2\sum _{i=1}^n\left(c_i\left(t\right)x_i^2\left(t\right)-x_i\left(t\right)\sum _{j=1}^n{a}_{i,j}\left(t\right)F_j\left(x_j\left(t\right)\right)-x_i\left(t\right)\sum _{j=1}^n{b}_{i,j}\left(t\right)G_j\left(x_j(t-\tau \left(t\right))\right)\right)\hfill \\ \le -2\left(\eta \left(t\right)-\xi \left(t\right)\right)\sum _{i=1}^n{x}_i^2\left(t\right).\hfill \end{array}$$

(24)

From inequality (24) and Lemma 3 the claim follows. ☐

Example 8.

Let $n=1$ and consider the scalar nonlinear FODNN

$${}_0^C{D}_t^{q}x\left(t\right)=-cx\left(t\right)+a\left(t\right)f\left(x\left(t\right)\right)+b\left(t\right)f\left(x(t-r)\right)+Ifort>0,$$

(25)

where $q\in (0,1)$, $r=const>0$, $I=0.5c,c>0$, the functions $a\in C\left(\right[0,\infty ),(\infty ,0\left]\right),b\in C\left(\right[0,\infty ),[0,\infty \left)\right):a\left(t\right)+b\left(t\right)\le \mu \left(t\right)\le 0$ and the activation function $f\left(t\right)$ is the Probit function or the Logit function (see Example 6). Then the Equation (25) has an equilibrium point $x^{*}=0.5$ (see Example 5). Neither of the activation functions are Lipschitz and Theorem 1 cannot be applied.

Let $t>0$ be such that ${sup}_{\mathsf{\Theta }\in [-r,t]}|x\left(\mathsf{\Theta }\right)|=|x\left(t\right)|$.

Consider the following possible cases:

-

Let $x\left(t\right)<0$ and $x(t-r)<-0.5<0$. Since $r>0$ we obtain $x(t-r)\ge x\left(t\right)$ and from the monotonicity property of the function f we get $f\left(x\right(t-r)+0.5)\ge f\left(x\right(t)+0.5)$ and $b\left(t\right)f\left(x\right(t-r)+0.5)x\left(t\right)\le b\left(t\right)f\left(x\right(t)+0.5)x\left(t\right)$. Then applying $xf(x+0.5)\ge x^2\ge 0,x\in \mathbb{R}$ we get

$$\begin{gathered} a\left(t\right)f(xu\left(t\right)+0.5)x\left(t\right)+b\left(t\right)f(x(t-r)+0.5)x\left(t\right) \\ \le \left(a\left(t\right)+b\left(t\right)\right)f(x\left(t\right)+0.5)x\left(t\right)\le \mu \left(t\right)x^2\left(t\right). \end{gathered}$$

-

Let $x\left(t\right)<0$ and $x(t-r)>-0.5$. Then $f\left(x\right(t-r)+0.5)\ge 0$ and applying $xf(x+0.5)\ge x^2\ge 0,x\in \mathbb{R}$ we obtain

$$\begin{gathered} a\left(t\right)f(x\left(t\right)+0.5)x\left(t\right)+b\left(t\right)f(x(t-r)+0.5)x\left(t\right) \\ \le a\left(t\right)f(x\left(t\right)+0.5)x\left(t\right)\le a\left(t\right)x^2\left(t\right)\le \mu \left(t\right)x^2\left(t\right). \end{gathered}$$

-

Let $x\left(t\right)>0$. Then $|x(t-r)|\le x\left(t\right)$ and from monotonicity property of the function f it follows $f\left(x\right(t-r)+0.5)\le f(|x(t-r)|+0.5)\le f\left(x\right(t)+0.5)$. Therefore,

$$\begin{gathered} a\left(t\right)f(x\left(t\right)+0.5)x\left(t\right)+b\left(t\right)f(x(t-r)+0.5)x\left(t\right) \\ \le \left(a\left(t\right)+b\left(t\right)\right)f(x\left(t\right)+0.5)x\left(t\right)\le \mu \left(t\right)x^2\left(t\right). \end{gathered}$$

Therefore, the Assumption 5 is satisfied. Then, according to Theorem 2 the equilibrium point $x^{*}=0.5$ of FODNN (25) is uniformly stable for all $c>0$.

4.3.3. Non-Lipschitz Activation Functions and Time-Depended Lyapunov Functions

In the case the function $\eta \left(t\right)$ in Assumption 6 not being large enough, we introduce

Assumption 7.

There exists a continuous positive function $p\left(t\right)\in C([0,\infty ),{\mathbb{R}}_{+})$ such that $0<\alpha \le p\left(t\right)\le \beta$ and the fractional derivative ${}_0^{RL}{D}_t^{q}p\left(t\right)$ exists for $t>0$.

In this case, Assumption 5 could be weakened to

Assumption 8.

There exists a function $\xi \in C({\mathbb{R}}_{+},\mathbb{R})$ such that for any function $\varphi \in C([-r,0],{\mathbb{R}}^n)$ and $t>0$ be such that $p\left(t\right){\sum }_{i=1}^n{\varphi }_i^2\left(0\right)={sup}_{s\in [-r,0]}p(t+s){\sum }_{i=1}^n{\varphi }_i^2\left(s\right)$ the inequality

$$\begin{array}{c}\sum _{i=1}^n{\varphi }_i\left(0\right)\sum _{j=1}^n\left(a_{ij}\left(t\right)(f_j({\varphi }_j\left(0\right)+x_j^{*})-f_j\left(x_j^{*}\right))+b_{ij}\left(t\right)\left(\right(g_j({\varphi }_j(-\tau \left(0\right)+x_j^{*})-g_j\left(x_j^{*}\right))\right)\hfill \\ \le \xi \left(t\right)\sum _{i=1}^n{\varphi }_i^2\left(0\right)\hfill \end{array}$$

(26)

holds.

Theorem 3.

Let the Assumptions 1, 6–8 be satisfied and

$$-p\left(t\right)\eta \left(t\right)+p\left(t\right)\xi \left(t\right)+0.5{}_0^{RL}{D}_t^{q}p\left(t\right)\le 0,t>0.$$

(27)

Then the equilibrium point $x^{*}$ of Caputo FODNN (15) is uniformly stable.

Proof. 

In this case the quadratic function $V(t,x)={\sum }_{j=1}^n{x}_i^2,x=(x_1,x_2,\cdots ,x_n)$ does not work.

Consider the function $V(t,x)=p\left(t\right){\sum }_{i=1}^n{x}_i^2,x=(x_1,x_2,\cdots ,x_n)$ where the function $p\left(t\right)$ is defined in Assumption 7. Then, according to Assumption 7 condition (i) of Lemma 4 is satisfied with ${\alpha }_1\left(u\right)=\alpha u$ and ${\alpha }_2\left(u\right)=\beta u$. Let $\varphi \in C([-r,0],{\mathbb{R}}^n)$ and $t>0$ be such that

$$V(t,\varphi \left(0\right))=p\left(t\right)\sum _{i=1}^n{\varphi }_i^2\left(0\right)=\underset{s\in [-r,0]}{sup}p(t+s)\sum _{i=1}^n{\varphi }_i^2\left(s\right)=\underset{s\in [-r,0]}{sup}V(t+s,\varphi \left(s\right)).$$

Then from Formula (4) and Example 6 we obtain for the Dini fractional derivative:

$$\begin{array}{c}{\mathcal{D}}_{\left(18\right)}^{+}V(t,\varphi \left(0\right);0)\hfill \\ \le 2p\left(t\right)\left(-\sum _{i=1}^n{c}_i\left(t\right){\varphi }_i^2\left(0\right)+\sum _{i=1}^n{\varphi }_i\left(0\right)\sum _{j=1}^n\left(a_{ij}\left(t\right)F_j\left({\varphi }_j\left(0\right)\right)+b_{ij}\left(t\right)G_j({\varphi }_j(-\tau \left(0\right))\right)\right)\hfill \\ +\left(\sum _{i=1}^n{\varphi }_i^2\left(0\right)\right){}_0^{RL}{D}^{q}p\left(t\right)\hfill \\ \le \left(-2p\left(t\right)\eta \left(t\right)+2p\left(t\right)\xi \left(t\right)+{}_0^{RL}{D}_t^{q}p\left(t\right)\right)\sum _{i=1}^n{\varphi }_i^2\left(0\right)\le 0.\hfill \end{array}$$

(28)

According to Remark 3 and inequality (28) we get the inequality

$${}_{\left(18\right)}^c{D}_{+}^{q}V(t,\varphi \left(0\right);0,{\psi }_0)<{\mathcal{D}}_{\left(18\right)}^{+}V(t,\varphi \left(0\right);0)\le 0,$$

i.e., condition (ii) of Lemma 4 is satisfied. Applying Lemma 4 we prove the claim. ☐

Example 9.

Let $n=1$ and consider the scalar FODNN (17) with $c\left(t\right)=\frac{0.55}{t^q\mathsf{\Gamma }(1-q)(E_q(-t^q)+0.1)}+\frac{0.005}{E_q(-t^q)+0.1}$, $I\left(t\right)\equiv 0$, $a,b\in C({\mathbb{R}}_{+},{\mathbb{R}}_{+}):a\left(t\right)+b\left(t\right)=\frac{0.45E_q(-t^q)}{E_q(-t^q)+0.1}$, the activation functions are the Continuous Tan-Sigmoid Function $f\left(u\right)=g\left(u\right)=tanh\left(u\right)=\frac{e^u-e^{-u}}{e^u+e^{-u}}$, $\tau \left(t\right)\equiv r>0$ and the equilibrium point $x^{*}=0$.

Theorem 1 is not applicable since the coefficient before x is not a constant.

Let $x\left(t\right)$ be a solution of FODNN (17). Then using the inequality $|f(x)|\le |x|,x\in \mathbb{R}$ we get $x\left(t\right)a\left(t\right)(f\left(x\left(t\right)\right)-f\left(0\right))\le a\left(t\right)x^2\left(t\right)$. In addition, let $t>0$ be such that ${\left(x\left(t\right)\right)}^2={sup}_{s\in [-r,0]}{\left(x\left(s\right)\right)}^2$ then $|f(x(t-r))|\le |x(t-r)|\le |x(t)|$ and $x\left(t\right)b\left(t\right)(f\left(x(t-r)\right)-f\left(0\right))\le b\left(t\right)|x\left(t\right)(f\left(x(t-r)\right)|\le b\left(t\right){\left(x\left(t\right)\right)}^2$ i.e., Assumption 5 is satisfied with $\xi \left(t\right)\equiv a\left(t\right)+b\left(t\right)$. However the inequality $\xi \left(t\right)=a\left(t\right)+b\left(t\right)=\frac{0.45E_q(-t^q)}{E_q(-t^q)+0.1}\le \eta \left(t\right)=c\left(t\right)=\frac{0.55}{t^q\mathsf{\Gamma }(1-q)(E_q(-t^q)+0.1)}+\frac{0.005}{E_q(-t^q)+0.1}$ is not satisfied (see Figure 4 for $q=0.8$). Therefore, Theorem 2 cannot be applied.

Consider the function $p\left(t\right)=(E_q(-t^q)+0.1)$. Then we get the inequality

$$\begin{array}{c}-p\left(t\right)\eta \left(t\right)+p\left(t\right)\xi \left(t\right)+0.5{}_0^{RL}{D}_t^{q}p\left(t\right)\hfill \\ =-(E_q(-t^q)+0.1)\left(\frac{0.55}{t^q\mathsf{\Gamma }(1-q)(E_q(-t^q)+0.1)}+\frac{0.005}{E_q(-t^q)+0.1}\right)\hfill \\ +(E_q(-t^q)+0.1)\frac{0.45E_q(-t^q)}{E_q(-t^q)+0.1}+0.5{}_0^{RL}{D}_t^q\left(E_q(-t^q)+0.1\right)\hfill \\ =-\frac{0.55}{t^q\mathsf{\Gamma }(1-q)}-0.005+0.45E_q(-t^q)+0.5\left(-E_q(-t^q)+\frac{1.1}{t^q\mathsf{\Gamma }(1-q)}\right)\hfill \\ =-\frac{0.55}{t^q\mathsf{\Gamma }(1-q)}-0.005+0.45E_q(-t^q)-0.5E_q(-t^q)+\frac{0.55}{t^q\mathsf{\Gamma }(1-q)}\hfill \\ =-0.005-0.05E_q(-t^q)=-0.05\left(E_q(-t^q)+0.1\right)\le -0.005.\hfill \end{array}$$

(29)

According to Theorem 3 the equilibrium point $x^{*}=0$ of Caputo FODNN (17) is uniformly stable.

Therefore, in the case that the activation functions are not Lipschitz and the coefficients $c_i$ are not constants in FODNN, we can use Lyapunov function depending directly on the time variable; its Caputo fractional Dini derivative is applicable to study the stability.

5. Conclusions

Several sufficient conditions are given for stability of the equilibrium point of Caputo fractional order neural networks with delays of order $0<q<1$. We study the general case when

-

the transmission delays are time variables;

-

the self-regulating parameters of all units are variable in time;

-

the functions of the connection between two neurons in the network are variable in time bounded functions;

-

the activation function is nonLipschitz (usually in the literature only the case of Lipschitz activation functions is studied).

The investigation is based on applying the fractional Lyapunov approach and various types of Lyapunov functions are discussed (not only the quadratic one as is usually considered in the literature). The presence of the fractional derivative in the model requires an appropriate defined and used derivative of Lyapunov functions among the fractional model. A brief overview of the existing literature and the main types of derivatives among Caputo fractional equations are presented. These types of derivatives are used as an apparatus to study stability properties of the considered Caputo fractional order neural networks for different types of activation functions. Many examples with different types of activation functions such as tanh function, Logit function, continuous Tan-Sigmoid function, are provided to illustrate the practical application of the obtained sufficient conditions.