The Multiple Frequency Conversion Sinusoidal Chaotic Neural Network and Its Application

Abstract

Aiming at the problem that the global search performance of a transiently chaotic neural network is not ideal, a multiple frequency conversion sinusoidal chaotic neural network (MFCSCNN) model is proposed based on the biological mechanism of the brain, including multiple functional modules and sinusoidal signals of different frequencies. In this model, multiple FCS functions and Sigmoid functions with different phase angles were used to construct the excitation function of neurons in the form of weighted sum. In this paper, the inverted bifurcation diagram, Lyapunov exponential diagram and parameter range of the model are given. The dynamic characteristics of the model are analyzed and applied to function optimization and combinatorial optimization problems. Experimental results show that the multiple frequency conversion sinusoidal chaotic neural network has better global search performance than the transient chaotic neural network and other related models.

1. Introduction

A transiently chaotic neural network (TCNN) is a type of neural network used to solve optimization problems [1,2]. The structure of a TCNN network is based on the traditional Hopfield neural network (HNN) by adding self-feedback connections, which makes the network show complex chaotic dynamic behavior [3,4,5]. The optimization mechanism of TCNN is the same as that of HNN, both of which adopt the gradient descent algorithm [6]. By introducing the chaotic dynamic (self-feedback) term, TCNN can make use of the ergodic property, pseudo-randomness and non-repeatability of chaos to avoid local optimization [7,8]. However, TCNN’s chaotic global search performance is still limited due to the parameter setting, excitation function, annealing function and other factors, resulting in not being ideal [9,10]. Therefore, scholars conduct comprehensive research from diverse perspectives to enhance the model’s performance.

The excitation function of TCNN is usually a monotone increasing Sigmoid function [2,5]. Shuai et al. [11] showed that the effective excitation function can select various shapes and should exhibit non-monotonic behavior, and then proposed a CNN model with odd symmetric excitation function. Potapov et al. [12] proposed that the non-monotonic excitation function is more likely to make the neural network produce chaotic dynamic behavior. Therefore, domestic and foreign scholars have carried out relevant research on this. Chen et al. [7] proposed a PLF-TCNN model for solving the traveling salesman problem using the piecewise linear function as a nonmonotonic excitation function. Uykan et al. [13,14] took the double Sigmoid function as the excitation function of neurons and proposed a DS-HNN model to solve the channel allocation optimization problem. Xu et al. [15] proposed a SSW-CNN model for solving combinatorial optimization problems by summing the Shannond wavelet and Sigmoid function as a non-monotonic excitation function. Zhang et al. [16] designed the Euler formula as a non-monotonic excitation function, and proposed a CHNN-APHM model for solving blind detection problems. The above models all propose different forms of non-monotonic excitation functions. Although the optimization ability of the model has been enhanced to some extent, they all lack a certain biological mechanism. According to the biological neurological characteristics of “the higher the frequency, the lower the amplitude” with α, β, δ, γ and θ brain waves in different states, Hu et al. [17,18,19] designed the FCS function, weighted it with the Sigmoid function, and formed the non-monotonic excitation function, and proposed the FCSCNN model. The model has a certain actual biological mechanism, but the human brain contains 14 to 16 billion neurons, and is divided into multiple functional modules, therefore, each module’s activity is different [20,21]. The brain waves are composed of different frequency of sinusoidal signals [22,23,24,25]. Therefore, a single FCS function cannot fully reflect the complex and varied mechanism of action of brain nerve cells.

The excitation function of the transient chaotic neural network conforms to the biological mechanism of neuroscience, and makes it have more abundant chaotic dynamic characteristics and better global search optimization performance. Based on the above theories and biological mechanisms, this paper combines multiple frequency conversion sinusoidal (FCS) functions with different phase angles and Sigmoid functions to form new non-monotonic excitation functions, which are a weighted sum. A new TCNN model, the multiple frequency conversion sinusoidal chaotic neural network (MFCSCNN) model, is proposed. This model maintains the overall monotonicity of the Sigmoid excitation function, and at the same time has variable local non-monotonicity, which can produce complex chaotic dynamic behavior, and is beneficial to make better use of the chaotic ergodic characteristics for global optimization. At the same time, the model is more consistent with the actual biological mechanism of neural information transmission in terms of function construction and phase angle introduction. In this paper, the inverse bifurcation diagram and Lyapunov exponent diagram [26,27,28] of chaotic neurons are provided, and their dynamic characteristics are analyzed. Then, the model is applied to function optimization and combinatorial optimization problems [29,30,31,32,33]. It is verified that the model has good global optimization performance and can overcome local optimization problems better than others.

2. MFCS Chaotic Neuron Model

2.1. MFCS Function

By introducing several FCS functions and adding phase angle parameters, this paper proposes a multi-frequency sine (MFCS) function, which is defined as follows:

$$S\left(u\right)={\displaystyle \sum _{n=1}^N{c}_n{A}_n\sin \left(\frac{u}{{\epsilon }_n}+{\varphi }_n\right)}={\displaystyle \sum _{n=1}^N{c}_n{A}_n\left(0\right)\cdot e^{-a_n\left|u\right|}}\cdot \sin \left(\frac{u}{{\epsilon }_n\left(0\right)\cdot e^{-b_n\left|u\right|}}+{\varphi }_n\right)$$

(1)

where n is the number of FCS functions, n = 1, 2, …, N; u is the function independent variable, representing the strength of brain activity; A_n, A_n(0) are the amplitude and the initial value of the amplitude (0 ≤ A(0) ≤ 1), respectively; ε_n is the steepness factor, representing the magnitude of the frequency of the sine function, ε_n(0) is the initial valuen of the steepness factor (ε_n(0) > 0); a_n, b_n are positive parameters; c_n is the weighting coefficient; and φ_n is the phase angle. When N = 2, A₁(0) = 0.8, ε₁(0) = 0.02, a₁ = 6, b₁ = 1, φ₁ = 0, c₁ = 0.25, A₂(0) = 0.4, ε₂(0) = 0.04, a₂ = 2, b₂ = 1.8, φ₂ = π/2, and c₂ = 0.35, the comparison graph of FCS and MFCS function, as shown in Figure 1, can be derived.

As shown in Figure 1, the MFCS function, composed of the weighted sum of two FCS functions, has more complex and variable amplitude-frequency characteristics than a single FCS function. With the increase of N value, by selecting different values of A(0), ε(0), φ, a, b, and c, the MFCS function will have a richer image waveform, which can better represent and reflect the biological mechanism of brainwave superposition, complexity, and diversity. MFCS and FCS function have the same parameter range and action characteristics, as shown in

Table 1. The parameters scope and impact characteristics of MFCS function.

Parameters	Range of Values	Action Characteristics
A(0)	[0, 1]	determine the FCS weight with c
ε(0)	[0.0044, 0.32]	control the lower bound of f; inversely proportional to f
φ	[0, 2π]	affect MFCS mutability
a	(0, ∞)	proportional to the rate at which A decreases
b	[0.56, 1.8]	control the upper bound of f; proportional to the rate at which f increases
c	[0, 1]	determine FCS weights with A(0)

.

Here, c·A(0) determines the weight of the FCS function; the greater the ε(0), the smaller the f; with the increase of |u|, the larger a is, the faster A decreases, and the larger b is, the faster f increases. As shown in Table 1, the frequency band and range of the MFCS function (0.497~100.311 Hz) are consistent with the frequency band and range of brainwave (0.5~100 Hz), and the biometric characteristics of the actual brainwave frequency are maintained.

2.2. MFCS Chaotic Neuron Model

In order to further analyze the dynamic characteristics and application of the MFCS function in the MFCSCNN model, the excitation function of neurons is constructed by weighted sum of the MFCS function and Sigmoid function, which is described as follows:

$$f\left(u\right)=S_0\left(u,{\epsilon }_0\right)+{\displaystyle \sum _{n=1}^N\left[c_n\cdot S_n\left(u,{\epsilon }_n\right)\right]}=\frac{1}{1+e^{-u/{\epsilon }_0}}+{\displaystyle \sum _{n=1}^N{c}_n{A}_n\sin \left(\frac{u}{{\epsilon }_n}+{\varphi }_n\right)}$$

(2)

where S₀ is the Sigmoid function, ε₀ is the steepness parameter of S₀ (ε₀ > 0), and other parameters are defined by the same Equation (1). When N is 1 and 2, respectively, the graph of the f(u) activation function, as shown in Figure 2, can be derived. The independent variable u is fixed at 0.5, and other parameters are set as shown in Figure 1, except b₁ and b₂. The three-dimensional image of f(u) changing with parameters b₁ and b₂ is shown in Figure 3.

As shown in Figure 2, the activation function with MFCS added has more obvious and intense non-monotonicity, while the global monotonicity of the Sigmoid function is not affected. As shown in Figure 3, the MFCS excitation function is more multidimensional and has a richer dynamic basis. When ε₀(0) = 0.08, N = 2, A₁(0) = 0.2, ε₁(0) = 0.1, a₁ = 6, b₁ = 1.8, φ₁ = 0, c₁ = 0.25, A₂(0) = 0.6, ε₂(0) = 0.2, a₂ = 3.5, b₂ = 0.6, φ₂ = 0.9π, c₂ = 0.35, Sigmoid, MFCS excitation function and its derivative images, as shown in Figure 4, can be derived.

As shown in Figure 4, by adjusting the values of the MFCS function’s parameters, A_n(0), ε_n(0), a_n, b_n, c_n, and φ_n, it is possible to control whether the derivative approaches zero. The derivative image of Sigmoid + MFCS has richer variations and possibilities than the Sigmoid excitation function, and the problem of gradient disappearance is not easy to see. All of the above provide a good prerequisite for neurons to produce more complex chaotic dynamic behavior. To sum up, according to Equation (2) and Figure 1, Figure 2, Figure 3 and Figure 4, it can be seen that larger c·A_n(0), a_n, and b_n, smaller ε_n(0) and random φ_n are more conducive to enhancing the non-monotonic degree of the neuronal excitation function. Therefore, this paper proposes a MFCS chaotic neuron model, which is described as follows:

$$x\left(t\right)=f\left(y\left(t\right)\right)$$

(3)

$$y\left(t+1\right)=ky\left(t\right)-z\left(t\right)\left[x\left(t\right)-I_0\right]$$

(4)

$$z\left(t+1\right)=\left(1-\beta \right)z\left(t\right)$$

(5)

$$f\left(u\right)=S_0\left(u,{\epsilon }_0\right)+{\displaystyle \sum _{n=1}^N\left(c_n\cdot S_n\left(u,{\epsilon }_n\right)\right)}$$

(6)

$$S_0\left(u,{\epsilon }_0\right)=1/\left(1+e^{-u/{\epsilon }_0}\right)$$

(7)

$$S_n\left(u,{\epsilon }_n\right)=A_n\sin \left(\frac{u}{{\epsilon }_n}+{\varphi }_n\right)=A_n\left(0\right)\cdot e^{-a_n\left|u\right|}\cdot \sin \left(\frac{u}{{\epsilon }_n\left(0\right)\cdot e^{-b_n\left|u\right|}}+r_n\cdot {\varphi }_n\right)$$

(8)

where y(t) is the internal state of the neuron, x(t) is the output of the neuron, z(t) is the self-feedback connection weight (z(t) > 0), k is the damping factor of the nerve diaphragm (0 ≤ k ≤1), I₀ is the positive parameter, β is the annealing attenuation factor of z(t) (0 ≤ β ≤1), r_n is the random number between [0, 1], and φ_n is 2π. The other parameters are defined by the same Equation (2).

2.3. Analysis of Dynamic Characteristics

There are also some techniques to present a strategy for identifying chaotic trajectories, such as inverse bifurcation diagram [10,18], Lyapunov index [17,26] and so on. In this paper, the dynamic characteristics of the MFCS chaotic neuron model are analyzed by drawing the inverse bifurcation diagram and calculating its Lyapunov index. The inverse bifurcation diagram can directly reflect the evolution process of neuronal dynamics and observe the state changes of chaos. The Lyapunov index can objectively quantify the nodes and intensity of chaos and periodic motion [3,26]. A positive Lyapunov exponent indicates that the model is in a chaotic state, and the greater the value, the stronger the degree of chaos [27,28]. The Lyapunov exponent definition is described as follows [17]:

$$\lambda =\underset{m\to \infty }{\lim }\frac{1}{m}{\displaystyle \sum _{i=0}^{m-1}\log \left|\frac{dy\left(t+1\right)}{dy\left(t\right)}\right|}$$

(9)

As shown from Equation (4):

$$\frac{dy\left(t+1\right)}{dy\left(t\right)}=k-z\left(t\right)\frac{dx\left(t\right)}{dy\left(t\right)}=k-z\left(t\right)\left[\frac{d{S}_0\left(y\left(t\right)\right)}{dy\left(t\right)}+{\displaystyle \sum _{n=1}^N{c}_n\frac{d{S}_n\left(y\left(t\right)\right)}{dy\left(t\right)}}\right]$$

(10)

where

$$\frac{d{S}_0\left(y\left(t\right)\right)}{dy\left(t\right)}=\frac{1}{{\epsilon }_0}{S}_0\left(y\left(t\right)\right)\left(1-S_0\left(y\left(t\right)\right)\right)$$

(11)

$$\begin{array}{l}\frac{d{S}_n\left(y\left(t\right)\right)}{dy\left(t\right)}=d\left[A_n\left(0\right)\cdot e^{-a_n\left|y\left(t\right)\right|}\cdot \sin \left(\frac{u}{{\epsilon }_n\left(0\right)\cdot e^{-b_n\left|y\left(t\right)\right|}}+{\varphi }_n\right)\right]/dy\left(t\right)\\ =\left\{\begin{array}{l}{A}_n\left(0\right)\cdot \left[-a{e}^{-a_{n}y\left(t\right)}\cdot \sin \left(\frac{y\left(t\right)e^{b_{n}y\left(t\right)}}{{\epsilon }_n\left(0\right)}+{\varphi }_n\right)+\frac{e^{-a_{n}y\left(t\right)}}{{\epsilon }_n\left(0\right)}\cos \left(\frac{y\left(t\right)e^{b_{n}y\left(t\right)}}{{\epsilon }_n\left(0\right)}+{\varphi }_n\right)\cdot \left(e^{b_{n}y\left(t\right)}+b_{n}y\left(t\right)e^{b_{n}y\left(t\right)}\right)\right],y>0\\ A_n\left(0\right)\cdot \left[a{e}^{a_{n}y\left(t\right)}\cdot \sin \left(\frac{y\left(t\right)e^{-b_{n}y\left(t\right)}}{{\epsilon }_n\left(0\right)}+{\varphi }_n\right)+\frac{e^{a_{n}y\left(t\right)}}{{\epsilon }_n\left(0\right)}\cos \left(\frac{y\left(t\right)e^{-b_{n}y\left(t\right)}}{{\epsilon }_n\left(0\right)}+{\varphi }_n\right)\cdot \left(e^{-b_{n}y\left(t\right)}-b_{n}y\left(t\right)e^{-b_{n}y\left(t\right)}\right)\right],y<0\end{array}\right.\\ =A_n\left(0\right)\cdot \left[-a{e}^{-a_n\left|y\left(t\right)\right|}\cdot \sin \left(\frac{y\left(t\right)e^{b_n\left|y\left(t\right)\right|}}{{\epsilon }_n\left(0\right)}+{\varphi }_n\right)+\frac{e^{-a_n\left|y\left(t\right)\right|}}{{\epsilon }_n\left(0\right)}\cos \left(\frac{y\left(t\right)e^{b_n\left|y\left(t\right)\right|}}{{\epsilon }_n\left(0\right)}+{\varphi }_n\right)\cdot \left(e^{b_n\left|y\left(t\right)\right|}+b_{n}y\left(t\right)e^{b_n\left|y\left(t\right)\right|}\right)\right]\\ =A_n\left(0\right)e^{-a_n\left|y\left(t\right)\right|}\cdot \left[\left(1+b_{n}y\left(t\right)\right)\frac{e^{b_n\left|y\left(t\right)\right|}}{{\epsilon }_n\left(0\right)}\cos \left(\frac{y\left(t\right)e^{b_n\left|y\left(t\right)\right|}}{{\epsilon }_n\left(0\right)}+{\varphi }_n\right)-a\cdot \sin \left(\frac{y\left(t\right)e^{b_n\left|y\left(t\right)\right|}}{{\epsilon }_n\left(0\right)}+{\varphi }_n\right)\right]\end{array}$$

(12)

The Lyapunov exponent calculation formula of the MFCS chaotic neuron model is shown in Equations (9)–(12). If the appropriate parameters are selected, MFCS neurons will exhibit abundant chaotic dynamic behavior. The neuron model basic parameter settings are as follows: k = 1, β = 0.005, ε₀ = 0.02, I₀ = 0.65, z(0) = 0.8; MFCS function related parameters setting are as follows: N = 2, A₁(0) = 0.8, ε₁(0) = 0.02, a₁ = 6, b₁ = 1, φ₁ = 0, A₂(0) = 0.4, ε₂(0) = 0.04, a₂ = 2, b₂ = 1.8, r₂·φ₂ = π/2. The output of the neuron x(t) is initialized randomly. x(t) is iteratively calculated according to the model Equations (3)–(8). The value of each x(t) are recorded. Inverse bifurcation diagram of the transient chaotic neuron (c₁ = c₂ = 0), FCS chaotic neuron (c₁ = 0.25, c₂ = 0), MFCS chaotic neuron (c₁ = 0.25, c₂ = 0.6) and time evolution diagram of the Lyapunov exponent are described as shown in Figure 5 by Matlab R2020a:

As shown in Figure 5, under the same parameter conditions, neuronal output x gradually changes from a chaotic state to a period-doubling state with the increase of iterations and the decrease of z (…, 4 cycles, 2 cycles, 1 cycle). The output range of the TCN, FCS, and MFCS chaotic neuron are [0, 1], [−0.0842, 1.0870], [−0.2006, 1.1562], respectively. In order to further prove our results quantitatively, we used the average Lyapunov exponents from 0 to 400 iterations to quantitatively compare the chaos characteristics of the two models. After calculation, the average Lyapunov exponents of the TCN, FCS, and MFCS chaotic neuron were −0.4942, 0.0239, 0.3753, respectively. MFCS has an inverted bifurcation plot with greater output range, better distribution uniformity, and higher and more positive Lyapunov exponents than FCS, transient neuron models. To sum up, the MFCS chaotic neuron model has richer and stronger chaotic dynamics, which lays a good theoretical foundation for its applications in intelligent optimization, neural computing, secure communication and so on.

3. Multiple Frequency Conversion Sinusoidal Chaotic Neural Network (MFCSCNN) Model

3.1. MFCSCNN Model

According to the MFCS chaotic neuron model above and the network structure of TCNN, the MFCSCNN model is constructed, which is described as follows:

$$x\left(t\right)=f\left(y\left(t\right)\right)$$

(13)

$$y_i\left(t+1\right)=k{y}_i\left(t\right)+\alpha \left[{\displaystyle \sum _{j=1,j\ne i}^N{w}_{ij}{x}_j\left(t\right)+I_i}\right]-z_i\left(t\right)\left[x_i\left(t\right)-I_0\right]$$

(14)

$$z\left(t+1\right)=\left(1-\beta \right)z\left(t\right)$$

(15)

$$f\left(u\right)=S_0\left(u,{\epsilon }_0\right)+{\displaystyle \sum _{n=1}^N\left(c_n\cdot S_n\left(u,{\epsilon }_n\right)\right)}$$

(16)

$$S_0\left(u,{\epsilon }_0\right)=1/\left(1+e^{-u/{\epsilon }_0}\right)$$

(17)

$$S_n\left(u,{\epsilon }_n\right)=A_n\sin \left(\frac{u}{{\epsilon }_n}+{\varphi }_n\right)=A_n\left(0\right)\cdot e^{-a_n\left|u\right|}\cdot \sin \left(\frac{u}{{\epsilon }_n\left(0\right)\cdot e^{-b_n\left|u\right|}}+r_n\cdot {\varphi }_n\right)$$

(18)

where α is the input positive proportion parameter, w_ij is the connection weight between neuron i and neuron j (w_ij = w_ji, w_ii = 0), I_i is the input threshold of neuron i, and the rest of the parameters are defined in the same way as the MFCS chaotic neuron model. The signal flow diagram and network structure diagram of the MFCSCNN model are shown in Figure 6:

To obtain good optimization performance of the model, it is necessary to properly select and balance the relationship between the basic parameters of the model and the MFCS parameter settings. The higher the complexity of the optimization problem, the stronger the non-monotony of MFCS is required. In the process of using the model, the choice of parameters is indeed a very critical factor, not only because of the initial value sensitivity of chaos, but also because the model has multiple parameters with different meanings. A relatively larger c_n·A_n(0), a_n, and b_n and a smaller ε_n(0) can be selected within the reference range in Table 1 [17,18,19]. Among the basic parameters, α represents the weight of the feedback signal received by the current neuron from other neurons, which is essentially used to balance the strength of chaotic dynamics and gradient descent [1,2,3,4]. z(0) and β mainly determine the initial value and decay rate of the self-feedback term (chaos term), respectively [1,2,3,4,5,6,7,8]. For the adjustment mechanism of α, z(0), and β, it is usually expected that the early stage is dominated by chaos search (improving the global search ability, avoiding local optimization), and the late stage is dominated by gradient convergence (improving the convergence speed). Appropriate parameter selection of the model determines whether the algorithm can find the optimal solution quickly and accurately, which is always the difficulty faced by the TCNN class optimization model. Therefore, in order to facilitate rapid, effective and clear research and the use of appropriate parameter settings, this paper summarizes all parameter settings and selection guidance by referring to many literatures [1,2,3,4,5,6,7,8,9,10,17,18,19] and the above experimental analysis and verification, as shown in

Table 2. The guidance of parameter settings for MFCSCNN model.

Parameters	Name	Theoretical Values	Experimental Values	Empirical Values
k [1,2]	Memory constant	(0, ∞)	[0, 1]	[0.9, 1]
α [1,2,3,4]	Positive parameter	(0, ∞)	(0, 1]	[0.005, 0.1]
β [1,2,3,4,5,8]	Annealing factor	[0, 1]	(0, 1)	[0.001, 0.05]
z(0) [6,7]	Initial value of self feedback	[0, 1]	[0.7, 1]	[0.6, 0.8]
I0 [7,8]	Positive parameter	(0, ∞)	[0, 1]	[0.45, 0.65]
ε0 [9,10]	Steepness parameter	[0, ∞)	[0.01, 0.5]	[0.01, 0.1]
An(0) [17]	Initial value of the amplitude	[0, ∞)	(0, 1.2]	[0.1, 0.8]
εn(0) [18]	Initial value of teepness	(0, ∞)	[0.0044, 0.32]	[0.01, 0.1]
an [18]	Positive parameter	(0, ∞)	[1, 10]	[3, 8]
bn [19]	Positive parameter	(0, ∞)	[0.56, 1.8]	[0.6, 1.5]
cn [19]	Weighting coefficient	(0, ∞)	[0, 1]	[0.2, 0.8]
r2	Random coefficient	[0, 1]	[0, 1]	[0, 1]
φn	Phase Angle	[0, 2π]	[0, 2π]	π or 2π

.

Table 2 shows the theoretical, experimental, and empirical values of the parameters. The theoretical values of these parameters are generally relatively certain, but the range is relatively large, and cannot play a good reference role in the actual experiment process. Experimental values are usually derived from references and experiments. Compared with the theoretical value, the parameter setting range of experimental values is relatively clear and narrow. It can ensure that the model can easily generate chaotic dynamics in these ranges. However, the range of this experimental value is still not accurate enough, which will make it difficult to find a more suitable parameter value to ensure the optimization ability of the model. Therefore, this paper provides a smaller range of parameters (empirical values) for the new model in the last column of Table 2. For users who have no experience in parameter selection, these empirical values can be used as a good reference.

The parameters shown in Table 2 restrict and influence each other. A change in one parameter will inevitably lead to a change of the dynamic characteristics in the overall model. If the change is caused by one parameter beyond the regulating ability of another parameter, it will lead to the invalidation of the parameters and the model. If any parameter is improperly selected, the algorithm cannot find the optimal solution or appear invalid solutions. The above provides the parameter evolution law and reference range of the MFCSCNN model, which is convenient to provide an effective reference for experiment setting when solving complex optimization problems by using the algorithm. The selection and change of MFCS parameters ultimately affect the non-monotonicity of the excitation function. If the non-monotonicity is too large, it will affect the stability of the model, and if it is too small, it will not produce the sufficient chaotic global search effect. The ideal parameter selection adjustment strategy should be based on the difficulty of the optimization problem and the current solution state to adjust the relevant parameter values in real time, which is also one of the key difficulties in this kind of model research.

3.2. Optimization Mechanism

The self-feedback term and non-monotonic excitation function are introduced into the MFCSCNN model based on the HNN structure to generate complex chaotic dynamics characteristics. With the decrease of z, the self-feedback term gradually attenuates to zero, and the model also evolves from chaotic search to the gradient descent state, and finally converges to the optimal solution [29,30]. Therefore, the optimization mechanism of MFCSCNN is the same as that of HNN, which maps the objective function of the problem to the energy function of the model and the dynamic evolution process of the model to the optimization process of the objective function [31]. When the model converges to the fixed point, the neuronal output is the solution of the current optimization problem [32]. According to the HNN optimization mechanism, the following relations exist:

$$\frac{d{y}_i}{dt}=-\frac{\partial E}{\partial x_i}$$

(19)

However, there is an additional energy term in the expression of the energy function due to the self-feedback term, which is also the key to generating chaotic ergodic search behavior (avoiding local optimality). The MFCSCNN model energy function is described as follows:

$$\begin{array}{l}E\left(t\right)=E_{Hop}+H\\ =-\frac{1}{2}{\displaystyle \sum _{\begin{array}{l}i=1\\ i\ne j\end{array}}^N{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne i\end{array}}^N{w}_{ij}{x}_i\left(t\right)x_j\left(t\right)-{\displaystyle \sum _{i=1}^N{I}_i{x}_i\left(t\right)}}} +\frac{1}{\tau }{\displaystyle \sum _{i=1}^N{\displaystyle {\int }_0^{x_i\left(t\right)}{f}^{-1}\left(\xi \right)}}d\xi +H\left(x_i,w_{ij},I_i\right)\end{array}$$

(20)

where H is the additional energy term, i = 1,2…, N, N is the number of neurons, w_ij is the connection weight between neuron i and neuron j, x_i is the output of the i th neuron, I_i is the threshold of the i th neuron, τ_i is the time constant of the i th neuron, and f⁻¹(·) is the inverse function of the activation function.

4. Application of the MFCSCNN Model for Optimization Problems

4.1. Application of the Model for Function Optimization

In order to verify the optimization performance of the algorithm, a classical function optimization problem is described as follows:

$$f\left(x_1,x_2\right)=4x_1^2-2.1x_1^4+x_1^6/3+x_1{x}_2-4x_2^2+4x_2^4$$

(21)

where f is the objective function (the optimal value is −1.0316285), and x₁, x₂ are the independent variables (the extreme value points are −0.089842, 0.71266, respectively). MFCSCNN is used to optimize the function, and the parameters are set as follows: k = 1, α = 0.03, β = 0.025, ε₀ = 0.02, I₀ = 0.65, z(0) = 0.8, A₁(0) = 0.4, ε₁(0) = 0.02, a₁ = 6, b₁ = 1, c₁ = 0.25, φ₁ = 0, A₂(0) = 0.8, ε₂(0) = 0.04, a₂ = 2, b₂ = 0.5, c₂ = 0.35, and φ₂ = π. The time evolution of the neuron output x₁, x₂ and the energy function E of the MFCSCNN model for optimizing function f is shown in Figure 7:

As shown in Figure 7, when the model evolved to 212 steps, E = −1.03162845, x₁ = −0.089834, x₂ = 0.71266, the time evolution diagram showed rich evolution processes of chaotic search and inverted bifurcations, and finally converged to the fixed point by gradient. During the experiment, it was found that small parameter changes will affect the dynamic evolution process of the whole model, which also confirms the initial value sensitivity of chaos. The experimental results show that MFCSCNN can solve the function optimization problem well. In order to better verify the optimization performance of the model, the complex Traveling Salesman Problem (TSP) in combinatorial optimization was selected for experiments.

4.2. Application of the Model for Combinatorial Optimization

The Traveling Salesman Problem (TSP) can be described as follows: suppose there are N cities providing their location and distance from each other, a closed path needs to be found; each city is only visited once, so to return to the starting city, the distance required for this path is the shortest [33,34]. The objective function of the TSP is as follows:

$$E=\frac{W_1}{2}\left\{{\displaystyle \sum _{i=1}^N{\left({\displaystyle \sum _{j=1}^N{x}_{ij}-1}\right)}^2+{\displaystyle \sum _{j=1}^N{\left({\displaystyle \sum _{i=1}^N{x}_{ij}-1}\right)}^2}}\right\}+\frac{W_1}{2}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{j=1}^N{\displaystyle \sum _{k=1}^N\left(x_{k,j+1}+x_{k,j-1}\right)x_{ij}{d}_{ik}}}}$$

(22)

The internal state equation of MFCS neurons for solving TSP is as follows:

$$\begin{array}{ll}{y}_{ij}\left(t+1\right)=& k{y}_{ij}\left(t\right)-z\left(t\right)\left(x_{ij}\left(t\right)-I_0\right)+\\ & \alpha \left\{-W_1\left[{\displaystyle \sum _{l\ne j}^N{x}_{il}\left(t\right)+{\displaystyle \sum _{k\ne i}^N{x}_{kj}\left(t\right)}}\right]-W_2\left[{\displaystyle \sum _{k\ne i}^N{d}_{ik}\left(x_{k,j+1}\left(t\right)+x_{k,j-1}\left(t\right)\right)}\right]+W_1\right\}\end{array}$$

(23)

where x_i0 = x_in, x_i(n+1)=x_i1, x_ij represents the output of the neuron, which represents the city i being accessed at the j th; W₁ and W₂ are the coupling parameters corresponding to the constrains and cost function of the tour length, respectively; and d_ij represents the distance between city i and city j.

As shown in Figure 8, TSP problems of cities 10, 30 and 75 are selected as experimental objects, and the shortest path lengths shown are 2.6776, 4.237406 and 5.434474, respectively [15,17,19,30].

Select the MFCSCNN model parameters as follows: k = 1, α = 0.025, β = 0.01, ε₀ = 0.02, I₀ = 0.65, z(0) = 0.8, A₁(0) = 0.1, ε₁(0) = 0.2, a₁ = 3, b₁ = 1.8, c₁ = 0.25, φ₁ = 0, A₂(0) = 0.5, ε₂(0) = 0.1, a₂ = 6, b₂ = 0.9, c₂ = 0.15, φ₂ = 0.8π, W₁ = 1, and W₂ = 1. By randomly initializing the value of x_ij, the model solves the output x_1,1 of a single neuron and the evolution diagram of the energy function for TSP of 10 cities, as shown in Figure 9.

As shown in Figure 9, the final energy function value is 1.3388 and the shortest path length is 2.6776. MFCSCNN can solve the TSP problem of 10 cities well. In order to further verify the ability of the model to solve more TSP problems with a higher number and complexity, the TSP problems of 30 cities were selected for experimental verification. The parameters of the MFCSCNN model were selected as follows: k = 1, α = 0.02, β = 0.005, ε₀ = 0.02, I₀ = 0.65, z(0) = 0.8, A₁(0) = 0.2, ε₁(0) = 0.2, a₁ = 4, b₁ = 1.5, c₁ = 0.25, φ₁ = 0, A₂(0) = 0.4, ε₂(0) = 0.1, a₂ = 2, b₂ = 0.8, c₂ = 0.15, φ₂ = 0.6π, W₁ = 1, and W₂ = 1. By randomly initializing the value of x_ij, the output x_1,1 of a single neuron and the evolution diagram of the energy function are shown in Figure 10 for TSP of 30 cities with MFCSCNN.

As shown from Figure 8, Figure 9 and Figure 10, MFCSCNN can solve the TSP problem well. In order to verify the optimization performance of MFCSCNN, the basic parameters of the model are fixed. The different intelligent optimization algorithms (HNN [4], TCNN [10], ITCNN [13], NCNN [35], FCSCNN [17]) are selected for comparative experiments. The values of x_ij and y_ij were randomly initialized. In total, 2000 and 200 independent experiments were carried out with the iteration times of each experiment being 1000, 2000, respectively. Statistical experimental results are shown in

Table 3. The results of the different models for 10-city TSP.

Model	NLP	NOP	RLP	RGM
HNN [4]	1582	930	79.10%	46.50%
TCNN [10]	1972	1809	98.60%	90.45%
ITCNN [13]	2000	1900	100%	95.00%
NCNN [35]	2000	1910	100%	95.50%
FCSCNN [17]	2000	1981	100%	99.05%
MFCSCNN	2000	1992	100%	99.60%

Note: Number of Legitimate Path, NLP; Number of Optimal Path, NOP; Rate of Legitimate Path, RLP; Rate of Global Minima, RGM. The best performance index are marked in bold.

and

Table 4. The results of the different models for 30-city TSP.

Model	NLP	NOP	RLP	RGM
HNN [4]	112	7	56.0%	3.5%
TCNN [10]	189	45	94.5%	22.5%
ITCNN [13]	196	52	98.0%	26.0%
NCNN [35]	195	60	97.5%	30.0%
FCSCNN [17]	193	61	96.5%	30.5%
MFCSCNN	195	68	97.5%	34.0%

The best performance index are marked in bold.

.

For the TSP problem of 75 cities, the optimal rate of most algorithms is low, and cannot find the optimal solution. Therefore, evaluation index J [36] is introduced to evaluate the optimization performance of different models, and the expression is shown as follows:

$$J=\frac{AVS-GM}{GM}\times 100\%$$

(24)

where AVS is the mean of the legal solution, and GM is the global optimal solution. The lower the value of J, the stronger the optimization performance. The values of x_ij and y_ij were randomly initialized. In total, 200 independent experiments were carried out with the iteration times of each experiment being 8000, respectively. Statistical experimental results are shown in

Table 5. The results of the different models for 75-city TSP.

Model	NLP	RLP	BS	J
HNN [4]	32	16.0%	6.4703	25.675%
TCNN [10]	103	51.5%	5.7617	13.534%
ITCNN [13]	125	62.5%	5.5884	10.802%
NCNN [35]	118	59.0%	5.6101	11.513%
FCSCNN [17]	136	68.0%	5.4345	8.621%
MFCSCNN	146	73.0%	5.4345	8.336%

Note: Best Solution, BS; Evaluation Index, J. The best performance index are marked in bold.

.

The richer the chaotic dynamic characteristics, the stronger the optimization ability of the model [37]. As shown from Table 3, Table 4 and Table 5, MFCSCNN has a better optimal ratio and accuracy than HNN, TCNN, ITCNN, NCNN and FCSCNN under the same basic parameter conditions. The larger the number of cities in TSP problem, the better to test the optimization ability of the model [38]. Furthermore, it has better performance when solving larger scale TSP problems. The introduction of more complex and significant non-monotonic excitation functions in MFCSCNN provides a rich global search basis for chaotic dynamics in the early stage. At the same time, MFCSCNN has more variable and complex function characteristics than other models, which also confirms the previous dynamic analysis results. Therefore, compared with other related algorithms, MFCSCNN shows better optimization performance.

5. Conclusions

In this paper, a new MFCS function is designed according to the biological mechanism of the brain wave composed of multiple sinusoidal frequency signals by introducing several FCS functions and adding phase angle parameters, and a new MFCSCNN model is proposed by combining it with the Sigmoid function to form the excitation function of chaotic neurons. By analyzing the activation function diagram, inverted bifurcation diagram and Lyapunov exponential diagram of chaotic neurons, it is proved that MFCS chaotic neurons have richer and more complex chaotic dynamic characteristics than TCN and FCS chaotic neurons, which lays a good foundation for global optimization. At the same time, according to the optimization mechanism and theoretical and experimental analysis, the parameter selection range of the model is provided. In order to further verify the optimization performance of the MFCSCNN model, the classical function optimization and TSP in NP problem with higher complexity were selected for the simulation experiments. The experimental results showed that the proposed MFCSCNN model has better optimization performance and accuracy than several existing optimization algorithms, especially in the optimization problems with higher complexity, which proves the effectiveness and feasibility of the model. The scope of the model can be expanded to address various optimization problems.