A New Neural Network Training Algorithm Based on Artificial Bee Colony Algorithm for Nonlinear System Identification

Abstract

Artificial neural networks (ANNs), one of the most important artificial intelligence techniques, are used extensively in modeling many types of problems. A successful training process is required to create effective models with ANN. An effective training algorithm is essential for a successful training process. In this study, a new neural network training algorithm called the hybrid artificial bee colony algorithm based on effective scout bee stage (HABCES) was proposed. The HABCES algorithm includes four fundamental changes. Arithmetic crossover was used in the solution generation mechanisms of the employed bee and onlooker bee stages. The knowledge of the global best solution was utilized by arithmetic crossover. Again, this solution generation mechanism also has an adaptive step size. Limit is an important control parameter. In the standard ABC algorithm, it is constant throughout the optimization. In the HABCES algorithm, it was determined dynamically depending on the number of generations. Unlike the standard ABC algorithm, the HABCES algorithm used a solution generation mechanism based on the global best solution in the scout bee stage. Through these features, the HABCES algorithm has a strong local and global convergence ability. Firstly, the performance of the HABCES algorithm was analyzed on the solution of global optimization problems. Then, applications on the training of the ANN were carried out. ANN was trained using the HABCES algorithm for the identification of nonlinear static and dynamic systems. The performance of the HABCES algorithm was compared with the standard ABC, aABC and ABCES algorithms. The results showed that the performance of the HABCES algorithm was better in terms of solution quality and convergence speed. A performance increase of up to 69.57% was achieved by using the HABCES algorithm in the identification of static systems. This rate is 46.82% for the identification of dynamic systems.

1. Introduction

ANN is one of the most important artificial intelligence techniques. It has been used to solve many real-world problems [1]. Its popularity is a result of its strong structure. Security, science, engineering, medical science, finance, education, energy, and manufacturing are some of the areas where ANN is used. One of the important capabilities of ANN is the learning feature. It can provide modeling of problems using existing data and can give information about data that it has never seen. The training process of ANN is very important. Successful modeling of a problem is directly related to the successful training process of ANN. One of the artificial intelligence approaches used extensively in ANN training is meta-heuristic algorithms [2,3,4].

Meta-heuristic algorithms have been used successfully in solving many real-world problems [5,6,7,8]. The success of meta-heuristic algorithms has increased the interest in this field. Currently, there are more than 200 meta-heuristic algorithms in the literature. The equilibrium optimizer (EO) [9], the marine predators algorithm (MPA) [10], the slime mould algorithm (SMA) [11], the reptile search algorithm (RSA) [12], the dandelion optimizer [13], the runge kutta optimizer (RUN) [14], and weighted mean of vectors (INFO) [15] are some of the current meta-heuristic algorithms recommended in 2020 and beyond. Artificial bee colony (ABC) [16,17], cuckoo search (CS) [18], particle swarm optimization (PSO) [19], differential evolution algorithm (DE) [20] and flower pollination algorithm (FPA) [21] are some of the very popular meta-heuristic algorithms proposed in recent years.

One of the popular meta-heuristic algorithms used in ANN training is the ABC algorithm. The ABC algorithm is used to solve many real-world problems [22]. The strong global and local convergence ability of the ABC algorithm is one of the important reasons for its popularity. This strong structure of the ABC algorithm has also enabled it to be used extensively in ANN-based studies. Some ANN studies based on the ABC algorithm are as follows: Karaboga and Akay [23] evaluated the performance of the ABC algorithm in ANN training and compared it with different algorithms. They reported that the ABC algorithm was successful in training ANN. Karaboga et al. [24] trained feed-forward neural networks (FFNNs) using the ABC algorithm. Ozturk and Karaboga [25] proposed a hybrid training algorithm based on Levenberg–Marquardt (LM) and ABC algorithm. Karaboga and Ozturk [26] evaluated its performance on classification problems by training FFNN with the ABC algorithm. Kaya [27] compared the performance of several meta-heuristic algorithms in ANN training for nonlinear system identification. It was reported that the ABC algorithm is among the most successful training algorithms. Uzlu et al. [28] used an approach based on the ABC algorithm and ANN for the prediction of hydroelectric generation in Turkey. They stated that their proposed approach was effective. Xu et al. [29] proposed a modified ABC algorithm to train FFNN. Kaya and Baştemur Kaya [30] proposed a novel neural network training algorithm called artificial bee colony algorithm based on effective scout bee stage (ABCES) for the identification of nonlinear static systems. Ghanem and Jantan [31] introduced a hybrid training algorithm based on the ABC algorithm and monarch butterfly optimization for cyberattack classification applications. Shah et al. [32] presented a quick gbest guided artificial bee colony algorithm to train FFNN for Saudi stock market prices prediction. They reported that their proposed approach was more effective than typical computational algorithms.

Many variants have been proposed to obtain more effective solutions with the ABC algorithm. At the same time, hybrid approaches have also been suggested using different meta-heuristic algorithms and some of their features. Yildiz [33] developed a new hybrid algorithm based on the ABC algorithm and the Taguchi method for a structural design optimization of a vehicle component and a multi tool milling optimization problem. Karaboga and Kaya [34] proposed a hybrid ABC algorithm using arithmetic crossover to train ANFIS. Jadon et al. [35] suggested a novel hybrid algorithm (HABCDE) based on the ABC algorithm and DE. The performance of HABCDE was compared with different variant and meta-heuristic approaches for solving various science and engineering optimization problems. Li et al. [36] presented a hybrid feature selection algorithm (MFABC) based on a discrete ABC algorithm for Parkinson’s disease diagnosis. The performance of MFABC was compared with meta-heuristic approaches such as PSO, GWO, GA, ABC, QPSO, DE, FIREFLY, BAT, FWA and BBO. It was reported that MFABC was effective. Kefayat et al. [37] suggested a hybrid optimization approach including ant colony optimization and the ABC algorithm for probabilistic optimal placement and sizing of distributed energy resources. Duan et al. [38] proposed a hybrid algorithm based on the ABC algorithm and the quantum evolutionary algorithm to solve continuous optimization problems. Awadallah et al. [39] used the hill climbing optimizer in the process of the employed bee and proposed a hybrid algorithm (HABC) based on the hill climbing optimizer and ABC algorithm for a nurse rostering problem. Apart from these, there are also different hybrid approaches [40,41,42,43,44,45,46].

In light of the above information, it can be seen that the ABC algorithm has an important area of use in ANN training. To improve the performance of the ABC algorithm in ANN training, variants possessing improved global and local convergence capability are needed. Therefore, in this study, the HABCES algorithm, which a powerful variant of the ABC algorithm, is proposed. For the analysis of the performance of the HABCES algorithm in ANN training, it was applied for modeling parity problems and the identification of nonlinear static and dynamic systems. This study makes important contributions to the literature and has many innovative aspects. These are summarized below:

• The study uses one of the most effective variants of the ABC algorithm proposed for ANN training. The HABCES algorithm is proven to be effective by comparing it with different variants.
• It is one of the most comprehensive studies on the identification of nonlinear static and dynamic systems.
• The study uses an important variant in which both global convergence and local convergence ability are developed. It can be used to solve many different real-world problems.
• Unlike other variants of ABC algorithms in the literature, the related algorithm is improved by using adaptive arithmetic crossover, adaptive step size, an adaptive limit parameter, and an effective scout bee solution generation mechanism.

2. Materials and Methods

2.1. Standard Artificial Bee Colony (ABC) Algorithm

The ABC algorithm models the foraging behavior of honey bees [16,17]. Unlike the behavior of real bees, the ABC algorithm makes some assumptions. The ABC algorithm includes three different types of bee, including employed bees, onlooker bees and scout bees. Half of a colony consists of employed bees. The other half includes onlooker bees. The position of a food source corresponds to a candidate solution of a problem. The nectar amount of a food source corresponds to the solution quality. The ABC algorithm has three basic control parameters. These are colony size, limit, and maximum number of generations.

In the ABC algorithm, the process starts with the random determination of the positions of the food sources using (1). $x_j^{min}$ is the lower bound and $x_j^{max}$ is the upper bound. $x_i$ corresponds to the ith solution. To produce new solutions in the employed bee onlooker bee phases, the solution generation mechanisms given in (2) are used. Here, ${\varnothing }_{ij}$ is a random number in the range [−1, 1] and k is an integer number in the range [1, number of employed bees].

$$x_{ij}=x_j^{min}+rand\left(0,1\right)\left(x_j^{max}-x_j^{min}\right)$$

(1)

$$v_{ij}=x_{ij}+{\varnothing }_{ij}\left(x_{ij}-x_{kj}\right)$$

(2)

$$P_i=\frac{fitnes{s}_i}{{{\displaystyle \sum }}_{j=1}^{SN}fitnes{s}_j}$$

(3)

A comparison is made as to whether the nectar amount of the candidate source is better than the previous one. If the nectar amount of the candidate source is better, the information of the previous source is deleted from the memory and the information of the candidate source is written. In the standard ABC algorithm, a selection process is carried out using the roulette wheel method. The selection probability of sources is computed by (3). Here, $fitnes{s}_i$ represents the quality of the ith source. SN is the number of employed bees.

The ABC algorithm has a failure counter. If the value of the failure counter reaches the limit value, a new solution is randomly generated by the scout bees using (1). This process is repeated until the maximum number of generations is reached.

2.2. Hybrid Artificial Bee Colony Algorithm Based on Effective Scout Bee Stage (HABCES)

The hybrid artificial bee colony algorithm based on effective scout bee stage (HABCES) improves both the global and local search capability of the standard ABC algorithm. The aABC algorithm [34] and ABCES algorithm [30] are two important variants of the ABC algorithm. The HABCES algorithm was created by combining the innovative features of these variants and adding new additional features.

To increase the local search capability of the ABC algorithm, two fundamental innovations were made in the structure of the aABC algorithm. Firstly, arithmetic crossover was used in the solution generation mechanisms of the employed and onlooker bee phases. An arithmetic crossover was applied between the instantaneous solution and the global best solution. The second is the adaptive adjustment of the step size according to the failure counter. To strengthen the local search capability of the HABCES algorithm, the solution search mechanism of the aABC algorithm specified in (4) and (5) was used. γ is the crossover rate and α is the adaptivity coefficient. One of the major drawbacks of the aABC algorithm is the manual setting of these parameters. The optimum values of the γ and α parameters may change according to the problem type. For the best results, it is necessary to try all alternatives. This disadvantage is eliminated with the HABCES algorithm. In each iteration, the values of γ and α are adjusted adaptively according to the number of iterations as noted in (6) and (7). iter is the current number of iterations and maxCycle is the maximum number of iterations. As can be understood from (6), the effect of the global best solution on the new solution increases as the number of iterations increases. Namely, better solutions are sought around the global best solution. At first, the effect of the global best solution is minimal. With this approach, new solutions are prevented from being similar to the global best solution. (7) shows the adaptive adjustment of the step size (α). α changes according to the number of iterations and approaches from 0 to 1. Its value is initially 0. It reaches 1 when iter and maxCycle are equal. This approach allows searching for a new solution in remote region of the existing solution in the initial iterations. As the number of iterations increases, the search region narrows.

$$v_{ij}=x_{ij}\gamma +x_{g_j}\left(1-\gamma \right)+B_i{\varnothing }_{ij}\left(x_{ij}-x_{kj}\right)$$

(4)

$$B_i={\left(\frac{1}{1+tria{l}_i}\right)}^{\alpha }$$

(5)

$$\gamma =rand\ast \frac{maxCycle-iter}{maxCycle}$$

(6)

$$\alpha =\frac{iter}{maxCycle}$$

(7)

$$Limit=1+w \times D \times FoodNumber \times \left(\frac{maxCycle-iter}{maxCycle}\right)$$

(8)

$$v_{ij}=\left\{\begin{array}{ccc}\hfill x_{ij}\epsilon +x_{g_j}\left(1-ϵ\right),& \hfill \hspace{1em}\hfill & r1<r2\hfill \\ \hfill x_{ij}+x_{ij} \delta ,& \hfill \hspace{1em}\hfill & other\hfill \end{array}\right.$$

(9)

$$r2=rand+\left(\frac{maxCycle-iter}{maxCycle}\right)$$

(10)

Limit is one of the important control parameters of the ABC algorithm. When the limit value reaches the failure counter, a random new solution is generated in the scout bee stage. Random generation of the new solution means abandoning previously achieved gains. This is a situation that negatively affects the global convergence ability of the algorithm. In the ABC algorithm, the value of the limit parameter is fixed. Large or small limit values can turn into a disadvantage depending on the development of the solution. The two main issues mentioned here are solved in the ABCES algorithm. As stated in (8), the limit value is dynamically adjusted depending on the current iteration number. Here, w is a random number in the range [0,1]. D is the number of parameters of the problem to be optimized. The ABCES algorithm utilizes a solution generation mechanism that uses the current solution and the global best solution instead of randomly determining the solutions in the scout bee phase. In (9) and (10), r1 and ϵ are random numbers in the range [0,1]. δ is the step size and is accepted as 0.01. In this way, it is ensured that the gains obtained are transferred to the new generations. At the same time, the global convergence capability of the algorithm has been increased. The two stated strengths of the ABCES algorithm are integrated into the HABCES algorithm to strengthen its global search capability.

In light of this information, the basic architecture of the HABCES algorithm can be summarized as follows:

• Arithmetic crossover is used in the solution generation mechanism of the employed and onlooker bee stages and an adaptive local search is adopted.
• Arithmetic crossover and adaptivity parameters are adjusted dynamically.
• Limit, one of the important control parameters, is dynamically adjusted depending on the number of iterations. Its value changes with each iteration.
• In the scout bee stage, instead of randomly generating new solutions, a strong solution generation mechanism including the global best solution and the current solution is used.

3. Results

3.1. Applications on the Solution of the Global Optimization Problems

In this section, the performance of the HABCES algorithm on solving global optimization problems was evaluated. For this purpose, the 43 benchmark test functions given in

Table 1. Benchmark functions used for solving global optimization problems.

Function	Function Name	Formulation
F1	Sphere	$f\left(x\right)={\displaystyle \sum _{i=1}^n}{x}_i^2$
F2	Exponential	$f\left(x\right)=-exp\left(-0.5{\displaystyle \sum _{i=1}^n}{x}_i^2\right)$
F3	High Conditioned Elliptic	$f\left(x\right)={\displaystyle \sum _{i=1}^n}{\left({10}^6\right)}^{\frac{i-1}{n-1}}{x}_i^2$
F4	Ackley	$f\left(x\right)=-20exp\left(-0.2\sqrt{{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{x}_i^2}\right)-exp\left({\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}\cos \left(2\pi x_i\right)\right)+20+e$
F5	Schwefels 1.2	$f\left(x\right)={\displaystyle \sum _{i=1}^n}{\left({\displaystyle \sum _{j=1}^i}{x}_j\right)}^2$
F6	Step	$f\left(x\right)={\displaystyle \sum _{i=1}^n}{\left(\lfloor x_i+0.5\rfloor \right)}^2$
F7	Sum of Different Powers	$f\left(x\right)={\displaystyle \sum _{i=1}^n}{\left|x_i\right|}^{i+1}$
F8	Griewank	$f\left(x\right)={\displaystyle \frac{1}{4000}}{\displaystyle \sum _{i=1}^n}{x}_i^2-{\displaystyle \prod _{i=1}^n}\cos \left({\displaystyle \frac{x_i}{\sqrt{i}}}\right)+1$
F9	Sum Squares	$f\left(x\right)={\displaystyle \sum _{i=1}^n}i{x}_i^2$
F10	Rastrigin	$f\left(x\right)={\displaystyle \sum _{i=1}^n}\left[x_i^2-10\cos \left(2\pi x_i\right)+10\right]$
F11	Penalized 1	$$\begin{gathered} f\left(x\right)={\displaystyle \frac{\pi }{n}}\left\{10{\sin }^2\left(\pi y_1\right)+{\displaystyle \sum _{i=1}^{n-1}}{\left(y_i-1\right)}^2\left[1+10{\sin }^2\left(\pi y_{i+1}\right)\right]+{\left(y_n-1\right)}^2\right\} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +{\displaystyle \sum _{i=1}^n}u\left(x_i, a, k, m\right) \\ {y}_i=1+{\displaystyle \frac{1}{4}}\left(x_i+1\right) \\ u\left(x_i, a, k, m\right)=\left\{\begin{array}{c}\begin{array}{cc}k{\left(x_i-a\right)}^m& x_i>a\end{array}\\ \begin{array}{cc}0& -a\le \end{array}{x}_i\le a\\ \begin{array}{cc}k{\left(-x_i-a\right)}^m& x_i<-a\end{array}\end{array}\right. \end{gathered}$$
F12	Penalized 2	$$\begin{gathered} f\left(x\right)=0.1\left\{{\sin }^2\left(3\pi x_1\right)+{\displaystyle \sum _{i=1}^{n-1}}{\left(x_i-1\right)}^2\left[1+{\sin }^2\left(3\pi x_{i+1}\right)\right]+{\left(x_n-1\right)}^2\left[1+{\sin }^2\left(2\pi x_n\right)\right]\right\} \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +{\displaystyle \sum _{i=1}^n}u\left(x_i, a, k, m\right) \end{gathered}$$
		$u\left(x_i, a, k, m\right)=\left\{\begin{array}{c}\begin{array}{cc}k{\left(x_i-a\right)}^m& x_i>a\end{array}\\ \begin{array}{cc}0& -a\le \end{array}{x}_i\le a\\ \begin{array}{cc}k{\left(-x_i-a\right)}^m& x_i<-a\end{array}\end{array}\right.$
F13	Rosenbrock	$f\left(x\right)={\displaystyle \sum _{i=1}^{n-1}}\left[100{\left(x_{i+1}-x_i^2\right)}^2+{\left(x_i-1\right)}^2\right]$
F14	Levy	$$\begin{gathered} f\left(x\right)={\sin }^2\left(\pi w_1\right)+{\displaystyle \sum _{i=1}^{n-1}}{\left(w_i-1\right)}^2\left[1+10{\sin }^2\left(\pi w_i+1\right)\right]+{\left(w_d-1\right)}^2+\left[1+{\sin }^2\left(2\pi w_d\right)\right] \\ {w}_i=1+{\displaystyle \frac{x_i-1}{4}} \end{gathered}$$
F15	Zakharov	$f\left(x\right)={\displaystyle \sum _{i=1}^n}{x}_i{}^2+{\left({\displaystyle \sum _{i=1}^n}0.5i{x}_i\right)}^2+{\left({\displaystyle \sum _{i=1}^n}0.5i{x}_i\right)}^4$
F16	Rotated Hyper-Ellipsoid	$f\left(x\right)={\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^i}{x}_j^2$
F17	Styblinski–Tang	$f\left(x\right)={\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^n}\left(x_i^4-16x_i^2+5x_i\right)$
F18	Cigar	$f\left(x\right)=x_1{}^2+{10}^6{\displaystyle \sum _{i=2}^n}{x}_i{}^2$
F19	Qing	$f\left(x\right)={\displaystyle \sum _{i=1}^n}{\left(x_i^2-i\right)}^2$
F20	Yang 1	$f\left(x\right)={\displaystyle \sum _{i=1}^n}{\epsilon }_i{\left|x_i\right|}^i$
F21	Bartels Conn	$f\left(x\right)=\left|x_1^2+x_2^2-x_1{x}_2\right|+\left|\sin \left(x_1\right)\right|+\left|\cos \left(x_2\right)\right|$
F22	Beale	$f\left(x\right)={\left(1.5-x_1+x_1{x}_2\right)}^2+{\left(2.25-x_1+x_1{x}_2{}^2\right)}^2+{\left(2.625-x_1+x_1{x}_2{}^3\right)}^2$
F23	Bohachevsky 1	$f\left(x\right)=x_1^2+2x_2^2-0.3\cos \left(3\pi x_1\right)-0.4\cos \left(4\pi x_2\right)+0.7$
F24	Bohachevsky 2	$f\left(x\right)=x_1^2+2x_2^2-0.3\cos \left(3\pi x_1\right)\cos \left(4\pi x_2\right)+0.3$
F25	Camel Function—Three Hump	$f\left(x\right)=2x_1^2-1.05x_1^4+{\displaystyle \frac{x_1^6}{6}}+x_1{x}_2+x_2^2$
F26	Egg Crate	$f\left(x\right)=x_1^2+x_2^2+25\left({\sin }^2\left(x_1\right)+{\sin }^2\left(x_2\right)\right)$
F27	Himmelblau	$f\left(x\right)={\left(x_1^2+x_2-11\right)}^2+{\left(x_1+x_2{}^2-7\right)}^2$
F28	Leon	$f\left(x\right)=100{\left(x_2-x_1^2\right)}^2+{\left(1-x_1\right)}^2$
F29	Matyas	$f\left(x\right)=0.26\left(x_1^2+x_2^2\right)-0.48x_1{x}_2$
F30	Mishra 3	$f\left(x\right)=\sqrt{\left|\cos \sqrt{\left|x_1^2+x_2^2\right|}\right|}+0.01\left(x_1+x_2\right)$
F31	Price 1	$f\left(x\right)={\left(\left|x_1\right|-5\right)}^2+{\left(\left|x_2\right|-5\right)}^2$
F32	Price 2	$f\left(x\right)=1+{\sin }^2{x}_1+{\sin }^2{x}_2-0.1e^{-x_1^2-x_2^2}$
F33	Scahffer 1	$f\left(x\right)=0.5+{\displaystyle \frac{{\sin }^2{\left(x_1^2+x_2^2\right)}^2-0.5}{1+0.001{\left(x_1^2+x_2^2\right)}^2}}$
F34	Scahffer 2	$f\left(x\right)=0.5+{\displaystyle \frac{{\sin }^2{\left(x_1^2-x_2^2\right)}^2-0.5}{1+0.001{\left(x_1^2+x_2^2\right)}^2}}$
F35	Trecanni	$f\left(x\right)=x_1^4+4x_1^3+4x_1^2+x_2^2$
F36	Branin’s RCOS 1	$f\left(x\right)={\left(x_2-{\displaystyle \frac{5.1x_1^2}{4{\pi }^2}}+{\displaystyle \frac{5x_1}{\pi }}-6\right)}^2+10\left(1-{\displaystyle \frac{1}{8\pi }}\right)\cos \left(x_1\right)+10$
F37	Easom	$f\left(x\right)=-\cos \left(x_1\right)\cos \left(x_2\right)e^{\left[-{\left(x_1-\pi \right)}^2-{\left(x_2-\pi \right)}^2\right]}$
F38	Six Hump Camel Function	$f\left(x\right)=\left(4-2.1x_1^2+{\displaystyle \frac{x_1^4}{3}}\right)x_1^2+x_1{x}_2+\left(-4+4x_2^2\right)x_2^2$
F39	Cube	$f\left(x\right)=100{\left(x_2-x_1^3\right)}^2+{\left(1-x_1\right)}^2$
F40	Hartmann 3	$f\left(x\right)=-{\displaystyle \sum _{i=1}^4}{a}_i{e}^{\left(-{\displaystyle {\sum }_{j=1}^3}{A}_{ij}{\left(x_j-P_{ij}\right)}^2\right)}$
F41	Hartmann 6	$f\left(x\right)=-{\displaystyle \sum _{i=1}^4}{a}_i{e}^{\left(-{\displaystyle {\sum }_{j=1}^6}{A}_{ij}{\left(x_j-P_{ij}\right)}^2\right)}$
F42	Colville	$$\begin{gathered} f\left(x\right)=100{\left(x_1-x_2^2\right)}^2+{\left(1-x_1\right)}^2+90{\left(x_4-x_3^2\right)}^2+{\left(1-x_3\right)}^2 \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +10.1\left({\left(x_2-1\right)}^2+{\left(x_4-1\right)}^2\right)+19.8\left(x_2-1\right)\left(x_4-1\right) \end{gathered}$$
F43	Miele Cantrell	$f\left(x\right)={\left(e^{-x_1}-x_2\right)}^4+100{\left(x_2-x_3\right)}^6+{\left(\tan \left(x_3-x_4\right)\right)}^4+x_1^8$

were used. The first 20 (F1–F20) test functions are suitable for high-dimensional problems. The other test functions (F21–F43) are low-dimensional optimization problems. For high-dimensional problems, colony size and maximum number of generations were taken as 50 and 500, respectively. The maximum number of generations for low-dimensional optimization problems is 100. Each application was run 30 times, and the mean fitness values were calculated.

The performance of the HABCES algorithm was compared with the standard ABC, aABC and ABCES algorithms. The results obtained for the first 20 test functions are given in

Table 2. Comparison of the results of the ABC, aABC, ABCES and HABCES algorithms for solving high-dimensional global optimization problems.

Function	Search Range	D	Optimal Value	ABC		ABCES		aABC		HABCES (Proposed)
				Mean	Std.	Mean	Std.	Mean	Std.	Mean	Std.
F1	[−5, 5]	30	0	3.0 × 10−4	1.1 × 10−3	2.5 × 10−6	2.4 × 10−6	2.2 × 10−5	6.7 × 10−5	5.7 × 10−12	8.2 × 10−12
F2	[−1, 1]	30	−1	−0.999972	8.4 × 10−5	−1	3.0 × 10−7	−0.999995	1.0 × 10−5	−1	0
F3	[−10, 10]	30	0	4.8 × 10−1	1.15215	2.4 × 10−2	7.2 × 10−2	2.7 × 10−1	6.9 × 10−1	9.0 × 10−9	2.0 × 10−8
F4	[−32, 32]	30	0	5.3 × 10−1	6.0 × 10−1	1.0 × 10−1	7.5 × 10−2	3.7 × 10−1	1.59738	3.0 × 10−5	1.4 × 10−5
F5	[−10, 10]	30	0	1.8 × 102	3.6 × 101	7.7 × 101	3.1 × 101	1.1 × 102	2.8× 101	4.7 × 101	2.1 × 101
F6	[−100, 100]	30	0	1.3 × 10−1	4.3 × 10−1	0	0	2.7 × 10−1	5.7 × 10−1	0	0
F7	[−1, 1]	30	0	9.4 × 10−9	2.9 × 10−8	1.5 × 10−12	2.8 × 10−12	6.9 × 10−12	2.6 × 10−11	9.1 × 10−18	8.0 × 10−18
F8	[−600, 600]	30	0	1.7 × 10−1	2.8 × 10−1	1.7 × 10−2	2.2 × 10−2	5.3 × 10−2	6.7 × 10−2	2.5 × 10−3	6.8 × 10−3
F9	[−10, 10]	30	0	2.8 × 10−1	1.48421	1.2 × 10−5	9.3 × 10−6	2.1 × 10−3	6.8 × 10−3	2.4 × 10−10	7.2 × 10−10
F10	[−5, 5]	30	0	4.46395	1.67945	5.64602	1.7277	6.35917	2.26352	6.9 × 10−2	2.3 × 10−1
F11	[−50, 50]	30	0	1.8 × 10−4	4.0 × 10−4	1.9 × 10−5	1.6 × 10−5	4.3 × 10−5	9.6 × 10−5	5.9 × 10−11	2.0 × 10−10
F12	[−50, 50]	30	0	4.9 × 10−3	2.0 × 10−2	1.4 × 10−4	1.0 × 10−4	8.3 × 10−4	2.5 × 10−3	2.2 × 10−10	2.9 × 10−10
F13	[−5, 5]	30	0	3.2 × 101	2.8 × 101	2.7 × 101	2.5 × 101	2.4 × 101	2.0 × 101	2.2 × 101	2.1 × 101
F14	[−10, 10]	30	0	7.1 × 10−4	2.1 × 10−3	3.8 × 10−5	3.9 × 10−5	7.3 × 10−5	1.3 × 10−4	1.1 × 10−10	2.3 × 10−10
F15	[−5, 10]	30	0	3.1 × 102	3.6 × 101	9.3 × 101	3.4 × 101	2.3 × 102	3.9 × 101	8.0 × 101	2.5 × 101
F16	[−100, 100]	30	0	1.33847	7.19865	1.4 × 10−5	1.8 × 10−5	2.0 × 10−3	4.7 × 10−3	1.2 × 10−10	2.4 × 10−10
F17	[−5, 5]	30	−1174.98	−1142.88	33.1328	−1171.84	6.70843	−1174.18	2.03502	−1174.98	0
F18	[−10, 10]	30	0	2.9 × 101	6.2 × 101	8.7 × 10−2	2.7 × 10−1	3.0 × 101	1.1 × 102	3.1 × 10−7	3.0 × 10−7
F19	[−100, 100]	30	0	3.92817	6.78767	6.3 × 10−1	6.3 × 10−1	1.9 × 10−1	3.7 × 10−1	2.7 × 10−4	6.8 × 10−4
F20	[−5, 5]	30	0	5.7 × 10−2	3.5 × 10−2	1.5 × 10−3	1.1 × 10−3	4.1 × 10−2	1.6 × 10−2	2.3 × 10−4	2.6 × 10−4

. In addition to the results, Table 2 also includes the search range, dimensions, and optimal values. The best results on all test functions were obtained with the proposed HABCES algorithm. A value of 5.7 × 10⁻² was reached using the HABCES algorithm for F1. For the ABCES and aABC algorithms, the fitness values were at 10⁻⁶ and 10⁻⁵ levels. Only the HABCES algorithm reached the optimal value for F2. After the HABCES algorithm, ABCES had the best fitness value, with a standard deviation value of 3.0 × 10⁻⁷. Although very high error values were obtained with the ABC, ABCES and aABC algorithms for F3, very successful results were found when utilizing the HABCES algorithm. The best fitness value for F4 was obtained by using HABCES algorithm as 3.0 × 10⁻⁵. Due to the difficulty of the problem for F5, high fitness values were generally achieved in all algorithms. The best result was obtained using the HABCES algorithm. The optimal value of F6 is 0, and the optimal value was achieved by the HABCES and ABCES algorithms. Interestingly, the least successful result was obtained using the aABC algorithm for this function. The best result for F7 was found to be 9.1 × 10⁻¹⁸ and it was obtained using the HABCES algorithm. The fitness value of the HABCES algorithm for F8 was at the 10⁻³ level. For other algorithms, it was greater than 10⁻³. The HABCES algorithm was very successful for F9. Its fitness value was 2.4 × 10⁻¹⁰. The closest result for F9 was obtained using the ABCES algorithm. The fitness value of the ABCES algorithm was 1.2 × 10⁻⁵. While the fitness value of the HABCES algorithm was at the 10⁻² level for F10, it was greater than 1 for the other algorithms. The best result after the HABCES algorithm was obtained when using the ABC algorithm. The best fitness value found for F11 was 5.9 × 10⁻¹¹ and was obtained when utilizing the HABCES algorithm. A value of 2.2 × 10⁻¹⁰ was obtained by using the HABCES algorithm for F12. The fitness values of the other algorithms were greater than 10⁻⁵. The fitness values found for F13 was between 22 and 32. The best result was achieved when using the HABCES algorithm. The least successful result was obtained with ABC algorithm.

The most successful algorithm for F14 was the HABCES algorithm. Generally, large fitness values were achieved for F15. The best result was 8.0 × 10¹ and was obtained when using the HABCES algorithm. The most successful algorithm after HABCES was ABCES. Its fitness value was 9.3 × 10¹. For F16, the difference between the fitness values of the HABCES algorithm and the ABC algorithm was very large. The HABCES algorithm was better than the other algorithms tested here. The optimal value of F17 was −1174.98. Only the HABCES algorithm reached this value. The best result after the HABCES algorithm was obtained with aABC algorithm. The best fitness value for F18 was found to be 3.1 × 10⁻⁷, and was obtained when using the HABCES algorithm. For F19 and F20, the HABCES algorithm was more effective than the other algorithms.

The results obtained for functions between F21 and F43 are presented in

Table 3. Comparison of the results of the ABC, aABC, ABCES and the HABCES algorithms when solving low-dimensional global optimization problems.

Function	Search Range	D	Optimal Value	ABC		ABCES		aABC		HABCES (Proposed)
				Mean	Std.	Mean	Std.	Mean	Std.	Mean	Std.
F21	[−500, 500]	2	1	1	0	1	0	1	0	1	0
F22	[−4, 4]	2	0	1.5 × 10−3	1.5 × 10−3	3.5 × 10−4	7.1 × 10−4	1.2 × 10−4	2.2 × 10−4	4.4 × 10−6	1.4 × 10−5
F23	[−100, 100]	2	0	7.1 × 10−15	3.5 × 10−14	0	0	0	0	0	0
F24	[−100, 100]	2	0	1.1 × 10−9	4.8 × 10−9	2.3 × 10−10	8.5 × 10−10	4.8 × 10−14	2.6 × 10−13	0	0
F25	[−5, 5]	2	0	1.7 × 10−9	5.8 × 10−9	1.4 × 10−10	5.4 × 10−10	1.1 × 10−17	8.1 × 10−18	2.9 × 10−18	3.1 × 10−18
F26	[−5, 5]	2	0	1.9 × 10−17	6.5 × 10−17	4.9 × 10−18	4.7 × 10−18	7.5 × 10−18	8.4 × 10−18	1.5 × 10−18	1.4 × 10−18
F27	[−5, 5]	2	0	1.5 × 10−7	3.5 × 10−7	3.3 × 10−6	1.3 × 10−5	3.7 × 10−11	2.0 × 10−10	3.1 × 10−18	2.9 × 10−18
F28	[−1, 1]	2	0	1.1 × 10−2	2.6 × 10−2	2.7 × 10−2	4.1 × 10−2	1.1 × 10−3	1.9 × 10−3	5.9 × 10−3	1.6 × 10−2
F29	[−10, 10]	2	0	1.8 × 10−3	3.8 × 10−3	1.5 × 10−4	2.7 × 10−4	3.2 × 10−5	4.7 × 10−5	7.5 × 10−6	1.1 × 10−5
F30	[−10, 10]	2	−0.18465	−0.152569	0.027513	−0.175058	0.0269035	−0.148548	0.0326784	−0.186899	0.0396051
F31	[−10, 10]	2	0	8.8 × 10−14	2.4 × 10−13	8.6 × 10−11	1.8 × 10−10	1.7 × 10−17	2.1 × 10−17	1.7 × 10−18	1.7 × 10−18
F32	[−10, 10]	2	0.9	0.90009	4.5 × 10−4	0.9	5.6 × 10−7	0.9	4.4× 10−16	0.906667	2.5 × 10−2
F33	[−100, 100]	2	0	0	0	0	0	0	0	0	0
F34	[−100, 100]	2	0	5.6 × 10−14	2.3 × 10−13	7.4 × 10−18	2.8 × 10−17	0	0	0	0
F35	[−2, 2]	2	0	2.1 × 10−18	2.8 × 10−18	4.9 × 10−19	8.1 × 10−19	5.1 × 10−19	6.2 × 10−19	2.5 × 10−20	4.7 × 10−20
F36	[−5, 15]	2	0.397887	0.397887	6.0 × 10−7	0.397887	3.40 × 10−7	0.397887	1.67 × 10−16	0.397887	1.67 × 10−16
F37	[−10, 10]	2	−1	−1	0	−1	0	−1	0	−1	0
F38	[−5, 5]	2	−1.03163	−1.03163	0	−1.03163	0	−1.03163	0	−1.03163	0
F39	[−10, 10]	2	0	2.7 × 10−2	3.4 × 10−2	5.5 × 10−3	9.1 × 10−3	1.7 × 10−3	3.5 × 10−3	1.2 × 10−3	2.4 × 10−3
F40	[0, 1]	3	−3.86278	−3.86278	4.4 × 10−16	−3.86278	4.4 × 10−16	−3.86278	4.4 × 10−16	−3.86278	4.4 × 10−16
F41	[0, 1]	6	−3.32237	−3.32232	7.0 × 10−5	−3.32231	1.1 × 10−4	−3.3031	4.3 × 10−2	−3.29058	5.3 × 10−2
F42	[−10, 10]	4	0	2.41054	2.5185	7.1 × 10−1	6.8 × 10−1	8.4 × 10−1	8.1 × 10−1	4.1 × 10−1	5.3 × 10−1
F43	[−10, 10]	4	0	1.1 × 10−3	3.9 × 10−3	2.3 × 10−5	5.8 × 10−5	7.9 × 10−4	4.1 × 10−3	1.8 × 10−7	4.6 × 10−7

. The optimum value was achieved with all algorithms for F21, F33, F37 and F38. The best result for F22 was found to be 4.4 × 10⁻⁶, and was obtained when using the HABCES algorithm. The optimum value was achieved when utilizing the ABCES, aABC and the HABCES algorithms for F23. The fitness value of the ABC algorithm was 7.1 × 10⁻¹⁵. The best result for F24 was found with the HABCES algorithm. The HABCES algorithm achieved the optimal value here. After the HABCES algorithm, the best result was obtained when using the aABC algorithm. Effective fitness values were obtained when using the HABCES and aABC algorithms for F25. Although the HABCES algorithm had the best fitness value for F26, the results obtained when using the other algorithms were also successful. The HABCES algorithm was better than the other algorithms for F27. The best fitness value for F28 was obtained when using the aABC algorithm. After the aABC algorithm, the best result was found by utilizing the HABCES algorithm. The best results for F29 were obtained via the HABCES and aABC algorithms. The fitness values achieved were 7.5 × 10⁻⁶ and 3.2 × 10⁻⁵, respectively. The optimal value of F30 is −0.18465. The closest result to the optimal value was obtained when using the HABCES algorithm. The best result for F31 was found to be 1.7 × 10⁻¹⁸, and was obtained when using the HABCES algorithm. The least successful algorithm was ABCES. The optimal value of F32 is 0.9. The best result for F32 was obtained by using the aABC algorithm. The fitness value of the HABCES algorithm was 0.906667. The fitness values obtained with the aABC and HABCES algorithms for F34 were both 0. Other algorithms were less successful. Generally effective fitness values were obtained for F35. The most successful was the HABCES algorithm. The best fitness value for F36 was found to be 0.397887 by utilizing the aABC and the HABCES algorithms. The fitness value of the HABCES algorithm for F39 was 1.2 × 10⁻³, and was the best result. For F40, the fitness value of all algorithms is the same. The best value for F41 was obtained when using the ABC algorithm. For F42, the best fitness value was found to be 4.1 × 10⁻¹, and was obtained with the HABCES algorithm. The performance of the HABCES algorithm for F43 was better than that of other algorithms. The fitness value obtained when using the HABCES algorithm was 1.8 × 10⁻⁷. The best result after the HABCES algorithm was found with the ABCES algorithm.

The fitness values obtained on 43 test functions showed that the performance of the HABCES algorithm was the best. However, solution quality is not the only criterion. The convergence graphs of the algorithms also give an idea of their performance. In Figure 1 and Figure 2, the convergence of the ABC, aABC, ABCES and the HABCES algorithms is compared for the first 20 test functions. For F1, F3, F4, F7, F9, F14, F16, F17, F18 and F19, the HABCES algorithm showed a fast convergence and reached the effective solution in a short time. For F2, all algorithms approached the optimal value in a short time. For F5, F6, F8, F10, F11, F12 and F20, the initial convergence speeds of the algorithms are similar. With increasing number of generations, the convergence speed of the HABCES algorithm became more effective than the others. For F13, the convergence speeds of the algorithms are close to one another. Due to the structure of F15, the convergence graphs of the algorithms showed a different appearance. The convergence graphics for the functions between F21 and F32 are compared in Figure 3. The HABCES algorithm converged rapidly for F22, F23, F24, F27, F29, F30 and F31. For F21, the convergence speeds of the algorithms are close to one another. For F25 and F26, the HABCES algorithm converged rapidly in a short time. Then, the speed of convergence decreased. For F28, the convergence speed of the HABCES algorithm was fast at first, and then decreased. The convergence graphics for the functions between F33 and F43 are compared in Figure 4. The HABCES algorithm showed fast convergence for F33, F34 and F35. In general, the convergence speeds were close for F36, F37, F38, F39 and F40. For F41, the HABCES algorithm reached a good solution in a short time. For F42, the convergence speed of the HABCES algorithm was better than for the others. For F43, it was seen that the convergence of the ABCES and HABCES algorithms was more effective. When the convergence graphs and solution qualities were evaluated together, it was concluded that the performance of the HABCES algorithm was better than the others.

3.2. Applications for the Training of the Neural Network

In this section, the performance of the HABCES algorithm regarding neural network training was evaluated. In this context, the algorithm was applied to three different groups of problems. These were the modeling of parity problems, the identification of nonlinear static systems, and the identification of nonlinear dynamic systems. The mean square error (MSE) was used as the error metric. Each application was run 30 times, and the mean error was calculated. The summing function was used as the transfer function. The sigmoid function was chosen as the activation function. The maximum number of generations was considered as the stopping criterion.

The results obtained when using the ABC, aABC, ABCES and the proposed HABCES algorithms for modeling of XOR, 3-bit parity and 4-bit parity problems are compared in

Table 4. Comparison of the results of the ABC, aABC, ABCES and HABCES algorithms when solving parity problems.

Examples	Number of Generations	Network Structure	ABC		ABCES		aABC		HABCES (Proposed)
			Mean	Std.	Mean	Std.	Mean	Std.	Mean	Std.
XOR	50	2-5-1	0.0270955	0.0653951	0.0134276	0.024631	0.0106609	0.0167402	0.00406869	0.00643696
		2-10-1	0.00440856	0.00875478	0.00370912	0.0070464	0.00124831	0.00196459	0.00051696	0.00079469
3 bit parity	100	3-5-1	0.0370702	0.0325831	0.0413162	0.0306131	0.031667	0.0318723	0.023972	0.0280218
		3-10-1	0.0252005	0.0380509	0.0170671	0.0263739	0.0200067	0.0234835	0.00855215	0.00924428
4 bit parity	100	4-5-1	0.128162	0.0415436	0.135707	0.0390414	0.128552	0.0290362	0.118425	0.0281676
		4-10-1	0.104024	0.0425457	0.115735	0.0388215	0.10854	0.0391539	0.0913625	0.0297292

. For each problem, 5 and 10 neurons were used in the hidden layer. For the XOR problem, the number of generations was taken to be 50. In the others, it was 100. A colony size of 20 was adopted. In the XOR problem, increasing numbers of neurons decreased the error value. The error values obtained when using the HABCES algorithm for 2-5-1 and 2-10-1 network structures were 0.00406869 and 0.000516968, respectively. After the HABCES algorithm, the best results were found with the ABCES algorithm. As in the XOR problem, the increase in the number of neurons improved the quality of the solution in the 3-bit parity problem. The best result obtained was 0.00855215, which was achieved when using the HABCES algorithm with the 3-10-1 network structure. The 4-bit parity problem was more difficult than the other problems. This situation can also be observed from the error values found. The error values found for the 4-5-1 and 4-10-1 network structures when using the HABCES algorithm were 0.118425 and 0.0913625, respectively. The performance of the other algorithms was worse. Table 4 shows that the solution quality of the HABCES algorithm was better for all problem and network structures. Figure 5 provides a comparison of the convergences of the ABC, aABC, ABCES and HABCES algorithms when solving parity problems. The fastest and most efficient convergence for each problem was achieved using the HABCES algorithm.

Secondly, the performance of the HABCES algorithm was analyzed with respect to the identification of nonlinear static systems. The information regarding the nonlinear static systems used is given in

Table 5. Information on the nonlinear static systems used.

System	Equation	Inputs	Output	Number of Training/Test Data	Range
S1	$y=2\sin \left(\pi x_1\right)$	x1	y	80/20	[0, 1]
S2	$y=10.391\left\{\left(x_1-0.4\right)\left(x_2-0.6\right)+0.36\right\}$	x1, x2	y	80/20	[0, 1]
S3	$y=\tanh \left(x_1+x_2-11\right)$	x1, x2	y	80/20	[0, 1]
S4	$y=1+x_1{}^{0.5}+x_2{}^{-1}+x_3{}^{-1.5}$	x1, x2, x3	y	173/43	[1, 6]
S5	$y={\left(x_1-5.5\right)}^2+{\left(x_2-5.5\right)}^2+x_3{}^2$	x1, x2, x3	y	100/25	[0, 1]
S6	$y=e^{2x_1\sin \left(\pi x_4\right)}+\sin \left(x_2{x}_3\right)$	x1, x2, x3, x4	y	100/25	[0.25, 0.25]

. S1 has one input. S2 and S3 have two inputs. S4 and S5 have three inputs. S6 has four inputs. All systems have one output. Neural networks with 4, 8 and 12 neurons in the hidden layer were used for each system. Colony size and the maximum number of generations belonging to the training algorithms were taken as 20 and 1000, respectively. In general, 80% of the data were used for the training process. The rest was reserved for the testing process. The results obtained when using the HABCES algorithms for nonlinear static system identification are given in

Table 6. The results obtained when using the HABCES algorithm for nonlinear static system identification.

System	Network Structure	Train		Test
		Mean	Sd.	Mean	Sd.
S1	1-4-1	0.00150	0.00061	0.00258	0.00086
	1-8-1	0.00109	0.00050	0.00201	0.00086
	1-12-1	0.00072	0.00031	0.00161	0.00068
S2	2-4-1	0.00128	0.00063	0.00691	0.00312
	2-8-1	0.00073	0.00063	0.00521	0.00494
	2-12-1	0.00058	0.00027	0.00453	0.00233
S3	2-4-1	0.00014	0.00007	0.00297	0.00173
	2-8-1	0.00016	0.00011	0.00332	0.00213
	2-12-1	0.00025	0.00012	0.00404	0.00191
S4	3-4-1	0.00214	0.00073	0.00253	0.00075
	3-8-1	0.00154	0.00068	0.00207	0.00078
	3-12-1	0.00174	0.00085	0.00212	0.00109
S5	3-4-1	0.00082	0.00050	0.00463	0.00391
	3-8-1	0.00085	0.00053	0.00893	0.00996
	3-12-1	0.00129	0.00085	0.01145	0.00980
S6	4-4-1	0.00148	0.00053	0.001563	0.00086
	4-8-1	0.00157	0.00044	0.00186	0.00071
	4-12-1	0.00164	0.00056	0.00229	0.00081

. It can be seen that both the training and test results for S1 improve in parallel with increasing numbers of neurons. The best results were found when using the 1-12-1 network structure. The best training and test error values were 0.00072 and 0.00161, respectively. The effect of the number of neurons on performance was the same for S2 as it was for S1. The best training error value found was 0.00058. Additionally, the best test error value obtained was 0.00453. Increasing the number of neurons in S3 worsened the solution quality. The best error values for the training and testing process were found when using the 2-4-1 network structure. The error values were 0.00014 and 0.00297, respectively. In S4, the 3-8-1 network structure was more effective than the others. The training error value of the 3-8-1 network structure was 0.00154. The test result was also 0.00207. The best results for S5 were obtained with a network including four neurons in the hidden layer, similarly to the case of S3. The training and test error values for S5 were 0.00082 and 0.00463, respectively. Reducing the number of neurons in S6 improved performance. Effective results were achieved with the 4-4-1 network structure. The best training error value for S6 was 0.00148. Additionally, the test error value was 0.001563.

Table 7. Comparison of the results of the ABC, aABC, ABCES and HABCES algorithms for nonlinear static system identification.

Examples	Network Structure	ABC		ABCES		aABC		HABCES (Proposed)
		Train	Test	Train	Test	Train	Test	Train	Test
S1	1-12-1	0.00099	0.00169	0.00100	0.00187	0.00086	0.00153	0.00072	0.00161
S2	2-12-1	0.00110	0.00590	0.00088	0.00585	0.00086	0.00480	0.00058	0.00453
S3	2-4-1	0.00046	0.00459	0.00031	0.00386	0.00019	0.00354	0.00014	0.00297
S4	3-8-1	0.00235	0.00278	0.00180	0.00220	0.00170	0.00211	0.00154	0.00207
S5	3-4-1	0.00183	0.00709	0.00128	0.00570	0.00118	0.01009	0.00082	0.00463
S6	4-4-1	0.00208	0.00227	0.00161	0.00167	0.00152	0.00152	0.00148	0.00156

compares the results obtained when using the ABC, aABC, ABCES and HABCES algorithms for nonlinear static system identification. For all systems, the best training error value was found when using the HABCES algorithm. Other than for S1 and S6, the best test results were obtained when using the HABCES algorithm. The best test results for S1 and S6 were obtained by using the aABC algorithm.

Table 8. Performance increase achieved when using HABCES for nonlinear static system identification.

System	ABC-HABCES		aABC-HABCES		ABCES-HABCES
	Train (%)	Test (%)	Train (%)	Test (%)	Train (%)	Test (%)
S1	27.27	4.73	28.00	13.90	16.27	-
S2	47.27	23.22	34.09	22.56	32.56	5.62
S3	69.57	35.29	54.84	23.05	26.31	16.10
S4	34.47	25.54	14.44	5.91	9.41	1.90
S5	55.19	34.70	35.94	18.77	30.51	54.11
S6	28.85	31.27	8.07	6.58	2.63	-

shows the increase in performance achieved when using the HABCES algorithm. The HABCES algorithm provided a performance improvement of up to 69.57% compared to using the ABC algorithm for the training process. For the test process, this rate was 35.29%. With respect to the aABC algorithm, performance improvements of up to 54.84% (training) and 23.05% (test), respectively, were achieved. When comparing the HABCES algorithm with the ABCES algorithm, effective performance increases can be observed. A performance increase of up to 32.56% was achieved for training. For the test process, this rate was 54.11%. Figure 6 presents a comparison of the convergences of the ABC, aABC, ABCES and HABCES algorithms. For all systems, the HABCES algorithm obtained effective results by achieving fast convergence. In particular, the convergence speeds for S1, S2, S3, S5 and S6 were much better than those of the other algorithms. In some iterations of S4, the ABCES and the HABCES algorithms came close to one another. A comparison of the real output and the predicted output for all nonlinear static systems is given in Figure 7. It can be seen that the real and predicted outputs overlap with each other for all systems. This is an indication that the HABCES algorithm can successfully solve the corresponding problems.

Thirdly, the algorithms were applied for the identification of nonlinear dynamic systems. Eight nonlinear dynamic systems were used. The nonlinear dynamic systems utilized are presented in

Table 9. Nonlinear dynamic systems used.

System	Equation	Inputs	Output	Number of Training/Test Data
D1	$$\begin{gathered} y\left(k+1\right)={\displaystyle \frac{y\left(k\right)\left[y\left(k-1\right)+2\right]\left[y\left(k\right)+2.5\right]}{8.5+{\left[y\left(k\right)\right]}^2+{\left[y\left(k-1\right)\right]}^2}}+u\left(k\right) \\ u\left(k\right)=\left\{\begin{array}{cc}\hfill 2\cos \left(2\pi k/100\right),& \hspace{1em}k\le 200\hfill \\ \hfill 1.2\sin \left(2\pi k/20\right),& \hspace{1em}200<k\le 500\hfill \end{array}\right. \end{gathered}$$	$y\left(k\right),$ $y\left(k-1\right), u\left(k\right)$	$y\left(k+1\right)$	400/100
D2	$$\begin{gathered} y\left(k+1\right)={\displaystyle \frac{y\left(k\right)y\left(k-1\right)\left[y\left(k\right)+2.5\right]}{1+{\left[y\left(k\right)\right]}^2+{\left[y\left(k-1\right)\right]}^2}}+u\left(k\right) \\ u\left(k\right)=\sin \left(2\pi k/25\right) \end{gathered}$$	$y\left(k\right),$ $y\left(k-1\right), u\left(k\right)$	$y\left(k+1\right)$	200/50
D3	$$\begin{gathered} y\left(k+1\right)={\displaystyle \frac{y\left(k\right)}{1+y{\left(k\right)}^2}}+u{\left(k\right)}^3 \\ u\left(k\right)=\sin \left(2\pi k/100\right) \end{gathered}$$	$y\left(k\right), u\left(k\right)$	$y\left(k+1\right)$	200/50
D4	$$\begin{gathered} y\left(k+1\right)={\displaystyle \frac{y\left(k\right)}{1+y\left(k-1\right)}}+u{\left(k\right)}^3 \\ u\left(k\right)=\sin \left(\pi k/25\right) \end{gathered}$$	$y\left(k\right),$ $y\left(k-1\right), u\left(k\right)$	$y\left(k+1\right)$	200/50
D5	$$\begin{gathered} y\left(k+1\right)={(1+y{\left(k\right)}^2)}^{-1}\left(-0.9y\left(k\right)+u\left(k\right)\right) \\ u\left(k\right)=0.75\sin \left(\pi \left(k-1\right)/180\right) \end{gathered}$$	$y\left(k\right), u\left(k\right)$	$y\left(k+1\right)$	400/100
D6	$$\begin{gathered} y\left(k+1\right)=0.72y\left(k\right)+0.025y\left(k-1\right)u\left(k\right)+0.01u{\left(k-1\right)}^2+0.2u\left(k-2\right) \\ u(k) \\ =\left\{\begin{array}{ll} \sin (\pi k/25),& \hspace{1em}\hspace{1em} k<250\\ 1.0,& 250\le k<500\\ -1.0,& 500\le k<750\\ 0.3\sin (\pi k/25)+0.1\sin (\pi k/32)+0.6\sin (\pi k/10),& 750\le k\le 1000\end{array}\right. \end{gathered}$$	$y\left(k\right),$ $y\left(k-1\right), u\left(k\right)$, $u\left(k-1\right)$,	$y\left(k+1\right)$	900/100
D7	$$\begin{gathered} y\left(k+1\right)=0.3y\left(k\right)+0.6y\left(k-1\right)+N\left(u\left(k\right)\right) \\ N\left(u\right)=0.6\sin \left(\pi u\right)+0.3\sin \left(3\pi u\right)+0.1\sin \left(5\pi u\right) \\ u\left(k\right)=\sin {\displaystyle \frac{2\pi k}{250}} \end{gathered}$$	$y\left(k\right),$ $y\left(k-1\right), u\left(k\right)$	$y\left(k+1\right)$	400/100
D8	$$\begin{gathered} y\left(k+1\right)={\displaystyle \frac{y\left(k\right)y\left(k-1\right)y\left(k-2\right)u\left(k-1\right)\left[y\left(k-2\right)-1\right]+u\left(k\right)}{1+y{\left(k-1\right)}^2+y{\left(k-2\right)}^2}} \\ u\left(k\right)=\left\{\begin{array}{cc}\hfill \sin \left(2\pi k/250\right),& \hspace{1em}k\le 500\hfill \\ 0.8\sin \left(2\pi k/250\right)+0.2\sin \left(2\pi k/25\right),& \hspace{1em}k>500\end{array}\right. \end{gathered}$$	$y\left(k\right), u\left(k\right)$	$y\left(k+1\right)$	800/200

. D3, D5 and D8 consist of two inputs. D1, D2, D4 and D7 have three inputs. D6 consists of four inputs. Except for D6, 80% of the data were used for training. The rest were designated for the testing process. For D6, these rates were 90% and 10%, respectively. Network structures with 4, 8 and 12 neurons in the hidden layer were utilized. The colony size and maximum number of generations used were 20 and 1000, respectively, as in the identification of nonlinear static systems. The results obtained using the HABCES algorithms in nonlinear dynamic system identification are given in

Table 10. The results obtained by using the HABCES algorithms in nonlinear dynamic system identification.

System	Network Structure	Train		Test
		Mean	Sd.	Mean	Sd.
D1	3-4-1	0.00051	0.00024	0.00088	0.00032
	3-8-1	0.00039	0.00015	0.00072	0.00030
	3-12-1	0.00052	0.00022	0.00106	0.00075
D2	3-4-1	0.00084	0.00026	0.00093	0.00031
	3-8-1	0.00075	0.00024	0.00096	0.00053
	3-12-1	0.00066	0.00023	0.00082	0.00041
D3	2-4-1	0.00070	0.00021	0.00095	0.00046
	2-8-1	0.00057	0.00009	0.00070	0.00022
	2-12-1	0.00067	0.00017	0.00085	0.00039
D4	3-4-1	0.00426	0.00078	0.00440	0.00081
	3-8-1	0.00348	0.00038	0.00367	0.00048
	3-12-1	0.00354	0.00040	0.00367	0.00052
D5	2-4-1	0.00026	0.00012	0.00026	0.00014
	2-8-1	0.00024	0.00013	0.00024	0.00011
	2-12-1	0.00026	0.00011	0.00030	0.00022
D6	4-4-1	0.00067	0.00031	0.00105	0.00045
	4-8-1	0.00079	0.00029	0.00133	0.00048
	4-12-1	0.00086	0.00042	0.00121	0.00063
D7	3-4-1	0.00035	0.00017	0.00063	0.00031
	3-8-1	0.00030	0.00014	0.00050	0.00022
	3-12-1	0.00034	0.00020	0.00059	0.00031
D8	2-4-1	0.00021	0.00009	0.00040	0.00011
	2-8-1	0.00017	0.00006	0.00031	0.00008
	2-12-1	0.00017	0.00006	0.00035	0.00011

. The best training and test error values for D1 were obtained using the 3-8-1 network structure. The related error values were 0.00039 and 0.00072, respectively. Increasing the number of neurons in D2 decreased the error value. The best results were obtained when using the 3-12-1 network structure. The training error value for the 3-12-1 network structure was 0.00066. The test error value was 0.00082. The most effective results for D3 and D4 were obtained when using network structures featuring eight neurons in the hidden layer. The training error values of D3 and D4 were 0.00057 and 0.00348, respectively. In addition, their test error values were 0.00070 and 0.00367, respectively. For D5, effective results were generally found when using all network structures. The 2-8-1 network structure was slightly more successful than the others. Increasing the number of neurons in D6 decreased performance. Namely, the results got worse. The best training and testing results were obtained when using the 4-4-1 network structure. The best training error value of D6 was 0.00067. The test error value was also 0.00105. The 3-8-1 network structure was effective for D7. There was a clear difference between the error values obtained for 3-8-1 and those obtained for the other network structures. The best training and test error values for D7 were 0.00030 and 0.00050, respectively. According to the training and test results in D8, the 2-8-1 network structure is more effective. The best training error value obtaind for D8 was 0.00017. The test error value was 0.00031. The performance of the HABCES algorithm was compared with the ABC, aABC and ABCES algorithms, and the results are presented in

Table 11. Comparison of the results of the ABC, aABC, ABCES and HABCES algorithms for nonlinear dynamic system identification.

Examples	Network Structure	ABC		ABCES		aABC		HABCES (Proposed)
		Train	Test	Train	Test	Train	Test	Train	Test
D1	3-8-1	0.00061	0.00095	0.00046	0.00087	0.00047	0.00097	0.00039	0.00072
D2	3-12-1	0.00092	0.00105	0.00082	0.00090	0.00083	0.00101	0.00066	0.00082
D3	2-8-1	0.00070	0.00090	0.00070	0.00098	0.00060	0.00077	0.00057	0.00070
D4	3-8-1	0.00372	0.00389	0.00370	0.00384	0.00367	0.00375	0.00348	0.00367
D5	2-4-1	0.00039	0.00043	0.00034	0.00040	0.00028	0.00029	0.00026	0.00026
D6	4-4-1	0.00126	0.00143	0.00108	0.00160	0.00079	0.00110	0.00067	0.00105
D7	3-8-1	0.00049	0.00067	0.00041	0.00064	0.00036	0.00056	0.00030	0.00050
D8	2-8-1	0.00028	0.00045	0.00023	0.00039	0.00019	0.00037	0.00017	0.00031

. It can be seen that the HABCES algorithm is better than the others with respect to the values of training and test error for all systems. A serious increase in performance was achieved when using the HABCES algorithm. The performance increase rates according to the systems are given in

Table 12. Performance increase achieved using HABCES for nonlinear dynamic system identification.

System	ABC-HABCES		aABC-HABCES		ABCES-HABCES
	Train (%)	Test (%)	Train (%)	Test (%)	Train (%)	Test (%)
D1	36.06	24.21	15.21	17.24	17.02	25.77
D2	28.26	21.90	19.51	8.89	20.48	18.81
D3	18.57	22.22	18.57	28.57	5.00	9.09
D4	6.45	5.65	5.94	4.42	5.17	2.13
D5	33.33	39.53	23.53	35.00	7.14	10.34
D6	46.82	26.57	37.96	34.37	15.19	4.55
D7	38.77	25.37	26.82	21.87	16.66	10.71
D8	39.28	31.11	26.08	20.51	10.52	16.21

. With the HABCES algorithm, a performance increase of up to 46.82% was achieved in the training process compared to when using the ABC algorithm. This ratio was 39.53% for the test. Compared to the aABC algorithm, the HABCES algorithm achieved performance improvements of up to 37.96% and 35.00%, respectively, for the training and testing processes. At the same time, the HABCES algorithm was more successful than the ABCES algorithm. With respect to the ABCES algorithm, a 20.48% performance increase was achieved in the training process. For the testing process, this ratio was 25.77%. Figure 8 presents a comparison of the convergences of the ABC, aABC, ABCES and HABCES algorithms. In D1, D2, D4, D6, D7 and D8, the HABCES algorithm showed a rapid convergence and reached effective solutions in a short time. In D3 and D5, the HABCES and ABCES algorithms had good convergence. A comparison of the real output and the predicted output for nonlinear dynamic system identification is given in Figure 9. It can be understood from the overlapping output graphics that successful modeling for all systems was achieved when using the HABCES algorithm.

4. Discussion

This study aims to develop a variant with strong global and local search capability for neural network training. The HABCES algorithm is a hybrid algorithm consisting of a combination of the aABC algorithm and ABCES algorithm. The strengths of both algorithms have been taken, making the HABCES algorithm a powerful algorithm with effective global and local convergence capability.

The arithmetic crossover feature was adapted for the HABCES algorithm, and knowledge of the global best solution was used. At the same time, the width of the search area was determined adaptively. This adaptive feature generally resulted in improved solution quality at each iteration. These two features provided a strong convergence feature. In particular, the determination of the solutions in the scout bee phase by means of an effective solution generation mechanism, rather than being randomly generated, ensured the preservation of the gains obtained. All of these changes strengthened the convergence ability of the HABCES algorithm and improved its solution quality. Although the HABCES algorithm was developed as a neural network training algorithm, it should be emphasized that it was also successful at solving global optimization problems. This fact expands the usage area of the algorithm. To interpret the performance of the HABCES algorithm with respect to different variants, it was compared with the standard ABC, aABC and ABCES algorithms. The results showed that the HABCES algorithm was more successful than the aABC and ABCES algorithms both at solving global optimization problems and at training the neural network.

The HABCES algorithm was first applied to solving global optimization problems. Forty-three benchmark test functions were used. These test functions were examined under two groups: high-dimensional and low-dimensional problems. High-dimensional problems can be more difficult or complex due to their higher number of dimensions. Due to the fast convergence of the HABCES algorithm, it was found that it was able to approach the optimal solution within a short period of time. The other variants fell short of the performance achieved by the HABCES algorithm. For low-dimensional optimization problems, it may be easier to reach the optimal result in a low number of iterations. Therefore, with increasing difficulty or size of a problem, the success of the HABCES algorithm could be more clearly observed.

In the training of neural networks, the algorithm was applied for the modeling of parity problems, the identification of nonlinear static systems, and the identification of nonlinear dynamic systems. The amount of data used in the training process for solving XOR, 3-bit parity, and 4-bit parity problems was limited. At the same time, the problem became more complex with increasing numbers of inputs. Parallel to this situation, the error value also increased. For these problems, there was a significant performance advantage compared to the other variants of the HABCES algorithm. In particular, the results were obtained with a low number of iterations. The fact that the HABCES algorithm was able to achieve an effective solution in a short time is an important indicator of its success.

Most of the problems encountered in daily life exhibit nonlinear behavior. Therefore, it is important to identify nonlinear systems. Nonlinear systems are inherently difficult problems. Effective training algorithms are required in order to obtain successful results. Nonlinear systems can be static or dynamic. In this study, the algorithm was applied for both system types. It can be observed that the HABCES algorithm was able to achieve effective results in a short period of time for the identification of nonlinear systems. When examining the output graphs obtained for these systems, the HABCES algorithm can be seen to have been successful. The graphs of real and predicted output overlap. The success of the HABCES algorithm at performing neural network training for the identification of these systems shows that it can also be used for solving different problems. The aABC algorithm stands out as a result of its arithmetic crossover and adaptive step size. The ABCES algorithm also stands out due to its adaptive limit value, as well as its solution generation mechanism at the scout bee stage. These features affect the result differently. In the identification of static systems, it can be seen that adaptive limit values and the effective scout bee stage have a more significant positive effect on the results than the arithmetic crossover and adaptive step size. In the identification of dynamic systems, different system behaviors are observed depending on the type of system.

One of the important parameters is standard deviation. In all problem types, effective standard deviation values were obtained in parallel with the solution. Each application was repeated 30 times, and the mean error/fitness values were obtained. The effective standard deviation values show that the obtained results can be achieved again within a certain tolerance. In particular, meta-heuristic algorithms usually start with random solutions. The closeness of the obtained results to one another also increases the confidence in the algorithm.

It can be observed that the network structures created in these applications affect the performance. In particular, an increase in the number of neurons resulted in an increase in performance in some cases, while decreasing the quality of the solution in other cases. This is related to the difficulty of the problems.

This study has limitations in terms of population size, maximum number of generations, network structures, and problem types. A value of 20, one of the values for which the ABC algorithm is effective, was used as the colony size. For the maximum number of generations, different values were selected depending on the problem type. In neural network training, it is known that the network structure affects performance. In this study, the applications were realized using a limited number of network structures for each problem. Two different types of problem were used. These were the solving of global optimization problems and the training of a neural network. The success of the HABCES algorithm should be evaluated within the context of these limitations.

5. Conclusions

This study proposed a new variant referred to as the HABCES algorithm for neural network training. Effective solution generation mechanisms were used in the employed bee, onlooker bee, and scout bee stages of the HABCES algorithm. The algorithm basically incorporates four changes. The first is the usage of arithmetic crossover between the current solution and the global best solution. Secondly, an adaptive approach was used in the selection of the local search area. Specifically, it changes depending on the number of generations. This can be thought of as a dynamic step size. Third, the limit control parameter was dynamically adjusted depending on the generation number. Fourth, in the scout bee stage, new solutions were created using an effective solution generation mechanism instead of randomly generating them.

Unlike the standard ABC algorithm, the HABCES algorithm was able to reach effective solutions in a short time as a result of these four fundamental innovations. In particular, as the dimensions and difficulty of the problem increased, the success of the HABCES algorithm could be more clearly observed. This was particularly evident with the solution of global optimization problems. At the same time, the HABCES algorithm was generally more successful than the standard ABC, ABC and ABCES algorithms for solving global optimization problems.

To create effective models with ANN, a successful training algorithm is required. Therefore, this study focused on developing a more successful algorithm for ANN training. In ANN training, there are different parameters that affect performance in addition to the training algorithm. Number of inputs, number of layers, number of neurons, activation function, and transfer functions are among them. This was observed in application. The structure of the neural network affected the performance. This effect varied depending on the type of problem. In some problems, increasing the number of neurons improved the solution quality. In others, the opposite situation was observed. Within its limitations, the HABCES algorithm seemed to be effective on the basis of convergence graphs, output graphs, and simulation results.

Nonlinear systems are inherently complex and difficult. Effective results were obtained when using the HABCES algorithm for the identification of the related systems. In particular, it had a significant advantage over variants such as aABC and ABCES. In nonlinear static systems, although it varies according to the system, the training performance increased by up to 54.84% with reference to the aABC algorithm. In comparison with the ABCES algorithm, this improvement reached 32.56%. Similar success was achieved for nonlinear dynamic systems. For nonlinear dynamic systems, these improvements were 37.96% and 20.48%, respectively. These rates increased even more with respect to the standard ABC algorithm. Compared to the ABC algorithm, performance increases of up to 69.57% and 46.82% were achieved.

The proposed variant was used for ANN training for the identification of nonlinear systems in this study. As is widely known, ANN is used in modeling of many systems. In future studies, the HABCES algorithm can be used as a training algorithm for modeling different systems, or new variants can be suggested by creating new updates to increase the performance of the HABCES algorithm.