Enhanced Brain Storm Optimization Algorithm Based on Modified Nelder–Mead and Elite Learning Mechanism

Abstract

Brain storm optimization algorithm (BSO) is a popular swarm intelligence algorithm. A significant part of BSO is to divide the population into different clusters with the clustering strategy, and the blind disturbance operator is used to generate offspring. However, this mechanism is easy to lead to premature convergence due to lacking effective direction information. In this paper, an enhanced BSO algorithm based on modified Nelder–Mead and elite learning mechanism (BSONME) is proposed to improve the performance of BSO. In the proposed BSONEM algorithm, the modified Nelder–Mead method is used to explore the effective evolutionary direction. The elite learning mechanism is used to guide the population to exploit the promising region, and the reinitialization strategy is used to alleviate the population stagnation caused by individual homogenization. CEC2014 benchmark problems and two engineering management prediction problems are used to assess the performance of the proposed BSONEM algorithm. Experimental results and statistical analyses show that the proposed BSONEM algorithm is competitive compared with several popular improved BSO algorithms.

1. Introduction

In recent years, as an effective class of optimization methods, swarm intelligence (SI) optimization algorithms are becoming more and more popular in solving complex practical problems in the fields of scientific research and engineering applications. Based on the concepts of self-organized coordination in natural swarm systems, the SI algorithms transfer information through communication with multiple local individuals to reach a common global goal collaboratively. Therefore, the collective behaviors are the core design of swarm intelligence algorithms for optimization problems [1]. For the SI algorithms, the solutions of the optimization problem are regarded as individuals of the population, and the population evolves continually via group collaboration. Some popular SI algorithms include ant colony optimization (ACO) [2], particle swarm optimization (PSO) [3], artificial bee colony optimization (ABC) [4], biogeography-based optimization (BBO) [5], cuckoo search (CS) [6], bat algorithm (BA) [7], etc. Moreover, some new optimization algorithms are proposed to solve optimization problems, such as an improved moth-flame optimization algorithm(I-MFO) [8], arithmetic optimization algorithm (AOA) [9], migration-based moth-flame optimization algorithm (M-MFO) [10], aquila Optimizer (AO) [11], red fox optimization algorithm (RFO) [12], remora optimization algorithm (ROA) [13], and chaos cloud quantum bat hybrid optimization algorithm (CCQBA) [14].

The brain storm optimization algorithm (BSO) [15] proposed by Shi in 2011 imitates the thinking and learning process of human beings to solve complex tasks. In the BSO, the population (ideas or solutions) is divided into several clusters. On the one hand, each parent idea is generated by learning from other ideas in intracluster or intercluster. On the other hand, the new idea is generated by a random search operator on the parent idea. Research findings indicate that the BSO algorithm has been successfully applied to solve optimization problems in different fields, such as feature selection [16], power system fault diagnosis [17], data classification [18], stock price forecasting [19], visual fixation prediction [20], Loney’s solenoid problem [21], electromagnetic applications [22], networks [23], and multimodal optimization [24].

In the BSO, the challenge can be mainly attributed to the effective balance mechanism between exploration and exploitation, and overexploitation may lead to premature convergence [25]. Then, different variants of BSO are proposed to improve the diversity or convergence of the algorithm, which can be mainly divided into two categories.

One category focuses on developing effective clustering methods. K-means is one of the popular clustering methods. For example, Bezdan et al. [26] proposed a hybrid swarm intelligence algorithm with a K-means algorithm to solve text clustering. The original BSO algorithm uses K-means clustering method, which has a high computational cost. In order to reduce the computational complexity, Zhan et al. [27] proposed a simple grouping method (SGM), where the seed of each group keeps unchanged to alleviate computational burden. Zhu et al. [28] proposed K-medians clustering method, which takes the median of the individuals, rather than the average value, as the new cluster center. Cao et al. [29] proposed a random grouping strategy, where a new dynamic step-size parameter control strategy is employed in the idea generation process. Chen et al. [30] proposed the role-playing strategy, which divides all ideas into three categories, namely innovative ideas, conservative ideas and common ideas. Shi [31] proposed a novel algorithm called brain storm optimization algorithm in objective space (BSO-OS). In the BSO-OS, instead of using the clustering algorithm, the convergent operation is implemented in the objective space to divide all individuals into two categories, namely elitists and normal.

The other category focuses on designing new learning strategies. To solve the problems, such as that the transfer function used in the BSO may not effectively capture the search features well, and the global search ability is poor when the search range is large, the idea difference strategy (IDS) is designed in a modified brain storm optimization (MBSO) algorithm [27]. Zhou et al. [32] proposed a step-size adaptive strategy, which is dynamically adjusted according to the current state of the individuals. If the individuals are aggregated, the step-size is relatively small. Otherwise, the step-size is relatively large. Cheng et al. [33] proposed the partial re-initialization strategy to alleviate premature convergence in the BSO algorithm. In detail, a certain number of solutions are re-initialized at each given iteration. Moreover, the number of re-initialized solutions decreases linearly at each re-initialization. Shen et al. [34] proposed an adaptive learning strategy, which guides the individual to learn from the global best individual and its cluster center, so as to improve the search ability and stability of the proposed algorithm. El-Abd proposed a global-best brain storm optimization algorithm (GBSO) [35], which uses the global-best idea information to update the population.

The basic BSO algorithm has the following disadvantages. (1) The greedy selection strategy is used to select the next population. If the population evolution deviates from the direction of global optimal solution, it may increase the possibility of falling into the local optimal solution. (2) The random noise disturbance operator [36] is used to generate new ideas; however, it is difficult for the operator to make use of potential valuable information in the search process. (3) The construction of S_select depends largely on the center of a cluster. In some worse cases, a cluster with a local optimal solution may cause a premature convergence. Therefore, it is necessary to improve the exploitation ability of the algorithm to record the historical optimal solution and search near the historical optimal solution.

Some weaknesses of the basic BSO have been overcome by proposing new clustering method or designing new learning strategy. However, the imbalance of exploration and exploitation is still a key issue for the performance of the algorithm. Moreover, to the best of our knowledge, the Nelder–Mead simplex search method [37] has not been used in the BSO before. Based on the above discussion, we propose an enhanced BSO algorithm based on modified Nelder–Mead and elite learning mechanism (BSONME) to solve these weak points. The main contribution of the proposed BSONME algorithm are as follows.

(1)

The basic Nelder–Mead method is modified and used as a local searcher to explore more promising search direction for the population, so as to reduce the probability of falling into local optimum;

(2)

An elite learning mechanism based on Euclidean distance is proposed to update the population with a certain probability. The evolution information hidden in the elite individuals is integrated into the population, which improves the convergence of the algorithm;

(3)

The reinitialization strategy is employed when the population is at the stagnation stage. The well-directed and revolutionary jumping can help the population jump out of the local optimum with a large probability, so as to explore a new evolutionary direction;

(4)

CEC2014 contest benchmark problems and two engineering prediction problems are used to evaluate the performance of the proposed algorithm.

The structure of this paper is as follows. The original BSO algorithm and the Nelder–Mead method are briefly introduced in Section 2. The proposed BSONME algorithm is described in Section 3. In Section 4, experiments on CEC2014 benchmark problems are given. Moreover, two practical engineering problems are used to evaluate the performance of the proposed BSONME. Section 5 summarizes the whole work and puts forward the next research direction.

2. Preliminaries

2.1. Brain Storm Optimization Algorithm

Similar to other SI algorithms, the population is composed of N D-dimensional individuals in the BSO algorithm. Each individual X_i is termed as an idea, i.e., candidate solution. First, the population is initialized randomly. Then, the BSO algorithm performs a loop of optimization operations: clustering, parent idea construction, new idea generation, and selection.

(1)

Clustering: At each generation, the population is divided into M clusters according to K-means algorithm. Each cluster selects the best idea as its center;

(2)

Parent idea construction: According to the probability index, one or two clusters are used to construct the parent individual X_select.

One situation is that X_select is generated from two clusters. First, two different clusters are randomly selected based on the given probability. Next, there are two methods to generate the individual X_select. One method is that X_select is generated based on a linear combination of two centers from two different clusters. The other method is that X_select is generated based on a linear combination of two ideas from two different clusters.

The other situation is that X_select is generated from one cluster. The cluster is selected according to the roulette strategy. In other words, the more ideas in the cluster, the higher the probability of being selected. The probability of selecting jth cluster p_j is defined as follows:

$$p_j=\frac{\left|M_j\right|}{N}, j=1,\dots ,M$$

(1)

where M is the number of clusters, |M_j| is the size of the jth cluster, and N is the population size.

(3)

New idea generation: After parent idea X_select is generated, a new idea is generated according to the following formula:

$$X_{new,d}=X_{select,d}+\xi \times N\left(\mu ,\sigma \right)$$

(2)

where X_select,d is the dth dimension of the selected idea. X_new,d is the dth dimension of newly generated idea. N (μ, σ) is the Gaussian distribution function with mean μ and variance σ, and $\xi$ is defined as follows:

$$\xi =logsig\left(\frac{0.5\times T-t}{k}\right)\times rand \left(0,1\right)$$

(3)

where $T$ is the maximum number of iterations. $t$ is the current number of iterations, and $k$ is the slope coefficient, which is used to control the convergence speed of the algorithm. rand (0,1) is a number randomly distributed in the range of [0, 1]. logsig() is a sigmoid activation function.

It can be seen from Equation (3) that $\xi$ decreases gradually as the increase of the number of iterations. Thus, the algorithm focuses on exploration in the early stage of evolution, and focuses on exploitation in the later stage of evolution.

(4)

Selection: The greedy selection strategy is used to select the better one from the parent idea X_i and the newly generated idea X_new according to their fitness. The better one is kept as a parent idea in the next generation.

The pseudo code of BSO algorithm is described by Algorithm 1.

Algorithm 1 The BSO algorithm
1:	Randomly generate N individuals
2:	Evaluate N individuals
3:	while the termination criteria are not met do
4:	Divide N individuals into M clusters by K-means
5:	Select the cluster center for each cluster
6:	if rand < p1 then
7:	Randomly generated an individual to replace a selected cluster center
8:	end if
9:	for each individual Xi do
10:	if rand < pone then
11:	Select one cluster c according to Equation (1)
12:	if rand < pone_center then
13:	Xselect ← the center of the cluster c
14:	else
15:	Xselect ← Random select an individual from cluster c
16:	end if
17:	else
18:	Randomly select two clusters c1 and c2
19:	if rand < ptwo_center then
20:	Generate Xselect according to linear combination rule
21:	else
22:	Randomly select an individual from X1 and X2, respectively
23:	Generate Xselect according to linear combination rule
24:	end if
25:	end if
26:	Generate new idea Xnew according to Equation (2)
27:	if f(Xnew) < f(Xi)then
28:	Xi ←Xnew
29:	f(Xi) ← f(Xnew)
30:	end if
31:	end for
32:	end while

2.2. Nelder–Mead Method

The Nelder–Mead method [37], proposed by Nelder et al., is a simplex search method for multidimensional unconstrained minimization. The Nelder–Mead method includes four key operations: reflection, expansion, contraction and shrinkage.

At first, (n + 1) points x₁, x₂, …, x_n+1 are initialized in the D-dimension search space. These points are sorted in ascending order according to their fitness values.

The centroid point $\overline{x}$ composed of the first $n$ points is calculated according to the formula:

$$\overline{x}={{\displaystyle \sum }}_{i=1}^n\frac{x_i}{n}$$

(4)

Then, the reflection point x_r is given as follows:

$$x_r=\overline{x}+\rho \left(\overline{x}-x_{n+1}\right)$$

(5)

where $\rho >0$ is the reflection coefficient, and x_n+1 is the worst point.

Next, the expansion point x_e is calculate according to the formula:

$$x_e=\overline{x}+\chi \left(x_r-\overline{x}\right)$$

(6)

where χ > max{1, ρ} is the expansion coefficient.

If the fitness of the reflection point x_r is worse than that of the second worst x_n, it is necessary to perform outside contraction or inside contraction. Here, the fitness of a point refers to its function value. When the fitness of the reflection point x_r is better than that of the worst point x_n+1, the outside contraction operation is performed as follows.

$$x_{oc}=\overline{x}+\gamma \left(x_r-\overline{x}\right)$$

(7)

Otherwise, the inside contraction operation is performed as follows.

$$x_{ic}=\overline{x}-\gamma \left(\overline{x}-x_{n+1}\right)$$

(8)

where 0 < γ < 1 is the contraction coefficient.

If the fitness of the outside contraction point x_oc is worse than that of the reflection point x_r, or the fitness of the inside contraction point x_ic is worse than that of x_n+1, the shrink operation for ith point x_i is performed as follow.

$$v_i=x_1+\sigma \left(x_i-x_1\right)$$

(9)

where 0 < σ< 1 is the shrink coefficient. The detailed information of the above mentioned four coefficients can be found in [37].

3. Proposed Method

3.1. Motivations

As mentioned before, the convergent operation and the population generation strategy used in the BSO algorithm may not be able to effectively adjust the exploration and exploitation search [36]. Therefore, the variants of BSO are proposed, such as MBSO, the improved random grouping BSO (IRGBSO) [38], and BSO-OS. MBSO proposed a new population generation strategy. IRGBSO introduced the re-initialized ideas. BSO-OS developed an effective grouping method. In order to evaluate the performance of the variants of BSO, four representative problems from CEC2014 benchmark problems [39] are selected. The representative problems include 1 unimodal problem (F₂), 1 multimodal problem (F₁₂), 1 hybrid problem (F₁₈), and 1 composite problems (F₂₃). Four representative problems are given in Appendix A. The detailed information of four representative problems can be found in [39].

The convergence curves in Figure 1 illustrate the mean value of solution error (in logarithmic scale) of BSO, MBSO, IRGBSO and BSO-OS. Details of experimental parameters are given in

Table 1. Details of experimental parameters.

Population Size	100
Solution Error	F(x) − F(x*)
F(x)	Best fitness value
F(x*)	Real global optimization value
Run times	30
Dimension (D)	30
Termination Criterion	D × 10,000

.

Figure 1a shows that MBSO performs better than BSO, BSO-OS, and IRGBSO. It indicates that the population generation strategy proposed in MBSO is effective. Figure 1b,c show that BSO-OS performs better than BSO, MBSO and IRGBSO. It can be concluded that effective grouping method can improve the performance of the algorithm. Figure 1d shows that IRGBSO performs better than BSO, MBSO, and BSO-OS. It can be seen that the re-initialization strategy is effective for some optimization problems. Although MBSO, BSO-OS, and IRGBSO perform better than BSO on some optimization problems, it can be seen from Figure 1 that the convergence curves of these algorithms are close to those of BSO algorithm. Therefore, it is necessary to design more effective strategies to improve the performance of the algorithm.

The effectiveness of BSO in solving optimization problems depends on the idea updating strategy and idea clustering method, which can generate better offspring and guide the population towards a promising area. However, Figure 1 shows that different optimization problems require different strategies to explore or exploit. Moreover, to solve a specific problem, it may be better to fully consider the characteristics of the landscape where the parent idea is located and use different strategies to explore or exploit during different stages of the evolution. Figure 2 shows an example of the potential advantages of using the worst individual to generate a new individual. In Figure 2, the X-axis represents the search space, and the F(X)-axis represents the function value of the optimization problem. The black solid dot X_w indicates the worst individual in the current population. Obviously, X_w will be eliminated in the evolution process due to the greedy selection. The new individual X_new (red solid dot) deviates from the correct evolution direction due to the loss of diversity. Therefore, the population is easy to cluster together in the local optimum region after continuous iterative evolution. If we make a local search around X_w before it is eliminated, then $X_{ne{w}^{\prime }}$ (red solid dot) will be generated. Although the fitness of $X_{ne{w}^{\prime }}$ is worse than that of X_new, $X_{ne{w}^{\prime }}$ has the potential to be closer to the global optimal solution.

Motivated by these observations, we endeavor to propose an enhanced BSO algorithm based on modified Nelder–Mead (MNM) and elite learning mechanism. First, the Nelder–Mead method is modified and used as a local search engine to fully explore more promising evolutionary direction, which can improve the exploration ability of the population. Next, the elite learning mechanism is used to enhance the exploitation ability for the evolution of the population. Finally, when the population is stagnant, the opposition learning strategy is employed to re-initialize the parent individual, which can make the population diverge into large search area and increase possibility to jump out of the local optimum.

3.2. BSONME Optimization Algorithm

3.2.1. Modified Nelder–Mead Updating Strategy

In order to enhance the local exploration ability of the population, the modified Nelder–Mead updating strategy is used to generate a new individual. For an individual X_i, the process of generating a new individual X_NM is as follows. Firstly, the individuals are assumed to be organized on a ring topology in connection with their indices. The ith neighborhood includes n + 1 individuals, which is illustrated in Figure 3.

Then, the n + 1 individuals are sorted in ascending order according to their fitness values. $\overline{X}$ denotes the centroid of the ith neighborhood. X_best and X_w denote the best individual and the worst individual in the ith neighborhood, respectively. Next, the reflection transformation is used to generate a reflection individual X_newr according to the following equation. α is set to 2.

$$X_{newr}=\alpha \overline{X}+X_w$$

(10)

$$\overline{X}={{\displaystyle \sum }}_{i=1}^n\frac{X_i}{n}$$

(11)

If the fitness of the reflection individual X_newr is worse than that of X₁, X_newr is extended in the same direction by using the expansion transformation. The expansion individual X_newe is generated as follow.

$$X_{newe}=\overline{X}+\alpha \left(X_{newr}-\overline{X}\right)$$

(12)

If the fitness of the reflection individual X_newr is worse than that of all individuals in the neighborhood, the outside contraction individual X_oc or the inside contraction individual X_ic is calculated as follows:

$$X_{oc}=\overline{X}+\frac{\left(\overline{X}-X_w\right)}{\alpha }$$

(13)

$$X_{ic}=\frac{\overline{X}+X_w}{\alpha }$$

(14)

The pseudo code of modified Nelder–Mead updating strategy (NMS) is described by Algorithm 2.

Algorithm 2 Modified Nelder–Mead updating strategy
1:	Input the individuals in ith neighborhood
2:	while the termination criteria are not met do
3:	Sort the individuals in ascending order according to fitness
4:	Generate a reflection individual Xnewr according to Equation (10)
5:	if f(Xnewr) < f(Xbest) then
6:	Generate an expansion individual Xnewe according to Equation (12)
7:	if f(Xnewe) < f(Xnewr) then
8:	Xw ← Xnewe; f(Xw) ← f(Xnewe)
9:	else
10:	Xw ← Xnewr; f(Xw) ← f(Xnewr)
11:	end if
12:	else
13:	if f(Xnewr) < f(Xw) then
14:	Xw ← Xnewr; f(Xw) ← f(Xnewr)
15:	else
16:	if rand < 0.5 then
17:	Generate an outside contraction individual Xoc according to Equation (13)
18:	if f(Xoc) < f(Xw) then
19:	Xw ← Xoc; f(Xw) ← f(Xoc)
20:	end if
21:	else
22:	Generate an inside contraction individual Xic according to Equation (14)
23:	if f(Xic) < f(Xw) then
24:	Xw ← Xic; f(Xw) ← f(Xic)
25:	end if
26:	end if
27:	end if
28:	end if
29:	end whlie
30:	Sort the individuals in ascending order according to fintness
31:	Return Xbest and f(Xbest)

3.2.2. Elite Learning Mechanism

The individuals learn from the elite individual, which may lead to a fast converging of the algorithm. A typical case is the role of gbest, the historically best position of the population, used in PSO algorithm. However, for multimodal problems, if the gbest is a local optimum, the algorithm is easy to fall into premature convergence.

Figure 4 shows that the distance between X_best and gbest will significantly affect the result of updating gbest. As illustrated in Figure 4, gbest (red dot) is the historical optimal solution stored in the archive. X_best is a newly generated solution with better fitness than gbest. In Figure 4a, if gbest is updated by X_best, the algorithm will fall into local optimization with great probability. It is advisable not to update gbest to avoid premature convergence. In Figure 4b, X_best is far away from gbest. It is better to update gbest with X_best to reduce the possibility of falling into local optima.

To make full use of the advantages of elite individuals and avoid premature convergence, this paper proposed an elite learning mechanism based on Euclidean distance. Here, the elite individual is denoted as gbest. In the first generation, the individual with the best fitness is selected as gbest. Figure 5 is the landscape of a two-dimensional optimization problem. Figure 6 shows the different situations for triggering the update operation. In Figure 6, gbest (the red pentagram) is the historical optimum solution. Blue squares and black triangles represent better and worse individuals, respectively. R is the average of the distance between the elite individuals and the gbest. A and B are the optimal solutions of the current generation, which may be used to update gbest. If the optimum solution A (the red quadrangle) obtained in the current generation falls into the circle, the update operation is not triggered to avoid the population falling into local optimization. On the contrary, if the optimum solution B (the red quadrangle) obtained in the current generation falls outside the circle, the update operation is triggered to guide the population to explore a more promising region.

gbest is updated as follows.

$$gbes{t}_t=\{\begin{array}{c}{X}_{best,t} if Dis\left(X_{best,t},gbes{t}_{t-1}\right)>R\\ gbes{t}_{t-1} otherwise \end{array}$$

(15)

$$R=\frac{{{\displaystyle \sum }}_{i=1}^{\left|elites\right|-1}Dis\left(X_{i,t},gbes{t}_{t-1}\right)}{\left|elites\right|-1}$$

(16)

where Dis(·) is the Euclidean distance, and X_best,_t is the optimal solution at generation t; gbest_t−1 is the optimal solution at generation t − 1; and X_i,t is ith individual in the elite group, which is different from X_best,t. |elites| is the size of the elite group.

The elite learning mechanism is described as follows:

$$X_{new,t}=\{\begin{array}{c}gbes{t}_t+r_1\times \left(X_{a,t}-X_{b,t}\right) if rand<C\\ X_{select,t}+r_1\times \left(X_{a,t}-X_{b,t}\right) otherwise \end{array}$$

(17)

where X_select,t is the selected individual at generate t, X_a,t and X_b,t are two randomly selected individuals, and the parameter C increases gradually in the iterative process, so as to speed up the convergence of the algorithm. The parameter $C$ is defined as follow [35].

$$C=C_{min}+\frac{t}{T}\times \left(C_{max}-C_{min}\right)$$

(18)

where t and T represent the current generation and the maximum generation, respectively. C_min and C_max are set to 0.2 and 0.8, respectively.

3.2.3. Reinitialization Strategy

The reinitialization strategy is usually used to solve the problem of individual homogenization in the population caused by fast convergence. However, if the individual is initialized in the whole search space, the fitness of the generated individual is worse in a high probability. Then, the generated individual may easily be discarded in the evolution process. Moreover, in the later stage of evolution, the differences between individuals tend to narrow, hence, small scale disturbance on individuals cannot effectively ensure the population diversity. Research shows that opposition-based learning (OBL) can effectively accelerate the search or process because estimates and counter estimates are considered simultaneously in OBL [40]. Therefore, this paper uses OBL as the reinitialization strategy. Here, the reinitialization strategy is used according to the current convergence status. Specially, each individual is assigned a counter. If an individual is updated in each iteration, its counter is zero, otherwise its counter is incremented by 1. The reinitialization strategy is employed if the counter exceeds a given threshold.

Let X(x₁, x₂,…, x_D) be a solution in D-dimensional space, the opposite solution of X is recorded as $X^{\prime }\left(x_1^{\prime },x_2^{\prime },\dots ,x_D^{\prime }\right)$ and is defined as follows:

$$x{\prime }_i=a_i+b_i-x_i$$

(19)

where x_i ∈ [a_i, b_i], i = 1,2, …, D.

Figure 7 illustrates the flow chart of the BSONME algorithm.

The pseudo code of BSONME is shown in Algorithm 3. The probability of random disturbance to an individual is denoted as p₁. The probability of selecting the elite subpopulation is denoted as p₂. The probability of selecting different strategies to generate new individuals is denoted as p₃. r₁ is a number randomly distributed in the range of [0, 1]. TH denotes the threshold.

Algorithm 3 The BSONME algorithm
1:	Randomly generate N individuals
2:	Evaluate N individuals
3:	Randomly select an individual as gbest
4:	while the termination criteria are not met do
5:	Divide the population into elite population and non-elite population
6:	Update gbest according to Equation (15)
7:	if rand < p1 then
8:	Randomly select an individual to update a selected dimension
9:	end if
10:	for each individual Xi do
11:	if rand < p2 then
12:	if rand < p3 then
13:	Xselect ← Randomly select an individual from the elite subpopulation
14:	else
15:	Randomly select Xi and Xj from the elite subpopulation (i ≠ j)
16:	Xselect = r1 × Xi + (1 − r1) × Xj
17:	end if
18:	else
19:	if rand < p3 then
20:	Xselect ← Randomly select an individual from the nonelite subpopulation
21:	else
22:	Randomly select Xi and Xj from the non-elite subpopulation (i ≠ j)
23:	Xselect = r1 × Xi + (1 − r1) × Xj
24:	end if
25:	end if
26:	Generate new idea Xnew according to Equation (17)
27:	Evaluate Xnew
28:	if f(Xnew) < f(Xi) then
29:	XNMA ← perform NMS on Xi according to algorithm 2;
30:	if f(XNMA) < f(Xnew) then
31:	Xi ← XNMA; f(Xi) ← f(XNMA)
32:	else
33:	Xi ← Xnew; f(Xi) ← f(Xnew)
34:	end if
35:	counti = 0
36:	else
37:	if counti > TH
38:	Update Xi with reinitialization strategy according to Equation (19)
39:	counti = 0
40:	else
41:	counti = counti + 1
42:	end if
43	end if
44:	end for
45:	end while

4. Comparative Studies of Experiments

In this section, experimental comparisons are made with the original algorithms and some popular BSO variants. All algorithms are tested on the CEC2014 contest benchmark problems [39] and prediction problems in the real world. The compared algorithms are listed as follows:

• Particle swarm optimization (PSO) [3];
• Brain storm optimization algorithm (BSO) [15];
• Modified brain storm optimization (MBSO) [27];
• Brain storm optimization algorithm in objective space (BSO-OS) [31];
• Random grouping brain storm optimization algorithm (RGBSO) [29];
• Improved random grouping BSO (IRGBSO) [38];
• BSO with learning strategy (BSOLS) [41];
• Active learning brain storm optimization (ALBSO) [42];
• BSO with role-playing strategy (RPBSO) [30];
• Brain storm optimization algorithm with adaptive learning strategy (BSO-AL) [34];
• Our approach (BSONME).

4.1. Parameter Settings

In this section, CEC2014 contest benchmark problems and two prediction problems are used to evaluate the performance of eleven algorithms. Details of benchmark problems and experimental setup are given in

Table 2. Details of CEC2014 benchmark problems and experimental setup.

F1–F3	Unimodal Problems
F4–F16	Simple Multimodal Problems
F17–F22	Hybrid Problems
F23–F30	Composite Problems
[−100, 100]D	Search Range
CPU	Intel Core i7-5500 2.40 GHz
Application Software	Matlab R2016a
Termination Criterion	Maximum number of function evaluations (MaxFES)
MaxFES	D × 10,000
Dimension	D = 30
Independent run times	30
Population Size	100

.

The performance of each algorithm is appraised by the mean value of solution error [43], which is defined in Table 1. To verify the statistical significance of the experimental results, three test measures are used, and the details of these measures are given in

Table 3. Performance measure used in the tests.

Wilcoxon signed-rank test [44]	5% significance level
	Compare BSONME with other compared algorithms
Multiple-problem Wilcoxon’s test [45]	Show the significant differences of the compared algorithm
Friedman’s test [45]	Determine the ranking of all compared algorithms
†	The performance of BSONME is better than that of the corresponding algorithm.
≈	The performance of BSONME is similar to that of the corresponding algorithm.
−	The performance of BSONME is worse than that of the corresponding algorithm.

.

The parameter settings of different comparison algorithms are shown in

Table 4. Parameter settings of compared algorithms.

Algorithm	N	Parameter
PSO	100	c1 = 2, c2 = 2, wmax = 0.9, wmin = 0.4
BSO	100	m = 5, preplace = 0.2, pone = 0.8, ponecenter = 0.4, ptwocenter = 0.5
MBSO	100	m = 5, preplace = 0.2, pone = 0.8, pr = 0.005, ponecenter = 0.4, ptwocenter = 0.5
BSO-OS	100	perce = 0.1, preplace = 0.2, pone = 0.8
RGBSO	100	m = 5, pone = 0.8, ponecenter = 0.4, ptwocenter = 0.5
IRGBSO	100	m = 5, pone = 0.8, ponecenter = 0.4, ptwocenter = 0.5, threshold = 10, F = 0.5
BSOLS	100	m = 5, pone = 0.8, ponecenter = 0.4, ptwocenter = 0.5, pe = 0.1, pl = 0.1, q1 = 0.13
ALBSO	100	m = 5, pone = 0.8, dc = 0.5, ponecenter = 0.4, ptwocenter = 0.5
RPBSO	100	m = 3, pone = 0.5, pr = 0.005
BSO-AL	100	m = 5, pone = 0.8, ponecenter = 0.4, ptwocenter = 0.5
BSONME	100	p1 = 0.2, p2 = 0.2, p3 = 0.8, n = 4, TH = 50

.

4.2. Experiment I: Mathematical Benchmark Problems

Table 5. Experimental results of PSO, BSO, MBSO, BSO-OS, RGBSO, and BSONME on F₁–F₃.

F	PSO	BSO	MBSO	BSO-OS	RGBSO	BSONME
F1	7.66 × 106 †	1.75 × 106 †	5.95 × 104 †	1.69 × 106 †	2.30 × 106 †	2.76 × 104
F2	1.59 × 102 †	8.28 × 103 †	14.5 †	9.95 × 103 †	8.74 × 103 †	7.23 × 10−6
F3	4.17 × 102 †	1.71 × 104 †	5.23 × 10−1 †	5.71 × 103 †	5.47 × 104 †	8.80 × 10−2
†	3	3	3	3	3	\
−	0	0	0	0	0	\
≈	0	0	0	0	0	\

and

Table 6. Experimental results of IRGBSO, BSOLS, ALBSO, RPBSO, BSO-AL, and BSONME on F₁–F₃.

F	IRGBSO	BSOLS	ALBSO	RPBSO	BSO-AL	BSONME
F1	1.30 × 108 †	9.82 × 106 †	5.50 × 108 †	1.32 × 106 †	7.10 × 107 †	2.76 × 104
F2	3.49 × 109 †	1.62 × 104 †	3.31 × 1010 †	5.73 × 103 †	1.04 × 109 †	7.23 × 10−6
F3	3.17 × 104 †	2.95 × 104 †	9.83 × 104 †	1.96 × 103 †	8.34 × 104 †	8.80 × 10−2
†	3	3	3	3	3	\
−	0	0	0	0	0	\
≈	0	0	0	0	0	\

show the statistical results of 11 algorithms for solving three unimodal functions. The results show that BSONME significantly outperforms PSO, BSO, MBSO, BSO-OS, RGBSO, IRGBSO, BSOLS, ALBSO, RPBSO, and BSO-AL on 3, 3, 3, 3, 3, 3, 3, 3, 3, and 3 test problems, respectively. The proposed BSONME can beat other algorithms because the elite mechanism can guide the population move toward the more promising regions, so that the algorithm can find high-quality solutions.

In the case of F₄–F₁₆, the results show that BSONME perform the best on F₈, F₁₀, F₁₁, and F₁₆. BSONME outperforms PSO, BSO, MBSO, BSO-OS, RGBSO, IRGBSO, BSOLS, ALBSO, RPBSO, and BSO-AL on 5, 7, 6, 6, 7, 13, 10, 13, 7, and 12 test problems, respectively.

Table 5, Table 6,

Table 7. Experimental results of PSO, BSO, MBSO, BSO-OS, RGBSO, and BSONME on F₄–F₁₆.

F	PSO	BSO	MBSO	BSO-OS	RGBSO	BSONME
F4	1.61 × 102 †	85.8 †	4.72 †	65.3 †	83.9 †	8.06 × 10−1
F5	20.9 †	20.0 −	20.9 †	20.0 −	20.0 −	20.1
F6	11.0 −	29.3 †	3.24 −	20.7 ≈	31.6 †	20.8
F7	1.09 × 10−2 −	5.26 × 10−4 −	7.72 × 10−3 −	2.49 × 10−3 −	2.14 × 10−3 −	4.14 × 10−2
F8	18.7 †	1.39 × 102 †	31.4 †	12.9 †	1.36 × 102 †	1.63
F9	65.3 −	1.63 × 102 †	36.6 −	1.19 × 102 †	1.79 × 102 †	76.2
F10	5.75 × 102 †	4.27 × 103 †	1.57 × 103 †	1.63 × 102 †	4.27 × 103 †	80.1
F11	2.89 × 103 ≈	4.15 × 103 †	3.02 × 103 ≈	3.20 × 103 †	4.33 × 103 †	2.82 × 103
F12	1.78 †	2.01 × 10−2 −	2.41 †	1.66 × 10−2 −	4.30 × 10−2 −	2.62 × 10−1
F13	4.40 × 10−1 ≈	3.33 × 10−1 −	2.49 × 10−1 −	3.14 × 10−1 −	3.54 × 10−1 −	4.77 × 10−1
F14	2.99 × 10−1 ≈	2.18 × 10−1 −	2.85 × 10−1 ≈	2.06 × 10−1 −	2.10 × 10−1 −	2.83 × 10−1
F15	7.87 −	7.68 −	2.98 −	5.88 −	15.3 −	22.7
F16	11.1 ≈	12.6 †	11.6 †	11.2 †	12.8 †	11.0
†	5	7	6	6	7	\
−	4	6	5	6	6	\
≈	4	0	2	1	0	\

,

Table 8. Experimental results of IRGBSO, BSOLS, ALBSO, RPBSO, BSO-AL, and BSONME on F₄–F₁₆.

F	IRGBSO	BSOLS	ALBSO	RPBSO	BSO-AL	BSONME
F4	5.17 × 102 †	61.1 †	4.45 × 103 †	12.4 †	2.12 × 102 †	8.06 × 10−1
F5	20.3 †	20.7 †	21.1 †	21.0 †	20.7 †	20.1
F6	25.0 †	34.4 †	37.8 †	2.77 × 10−1 −	37.9 †	20.8
F7	37.1 †	1.82 × 10−1 †	3.19 × 102 †	2.47 × 10−4 −	5.83 †	4.14 × 10−2
F8	1.80 × 102 †	1.55 × 102 †	2.71 × 102 †	31.3 †	1.58 × 102 †	1.63
F9	2.05 × 102 †	1.97 × 102 †	3.06 × 102 †	54.7 −	2.03 × 102 †	76.2
F10	3.73 × 103 †	3.58 × 103 †	6.51 × 103 †	1.34 × 103 †	3.89 × 103 †	80.1
F11	4.12 × 103 †	3.91 × 103 †	6.95 × 103 †	6.25 × 103 †	4.30 × 103 †	2.82 × 103
F12	5.84 × 10−1 †	9.20 × 10−1 †	2.98 †	2.45 †	9.19 × 10−1 †	2.62 × 10−1
F13	8.33 × 10−1 †	4.79 × 10−1 ≈	5.17 †	3.46 × 10−1 −	3.84 × 10−1 −	4.77 × 10−1
F14	12.9 †	3.32 × 10−1 ≈	1.28 × 102 †	2.82 × 10−1 ≈	1.75 †	2.83 × 10−1
F15	70.0 †	22.2 ≈	1.17 × 104 †	15.3 −	2.81 × 104 †	22.7
F16	11.8 †	12.3 †	13.1 †	12.2 †	12.9 †	11.0
†	13	10	13	7	12	\
−	0	0	0	5	1	\
≈	0	3	0	1	0	\

,

Table 9. Experimental results of PSO, BSO, MBSO, BSO-OS, RGBSO, and BSONME on F₁₇–F₂₂.

F	PSO	BSO	MBSO	BSO-OS	RGBSO	BSONME
F17	6.43 × 105 †	1.21 × 105 †	8.73 × 103 †	8.87 × 104 †	2.05 × 105 †	4.19 × 103
F18	5.16 × 103 †	1.99 × 103 ≈	3.79 × 103 ≈	1.88 × 103 ≈	1.60 × 103 ≈	1.51 × 103
F19	10.1 †	18.7 †	5.94 −	13.8 †	24.8 †	9.40
F20	6.34 × 102 −	1.04 × 104 †	1.51 × 102 ≈	1.60 × 104 †	2.42 × 104 †	2.24 × 103
F21	1.47 × 105 †	6.03 × 104 †	1.01 × 104 †	7.91 × 104 †	1.01 × 105 †	1.62 × 103
F22	2.57 × 102 −	8.41 × 102 †	1.61 × 102 −	7.69 × 102 ≈	9.25 × 102 †	6.67 × 102
†	4	5	2	4	5	\
−	2	0	2	0	0	\
≈	0	1	2	2	1	\

,

Table 10. Experimental results of IRGBSO, BSOLS, ALBSO, RPBSO, BSO-AL, and BSONME on F₁₇–F₂₂.

F	IRGBSO	BSOLS	ALBSO	RPBSO	BSO-AL	BSONME
F17	2.59 × 106 †	7.29 × 105 †	3.45 × 107 †	4.03 × 104 †	7.18 × 106 †	4.19 × 103
F18	1.52 × 106 †	4.01 × 103 †	4.18 × 108 †	3.08 × 103 ≈	2.66 × 107 †	1.51 × 103
F19	58.2 †	21.1 †	2.49 × 102 †	5.13 −	20.6 †	9.40
F20	2.06 × 104 †	6.82 × 103 †	1.10 × 105 †	2.92 × 103 †	1.31 × 105 †	2.24 × 103
F21	4.63 × 105 †	2.73 × 105 †	9.88 × 106 †	1.25 × 104 †	4.14 × 106 †	1.62 × 103
F22	5.50 × 102 −	8.86 × 102 †	1.45 × 103 †	64.0 −	1.20 × 103 †	6.67 × 102
†	5	6	6	3	6	\
−	1	0	0	2	0	\
≈	0	0	0	1	0	\

,

Table 11. Experimental results of PSO, BSO, MBSO, BSO-OS, RGBSO, and BSONME on F₂₃–F₃₀.

F	PSO	BSO	MBSO	BSO-OS	RGBSO	BSONME
F23	3.16 × 102 †	3.15 × 102 †	3.15 × 102 †	3.14 × 102 †	2.00 × 102 ≈	2.00 × 102
F24	2.30 × 102 †	2.51 × 102 †	2.31 × 102 †	2.31 × 102 †	2.00 × 102 ≈	2.00 × 102
F25	2.08 × 102 †	2.21 × 102 †	2.04 × 102 †	2.18 × 102 †	2.00 × 102 ≈	2.00 × 102
F26	1.21 × 102 ≈	1.15 × 102 ≈	1.00 × 102 −	1.73 × 102 †	1.28 × 102 ≈	1.07 × 102
F27	5.84 × 102 †	8.44 × 102 †	4.22 × 102 †	7.32 × 102 †	2.00 × 102 ≈	2.00 × 102
F28	1.11 × 103 †	4.35 × 103 †	8.86 × 102 †	3.30 × 103 †	2.00 × 102 ≈	2.00 × 102
F29	7.92 × 105 †	3.97 × 105 †	1.34 × 103 †	1.48 × 103 †	2.00 × 102 ≈	2.00 × 102
F30	4.13 × 103 †	8.62 × 103 †	1.77 × 103 †	3.13 × 103 †	2.00 × 102 ≈	2.00 × 102
†	7	7	7	8	0	\
−	0	0	1	0	0	\
≈	1	1	0	0	8	\

and

Table 12. Experimental results of IRGBSO, BSOLS, ALBSO, RPBSO, BSO-AL, and BSONME on F₂₃–F₃₀.

F	IRGBSO	BSOLS	ALBSO	RPBSO	BSO-AL	BSONME
F23	2.00 × 102 ≈	3.20 × 102 †	5.03 × 102 †	2.00 × 102 ≈	2.00 × 102 ≈	2.00 × 102
F24	2.00 × 102 ≈	2.58 × 102 †	2.37 × 102 †	2.00 × 102 †	2.00 × 102 ≈	2.00 × 102
F25	2.00 × 102 ≈	2.21 × 102 †	2.17 × 102 †	2.00 × 102 ≈	2.00 × 102 ≈	2.00 × 102
F26	1.04 × 102 −	1.67 × 102 †	1.12 × 102 †	1.10 × 102 ≈	1.76 × 102 †	1.07 × 102
F27	2.00 × 102 ≈	1.02 × 103 †	1.11 × 103 †	2.86 × 102 †	2.41 × 102 ≈	2.00 × 102
F28	2.00 × 102 ≈	1.58 × 103 †	4.78 × 103 †	2.00 × 102 ≈	2.00 × 102 ≈	2.00 × 102
F29	2.00 × 102 ≈	2.88 × 102 †	3.23 × 105 †	1.64 × 103 †	2.19 × 105 †	2.00 × 102
F30	2.00 × 102 ≈	1.21 × 104 †	1.38 × 106 †	9.72 × 102 †	3.21 × 105 †	2.00 × 102
†	0	8	8	4	3	\
−	1	0	0	0	0	\
≈	7	0	0	4	5	\

show that BSONME performs well for most test problems. To further accurately evaluate the performance of each algorithm, the Wilcoxon’s test at the 5% significance level is employed based on the PR values.

Table 13. Results obtained by the Wilcoxon test for algorithm BSONME.

VS	R+	R–	Asymptotic p-Value
PSO	385.0	80.0	0.00165
BSO	440.0	25.0	0.000019
MBSO	350.0	115.0	0.015222
BSO-OS	433.0	32.0	0.000034
RGBSO	366.5	68.5	0.001226
IRGBSO	396.5	38.5	0.000104
BSOLS	461.0	4.0	0.000002
ALBSO	465.0	0.0	0.000002
RPBSO	367.0	98.0	0.005491
BSO-AL	455.0	10.0	0.000005

shows that BSONME provides higher R + values than R − values compared with PSO, BSO, MBSO, BSO-OS, RGBSO, IRGBSO, BSOLS, ALBSO, RPBSO, and BSO-AL. The p values of PSO, BSO, MBSO, BSO-OS, RGBSO, IRGBSO, BSOLS, ALBSO, RPBSO, and BSO-AL are less than 0.05, which means that BSONME is significantly better than these competitors. Moreover, the Friedman test is utilized to rank each one according to their mean fitness. The lower the ranking of an algorithm, the better its performance. As shown in

Table 14. Average ranking of the algorithms (Friedman).

Algorithm	Ranking
BSONME	2.9333
PSO	5.65
BSO	6.1833
MBSO	3.9167
BSO-OS	4.8167
RGBSO	5.4333
IRGBSO	6.6667
BSOLS	7.7
ALBSO	10.4667
RPBSO	4.15
BSO-AL	8.0833

, the overall ranking sequences for the compared algorithms are BSONME, MBSO, RPBSO, BSO-OS, RGBSO, PSO, BSO, IRGBSO, BSOLS, BSO-AL, and ALBSO.

In the case of the hybrid problems F₁₇–F₂₂, BSONME performs better than other compared algorithms on F₁₇, F₁₈, and F₂₁. BSONME wins PSO, BSO, MBSO, BSO-OS, RGBSO, IRGBSO, BSOLS, ALBSO, RPBSO, and BSO-AL on 5, 5, 2, 4, 5, 5, 6, 6, 3, and 6 test problems, respectively.

In the case of the composition problems F₂₃–F₃₀, the performance of BSONME is second only to that of IRGBSO. BSONME wins PSO, BSO, MBSO, BSO-OS, RGBSO, IRGBSO, BSOLS, ALBSO, RPBSO, and BSO-AL on 7, 7, 7, 8, 0, 0, 8, 8, 4, and 3 test problems, respectively.

To intuitively show the convergence of each algorithm, twelve benchmark problems, including unimodal problems, multimodal problems, hybrid problems, and composite problems, are selected to plot in Figure 8 and Figure 9. In Figure 8 and Figure 9, the vertical axis is the mean errors of fitness (in logarithmic scale) and horizontal axis is the number of function evaluations. Figure 8 and Figure 9 show that BSONME converges faster than other algorithms. Moreover, BSONME performs best in terms of the lowest average best-so-far solution on most test problems.

4.3. Result and Discussion

Table 5, Table 6, Table 7, Table 8, Table 9, Table 10, Table 11 and Table 12 show that the number of “† (better)” obtained by BSONME is more than that obtained by the compared algorithms. The average excellent rate of BSONME for 30 functions is 71.67% (${{\displaystyle \sum }}_{i=1}^{30}{b}_i/\left(30\times 10\right)$). The results of the multiple-problem Wilcoxon’s test and the Friedman test show that BSONME ranks first. The reason why BSONME can achieve better results is mainly due to the use of elite learning mechanism and reinitialization strategy. The elite learning mechanism helps to integrate the evolution information hidden in the superior individual into the population. The reinitialization strategy helps to improve the probability of algorithm jumping out of local optimization. We will discuss the performance of the proposed algorithm from the following three aspects.

4.3.1. Discussion on Elite Learning Mechanism and Reinitialization Strategy

To further illustrate the importance role of elite learning mechanism and reinitialization strategy, four representative problems are selected from CEC2014, including unimodal problem (F₃), multimodal problem (F₁₁), hybrid problem (F₁₉), and composite problem (F₃₀). In this experiment, three algorithms are compared, which are BSONME, BSO-RS (BSO with reinitialization strategy), and BSO-EL (BSO with elite learning mechanism), respectively. Figure 10 shows that the convergence performance of BSONME is better than that of BSO-RS and BSO-EL. This illustrates that elite learning mechanism and reinitialization strategy can effectively improve the performance of the algorithm.

4.3.2. Analysis of Population Diversity

Population diversity is one of the key factors affecting the exploration performance of the algorithm. In this section, the population diversity is evaluated by a moment of inertia (I_c) [46], which is defined as

$$I_c={{\displaystyle \sum }}_{i=1}^D{{\displaystyle \sum }}_{j=1}^N{\left(M_{ij}-c_i\right)}^2$$

(20)

$$c_i=\frac{1}{N}{{\displaystyle \sum }}_{j=1}^N{M}_{ij}$$

(21)

where I_c denotes the spreading of each individual from their centroid. c_i represents the centroid of the ith dimension, i = 1, 2, …, D.

Figure 11 shows the diversity convergence diagram of BSONME for solving some benchmark functions in CEC2014. It can be seen from Figure 11 that the reinitialization strategy used in BSONME ensures the diversity of the population in the process of population evolution.

4.3.3. The Overall Effectiveness of BSONME

This section evaluated the overall effectiveness (OE) of the BSONME and compared algorithms based on the experimental results reported in Table 5, Table 6, Table 7, Table 8, Table 9, Table 10, Table 11 and Table 12. OE is calculated as follow [47].

$$OE=\frac{N-L}{N}\times 100\%$$

(22)

where N is the total number of test functions. L is the number of losing tests for each algorithm. The results are shown in

Table 15. The overall effectiveness of the BSONME and other algorithms.

	PSO	BSO	MBSO	BSO-OS	RGBSO	IRGBSO	BSOLS	ALBSO	RPBSO	BSO-AL	BSONME
W/T/L	1/5/24	2/2/26	7/4/19	3/3/24	1/9/20	1/7/22	0/3/27	0/0/30	7/5/18	0/4/26	11/7/12
OE	20%	13%	37%	20%	33%	27%	10%	0%	40%	13%	60%

. Table 15 shows that BSONME has the highest overall effectiveness, which is 60%. This shows that BSONME is an effective algorithm.

4.4. Experiment II: Least Squares Support Vector Machine for Prediction Problems

In this section, we demonstrate the efficacy of BSONME against several existing BSO optimizers and state-of-the-art methods on the real-world problem. The least squares support vector machine (LSSVM) is used to solve the prediction problems.

4.4.1. Least Squares Support Vector Machine

LSSVM is a machine learning algorithm [48] proposed by Suykens et al. LSSVM can transform the optimization problem into solving equations, and take the least square linear system as the loss function. LSSVM has been widely applied in many fields. For example, Tian et al. [49] used LSSVM to predict short-term wind speed to solve the problem of turbines optimal scheduling and ensure the safe and stable operation of power grids. Song et al. [50] established the PSO-LSSVM model, in which PSO algorithm is used to optimize the parameters of LSSVM. Then, the PSO-LSSVM model is used to predict the energy consumption of underwater gliders.

Let {(x_i,y_i)|i = 1, …,n} be a training set, x_i is a D-dimensional input vector, and y_i is output data. The optimization problem is defined as follows:

$$\begin{gathered} min J\left(\omega ,\xi \right)=\frac{1}{2}{\omega }^T\omega +\frac{1}{2}\gamma {{\displaystyle \sum }}_{i=1}^N{\xi }^2 \\ s.t. y_i={\omega }^T\phi \left(x_i\right)+b+{\xi }_i \end{gathered}$$

(23)

where γ > 0 is the penalty parameter, ξ_i is the regression error, b is the bias coefficient, and φ(x) is a nonlinear mapping function.

The Lagrange function L is constructed to solve this optimization problem.

$$L=\frac{1}{2}{\omega }^T\omega +\frac{1}{2}\gamma {{\displaystyle \sum }}_{i=1}^N{\xi }^2-{{\displaystyle \sum }}_{i=1}^N{\alpha }_i\left({\omega }^T\phi \left(x_i\right)+b+{\xi }_i-y_i\right)$$

(24)

where α_i is the Lagrange multipliers.

In the paper, RBF is selected as the kernel function. RBF can be expressed as follows.

$$K\left(x,x_i\right)=exp\left(-\frac{r^2}{2{\sigma }^2}\right),\sigma >0$$

(25)

where σ is the width of the core, $r=||x-x_i||$.

The LSSVM model is described as:

$$y\left(x\right)={{\displaystyle \sum }}_{i=1}^N{\mathsf{\alpha }}_{i}K\left(x,x_i\right)+b$$

(26)

4.4.2. BSONME-LSSVM Model

The hyperparameters γ and σ have a great influence on the performance of LSSVM model, so how to optimize these two hyperparameters is the key problem. In this paper, we construct a BSONME-LSSVM model, in which BSONME algorithm is used to find the hyperparameters γ and σ with the smallest error measure. Here, the objective function is the mean square error function (f_MSE) is calculated by:

$$f_{MSE}\left(\gamma ,\sigma \right)=\sqrt{\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N{\left(y_i-{\widehat{y}}_i\right)}^2}$$

(27)

where y_i is the actual value and ${\widehat{y}}_i$ is the predicted value.

The flow chart of the proposed BSONME-LSSVM model is shown in Figure 12. The BSONME-LSSVM model finds the optimal hyperparameters through data preprocessing, initialization parameters and parameter optimization. Then, the LSSVM model is established with the optimal parameters, and the BSONME-LSSVM model is used to predict engineering problems.

4.4.3. Prediction of Pipeline Instantaneous Water Flow

Generally, urban and rural water consumption analysis can provide a reference for water plants to maintain the dynamic balance of the reservoir. Accurate prediction of pipeline instantaneous water flow can maintain the water level balance of the reservoir and effectively prevent the occurrence of water hammer.

In this section, the data of the first water plant in mishia Township, Payzawat County is used as the experimental data set. We extracted the 23-day instantaneous water flow data of the water plant. The instantaneous water flow mode of the water plant is recorded every 15 min, that is, 96 datapoints are recorded every day. A total of 2195 consecutive datapoints are used as experimental data set. For the selected experimental data, the former 1756 datapoints are selected to train the BSONME-LSSVM model, and the remaining 439 are used to verify the effectiveness of the model. The 12 consecutive units of time data are used to predict 1 unit of time data. The population size is set as 20; the ended condition of each algorithm is determined by the maximum number of iterations (MaxGen = 20). The results are shown in

Table 16. Comparisons between PSO, BSO, MBSO, BSO-OS, RGBSO, IRGBSO, RPBSO, and BSONME on MSE of pipeline instantaneous water flow.

Algorithm	Training Error		Testing Error
	Mean	SD	Mean	SD
PSO	6.20	5.89 × 10−3	5.68	9.08 × 10−4
BSO	6.20	1.27 × 10−2	5.68	3.85 × 10−3
MBSO	6.20	3.95 × 10−4	5.68	4.39 × 10−6
BSO-OS	6.21	1.59 × 10−2	5.68	6.98 × 10−3
RGBSO	6.19	2.61 × 10−2	5.68	1.20 × 10−2
IRGBSO	6.20	1.62 × 10−2	5.68	2.54 × 10−3
RPBSO	6.20	5.80 × 10−4	5.68	6.84 × 10−6
BSONME	6.20	2.50 × 10−3	5.68	9.90 × 10−5

in terms of Mean (mean value) and SD (standard deviation) of MSE (mean square error) obtained in the 20 independent runs for 8 optimization algorithms.

Table 16 shows that eight algorithms achieve the same results on the mean value of testing error. The mean value of training error obtained by BSONME is ranked second of eight algorithms. The standard deviation of training error and testing error obtained by BSONME is ranked third of eight algorithms.

4.4.4. Forecast of Fund Trend

Securities investment fund is a common way of investment and financial management. Accurate forecast of fund fluctuation can effectively avoid risks. Influenced by market orientation, policies, and other factors, the net value of the fund changes from time to time. Since there are many factors affecting the rise and fall of funds, in this section, in order to simplify the experiment, we only select the daily rise and fall of a fund with code 161,725 as the experimental data set (https://fund.eastmoney.com/, accessed on 13 May 2021) to predict the trend of the fund. The fund is an index stock fund with high risk. A total of 355 consecutive datapoints are used as experimental data set. For the selected experimental data, the former 284 datapoints are selected to train the BSONME-LSSVM model, and the remaining 71 are used to verify the effectiveness of the model. The five consecutive datapoints are used to predict the sixth. The population size and the ended condition of each algorithm is the same as the experimental setting in 4.4.3. The results are shown in

Table 17. Comparisons between PSO, BSO, MBSO, BSO-OS, RGBSO, IRGBSO, RPBSO, and BSONME on MSE of fund trend.

Algorithm	Training Error		Testing Error
	Mean	SD	Mean	SD
PSO	4.06 × 10−2	2.12 × 10−5	3.18 × 10−2	5.05 × 10−6
BSO	4.06 × 10−2	5.30 × 10−5	3.19 × 10−2	3.34 × 10−5
MBSO	4.06 × 10−2	2.29 × 10−7	3.18 × 10−2	7.51 × 10−8
BSO-OS	4.06 × 10−2	7.31 × 10−5	3.19 × 10−2	4.41 × 10−5
RGBSO	4.06 × 10−2	7.70 × 10−5	3.19 × 10−2	6.28 × 10−5
IRGBSO	4.06 × 10−2	7.15 × 10−5	3.19 × 10−2	3.62 × 10−5
RPBSO	4.06 × 10−2	1.24 × 10−6	3.18 × 10−2	5.91 × 10−7
BSONME	4.06 × 10−2	1.67 × 10−7	3.18 × 10−2	6.42 × 10−8

in terms of Mean (mean value) and SD (standard deviation) of MSE (mean square error) obtained in the 20 independent runs for 8 optimization algorithms.

Table 17 shows that eight algorithms achieve the same results on the mean value of training error. The standard deviation of training error, mean value and standard deviation of testing error obtained by BSONME is ranked first of eight algorithms. The experimental results indicate that BSONME is competitive with other seven algorithms.

5. Conclusions

In this paper, three improvement strategies are employed to improve the performance of BSO. The modified Nelder–Mead method is used as a local searcher, which fully considers the characteristics of the landscape where the parent idea located. The elite learning mechanism based on Euclidean distance is used to guide the population to move towards the promising area. Moreover, the reinitialization strategy is employed to help the population jump out of the local optimum with a large probability.

Thirty benchmark problems (CEC2014) and two engineering prediction problems (pipeline instantaneous water flow problem and fund trend problem) are used to verify the effectiveness of BSONME. The experimental results of Wilcoxon’s test and the Friedman’s test demonstrate that BSONME is competitive with some popular BSO variants in handling optimization problems. This demonstrates that the designed three improved strategies can effectively guide the population to explore and exploit, which makes the proposed algorithm achieve remarkable results.

Our future work will focus on extending BSONME to multimodal problems, multifactorial optimization problems, and multi-objective problems. Moreover, the performance of BSONME will be further improved to solve engineering problems more effectively.