An Improved Novel Global Harmony Search Algorithm Based on Selective Acceptance

Abstract

The novel global harmony search (NGHS) algorithm is proposed in 2010, and it is an improved harmony search (HS) algorithm which combines the particle swarm optimization (PSO) and the genetic algorithm (GA). One of the main differences between the HS and NGHS algorithms is that of using different mechanisms to renew the harmony memory (HM). In the HS algorithm, in each iteration, the new harmony is accepted and replaced the worst harmony in the HM while the fitness of the new harmony is better than the worst harmony in the HM. Conversely, in the NGHS algorithm, the new harmony replaces the worst harmony in the HM without any precondition. However, in addition to these two mechanisms, there is one old mechanism, the selective acceptance mechanism, which is used in the simulated annealing (SA) algorithm. Therefore, in this paper, we proposed the selective acceptance novel global harmony search (SANGHS) algorithm which combines the NGHS algorithm with a selective acceptance mechanism. The advantage of the SANGHS algorithm is that it balances the global exploration and local exploitation ability. Moreover, to verify the search ability of the SANGHS algorithm, we used the SANGHS algorithm in ten well-known benchmark continuous optimization problems and two engineering problems and compared the experimental results with other metaheuristic algorithms. The experimental results show that the SANGHS algorithm has better search ability than the other four harmony search algorithms in ten continuous optimization problems. In addition, in two engineering problems, the SANGHS algorithm also provided a competition solution compared with other state-of-the-art metaheuristic algorithms.

1. Introduction

In the past two decades, the interest in the research related to metaheuristic algorithms has increased significantly due to the fact that existing numerical methods do not allow for solving a number of complex optimization problems, such as discrete structural optimization [1], design of water distribution networks [2], and vehicle routing [3]. A metaheuristic algorithm is applied to simulate the randomness and rules in natural phenomena. For example, the genetic algorithm (GA) [4] imitates the process of the biological evolutionary and natural selection. The particle swarm optimization (PSO) [5] was inspired by the animal behavior, namely, the foraging behavior of bird flocking. In addition, there are many algorithms extending PSO and GA, such as the harmony search (HS) algorithm [6] and the honey bee mating optimization (HBMO) [7]. These algorithms have proved their ability to search for better solutions than numerical methods in extensive optimization problems [8,9,10]. However, compared with theses algorithms, the HS algorithm which imitates an improvisation process of a harmony has two features. Firstly, it has fewer mathematical requirements and is easy to implement. Secondly, each decision variable in a new solution vector could be generated independently by considering all of the existing solution vectors [11]. Due to these two features, the HS algorithm has stronger adaptability for many engineering problems and could generate competition results compared with the GA [11]. In addition, based on the above advantages, many modified versions of the HS algorithm are proposed to further improve the searching ability of the HS algorithm, such as the improved harmony search (IHS) algorithm [11], the self-adaptive global best harmony search (SGHS) algorithm [12], and the novel global harmony search (NGHS) algorithm [13]. These modified versions of the HS algorithm have better search ability than the original HS algorithm.

It is worth noting that, among these HS algorithms, the mechanism that is used to decide whether a new generated harmony (solution vector) is accepted and replaces the worst harmony in the harmony memory (HM) or not is different. The HM is a storage of the harmony. Specifically, the HS, IHS, and SGHS algorithms accept a new generated harmony only when it is better than the worst harmony in the current HM. This mechanism, we call it accept-the-better-only mechanism, reminded us of the tabu search (TS) algorithm [14] which also only accepts the better solution vector in the searching process. Conversely, in the NGHS algorithm, a new generated harmony is accepted and replaced the worst harmony in the HM without preconditions in each iteration. This mechanism, we call it accept-everything mechanism, is the same one used in the PSO and GA. In the PSO algorithm, in each iteration, after calculating the velocity of the particle, the particle will fly to the new position according to its velocity whether the fitness of the new position is better than the current position or not. However, while we were discussing these two acceptance mechanisms, we suddenly came up with an old algorithm, the simulated annealing (SA) algorithm [15]. In the SA algorithm, in each iteration, the new solution vector is accepted or not according to the acceptable probability (AP), which is calculated by the Boltzmann distribution. Thus, the SA algorithm inspired us to investigate whether the AP affects the search ability or not in the NGHS algorithm. Therefore, in this paper, we proposed the selective acceptance novel global harmony search (SANGHS) algorithm, which combined the selective acceptance mechanism with the NGHS algorithm. Moreover, we proposed a proportion formula to calculate the AP according to the convergence degree. To verify the searching performance of the SANGHS algorithm, we used the SANGHS algorithms in ten well-known optimization problems and two engineering problems and compared the experimental results with other 15 algorithms. The results revealed that the SANGHS algorithm outperformed other 15 algorithms.

The remainder of this paper is organized as follows. In Section 2, the HS, IHS, SGHS, and the NGHS algorithms are introduced. Section 3 describes the SANGHS algorithm. A considerable number of experiments were conducted to test the search ability of the proposed SANGHS algorithm, and the corresponding experiments results are analyzed in Section 4. Conclusions and future research directions are provided in Section 5.

2. HS, IHS, SGHS, and NGHS Algorithms

2.1. Harmony Search Algorithm

The harmony search was proposed by Geem in 2001 [6]. Unlike other metaheuristic algorithms used to simulate the reproductive or foraging processes in the biological world, such as PSO and GA, the harmony search algorithm is inspired by a jazz trio and simulates its improvisation process [6]. The HS algorithm introduces musical notes to represent decision variables in practical problems. The number of musical notes is determined by the number of decision variables. Each note is played by a musician. Harmony requires two or more notes to be played at the same time. Thereby, each harmony represents a set of decision values. The HS algorithm simulates this in an improvisation process; hence, if the new harmony is better than the worst harmony, the new harmony will be saved to the harmony memory, and the worst harmony will be discarded from the harmony memory. Conversely, the new harmony will not be committed, and the worst harmony will not be discarded.

The HS algorithm is superior to other heuristic algorithms in terms of solving continuous variable problems and combinatorial problems; for example, it is deemed to be more applicable that GA [6]. The HS algorithm works as follows [16].

• Step 1: Initialize the algorithm and problem parameters

The main parameters of the harmony search algorithm include the harmony memory size (HMS), the harmony memory considering rate (HMCR), the pitch adjusting rate (PAR), and the bandwidth (BW). HMS is the total number of harmonies, which can be stored in the harmony memory (HM). HMCR determines whether musicians create a new harmony based on a harmony in the HM or not. PAR determines whether musicians fine-tune the musical note of a harmony that was selected in the HM or not. The bandwidth BW is the range, in which musicians fine-tune the selected harmony. Another parameter is the maximum number of iterations (NI), which denotes how many improvisations musicians have made to define the best harmony.

In this step, the above parameters in the HS algorithm are determined. In addition, the current iteration (k) is set to 1. The HS algorithm is usually used in the D-dimensional optimization problem defined as minimization of $\mathrm{f}\left(\mathrm{x}\right)$ subject to $x_j\in \left[x_{jL},x_{jU}\right]\left(j=1,2\dots D\right)$. Here, $x_{jL}$ and $x_{jU}$ are the lower and upper bounds for the decision variable $x_j$.

• Step 2: Initialize the harmony memory (HM)

The number of harmony vectors stored in the HM is equal to HMS. A harmony vector is a set of decision variables. The initial values of the decision variable $x_{i,j}^{k=0} \left(i=1,2,\dots , HMS\right)$ are generated by Equation (1). In Equation (1), $r$ is a random number in the range [0,1]. Here, $x_{i,j}^k$ represents the jth decision variable in the ith harmony vector, when the current iteration is $k$. The harmony memory (HM) is shown in Equation (2):

$$x_{i,j}^0=x_{jL}+r\times \left(x_{jU}-x_{jL}\right)$$

(1)

$$\mathrm{HM}=\left[\begin{array}{cccc}{x}_{1,1}^0& x_{1,2}^0& \cdots & x_{1,D}^0\\ x_{2,1}^0& x_{2,2}^0& \cdots & x_{2,D}^0\\ ⋮& ⋮& \ddots & ⋮\\ x_{HMS,1}^0& x_{HMS,2}^0& \cdots & x_{HMS,D}^0\end{array}\right]$$

(2)

• Step 3: Improvise a new harmony

A new harmony vector $x_{new,j}^k=(x_{new,1}^k,x_{new,2}^k,\cdots ,x_{new,D}^k)$ is improvised by three mechanisms: memory consideration, pitch adjustment, and random selections. It is defined as in Algorithm 1.

Algorithm 1 The Improvisation of New Harmony in HS

1: For $j=1$ to $D$ do

2:  If $r_1\le HMCR$ then

3:    $x_{new,j}^k=x_{i,j}^k$ %memory consideration

4:    If $r_2\le PAR$ then

5:      $x_{new,j}^k=x_{new,j}^k-BW+r_3\times 2\times BW$ %pitch adjustment

6:     If $x_{new,j}^k>x_{jU}$ then

7:      $x_{new,j}^k=x_{jU}$

8:     Else if $x_{new,j}^k<x_{jL}$ then

9:      $x_{new,j}^k=x_{jL}$

10:     End

11:   End

12:  Else

13:   $x_{new,j}^k=x_{jL}+r_4\times \left(x_{jU}-x_{jL}\right)$ % random selection

14:  End

15: End

Here, $x_{new,j}^k$ is the jth decision variable of the new harmony improvised in the current iteration $k$. In the memory consideration mechanism, $i$ of the $x_{i,j}^k$ is an integer random selected in the range of [1, HMS]; $r_1$, $r_2$, $r_3$, and $r_4$ are random numbers in the region of [0,1].

• Step 4: Update the harmony memory

If the new harmony vector $x_{new,j}^k=(x_{new,1}^k,x_{new,2}^k,\cdots ,x_{new,D}^k)$ has better fitness compared with the worse fitness in the harmony memory, the new harmony vector $x_{new,j}^k$ will replace the worse harmony vector.

• Step 5: Check the stopping iteration

If the current iteration is equal to the stopping iteration (the maximum number of iterations is NI), the computation is terminated. Otherwise, set $k=k+1$ and go back to step 3.

2.2. Improved Harmony Search Algorithm

In the HS algorithm, the pitch adjustment mechanism is designed to improve the local search ability. However, the PAR and the BW are fixed in the search process, which means that the local search ability does not change in the early iteration and the final iteration. This is a reason why the HS algorithm has a limit to balance the global search ability and local search ability in the different stages of the search process. To mitigate this deficiency, Mahdavi et al. [11] proposed a new strategy to dynamically adjust PAR and BW. At the early stage of optimization, the small PAR values with the large BW enhance the ability of the global search and increase the diversity of the solution vectors. The large PAR and the small BW values improve the ability of the local search and make the algorithm converge to the optimal solution vector in the late stage of optimization. Therefore, PAR and BW are adjusted according to Equations (3) and (4), respectively:

$$PA{R}^k=PA{R}_{min}+\frac{\left(PA{R}_{max}-PA{R}_{min}\right)}{NI}\times k$$

(3)

$$B{W}^k=B{W}_{max}\times e^{\ln \left(\frac{B{W}_{min}}{B{W}_{max}}\right)\times \mathrm{k}/\mathrm{NI}}$$

(4)

In Equation (3), $PA{R}^k$ represents the pitch adjusting rate, when the current iteration is $k$. $PA{R}_{min}$ and $PA{R}_{max}$ are the minimum and maximum pitch adjusting rates in the process of optimization, respectively. Additionally, in Equation (4), $B{W}^k$ represents the bandwidth, when the current iteration is $k$. $B{W}_{max}$ and $B{W}_{min}$ are the maximum and the minimum bandwidth in the process of optimization.

2.3. Self-Adaptive Global Best Harmony Search Algorithm

In 2010, the SGHS algorithm was proposed by Pan et al. [12]. The SGHS algorithm and the HS algorithm differ between each other in three aspects. Firstly, the SGHS algorithm implies that the pitch adjusting mechanism should be computed before the memory consideration mechanism. In addition, the memory consideration mechanism only selects the best harmony in the HM other than selecting randomly. Thereby, the new harmony is improvised using Algorithm 2. Secondly, the parameters HMCR and PAR are dynamically adjusted by the normal distribution, and BW is adjusted in each iteration according to Equation (5). Furthermore, HMCR is generated by the mean $HMC{R}_{mean}$ that is in the range of [0.9, 1.0] and the static standard deviation (0.01). PAR is generated by the mean $PA{R}_{mean}$ that is in the region of [0.0, 1.0] and the static standard deviation (0.05). Finally, the SGHS algorithm introduces another parameter named the learning period (LP) to determine $HMC{R}_{mean}$ and $PA{R}_{mean}$. In a learning period, if the new harmony replaces the worse harmony in the HM, when the iteration is $k$, $HMC{R}^k$ and $PA{R}^k$ are recorded. When the learning period is satisfied, $HMC{R}_{mean}$ and $PA{R}_{mean}$ are recalculated by averaging the recorded $HMC{R}^k$ and $PA{R}^k$, respectively, then the recorded $HMC{R}^k$ and $PA{R}^k$ are discarded, and another new learning period will begin.

Algorithm 2 The Improvisation of New Harmony in SGHS

1: For $j=1$ to $D$ do

2:  If $r_1\le HMCR$ then

3:    $x_{new,j}^k=x_{i,j}^k-BW+r_2\times 2\times BW$ %pitch adjustment

4:    If $x_{new,j}^k>x_{jU}$ then

5:     $x_{new,j}^k=x_{jU}$

6:    Else if $x_{new,j}^k<x_{jL}$ then

7:     $x_{new,j}^k=x_{jL}$

8:    End

9:    If $r_3\le PAR$ then

10:     $x_{new,j}^k=x_{B,j}^k$ %memory consideration

11:    End

12:  Else

13:    $x_{new,j}^k=x_{jL}+r_4\times \left(x_{jU}-x_{jL}\right)$ % random selection

14:  End

15: End

Here, $x_{B,j}^k$ represents the jth decision variable of the best harmony in the HM, when the iteration is $k$. The meaning of other variables in Algorithm 2 is the same as in Algorithm 1:

$$B{W}^k=\{\begin{array}{c}B{W}_{max}-\frac{B{W}_{max}-B{W}_{min}}{NI}\times 2k if k<NI/2,\\ B{W}_{min} if k\ge NI/2,\end{array}$$

(5)

2.4. Noval Global Harmony Search Algorithm

In 2010, the NGHS algorithm was proposed by Zou et al. [13,17,18]. The algorithm proposed in their study combines the HS algorithm, GA, and the particle swarm optimization (PSO) algorithm. GA imitates a process of the biological evolution [4]. The genetic mutation probability ($p_m$) in GA was introduced to escape from the local optimum. Particle swarm optimization (PSO) imitates the foraging process of bird flocking [5]. The position update in the NGHS algorithm was inspired by the main characteristic of PSO implying that the position of the particles is affected by the best position of all particles [13].

Figure 1 illustrates the principle of the position update. $ste{p}_j^k=|x_{best,j}^k-x_{worse,j}^k|$ is an adaptive step of the jth decision variable in the current iteration $k$. The range between P and R is referred to as a truth region, which is centered at the global best harmony. In other words, the jth decision variable is generated randomly in a range of the truth region. In the early iteration, all solution vectors in the harmony memory are sporadic, and the truth region is wide, which means that the new harmony is generated randomly in a large range. It is beneficial to enhance the global search ability. In the late iteration, all solution vectors in the HM are inclined to move to the global optimum. Therefore, all solution vectors are concentrated within a small solution space, and the truth region is narrow, which enhances the local search ability of NGHS [13,17,18]. The NGHS algorithm works as follows.

• Step 1: Initialize the algorithm and problem parameters

The genetic mutation probability ($p_m$), the current iteration $k$, and the maximum number of iterations (NI) are set in this step. However, the harmony memory considering rate (HMCR), pitch adjusting rate (PAR), and the bandwidth (BW) are excluded from the NGHS algorithm.

• Step 2: Initialize the harmony memory (HM)

The initial values of the decision variable $x_{i,j}^{k=0}\left(i=1,,\dots ,HMS\right)$ are generated by Equation (1). The harmony memory (HM) is shown in Equation (2).

• Step 3: Improvise a new harmony

A new harmony vector $x_{new,j}^k=(x_{new,1}^k,x_{new,2}^k,\cdots ,x_{new,D}^k)$ is improvised by Algorithm 3.

Algorithm 3 The Improvisation of New Harmony in NGHS

1: For $j=1$ to $D$ do

2:   $x_R=2\times x_{best,j}^k-x_{worse,j}^k$

3:   If $x_R>x_{jU}$ then

4:     $x_R=x_{jU}$

5:   Else if $x_R<x_{jL}$ then

6:     $x_R=x_{jL}$

7:   End

8:   $x_{new,j}^k=x_{worse,j}^k+r_1\times \left(x_R-x_{worse,j}^k\right)$ %position update

9:   If $r_2\le p_m$ then

10:     $x_{new,j}^k=x_{jL}+r_3\times \left(x_{jU}-x_{jL}\right)$ % genetic mutation

11:  End

12: End

Here, $x_{best,j}^k$ and $x_{worse,j}^k$ are the best harmony and the worse harmony in the harmony memory, when the current iteration is $k$, respectively. $r_1$, $r_2$, and $r_3$ are random numbers in the range of [0,1]:

• Step 4: Update the harmony memory

The new harmony replaces the worse harmony in the HM, even if the worse harmony in the HM has better fitness than the new harmony.

• Step 5: Check the stopping iteration

If the current iteration $k$ is equal to the stopping iteration (the maximum number of iterations is NI), the algorithm is terminated. Otherwise, set $k=k+1$ and go back to step 3.

3. Selective Acceptance Novel Global Harmony Search Algorithm

Assad et al. [19] pointed that the effectiveness of evolutionary algorithms relies on the extent of balance between diversification, which implies the ability of an algorithm to explore the high performance regions of the search space, and intensification, which is the ability of an algorithm to search for better solution in the high performance regions during the optimizing process. In other words, the key to improving the searching ability of evolutionary algorithms is to enhance the extent of balance between diversification and intensification capabilities. It can be seen that, in Section 2, Step 3 (Improvise a new harmony) in the IHS, SGHS, or NGHS algorithm is gradually improved to balance the diversification and intensification capabilities. However, in Step 4, the mechanism of updating the HM is different in these algorithms. Accept-the-better-only mechanism is used in the HS, IHS, and the SGHS algorithm. The accept-everything mechanism is used in the NGHS algorithm. This difference inspired us to explore whether the mechanism in Step 4 could affect the balance of diversification and intensification capabilities and improve the searching ability of the NGHS algorithm. However, in the process of finding a good mechanism in Step 4, the mechanism that is used in the SA algorithm attracted our attention. The SA algorithm used Boltzmann distribution to calculate the acceptable probability dynamically in each iteration and determine to accept the new solution or not by the value of acceptable probability. Therefore, in this paper, we proposed the selective acceptance novel global harmony search (SANGHS) algorithm, which combined the selective acceptance mechanism with the NGHS algorithm.

The main difference between the SANGHS and the NGHS algorithms is that the SANGHS algorithm will accept and replace the worst harmony in the HM with the new harmony according to the selective acceptance mechanism. In this mechanism, the selective acceptance formula as shown in Equation (6), which is also dynamically adjusted in the optimization process as in the SA algorithm, is used to further balance the diversification and intensification capabilities. However, the selective acceptance formula is different from the Boltzmann distribution in the SA algorithm and does not need to use additional parameters. In Equation (6), $A{P}^k$ represents the acceptable probability in the current iteration $k$. $F_{new}^k$ represents the fitness value of the new harmony in the current iteration $k$. $F_{best}^k$ and $F_{worse}^k$ represent the fitness values of the best and the worst harmonies in the HM in the current iteration $k$, respectively. According to the selective acceptance formula in the SANGHS algorithm, if the fitness of the new harmony is better than the worst harmony in the HM, the AP is equal to 1. It means that the worst harmony will be replaced by the new harmony in the HM. However, if the fitness of the new harmony is worse than the worst harmony in the HM, the SANGHS algorithm will accept the new harmony according to the AP, which is calculated by the proportion formula. Proportion formula is the most important part of the selective acceptance formula. In proportion formula, $F_{worse}^k-F_{best}^k$ represents the convergence degree of all harmonies in the HM. In the early iterations, all harmonies in the HM are dispersed in a large solution space, and the value of $F_{worse}^k-F_{best}^k$ is a large value. Thus, it is more likely to accept a worse harmony and thereby expand the range of truth region in the SANGHS algorithm, which would contribute to improve the diversification capabilities. Conversely, in the late iterations, all harmonies in the HM have converged in a small solution space; the value of $F_{worse}^k-F_{best}^k$ is a small value. The HM is less likely to accept a worse harmony and thereby reduce the range of truth region, which makes the SANGHS algorithm have strong intensification capabilities:

$$A{P}^k=\{\begin{array}{cc}1& if F_{new}^k\le F_{worse}^k\\ \frac{\left(F_{worse}^k-F_{best}^k\right)}{\left(F_{new}^k-F_{best}^k\right)}& if F_{new}^k>F_{worse}^k\end{array}$$

(6)

In general, the SANGHS algorithm contains the following steps:

• Step 1: Initialize the algorithm and problem parameters

The genetic mutation probability ($p_m$), the current iteration $k$, and the maximum iteration (NI) are set in this step.

• Step 2: Initialize the harmony memory (HM)

The initial values of the decision variable $x_{i,j}^{k=0}\left(i=1,2,\dots ,HMS\right)$ are generated by Equation (1). The harmony memory (HM) is shown in Equation (2).

• Step 3: Improvise a new harmony

A new harmony vector $x_{new,j}^k=(x_{new,1}^k,x_{new,2}^k,\cdots ,x_{new,D}^k)$ is improvised by Algorithm 3. This step is the same as in the NGHS algorithm.

• Step 4: Calculate AP

If the new harmony has a better fitness value than the worst harmony in the HM, AP is set to 1. Otherwise, AP is calculated by Equation (6).

• Step 5: Update the harmony memory

Firstly, generate a random number in [0,1]. If the random number is smaller than the AP, the new harmony replaces the worst harmony in the HM. Otherwise, the new harmony is discarded.

• Step 6: Check the stopping iteration

If the current iteration $k$ is equal to the stopping iteration (the maximum number of iterations is NI), the algorithm is terminated. Otherwise, set $k=k+1$ and go back to Step 3.

4. Experimental Results and Analysis

4.1. Computing Environment Settings

In this study, we used Python 3.5.1 (64-bit) as the complier to implement the program to search for the trial solution. The solution-finding equipment comprised an Intel Core (TM) i5-8500 (3.0 GHz, 3.0 GHz) CPU, 8 GB of memory, and Windows 10 professional edition (64-bit) OS.

4.2. Ten Well-Known Benchmark Optimization Problems

To estimate the performance for the proposed acceptable probability mechanism, we considered ten well-known benchmark optimization problems [20,21] as follows. Among these problems, problems 1 to 4 are unimodal problems. Problems 5 to 10 are multimodal problems with several local optima, and the number of local optima is increased with the number of decision variables [12].

(1)

Sphere function (F1)

The Sphere function is defined as $minf\left(x_i\right)={\sum }_{i=1}^N{x}_i^2$, where the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-100,100\right].$

(2)

Schwedel’s problem 2.22 (F2)

This problem is defined as $minf\left(x_i\right)={\sum }_{i=1}^N\left|x_i\right|+{\prod }_{i=1}^N\left|x_i\right|$, where the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-10,10\right].$

(3)

Axis Parallel Function (F3)

This problem is defined as $\min f\left(x_i\right)={\sum }_{i=1}^{N}i{x}_i^2$, where the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-5.12,5.12\right].$

(4)

Quartic Function (F4)

This problem is defined as $\min f\left(x_i\right)={\sum }_{i=1}^N{x}_i^4$, where the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-1.28,1.28\right].$

(5)

Ackley’s function (F5)

The Ackley’s function is defined as $minf\left(x_i\right)=20+e-20exp\left(-0.2\sqrt{{\sum }_{i=1}^N\frac{x_i^2}{N}}\right)-exp\left(\frac{{\sum }_{i=1}^{N}cos\left(2\pi x_i\right)}{N}\right)$, where the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-32,32\right].$

(6)

Rastrigin function (F6)

The Rastrigin function is defined as $minf\left(x_i\right)={\sum }_{i=1}^N\left(x_i^2-10cos\left(2\pi x_i\right)+10\right)$, where the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-5.12,5.12\right].$

(7)

Schwefel’s problem 2.26 (F7)

This problem is defined as $minf\left(x_i\right)=418.9829N-{\sum }_{i=1}^N\left(x_{i}sin\left(\sqrt{\left|x_i\right|}\right)\right)$, where the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-500,500\right].$

(8)

Levy Function (F8)

This problem is defined as $minf\left(x_i\right)={\sin }^2\left(\mathsf{\pi }{w}_1\right)-{\sum }_{i=1}^{N-1}{\left({\mathrm{w}}_i-1\right)}^2\left[1+10si{n}^2\left(\pi w_i+1\right)\right]+{\left(w_N-1\right)}^2\left[1+si{n}^2\left(2\pi w_D\right)\right]$, where $w_i=1+\frac{x_i-1}{4}$ and the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-10,10\right].$

(9)

Bohachevsky function (F9)

This problem is defined as $minf\left(x_i\right)={\sum }_{i=1}^{N-1}\left[x_i^2+2x_{i+1}^2-0.3\cos \left(3\pi x_i\right)-0.4\cos \left(4\pi x_{i+1}\right)+0.7\right]$, where the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-15,15\right].$

(10)

Alpine 1 function (F10)

This problem is defined as $minf\left(x_i\right)={\sum }_{i=1}^N\left|x_i\sin \left(x_i\right)+0.1x_i\right|$, where the global optimum is 0, and the search space of $x_i\left(i=1,2,\cdots ,N\right)$ is $\left[-10,10\right].$

4.3. Comparison of Different Harmony Search Algorithms in Ten Benchmark Optimization Problems

To test the performance of the SANGHS algorithm for ten aforementioned optimization problems, we compared it with four harmony search algorithms, which were the HS, IHS, SGHS, and NGHS algorithms. The parameters of these algorithms are provided in

Table 1. Parameters of compared harmony search (HS) algorithms in ten optimization problems.

Algorithm	HS	IHS	SGHS	NGHS	SANGHS
HMS 1	5	5	5	5	5
HMCR 2	0.9	0.9	$HMC{R}_{mean}=0.98$	–	–
PAR 3	0.3	$PA{R}_{min}=0.01$ $PA{R}_{max}=0.99$	$PA{R}_{mean}=0.9$	–	–
BW 4	–	$B{W}_{min}=0.0001$ $B{W}_{max}=\left(x_{jU}-x_{jL}\right)/20$	$B{W}_{min}=0.0005$ $B{W}_{max}=\left(x_{jU}-x_{jL}\right)/10$	–	–
LP 5	–	–	100	–	–
$P_m$6	–	–	–	0.005	0.005

¹ HMS: the harmony memory size; ² HMCR: the harmony memory considering rate; ³ PAR: the pitch adjusting rate; ⁴ BW: the bandwidth; ⁵ LP: the learning period; ⁶ $P_m$: the genetic mutation probability.

[12,17,22,23]. We tested three different dimensions equal to 10, 30, and 100. The maximum number of iterations (NI) is set to 30,000 with D = 10, 60,000 with D = 30, and 150,000 with D = 100. Each problem was tested in 30 independent experiments. The experimental results for all problems with three different dimensions obtained by the four harmony search algorithms and the SANGHS algorithm are shown in

Table 2. Experimental results of five algorithms for 10 problems with D = 10.

No.	Algorithm	Min	Max	Mean	Std	No.	Algorithm	Min	Max	Mean	Std
F1	HS	5.2096 × 10−8	1.1157 × 10−6	2.3562 × 10−7	1.8902 × 10−7	F6	HS	8.3112 × 10−6	7.8214 × 10−5	3.3076 × 10−5	1.7982 × 10−5
	IHS	6.7803 × 10−9	2.3518 × 10−8	1.3575 × 10−8	3.7943 × 10−9		IHS	1.1191 × 10−6	4.0212 × 10−6	2.6004 × 10−6	6.4709 × 10−7
	SGHS	6.8178 × 10−12	1.4106 × 10−10	3.2650 × 10−11	2.5506 × 10−11		SGHS	1.1716 × 10−9	3.8150 × 10−1	1.2717 × 10−2	6.8481 × 10−2
	NGHS	1.0698 × 10−45	2.1604 × 10−36	8.2477 × 10−38	3.8833 × 10−37		NGHS	0.0000 × 100	0.0000 × 100	0.0000 × 100	0.0000 × 100
	SANGHS	9.6086 × 10−111	1.3918 × 10−77	4.6889 × 10−79	2.4976 × 10−78		SANGHS	0.0000 × 100	0.0000 × 100	0.0000 × 100	0.0000 × 100
F2	HS	3.2109 × 10−4	1.5971 × 10−3	7.7726 × 10−4	2.7372 × 10−4	F7	HS	1.2731 × 10−4	2.8515 × 10−2	1.0739 × 10−3	5.0957 × 10−3
	IHS	2.0318 × 10−4	3.6754 × 10−4	2.9700 × 10−4	3.2031 × 10−5		IHS	1.2728 × 10−4	1.2728 × 10−4	1.2728 × 10−4	4.0700 × 10−10
	SGHS	7.3565 × 10−6	1.8055 × 10−5	1.2027 × 10−5	2.7866 × 10−6		SGHS	1.2728 × 10−4	2.9759 × 10−4	1.3517 × 10−4	3.1705 × 10−5
	NGHS	3.8140 × 10−27	5.1898 × 10−21	2.3612 × 10−22	9.4731 × 10−22		NGHS	1.2728 × 10−4	1.2728 × 10−4	1.2728 × 10−4	8.5769 × 10−14
	SANGHS	1.4267 × 10−82	7.2433 × 10−54	3.5052 × 10−55	1.4044 × 10−54		SANGHS	1.2728 × 10−4	1.2728 × 10−4	1.2728 × 10−4	4.4839 × 10−14
F3	HS	1.4468 × 10−7	2.1934 × 10−6	8.0446 × 10−7	4.2722 × 10−7	F8	HS	2.0732 × 10−8	3.4077 × 10−7	8.0031 × 10−8	6.4199 × 10−8
	IHS	3.1842 × 10−8	9.8796 × 10−8	6.2079 × 10−8	1.8245 × 10−8		IHS	3.3581 × 10−9	1.0209 × 10−8	6.7715 × 10−9	1.7956 × 10−9
	SGHS	2.7923 × 10−11	7.2398 × 10−10	1.7525 × 10−10	1.4531 × 10−10		SGHS	1.7990 × 10−12	5.0988 × 10−11	1.3622 × 10−11	1.1253 × 10−11
	NGHS	4.4709 × 10−46	3.5344 × 10−39	2.3102 × 10−40	7.0896 × 10−40		NGHS	1.4998 × 10−32	1.4998 × 10−32	1.4998 × 10−32	5.4738 × 10−48
	SANGHS	3.8636 × 10−113	4.3798 × 10−83	2.3947 × 10−84	9.1559 × 10−84		SANGHS	1.4998 × 10−32	1.4998 × 10−32	1.4998 × 10−32	5.4738 × 10−48
F4	HS	2.2666 × 10−17	3.0752 × 10−14	7.0687 × 10−15	8.2181 × 10−15	F9	HS	1.1130 × 10−6	2.8915 × 10−5	7.0153 × 10−6	5.3517 × 10−6
	IHS	9.5725 × 10−18	6.5490 × 10−17	2.9787 × 10−17	1.3161 × 10−17		IHS	3.0585 × 10−7	9.1446 × 10−7	5.8587 × 10−7	1.4754 × 10−7
	SGHS	2.5924 × 10−23	4.7975 × 10−21	6.9747 × 10−22	1.1480 × 10−21		SGHS	3.2077 × 10−10	4.1452 × 10−9	1.3097 × 10−9	8.5282 × 10−10
	NGHS	1.9010 × 10−86	7.5124 × 10−67	2.5042 × 10−68	1.3485 × 10−67		NGHS	5.7250 × 10−49	2.0380 × 10−36	7.8672 × 10−38	3.6838 × 10−37
	SANGHS	1.7493 × 10−169	9.2806 × 10−124	3.2904 × 10−125	1.6640 × 10−124		SANGHS	2.0444 × 10−113	1.4776 × 10−69	4.9253 × 10−71	2.6524 × 10−70
F5	HS	1.7118 × 10−4	8.0222 × 10−4	4.6344 × 10−4	1.5000 × 10−4	F10	HS	1.1494 × 10−4	1.8959 × 10−3	4.9240 × 10−4	3.4263 × 10−4
	IHS	1.0584 × 10−4	1.6838 × 10−4	1.4686 × 10−4	1.5429 × 10−5		IHS	2.0995 × 10−5	3.2351 × 10−4	5.9924 × 10−5	7.1295 × 10−5
	SGHS	2.8256 × 10−6	1.2840 × 10−5	6.6281 × 10−6	2.4208 × 10−6		SGHS	5.6467 × 10−7	1.8978 × 10−5	5.9487 × 10−6	5.4367 × 10−6
	NGHS	6.6613 × 10−15	3.8636 × 10−14	1.8267 × 10−14	7.1015 × 10−15		NGHS	1.4433 × 10−15	1.5821 × 10−14	6.1063 × 10−15	3.0077 × 10−15
	SANGHS	6.6613 × 10−15	2.7978 × 10−14	1.4122 × 10−14	4.8842 × 10−15		SANGHS	8.3267 × 10−16	1.0658 × 10−14	4.9442 × 10−15	2.5741 × 10−15

,

Table 3. Experimental results of five algorithms for 10 problems with D = 30.

No.	Algorithm	Min	Max	Mean	Std	No.	Algorithm	Min	Max	Mean	Std
F1	HS	5.0536 × 10−1	6.4171 × 100	3.3124 × 100	1.5861 × 100	F6	HS	2.6666 × 10−2	2.0462 × 100	5.0288 × 10−1	6.0096 × 10−1
	IHS	2.1207 × 10−7	4.9163 × 10−7	3.5819 × 10−7	7.1968 × 10−8		IHS	4.0261 × 10−5	4.6986 × 100	1.3178 × 100	1.0352 × 100
	SGHS	1.5978 × 10−9	1.4781 × 10−8	4.8841 × 10−9	3.1202 × 10−9		SGHS	7.9528 × 10−7	9.9496 × 10−1	4.8052 × 10−2	1.8483 × 10−1
	NGHS	1.2081 × 10−17	6.1930 × 10−15	6.6153 × 10−16	1.3269 × 10−15		NGHS	0.0000 × 100	4.7713 × 10−12	1.7243 × 10−13	8.5424 × 10−13
	SANGHS	2.3793 × 10−66	2.4404 × 10−37	8.1347 × 10−39	4.3807 × 10−38		SANGHS	0.0000 × 100	0.0000 × 100	0.0000 × 100	0.0000 × 100
F2	HS	4.5085 × 10−2	2.0043 × 10−1	7.6210 × 10−2	3.7203 × 10−2	F7	HS	8.1324 × 100	5.2395 × 101	2.0713 × 101	9.5235 × 100
	IHS	1.7655 × 10−3	4.4854 × 10−3	2.3905 × 10−3	4.6485 × 10−4		IHS	3.8186 × 10−4	1.7516 × 100	6.3827 × 10−2	3.1460 × 10−1
	SGHS	8.1564 × 10−5	1.9686 × 10−4	1.5091 × 10−4	2.9858 × 10−5		SGHS	3.8277 × 10−4	7.8715 × 10−2	1.7508 × 10−2	1.9421 × 10−2
	NGHS	3.2148 × 10−10	8.7017 × 10−9	2.0345 × 10−9	1.7972 × 10−9		NGHS	3.8183 × 10−4	3.8183 × 10−4	3.8183 × 10−4	8.1680 × 10−12
	SANGHS	1.3727 × 10−44	7.3787 × 10−28	2.4752 × 10−29	1.3243 × 10−28		SANGHS	3.8183 × 10−4	3.8183 × 10−4	3.8183 × 10−4	1.0918 × 10−13
F3	HS	1.5858 × 10−3	4.8414 × 10−3	2.9867 × 10−3	7.1956 × 10−4	F8	HS	5.1397 × 10−5	1.6213 × 10−4	1.0240 × 10−4	2.2064 × 10−5
	IHS	8.1295 × 10−6	1.3227 × 10−3	1.7068 × 10−4	3.1223 × 10−4		IHS	1.1278 × 10−7	4.7712 × 10−6	8.2210 × 10−7	1.0175 × 10−6
	SGHS	8.4500 × 10−9	5.2306 × 10−8	2.1700 × 10−8	1.1042 × 10−8		SGHS	3.0351 × 10−10	1.4228 × 10−9	7.3604 × 10−10	2.3859 × 10−10
	NGHS	3.4329 × 10−19	3.4291 × 10−16	1.8533 × 10−17	6.0640 × 10−17		NGHS	9.6185 × 10−20	7.8833 × 10−17	8.7557 × 10−18	1.8706 × 10−17
	SANGHS	9.6420 × 10−70	2.0955 × 10−48	7.4888 × 10−50	3.7563 × 10−49		SANGHS	1.4998 × 10−32	1.4998 × 10−32	1.4998 × 10−32	5.4738 × 10−48
F4	HS	9.2705 × 10−10	4.8468 × 10−9	3.1521 × 10−9	9.6498 × 10−10	F9	HS	7.5871× 10−3	3.2014× 100	1.0965× 100	1.0189× 100
	IHS	2.3766 × 10−15	2.5488 × 10−14	8.1771 × 10−15	4.4175 × 10−15		IHS	8.6839× 10−6	2.1440× 100	5.3391× 10−1	5.8893× 10−1
	SGHS	1.9509 × 10−20	1.6666 × 10−18	4.2398 × 10−19	4.2829 × 10−19		SGHS	3.1568× 10−8	1.2410× 10−7	7.9116× 10−8	2.4154× 10−8
	NGHS	1.8030 × 10−40	6.0162 × 10−35	4.2247 × 10−36	1.1809 × 10−35		NGHS	4.2058× 10−19	1.3275× 10−14	1.0821× 10−15	2.8017× 10−15
	SANGHS	2.6142 × 10−94	3.9928 × 10−69	1.3328 × 10−70	7.1669 × 10−70		SANGHS	6.1353 × 10−65	6.3460 × 10−33	2.3282 × 10−34	1.1387 × 10−33
F5	HS	1.5649 × 10−2	1.7072 × 100	6.7498 × 10−1	3.9769 × 10−1	F10	HS	2.4004 × 10−2	1.9367 × 10−1	1.0981 × 10−1	4.6542 × 10−2
	IHS	3.2084 × 10−4	1.3089 × 100	3.6401 × 10−1	4.4719 × 10−1		IHS	2.1852 × 10−2	9.7546 × 10−2	5.3262 × 10−2	1.9894 × 10−2
	SGHS	2.5647 × 10−5	4.6874 × 10−5	3.2244 × 10−5	5.0051 × 10−6		SGHS	1.2359 × 10−5	1.4067 × 10−3	1.0235 × 10−4	2.4394 × 10−4
	NGHS	7.4210 × 10−10	1.4597 × 10−8	4.6791 × 10−9	3.5652 × 10−9		NGHS	1.0805 × 10−10	3.0740 × 10−9	8.6347 × 10−10	6.8488 × 10−10
	SANGHS	3.8636 × 10−14	2.9798 × 10−13	6.2321 × 10−14	4.5175 × 10−14		SANGHS	4.4409 × 10−15	2.1261 × 10−14	1.1208 × 10−14	4.4960 × 10−15

and

Table 4. Experimental results of five algorithms for 10 problems with D = 100.

No.	Algorithm	Min	Max	Mean	Std	No.	Algorithm	Min	Max	Mean	Std
F1	HS	8.7644 × 103	1.6480 × 104	1.2203 × 104	1.5403 × 103	F6	HS	2.2313 × 102	2.9442 × 102	2.5974 × 102	2.1788 × 101
	IHS	9.1067 × 103	1.4800 × 104	1.2428 × 104	1.0779 × 103		IHS	1.9487 × 102	2.8076 × 102	2.4522 × 102	2.0779 × 101
	SGHS	8.7681 × 10−1	2.6029 × 100	1.3666 × 100	4.2374 × 10−1		SGHS	1.5762 × 100	8.5459 × 100	4.0935 × 100	1.5444 × 100
	NGHS	4.8032 × 10−4	1.0576 × 10−3	7.3694 × 10−4	1.6006 × 10−4		NGHS	1.5643 × 10−3	2.0192 × 100	4.7179 × 10−1	6.1547 × 10−1
	SANGHS	5.1238 × 10−23	7.0729 × 10−15	4.3965 × 10−16	1.3365 × 10−15		SANGHS	0.0000 × 100	1.3856 × 10−13	6.5133 × 10−15	2.4832 × 10−14
F2	HS	5.0156 × 101	7.3451 × 101	6.1118 × 101	4.9536 × 100	F7	HS	4.5600 × 103	7.0794 × 103	5.8498 × 103	5.8361 × 102
	IHS	5.1843 × 101	6.8563 × 101	6.0248 × 101	4.3861 × 100		IHS	4.4499 × 103	6.5893 × 103	5.5203 × 103	4.9043 × 102
	SGHS	8.1631 × 10−2	4.1138 × 10−1	2.4863 × 10−1	8.1339 × 10−2		SGHS	8.9612 × 100	3.0848 × 101	1.7210 × 101	5.3463 × 100
	NGHS	9.9950 × 10−3	1.6514 × 10−2	1.3619 × 10−2	1.5409 × 10−3		NGHS	3.9466 × 10−3	1.1539 × 10−1	1.6436 × 10−2	2.4948 × 10−2
	SANGHS	2.2360 × 10−12	2.2218 × 10−3	1.9072 × 10−4	5.5550 × 10−4		SANGHS	1.2728 × 10−3	1.2728 × 10−3	1.2728 × 10−3	4.8314 × 10−12
F3	HS	9.4087 × 102	1.5941 × 103	1.1913 × 103	1.6577 × 102	F8	HS	2.4107 × 101	3.3570 × 101	2.9300 × 101	2.8794 × 100
	IHS	9.2595 × 102	1.5737 × 103	1.1718 × 103	1.4788 × 102		IHS	2.3183 × 101	3.6516 × 101	2.9135 × 101	3.1786 × 101
	SGHS	6.4869 × 10−5	5.3603 × 10−3	8.8126 × 10−4	1.2161 × 10−3		SGHS	4.4458 × 10−7	4.0334 × 10−4	2.2728 × 10−5	8.2115 × 10−5
	NGHS	3.3424 × 10−5	1.1541 × 10−4	6.4369 × 10−5	1.9242 × 10−5		NGHS	1.4748 × 10−6	9.2715 × 10−6	3.6238 × 10−6	1.4865 × 10−6
	SANGHS	7.2707 × 10−22	1.4056 × 10−13	7.1676 × 10−15	2.6377 × 10−14		SANGHS	4.8189 × 10−31	9.5129 × 10−17	3.2482 × 10−18	1.7065 × 10−17
F4	HS	1.3307 × 10−1	2.6181 × 10−1	1.9974 × 10−1	3.4152 × 10−2	F9	HS	7.0005 × 102	1.1205 × 103	9.0061 × 102	1.0327 × 102
	IHS	1.3301 × 10−1	2.8520 × 10−1	2.0499 × 10−1	3.3111 × 10−2		IHS	7.3029 × 102	1.1882 × 103	9.1884 × 102	1.0628 × 102
	SGHS	4.0943 × 10−15	1.1631 × 10−14	7.5656 × 10−15	2.0292 × 10−15		SGHS	2.5895 × 10−1	4.6678 × 100	2.0793 × 100	1.0991 × 100
	NGHS	8.7725 × 10−16	7.2065 × 10−15	2.3953 × 10−15	1.7313 × 10−15		NGHS	3.4107 × 10−4	1.4657 × 10−3	7.7456 × 10−4	2.5196 × 10−4
	SANGHS	6.1762 × 10−46	4.0280 × 10−31	1.4178 × 10−32	7.2252 × 10−32		SANGHS	2.8823 × 10−22	1.6936 × 10−15	1.6201 × 10−16	3.2735 × 10−16
F5	HS	1.0843 × 101	1.2640 × 101	1.1775 × 101	4.3937 × 10−1	F10	HS	1.9740 × 101	2.6059 × 101	2.2513 × 101	1.6612 × 100
	IHS	1.1199 × 101	1.2544 × 101	1.1869 × 101	3.3008 × 10−1		IHS	1.6309 × 101	2.6621 × 101	2.1674 × 101	2.1494 × 100
	SGHS	3.2093 × 10−2	6.5191 × 10−1	2.4740 × 10−1	1.3506 × 10−1		SGHS	9.7570 × 10−2	2.0103 × 10−1	1.4311 × 10−1	2.6382 × 10−2
	NGHS	2.4194 × 10−3	5.1098 × 10−3	3.4730 × 10−3	5.5824 × 10−4		NGHS	3.1071 × 10−3	7.3450 × 10−3	5.1084 × 10−3	1.1414 × 10−3
	SANGHS	2.1347 × 10−12	2.4401 × 10−9	2.9843 × 10−10	6.6605 × 10−10		SANGHS	1.7615 × 10−13	8.6512 × 10−4	3.8048 × 10−5	1.5973 × 10−4

. Mean and Std represent the average and the standard deviation of 30 independent experiments, respectively, as provided in Table 2, Table 3 and Table 4. Min and Max represent the minimum and maximum fitness values in 30 independent experiments.

In Table 2, Table 3 and Table 4, several experimental results are given. Firstly, in all problems (F1 to F10) with three different dimensions (D = 10, 30, and 100), the SANGHS algorithm has smaller minimum (Min) and maximum (Max) fitness values than other four HS algorithms. For example, in problem 1 with D = 10, SANGHS has the smallest minimum fitness values (9.6086 × 10⁻¹¹¹) and the smallest maximum fitness values (1.3918 × 10⁻⁷⁷). Secondly, the SANGHS algorithm has a smallest average (Mean) fitness value. For example, in problem 1 with D = 10, SANGHS has the smallest average fitness values (4.6889 × 10⁻⁷⁹). This value is far less than the second one (8.2477 × 10⁻³⁸). The above two experimental results reveal that the SANGHS algorithm has a better search ability than other four harmony search algorithms. Lastly, in the above problems (F1 to F10) with D = 10, 30, and 100, the SANGHS algorithm has a smallest standard deviation (Std) value. For example, in problem 1 with D = 10, SANGHS has the smallest standard deviation values (2.4976 × 10⁻⁷⁸), and this value is also far less than the second one (3.8833 × 10⁻³⁷). It means that the search ability of the SANGHS algorithm is more robust than other four harmony search algorithms.

In addition, to further explore the difference in the performance of the SANGHS algorithm and other four harmony search algorithms, the experiment results were also analyzed through the Wilcoxon rank-sum test [24,25] as shown in

Table 5. p-value of Wilcoxon rank-sum test in ten problems.

D	No.	SANGHS vs. HS	SANGHS vs. IHS	SANGHS vs. SGHS	SANGHS vs. NGHS
10	F1	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F2	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F3	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F4	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F5	7.8779 × 10−12	7.8779 × 10−12	7.8779 × 10−12	1.1293 × 10−02
	F6	6.0589 × 10−13	6.0589 × 10−13	6.0589 × 10−13	–
	F7	9.1371 × 10−12	9.1371 × 10−12	9.1371 × 10−12	8.1647 × 10−04
	F8	6.0589 × 10−13	6.0589 × 10−13	6.0589 × 10−13	–
	F9	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F10	1.5071 × 10−11	1.5071 × 10−11	1.5071 × 10−11	6.3886 × 10−02
30	F1	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F2	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F3	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F4	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F5	1.4019 × 10−11	1.4019 × 10−11	1.4019 × 10−11	1.4019 × 10−11
	F6	6.0589 × 10−13	6.0589 × 10−13	6.0589 × 10−13	2.2246 × 10−12
	F7	1.3835 × 10−11	1.3835 × 10−11	1.3835 × 10−11	1.3661 × 10−11
	F8	6.0589 × 10−13	6.0589 × 10−13	6.0589 × 10−13	6.0589 × 10−13
	F9	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F10	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
100	F1	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F2	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F3	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F4	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F5	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F6	1.0142 × 10−11	1.0142 × 10−11	1.0142 × 10−11	1.0142 × 10−11
	F7	1.4688 × 10−11	1.4688 × 10−11	1.4688 × 10−11	1.4688 × 10−11
	F8	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F9	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11
	F10	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11	1.5099 × 10−11

. In Table 5, the p-value, which is greater than 0.05, is bold. Based on the p-value, the SANGHS algorithm is significantly better than the other four HS algorithms in all problems with three different dimensions, except in problem 10 (F10) with D = 10 (6.3886 × 10⁻²).

Figure 2 shows the typical convergence graphs of five different algorithms for ten problems with D = 30. In Figure 2a–d which are unimodal problems, as the number of iterations increases, the SANGHS algorithm converges faster than other four harmony search algorithms. It means that the SANGHS algorithm has a strong local search ability in unimodal problems. In Figure 2e,f,h,j which are multimodal problems, we can easily observe that the SANGHS algorithm is more effective in escaping from the local optima and have a better convergence ability compared with the other four harmony search algorithms. In other words, the SANGHS algorithm has a better balance between the global search ability and the local search ability in multimodal problems.

4.4. Two Benchmarks’ Engineering Optimization Problems

In this section, to verify the searching performance of the SANGHS algorithm further, we used the SANGHS algorithm to solve two benchmark engineering problems and compared the experimental results with previous references [26,27,28,29,30,31,32,33,34,35,36,37,38].

4.4.1. Tension/Compression Spring Design Optimization Problem

The first engineering problem is the tension/compression spring design engineering problem. The main purpose of this problem is to minimize the spring weight by determining three design variables when four constraints are satisfied. Three design variables are the wire diameter (d), mean coil diameter (D), and the number of active coils (N). Four constraints are about the shear stress, surge frequency, and the minimum deflection. One objective function, three design variables, and four constraints are shown in Figure 3.

We used the HS, IHS, SGHS, NGHS, and the SANGHS algorithms to search the best result for this problem.

Table 6. Parameters of compared harmony search (HS) algorithms in the tension/compression spring design engineering problem.

Algorithm	HS	IHS [11]	SGHS	NGHS	SANGHS
HMS 1	4	4	4	4	4
HMCR 2	0.8	0.95	$HMC{R}_{mean}=0.98$	–	–
PAR 3	0.3	$PA{R}_{min}=0.35$ $PA{R}_{max}=0.99$	$PA{R}_{mean}=0.9$	–	–
BW 4	–	$B{W}_{min}=0.0005$ $B{W}_{max}=0.05$	$B{W}_{min}=0.0005$ $B{W}_{max}=\left(x_{jU}-x_{jL}\right)/10$	–	–
LP 5	–	–	100	–	–
$P_m$6	–	–	–	0.006	0.008
NI 7	50,000	50,000	50,000	50,000	50,000

¹ HMS: the harmony memory size; ² HMCR: the harmony memory considering rate; ³ PAR: the pitch adjusting rate; ⁴ BW: the bandwidth; ⁵ LP: the learning period; ⁶ $P_m$: the genetic mutation probability; ⁷ NI: the maximum number of iterations.

shows the parameters of these five HS algorithms. The best results obtained by these five algorithms are shown in

Table 7. Experimental results in the tension/compression spring design engineering problem.

Algorithm	d	D	N	Optimal Cost
HHO [26]	0.05179639	0.35930536	11.13885900	0.0126654
GWO [27]	0.05169000	0.35673700	11.28885000	0.0126663
RO [28]	0.05137000	0.34909600	11.76279000	0.0126786
CDE [29]	0.05160900	0.35471400	11.41083100	0.0126702
CPSO [30]	0.05172800	0.35764400	11.24454300	0.0126747
GA3 [31]	0.05198900	0.36396500	10.89052200	0.0126810
ES [32]	0.05164300	0.35536000	11.39792600	0.0126980
GSA [33]	0.05027600	0.32368000	13.52541000	0.0127022
MFO [34]	0.05199446	0.36410932	10.86842186	0.0126669
CWCA [35]	0.05170910	0.35710734	11.27082577	0.0126716
LAFBA [36]	0.05166300	0.35607400	11.33340000	0.0126720
HS	0.05152035	0.35266977	11.53040649	0.0126659
IHS	0.05129291	0.34725895	11.86597566	0.0126683
SGHS	0.05149684	0.35211045	11.56429455	0.0126659
NGHS	0.05148116	0.35173679	11.58707075	0.0126660
SANGHS	0.05162828	0.35525732	11.37510196	0.0126653

. Moreover, as shown in Table 7, to further verify the searching performance of SANGHS algorithm, the best result of SANGHS algorithm is also compared with other state-of-the-art algorithms which have been applied to solve this problem.

In Table 7, it can be seen that, compared with the HS, IHS, SGHS, and the NGHS algorithm, the SANGHS algorithm provides a best fitness value for the tension/compression spring design engineering problem. It is worth noting that the SANGHS algorithm also provides a very competitive result compared with the other state-of-the-art algorithms in previous works.

4.4.2. Welded Beam Design Optimization Problem

The second problem is the welded beam design engineering problem. In this problem, there are four variables that need to be determined that are the thickness of the weld ($x_1$), the length of the clamped bar ($x_2$), the height of the bar ($x_3$), and the thickness of the bar ($x_4$). The main purpose of this problem is to minimize the manufacturing cost when seven constraints are satisfied. The constraints include limits on the shear stress, bending stress, buckling load and end deflection, and several size constraints. The welded beam and its design variables are shown in Figure 4. These seven constraints and the objective function of manufacturing cost are defined as follows.

We also used the HS, IHS, SGHS, NGHS, and the SANGHS algorithms to search the best result for this problem.

Table 8. Parameters of compared harmony search (HS) algorithms in the welded beam design engineering problem.

Algorithm	HS	IHS [11]	SGHS	NGHS	SANGHS
HMS 1	8	8	8	8	8
HMCR 2	0.8	0.95	$HMC{R}_{mean}=0.98$	–	–
PAR 3	0.3	$PA{R}_{min}=0.45$ $PA{R}_{max}=0.99$	$PA{R}_{mean}=0.9$	–	–
BW 4	–	$B{W}_{min}=0.0005$ $B{W}_{max}=2.5$	$B{W}_{min}=0.0005$ $B{W}_{max}=\left(x_{jU}-x_{jL}\right)/10$	–	–
LP 5	–	–	100	–	–
$P_m$6	–	–	–	0.015	0.014
NI 7	200,000	200,000	200,000	200,000	200,000

¹ HMS: the harmony memory size; ² HMCR: the harmony memory considering rate; ³ PAR: the pitch adjusting rate; ⁴ BW: the bandwidth; ⁵ LP: the learning period; ⁶ $P_m$: the genetic mutation probability; ⁷ NI: the maximum number of iterations.

shows the parameters of these five HS algorithms. The best results obtained by these five algorithms are shown in

Table 9. Experimental results in the welded beam design engineering problem.

Algorithms	h	l	t	b	Optimal Cost
HHO [26]	0.20403900	3.53106100	9.02746300	0.20614700	1.73198796
RO [28]	0.20368700	3.52846700	9.00423300	0.20724100	1.73534560
CDE [29]	0.20313700	3.54299800	9.03349800	0.20617900	1.73346218
ES [32]	0.19974200	3.61206000	9.03750000	0.20608200	1.73729731
GSA [33]	0.18212900	3.85697900	10.00000000	0.20237600	1.87994703
LAFBA [36]	0.18470619	3.64265569	9.13489736	0.20525405	1.72870000
GA1 [37]	0.24890000	6.17300000	8.17890000	0.25330000	2.43116000
RANDOM [38]	0.45750000	4.73130000	5.08530000	0.66000000	4.11855504
DAVID [38]	0.24340000	6.25520000	8.29150000	0.24440000	2.38410685
SIMPLEX [38]	0.27920000	5.62560000	7.75120000	0.27960000	2.53072583
APPOX [38]	0.24440000	6.21890000	8.29150000	0.24440000	2.38145339
HS	0.20570604	3.47100746	9.03659628	0.20573099	1.72489123
IHS	0.20573721	3.47045570	9.03644813	0.20574489	1.72494527
SGHS	0.20547623	3.47814281	9.03104463	0.20598515	1.72646956
NGHS	0.20573640	3.47040079	9.03647449	0.20573644	1.72487683
SANGHS	0.20572954	3.47049090	9.03662388	0.20572964	1.72485245

. Moreover, as shown in Table 9, to further verify the searching performance of SANGHS algorithm, the best result of the SANGHS algorithm is also compared with other state-of-the-art algorithms that have been applied to solve this problem.

In Table 9, it can be seen that, compared with the HS, IHS, SGHS, and the NGHS algorithm, the SANGHS algorithm provides a minimum manufacturing cost for the welded beam design engineering problem. It is worth noting that the SANGHS algorithm also provides a very competitive result compared with other state-of-the-art algorithms in previous works.

5. Conclusions

In this paper, we presented a novel algorithm, called the SANGHS algorithm, which combines the NGHS algorithm with a selective acceptance mechanism. In the SANGHS algorithm, the acceptable probability (AP) is generated by a selective acceptance formula in each iteration. Based on the value of AP, the SANGHS algorithm will determine whether to accept the worse trial solution or not. Moreover, to verify the searching performance of the SANGHS algorithm, we used the SANGHS algorithm in ten well-known benchmark continuous optimization problems and two engineering problems and compared the experimental results with other state-of-the-art metaheuristic algorithms. The experimental results provide two findings that are worth noting.

Firstly, it was found that the SANGHS algorithm could result in improved searching performance as opposed to the NGHS algorithm. In the SANGHS algorithm, the selective acceptance mechanism dynamically adjusts the acceptable probability according to the selective acceptance formula. The most important part of the selective acceptance formula is proportion formula, and the main concept of the proportion formula is convergence degree in the HM. Hence, in Equation (6), $F_{worse}^k-F_{best}^k$ represents the convergence degree of all harmonies in the HM. In the early iterations, all solutions have not yet converged in the HM. Therefore, the value of $F_{worse}^k-F_{best}^k$ will be a larger value, and then there is a larger acceptable probability to accept a worse harmony. In the next iteration, there is a higher probability to generate a new harmony in a larger truth region. In other words, there is a strong global exploration ability in the early iterations. However, in the late iterations, the value of $F_{worse}^k-F_{best}^k$ will be a smaller value while all solutions have converged in the HM. Therefore, there is a smaller acceptable probability to accept a worse harmony. In the next iteration, there is a higher probability to generate a new harmony in a smaller truth region. In other words, there is a strong local exploitation ability in the later iterations. According to the experimental results, the proposed proportion mechanism can balance the global exploration and local exploitation ability in the NGHS algorithm.

Finally, in ten considered benchmark optimization problems with three different dimensions, based on the results of the Wilcoxon rank-sum test and the convergence graphs, the searching performance of the SANGHS algorithm was better than other four HS algorithms significantly. Especially in Figure 2, we can easily observe that the SANGHS algorithm had the better searching performance and convergence ability than other algorithms in all problems. Conversely, the HS, IHS, and the SGHS algorithms easily fall within the local optimum, and unlikely escape it. Moreover, in two engineering optimization problems, the SANGHS algorithm could not only search for a better solution compared with other four HS algorithms, but also could provide a competition result to compare with other state-of-the-art algorithms. In conclusion, the SANGHS algorithm is a more efficient and effective algorithm.