Feed-Forward Neural Networks Training with Hybrid Taguchi Vortex Search Algorithm for Transmission Line Fault Classification

Abstract

In this study, the hybrid Taguchi vortex search (HTVS) algorithm, which exhibits a rapid convergence rate and avoids local optima, is employed as a new training algorithm for feed-forward neural networks (FNNs) and its performance was analyzed by comparing it with the vortex search (VS) algorithm, the particle swarm optimization (PSO) algorithm, the gravitational search algorithm (GSA) and the hybrid PSOGSA algorithm. The HTVS-based FNN (FNNHTVS) algorithm was applied to three datasets (iris classification, wine recognition and seed classification) taken from the UCI database (the machine learning repository of the University of California at Irvine) and to the 3-bit parity problem. The obtained statistical results were recorded for comparison. Then, the proposed algorithm was used for fault classification on transmission lines. A dataset was created using 735 kV, 60 Hz, 100 km transmission lines for different fault types, fault locations, fault resistance values and fault inception angles. The FNNHTVS algorithm was applied to this dataset and its performance was tested in comparison with that of other classifiers. The results indicated that the performance of the FNNHTVS algorithm was at least as successful as that of the other comparison algorithms. It has been shown that the FNN model trained with HTVS can be used as a capable alternative algorithm for the solution of classification problems.

1. Introduction

An artificial neural network (ANN) is a computational model based on the human nervous system and it is a useful modeling tool. For this reason, ANNs have been researched with interest in many disciplines such as engineering, finance, technology, etc. ANN structures inspired by biological neural networks have been developed and used in classification [1,2,3], signal processing [4,5] and prediction tasks [6,7,8], as well as in various other studies [9,10,11,12]. In the successful use of an ANN, it is important to choose the training algorithm, the activation function in the neurons, the neural network structure and the parameters (weights and biases) correctly. The training algorithms used in the training of networks aim to create a suitable network structure for the problem by finding the optimal weights and bias parameters. For example, studies have been conducted in an attempt to find the optimal weights and biases by keeping the network topology and activation function constant [13,14,15].

There is a need for a training set that includes suitable features for network training. The parameters of the network are regulated by the training algorithms using the training data [16]. In this context, the main purpose of network training is to ensure harmony between network output and real output by means of training algorithms.

There are many algorithms and methods in the literature that can be used in ANN training. The most commonly used mathematical methods are the back-propagation (BP) [17], gradient descent (GD) [18], conjugate gradient (CG) [19] and Levenberg–Marquardt (LM) methods [20]. Many heuristic algorithms can be used to construct the appropriate network in FNN training.

In [14], the PSOGSA algorithm was proposed for FNN training. The obtained results were compared with the PSO-based FNN (FNNPSO). It has been observed that the PSOGSA-based FNN (FNNPSOGSA) algorithm produces better results compared to the PSO-based FNN (FNNPSO) and GSA-based FNN (FNNGSA) algorithms.

A study was conducted to investigate the effectiveness of the use of the VS algorithm in FNN training [13]. In [13], the performance of the VS-based FNN (FNNVS) was compared with the performance of an FNN trained with other optimization algorithms using different classification problems. The obtained results showed that the VS algorithm can be used in FNN training. Furthermore, the discrete-continuous version of the vortex search algorithm was used to determine the sizes and locations of PV sources [21]. In [22], the optimal selection of conductors in three-phase distribution networks was performed through the use of a discrete version of the vortex search algorithm.

It can be used in models obtained as a result of hybridizing classical training algorithms and heuristic optimization algorithms in ANN training. In [23], a new method was presented, based on hybridizing the artificial bee colony (ABC) algorithm and the LM algorithm (ABC-LM). The authors carried out this study to prevent the LM algorithm from getting stuck on local minimums and the ABC algorithm converged slowly to global minimums.

Heidari et al. [24] presented a stochastic training algorithm in their study. They suggested that the grasshopper optimization algorithm (GOA) performed well in the solution of optimization problems and could also be used in the training of multilayer perceptron (MLP) neural networks. The GOAMLP model was compared with other efficient algorithms using five different classification problems. The authors stated that the use of GOAMLP contributed to obtaining accurate classification performance.

In [25], the dragonfly algorithm (DA) was used in FNN training. Experiments were conducted on classification problems and a civil engineering problems. The obtained results showed that the DA was quite successful in FNN training. Additionally, they tried to emphasize the avoidance of the local optima.

In [26], weights and biases parameters of the FNN were optimized by means of the whale optimization algorithm (WOA). Within the scope of the study, comparisons were made with different algorithms through classification problems. The authors stated that it performed better in terms of its avoidance of the local optimum and its convergence rate. In addition to the studies mentioned above, other studies have been conducted using optimization algorithms for ANN training. These include studies of the krill-herd algorithm (KHA) [27], the cuckoo search (CS) algorithm [28] and the the symbiotic organism search (SOS) algorithm [29].

Table 1. Brief summary of algorithms used in the literature for FNN training.

Reference	Algorithm
Faris, H. et al., 2016 [1]	Multi-Verse Optimizer
Sag, T. et al., 2021 [13]	Vortex Search Algorithm
Mirjalili, S. et al., 2012 [14]	Hybrid PSO-GSA algorithm
Pashaei, E. et al., 2021 [15]	Enhanced Black Hole Algorithm
Hagan, M.T. et al., 1994 [20]	Marquardt Algorithm
Ozturk, C. et al., 2011 [23]	Hybrid Artificial Bee Colony Algorithm
Heidari, A.A. et al., 2019 [24]	Grasshopper Optimization Algorithm
Gulcu, S., 2022 [25]	Dragonfly Algorithm
Aljarah, I. et al., 2018 [26]	Whale Optimization Algorithm
Lari, N.S. et al., 2014 [27]	Krill-Herd Algorithm
Jiao-hong, Y. et al., 2014 [28]	Cuckoo Search Algorithm
Wu, H. et al., 2016 [29]	Symbiotic Organisms Search Algorithm
Zhang, J.R. et al., 2007 [30]	Hybrid PSO-BP Algorithm
Mirjalili, S. et al., 2014 [31]	Biogeography-based Optimizer

presents some algorithms used in FNN training. The main purpose of these studies was to train the FNN structure in the best way. The main difference between these studies is that they used different algorithms from one another. The algorithm presented in this study is different from these, and it was also used for transmission line fault classification.

To the best of our knowledge, this is the first study conducted on HTVS-based FNN training. In this study, our main purpose was not to find the most suitable FNN structure for a test problem or to obtain the smallest error value that could be achieved. Rather, the primary purpose of this study was to present the use of the HTVS algorithm [32] in FNN training and to compare its performance with that of the VS [33], PSO [34], PSOGSA [14] and GSA [35]. Therefore, 3-bit parity, iris classification, wine recognition and seed classification benchmark datasets were used for performance comparisons. In order to show that the HTVS algorithm had a competitive character compared to other algorithms used in FNN training, tests were conducted using different hidden neuron numbers in the FNN structure.

The second main purpose of the study was to show that the proposed algorithm can be used in fault classification on transmission lines. For this purpose, a transmission line of 735 kV, 60 Hz and 100 km longwas modeled as frequency-dependent with the help of Matlab/Simulink. Fault data were produced and recorded on the modeled transmission line. Using these data, the FNNHTVS algorithm and the optimization algorithm-based FNNVS, FNNPSO, FNNPSOGSA and FNNGSA algorithms were compared. Additionally, the performance of the proposed algorithm was compared with that of classifiers such as a support vector machine (SVM), the K-nearest neighbor (KNN) method and an FNN with LM and Naive Bayes (NB). The results showed that the FNNHTVS algorithm was quite successful.

The main contributions of this study are briefly listed as follows.

• The HTVS algorithm is presented for the first time as an alternative algorithm to overcome slow convergence and local optimum problems in FNN training.
• The effectiveness of the HTVS algorithm in FNN training is demonstrated.
• It has been proven that the FNNHTVS structure can achieve results comparable to and better than those of other successful algorithms in classification studies.
• It has been shown that the FNNHTVS algorithm can be used as an alternative algorithm for transmission line short-circuit fault classification tasks.

The remainder of this paper is organized as follows. In Section 2 we explain the basics concept of the FNN, the HTVS algorithm and FNN training using HTVS. In Section 3 we present the experimental results and a discussion of the performance of the algorithms. In Section 4 we present an evaluation of the performance of FNNHTVS in fault classification. In Section 5, we present our conclusions.

2. Basic Principles

2.1. Feed-Forward Neural Network

FNNs are neural networks that have forward data flow. They are frequently used in classification and regression problems. Neurons are represented as processing units for FNNs. In the structure of each neuron, there are activation functions that may be radial, linear, sigmoid, etc. Neurons generate output data based on input data using activation functions. In FNNs, neurons in each layer are fed only by the neurons of the previous layer. Neurons are arranged in layers and the outputs of neurons in one layer are input to the next layer over weights. The input layer transmits the information it receives from the external environment to the neurons in the next layer without making any changes [36]. The first layer is the input, the last layer is the output and the layers between these two are called hidden layers. Figure 1 presents the basic structure of a three-layer FNN. The number of neurons in the input and output layer varies depending on the nature of the problem. The number of hidden layers and the number of neurons in the hidden layers are chosen according to the complexity of the problem being studied [13]. The overall goal of network training is to minimize the difference between the target output and the achieved output. The FNN training process is completed by updating the weights in its structure and the bias parameters used to balance these weights in each cycle.

2.2. HTVS Algorithm

HTVS is an optimization algorithm created by hybridizing the VS algorithm and the Taguchi orthogonal array approach (TOAA) [32]. This algorithm has shown successful results in optimization problems [32]. For this algorithm the use of orthogonal arrays (OAs) in the population generation phase is preferable. Since there are OAs in the structure of the HTVS algorithm, the computational cost is slightly higher than that of VS. However, the disadvantages of VS, such as its slow convergence and the fact that it can become trapped in local minima, are compensated for in this way. In the HTVS algorithm, randomly generated candidate solutions are evenly distributed in the search space via TOAA. The developed candidate solutions are used for the VS algorithm. HTVS is an optimization algorithm that can achieve highly effective results using fewer iterations.

The working principle of HTVS can be briefly explained as follows. Firstly, candidate solutions are distributed through TOAA. In the OA, columns represent the parameters that need to be optimized. Each row describes a possible combination of the level values for these parameters. The problem size and OA columns are compatible with each other. Secondly, OA-related level values are determined for each candidate solution in order to improve the candidate solutions produced. Optimum level values are determined for each candidate solution and they are selected for OA training. Finally, the optimized candidate solutions are sent to the VS search space (circle). The best solution produced by the candidate solutions is determined as the best of that iteration. If the solution obtained as a result of an iteration is better than the previous results, it is saved and kept as the best solution. These operations are performed until a specified number of iterations has been reached. The pseudo-code of the HTVS is presented in Algorithm 1. At the beginning of the algorithm, necessary definitions, such as problem boundaries, size, number of iterations, reduction ratio coefficient, etc., are set. The desired OA is created according to the problem dimensions. Then, candidate solutions are created and checked to see if they are within the boundaries. Each level value is determined for each candidate solution. These level values are associated with the OA. Optimum level values are determined for each parameter. The level difference is reduced by means of the reduction ratio coefficient and this process is continued until the target error value is reached. Thus, the candidate solutions are improved. The improved candidate solutions are sent to the vortex circle for examination. If the iteration’s best value is better than the global best value, the iteration’s best value is selected and recorded as the global best value. Then, the radius of the vortex circle is updated and reduced. The algorithm’s steps continue until the maximum iteration number is reached. More detailed information about the HTVS and VS algorithms can be found in [32,33], respectively.

Algorithm 1: HTVS Algorithm.

2.3. FNN Training Using HTVS

In this section, the basics of using the HTVS algorithm in FNN training are explained. In this study, optimal weights and biases were selected to improve the FNN’s performance. The activation function and FNN structure remained constant. By means of the optimum values obtained, we ensured that the FNN reached the minimum error. In order to create the FNNHTVS structure discussed in this article, the fitness function and encoding strategy should be determined. A fitness function must be defined based on the mean square error (MSE) value in the FNN output.

The fitness function is produced as in [30]. For an FNN with a structure as in Figure 2, the fitness function is calculated by following the steps outlined below, where n is the number of inputs, $h_n$ is the number of hidden layer nodes and $o_n$ is equal to the output number. To calculate each hidden node,

$$f\left(x_l\right)=1/\left(1+exp\left(-\left(\sum _{k=1}^n{w}_{kl}·x_l-b_l\right)\right)\right),l=1,2,\dots ,h_n$$

(1)

where $w_{kl}$ is the connection weight from input nodes to hidden layer nodes. $b_l$ stands for the hidden layer node bias and $x_l$ is the lth input for the network. After calculating the output of the hidden nodes, the output is evaluated as follows:

$$ou{t}_m=\sum _{i=1}^{h_n}{w}_{mi}·f\left(h_i\right)-b_m,m=1,2,\dots ,o_n$$

(2)

where $w_{mi}$ is the weight value from the hidden layer nodes to the output layer nodes and $b_m$ is used to express the output layer nodes’ biases. MSE is determined as follows:

$$MSE=\frac{1}{n_s}\sum _{q=1}^{n_s}{\left(\sum _{i=1}^{o_n}ou{t}_i^z-t_i^z\right)}^2$$

(3)

where $t_i$ is equal to the real value, $n_s$ is the number of samples used for training and z stands for an output node.

After the fitness function was created, the coding strategy was chosen. In this study, the matrix encoding strategy, used in studies related to FNN training, was chosen. The candidate solution matrix (CSM) consists of the combination of the four matrices described below. Figure 2 shows the weights and biases for an FNN with a 3-2-1 topology.

CSM is performed as follows:

$$CSM=\left[\begin{array}{ccccccc}weigh{t}_1& & weigh{t}_2^{\prime }& & bia{s}_1& & bia{s}_2\end{array}\right]$$

$$Weigh{t}_1=\left[\begin{array}{ccc}{w}_{11}& w_{21}& w_{31}\\ w_{12}& w_{22}& w_{32}\end{array}\right],\hspace{1em}Bia{s}_1=\left[\begin{array}{c}{b}_1\\ b_2\end{array}\right]$$

$$Weigh{t}_2^{\prime }=\left[\begin{array}{c}{w}_{51}\\ w_{61}\end{array}\right],Bia{s}_2=\left[\begin{array}{c}{b}_3\end{array}\right]$$

where $Weigh{t}_1$ is the weight matrix from the input layer to the hidden layer, $Bia{s}_1$ is the hidden layer node bias matrix, $Weigh{t}_2^{\prime }$ is the transpose weight matrix for from the hidden layer to the output layer and $Bia{s}_2$ is the output layer node bias matrix.

After the fitness function and coding strategy are determined, a mesh structure suitable for the data set is determined. The steps of the FNNHTVS algorithm are followed in order to find the best values of the weights and biases. A flowchart diagram of the FNNHTVS algorithm is shown in Figure 3.

The FNN structure is determined for the classification dataset. HTVS parameters are defined, such as the maximum iteration number, the initial range of candidate solutions, the population size, etc. The dimensions of the problem are equal to the total number of weights and biases. To achieve the best values of weights and biases, candidate solutions are constructed depending on the OA and improved candidate solutions are determined. The feed-forward calculation is first applied for each sample in the dataset. Then, the errors, which are the difference between the calculated and desired values, are found. Finally, MSE is calculated. The best candidate solutions (CSs) are selected and parameters are updated according to rules. FNNHTVS continues until meeting the end criterion.

3. Validation of the FNNHTVS via Benchmark Datasets

In this section, the proposed FNNHTVS training algorithm is compared with the FNNVS, FNNPSO, FNNGSA and FNNPSOGSA algorithms. All algorithms are run on FNNs with the same structure. To analyze the performance of the FNNHTVS algorithm and to compare it with other algorithms, four frequently used classification problems were selected. These are iris classification, wine recognition, seed classification and the 3-bit parity problem. The first three of these were taken from the UCI machine learning repository of the University of California at Irvine [37]. The fourth problem is the 3-bit parity problem. The input and output values related to this problem are given in

Table 2. 3-bit parity problem.

Input	000	001	010	011	100	101	110	111
Output	0	1	1	0	1	0	0	1

.

The problems chosen for comparison are classification problems that are frequently used in the literature [13,14,15]. The features and class numbers related to the problems are expressed in

Table 3. Dataset information.

Problem	N. of Features	N. of Classes	N. of Samples
3-bit parity	3	2	8
Seeds	7	3	210
Iris	4	3	150
Wine	13	3	178

.

The parameters common to all algorithms were kept the same. For all algorithms, the population size and maximum iteration were set to 30 and 100, respectively. An initial range of candidate solutions of [−50, 50] was preferred so that all the training algorithms could search within a wider space. Additionally, these algorithms contain user-controlled parameters.

Table 4. Special parameters for each algorithm.

Algorithm	Parameter	Value
HTVS	Level difference	0.8
PSO	$C_1$ and $C_2$ constants	2
	Inertia weights	[0.9, 0.5]
GSA	a	20
	Gravitational constant	1
	Initial acceleration and mass	0
PSOGSA	$C_1^{\prime }$ and $C_2^{\prime }$ constants	1
	Gravitational constant	1
	Inertia weights	[0.9, 0.5]

presents these parameters.

In this study, a network structure with 1 input, 1 hidden and 1 output (i-h-o) layer was selected. The Sigmoid function was determined for each node as the activation function. The algorithms were compared using benchmark datasets for 11 different numbers of hidden nodes. The algorithms were run until they reached the maximum number of iterations. Each algorithm was run 30 different times for each case. MSE was chosen as the comparison parameter and the mean, standard deviation (std. dev.) and best and worst values of the obtained data were recorded. These recorded statistical values provided information for the comparison. However, the Wilcoxon signed rank (WSR) pairwise comparison test was also applied to make a stronger comparison. The WSR test was used to determine which of the two comparing methods was superior. In this study, the statistical significance value was 0.05 for the WSR test. For each problem, the FNNHTVS algorithm was compared with other algorithms separately and measures of superiority, equality and loss were noted. Detailed information about the WSR test can be found in [38].

3.1. 3-Bit Parity Problem

The 3-bit parity problem is a frequently used nonlinear problem. It is an important problem used to measure the performance of training algorithms against nonlinear problems. In the three-input, single-output 3-bit parity problem, if the number of ones in the inputs is odd, the output is one; if even, the output is zero. The input and output sets of this problem are expressed in Table 2. $H_n$ is the number of hidden nodes with $H_n$= 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 20, 30. For the 3-bit parity problem, a 3-$H_n$-1 FNN structure is used. This structure has a total of ($5H_n+1$) parameters, $4H_n$ weights and $H_n+1$ biases, and the parameter range is taken as [−50, 50]. Algorithms were evaluated based on the mean, standard deviation and the best and worst value of MSE. The statistical results obtained after 30 independent runs are shown in

Table A1. Statistical results (MSE) for the 3-bit parity problem.

Hidden Nodes	Parameters	FNNHTVS	FNNVS	FNNPSO	FNNPSOGSA	FNNGSA
4	mean	2.46929 $\times {10}^{-2}$	1.74843 $\times {10}^{-1}$	2.82043 $\times {10}^{-1}$	2.73659 $\times {10}^{-1}$	4.82205 $\times {10}^{-1}$
	std. dev.	4.74911 $\times {10}^{-2}$	1.51402 $\times {10}^{-1}$	9.26122 $\times {10}^{-2}$	8.93695 $\times {10}^{-2}$	1.73568 $\times {10}^{-2}$
	best	7.55448 $\times {10}^{-21}$	1.39264 $\times {10}^{-14}$	1.12847 $\times {10}^{-1}$	1.24014 $\times {10}^{-1}$	4.24011 $\times {10}^{-1}$
	worst	1.60879 $\times {10}^{-1}$	4.58333 $\times {10}^{-1}$	4.18578 $\times {10}^{-1}$	4.57686 $\times {10}^{-1}$	5.04945 $\times {10}^{-1}$
5	mean	9.63464 $\times {10}^{-3}$	1.26501 $\times {10}^{-1}$	2.75948 $\times {10}^{-1}$	2.38619 $\times {10}^{-1}$	4.82409 $\times {10}^{-1}$
	std. dev.	3.28141 $\times {10}^{-2}$	1.30249 $\times {10}^{-1}$	7.29964 $\times {10}^{-2}$	7.01424 $\times {10}^{-2}$	1.95679 $\times {10}^{-2}$
	best	6.62840 $\times {10}^{-39}$	5.06917 $\times {10}^{-21}$	9.35993 $\times {10}^{-2}$	1.27838 $\times {10}^{-1}$	3.94271 $\times {10}^{-1}$
	worst	1.74005 $\times {10}^{-1}$	3.75015 $\times {10}^{-1}$	4.18856 $\times {10}^{-1}$	4.29496 $\times {10}^{-1}$	5.01850 $\times {10}^{-1}$
6	mean	1.07093 $\times {10}^{-2}$	1.63833 $\times {10}^{-1}$	2.53330 $\times {10}^{-1}$	2.35092 $\times {10}^{-1}$	4.78239 $\times {10}^{-1}$
	std. dev.	2.93566 $\times {10}^{-2}$	1.46849 $\times {10}^{-1}$	9.22590 $\times {10}^{-2}$	8.21892 $\times {10}^{-2}$	2.08078 $\times {10}^{-2}$
	best	8.82973 $\times {10}^{-27}$	5.81165 $\times {10}^{-16}$	5.41118 $\times {10}^{-2}$	1.12504 $\times {10}^{-1}$	4.12376 $\times {10}^{-1}$
	worst	1.25001 $\times {10}^{-1}$	4.16674 $\times {10}^{-1}$	3.85710 $\times {10}^{-1}$	4.10193 $\times {10}^{-1}$	5.02324 $\times {10}^{-1}$
7	mean	3.89732 $\times {10}^{-4}$	1.52462 $\times {10}^{-1}$	2.59321 $\times {10}^{-1}$	2.13170 $\times {10}^{-1}$	4.73091 $\times {10}^{-1}$
	std. dev.	1.94526 $\times {10}^{-3}$	1.48291 $\times {10}^{-1}$	7.65263 $\times {10}^{-2}$	8.89791 $\times {10}^{-2}$	2.78423 $\times {10}^{-2}$
	best	3.08697 $\times {10}^{-43}$	7.01922 $\times {10}^{-24}$	1.30899 $\times {10}^{-2}$	6.36175 $\times {10}^{-2}$	4.03514 $\times {10}^{-1}$
	worst	1.06678 $\times {10}^{-2}$	5.00000 $\times {10}^{-1}$	4.33768 $\times {10}^{-1}$	4.52949 $\times {10}^{-1}$	5.01753 $\times {10}^{-1}$
8	mean	1.55137 $\times {10}^{-2}$	1.03572 $\times {10}^{-1}$	2.76456 $\times {10}^{-1}$	2.16024 $\times {10}^{-1}$	4.73142 $\times {10}^{-1}$
	std. dev.	4.06209 $\times {10}^{-2}$	1.16576 $\times {10}^{-1}$	1.03673 $\times {10}^{-1}$	8.54150 $\times {10}^{-2}$	1.92504 $\times {10}^{-2}$
	best	1.62182 $\times {10}^{-39}$	3.26241 $\times {10}^{-31}$	3.50026 $\times {10}^{-2}$	7.82187 $\times {10}^{-2}$	4.30022 $\times {10}^{-1}$
	worst	1.25328 $\times {10}^{-1}$	3.75000 $\times {10}^{-1}$	4.20465 $\times {10}^{-1}$	4.10480 $\times {10}^{-1}$	5.08010 $\times {10}^{-1}$
9	mean	8.33333 $\times {10}^{-3}$	1.41648 $\times {10}^{-1}$	2.87803 $\times {10}^{-1}$	2.14414 $\times {10}^{-1}$	4.62800 $\times {10}^{-1}$
	std. dev.	3.17135 $\times {10}^{-2}$	1.26004 $\times {10}^{-1}$	8.33543 $\times {10}^{-2}$	7.56429 $\times {10}^{-2}$	3.88811 $\times {10}^{-2}$
	best	3.14792 $\times {10}^{-40}$	4.13176 $\times {10}^{-19}$	1.44202 $\times {10}^{-1}$	7.93862 $\times {10}^{-2}$	3.85171 $\times {10}^{-1}$
	worst	1.25000 $\times {10}^{-1}$	3.75000 $\times {10}^{-1}$	4.33781 $\times {10}^{-1}$	3.66044 $\times {10}^{-1}$	5.15740 $\times {10}^{-1}$
10	mean	1.66667 $\times {10}^{-2}$	1.63635 $\times {10}^{-1}$	2.59155 $\times {10}^{-1}$	2.24921 $\times {10}^{-1}$	4.49624 $\times {10}^{-1}$
	std. dev.	4.32182 $\times {10}^{-2}$	1.04256 $\times {10}^{-1}$	9.58570 $\times {10}^{-2}$	6.07222 $\times {10}^{-2}$	3.93542 $\times {10}^{-2}$
	best	4.47502 $\times {10}^{-48}$	2.34936 $\times {10}^{-16}$	5.34658 $\times {10}^{-2}$	7.80592 $\times {10}^{-2}$	3.52046 $\times {10}^{-1}$
	worst	1.25000 $\times {10}^{-1}$	3.75000 $\times {10}^{-1}$	4.06478 $\times {10}^{-1}$	3.48740 $\times {10}^{-1}$	5.09670 $\times {10}^{-1}$
12	mean	1.71400 $\times {10}^{-2}$	1.00002 $\times {10}^{-1}$	2.37016 $\times {10}^{-1}$	2.02146 $\times {10}^{-1}$	4.58024 $\times {10}^{-1}$
	std. dev.	4.31070 $\times {10}^{-2}$	1.10837 $\times {10}^{-1}$	8.66082 $\times {10}^{-2}$	8.98497 $\times {10}^{-2}$	3.15063 $\times {10}^{-2}$
	best	1.19825 $\times {10}^{-62}$	6.10052 $\times {10}^{-25}$	2.31919 $\times {10}^{-2}$	7.19333 $\times {10}^{-2}$	3.84548 $\times {10}^{-1}$
	worst	1.25000 $\times {10}^{-1}$	3.75000 $\times {10}^{-1}$	4.03867 $\times {10}^{-1}$	3.95908 $\times {10}^{-1}$	5.13763 $\times {10}^{-1}$
15	mean	8.33333 $\times {10}^{-3}$	1.37322 $\times {10}^{-1}$	2.75541 $\times {10}^{-1}$	1.63485 $\times {10}^{-1}$	4.39196 $\times {10}^{-1}$
	std. dev.	3.17135 $\times {10}^{-2}$	1.14827 $\times {10}^{-1}$	8.07514 $\times {10}^{-2}$	7.68703 $\times {10}^{-2}$	4.77070 $\times {10}^{-2}$
	best	1.89087 $\times {10}^{-75}$	9.42598 $\times {10}^{-24}$	7.19125 $\times {10}^{-2}$	2.77272 $\times {10}^{-2}$	3.30466 $\times {10}^{-1}$
	worst	1.25000 $\times {10}^{-1}$	3.75000 $\times {10}^{-1}$	4.42468 $\times {10}^{-1}$	3.38089 $\times {10}^{-1}$	5.07388 $\times {10}^{-1}$
20	mean	4.16667 $\times {10}^{-3}$	1.70834 $\times {10}^{-1}$	2.34259 $\times {10}^{-1}$	2.49060 $\times {10}^{-1}$	4.93816 $\times {10}^{-1}$
	std. dev.	2.28218 $\times {10}^{-2}$	8.97992 $\times {10}^{-2}$	8.78904 $\times {10}^{-2}$	1.58705 $\times {10}^{-1}$	1.58475 $\times {10}^{-1}$
	best	9.27099 $\times {10}^{-100}$	3.63070 $\times {10}^{-29}$	5.42893 $\times {10}^{-2}$	6.75910 $\times {10}^{-4}$	1.09362 $\times {10}^{-1}$
	worst	1.25000 $\times {10}^{-1}$	2.50000 $\times {10}^{-1}$	4.37647 $\times {10}^{-1}$	5.03318 $\times {10}^{-1}$	8.68272 $\times {10}^{-1}$
30	mean	8.41417 $\times {10}^{-73}$	1.91668 $\times {10}^{-1}$	2.69017 $\times {10}^{-1}$	3.28764 $\times {10}^{-1}$	5.57519 $\times {10}^{-1}$
	std. dev.	4.60821 $\times {10}^{-72}$	1.34282 $\times {10}^{-1}$	8.80375 $\times {10}^{-2}$	1.54711 $\times {10}^{-1}$	8.00067 $\times {10}^{-2}$
	best	1.69134 $\times {10}^{-166}$	8.16898 $\times {10}^{-38}$	4.92869 $\times {10}^{-2}$	7.18583 $\times {10}^{-5}$	3.75444 $\times {10}^{-1}$
	worst	2.52403 $\times {10}^{-71}$	5.00000 $\times {10}^{-1}$	4.18743 $\times {10}^{-1}$	6.25000 $\times {10}^{-1}$	6.93499 $\times {10}^{-1}$

.

Looking at Table A1 from a general perspective, it can be observed that the FNNHTVS algorithm performed better than the other compared algorithms. The proposed algorithm for all hidden nodes obtained the best mean MSE values. This indicates that it effectively escaped the local minimum. We determined that the FNNGSA algorithm had the lowest standard deviations, except for hidden nodes 7, 15, 20 and 30. When the best and worst MSE values were examined, we found that the best values belonged to the FNNHTVS algorithm. The closest follower of the FNNHTVS algorithm was the FNNVS algorithm.

Additionally, the WSR test results are presented in

Table 5. WSR test results for the 3-bit parity problem.

Method	Winner (FNNHTVS/Method 2)	Equal
FNNHTVS vs. FNNVS	11/0	0
FNNHTVS vs. FNNPSO	11/0	0
FNNHTVS vs. FNNPSOGSA	11/0	0
FNNHTVS vs. FNNGSA	11/0	0

. The Winner column in Table 5 shows in how many cases (11 different hidden nodes) the two compared algorithms outperformed each other. The column specified as Equal shows the number of cases where the algorithms could not outperform each other. As a result of the paired comparisons, the superiority of the FNNHTVS algorithm can be observed. Within the framework of the results, the effectiveness of the proposed training algorithm for this nonlinear problem has been shown.

3.2. Iris Classification Problem

The iris dataset is the best-known and most commonly used dataset in the pattern recognition literature [37]. The dataset consists of four inputs and three classes. The dataset contains a total of 150 samples, fifty for each class. The first class is classified as Iris setosa, the second class is Iris Versicolor and the third class is Iris Virginica. For the iris classification problem, a 4-$H_n$-3 FNN structure is used. This structure has a total of $8H_n+3$ parameters, 7$H_n$ weights and $H_n+3$ biases, and the initial parameter range is taken as [−50, 50]. The statistical results obtained after 30 independent runs are presented in

Table A2. Statistical results (MSE) in the iris classification problem.

Hidden Nodes	Parameters	FNNHTVS	FNNVS	FNNPSO	FNNPSOGSA	FNNGSA
4	mean	1.14862 $\times {10}^{-1}$	1.68751 $\times {10}^{-1}$	2.01325 $\times {10}^{-1}$	2.15957 $\times {10}^{-1}$	4.64779 $\times {10}^{-1}$
	std. dev.	1.06685 $\times {10}^{-1}$	1.50803 $\times {10}^{-1}$	5.22724 $\times {10}^{-2}$	3.53812 $\times {10}^{-2}$	5.46828 $\times {10}^{-2}$
	best	2.99539 $\times {10}^{-2}$	1.33366 $\times {10}^{-2}$	8.45694 $\times {10}^{-2}$	1.46577 $\times {10}^{-1}$	3.85026 $\times {10}^{-1}$
	worst	3.65792 $\times {10}^{-1}$	3.86443 $\times {10}^{-1}$	3.10293 $\times {10}^{-1}$	2.79219 $\times {10}^{-1}$	6.06090 $\times {10}^{-1}$
5	mean	6.34653 $\times {10}^{-2}$	1.62361 $\times {10}^{-1}$	1.84219 $\times {10}^{-1}$	1.92374 $\times {10}^{-1}$	4.34723 $\times {10}^{-1}$
	std. dev.	4.37378 $\times {10}^{-2}$	1.43383 $\times {10}^{-1}$	5.23417 $\times {10}^{-2}$	4.11192 $\times {10}^{-2}$	3.80232 $\times {10}^{-2}$
	best	2.64633 $\times {10}^{-2}$	1.86090 $\times {10}^{-2}$	8.53060 $\times {10}^{-2}$	1.04880 $\times {10}^{-1}$	3.79576 $\times {10}^{-1}$
	worst	2.45174 $\times {10}^{-1}$	3.66705 $\times {10}^{-1}$	2.72920 $\times {10}^{-1}$	2.80629 $\times {10}^{-1}$	5.14518 $\times {10}^{-1}$
6	mean	4.68511 $\times {10}^{-2}$	1.36568 $\times {10}^{-1}$	1.81852 $\times {10}^{-1}$	1.84580 $\times {10}^{-1}$	4.27138 $\times {10}^{-1}$
	std. dev.	1.91529 $\times {10}^{-2}$	1.41366 $\times {10}^{-1}$	4.42480 $\times {10}^{-2}$	4.06917 $\times {10}^{-2}$	5.56950 $\times {10}^{-2}$
	best	2.63022 $\times {10}^{-2}$	1.35111 $\times {10}^{-2}$	9.05435 $\times {10}^{-2}$	9.49799 $\times {10}^{-2}$	3.05662 $\times {10}^{-1}$
	worst	1.20172 $\times {10}^{-1}$	4.73654 $\times {10}^{-1}$	2.70677 $\times {10}^{-1}$	2.52811 $\times {10}^{-1}$	5.67574 $\times {10}^{-1}$
7	mean	5.64944 $\times {10}^{-2}$	1.16451 $\times {10}^{-1}$	1.92708 $\times {10}^{-1}$	1.81135 $\times {10}^{-1}$	4.10110 $\times {10}^{-1}$
	std. dev.	6.35361 $\times {10}^{-2}$	1.27068 $\times {10}^{-1}$	5.36873 $\times {10}^{-2}$	4.98007 $\times {10}^{-2}$	6.73658 $\times {10}^{-2}$
	best	1.33333 $\times {10}^{-2}$	1.62844 $\times {10}^{-2}$	1.15527 $\times {10}^{-1}$	9.28957 $\times {10}^{-2}$	2.99464 $\times {10}^{-1}$
	worst	3.46725 $\times {10}^{-1}$	3.56866 $\times {10}^{-1}$	3.25583 $\times {10}^{-1}$	2.90122 $\times {10}^{-1}$	5.91297 $\times {10}^{-1}$
8	mean	7.25993 $\times {10}^{-2}$	1.13436 $\times {10}^{-1}$	1.60370 $\times {10}^{-1}$	1.60365 $\times {10}^{-1}$	4.09623 $\times {10}^{-1}$
	std. dev.	6.10813 $\times {10}^{-2}$	1.09866 $\times {10}^{-1}$	5.21421 $\times {10}^{-2}$	3.76046 $\times {10}^{-2}$	5.45519 $\times {10}^{-2}$
	best	2.15981 $\times {10}^{-2}$	2.29368 $\times {10}^{-2}$	6.82389 $\times {10}^{-2}$	9.08569 $\times {10}^{-2}$	3.31659 $\times {10}^{-1}$
	worst	2.50296 $\times {10}^{-1}$	3.58895 $\times {10}^{-1}$	2.64006 $\times {10}^{-1}$	2.29685 $\times {10}^{-1}$	5.58969 $\times {10}^{-1}$
9	mean	3.94528 $\times {10}^{-2}$	1.53900 $\times {10}^{-1}$	1.48810 $\times {10}^{-1}$	1.75455 $\times {10}^{-1}$	3.71082 $\times {10}^{-1}$
	std. dev.	1.30927 $\times {10}^{-2}$	1.30269 $\times {10}^{-1}$	3.95891 $\times {10}^{-2}$	4.77214 $\times {10}^{-2}$	4.59631 $\times {10}^{-2}$
	best	1.98988 $\times {10}^{-2}$	2.62890 $\times {10}^{-2}$	7.89208 $\times {10}^{-2}$	1.02002 $\times {10}^{-1}$	2.87191 $\times {10}^{-1}$
	worst	7.31698 $\times {10}^{-2}$	3.73252 $\times {10}^{-1}$	2.53782 $\times {10}^{-1}$	2.80107 $\times {10}^{-1}$	4.96640 $\times {10}^{-1}$
10	mean	3.51741 $\times {10}^{-2}$	1.34166 $\times {10}^{-1}$	1.52510 $\times {10}^{-1}$	1.75090 $\times {10}^{-1}$	3.96100 $\times {10}^{-1}$
	std. dev.	1.04808 $\times {10}^{-2}$	1.20707 $\times {10}^{-1}$	4.31790 $\times {10}^{-2}$	3.58333 $\times {10}^{-2}$	5.75770 $\times {10}^{-2}$
	best	1.33428 $\times {10}^{-2}$	2.55888 $\times {10}^{-2}$	7.93357 $\times {10}^{-2}$	9.93091 $\times {10}^{-2}$	2.97023 $\times {10}^{-1}$
	worst	6.02462 $\times {10}^{-2}$	3.67249 $\times {10}^{-1}$	2.64466 $\times {10}^{-1}$	2.39029 $\times {10}^{-1}$	4.91150 $\times {10}^{-1}$
12	mean	3.74717 $\times {10}^{-2}$	1.35911 $\times {10}^{-1}$	1.47051 $\times {10}^{-1}$	1.93487 $\times {10}^{-1}$	3.85262 $\times {10}^{-1}$
	std. dev.	1.29253 $\times {10}^{-2}$	1.18291 $\times {10}^{-1}$	3.77622 $\times {10}^{-2}$	1.31807 $\times {10}^{-1}$	6.21025 $\times {10}^{-2}$
	best	1.33333 $\times {10}^{-2}$	2.05661 $\times {10}^{-2}$	6.78784 $\times {10}^{-2}$	9.12593 $\times {10}^{-2}$	3.05040 $\times {10}^{-1}$
	worst	8.38928 $\times {10}^{-2}$	3.53335 $\times {10}^{-1}$	2.06790 $\times {10}^{-1}$	8.24107 $\times {10}^{-1}$	5.65557 $\times {10}^{-1}$
15	mean	6.38422 $\times {10}^{-2}$	1.08019 $\times {10}^{-1}$	1.48758 $\times {10}^{-1}$	2.39809 $\times {10}^{-1}$	4.09646 $\times {10}^{-1}$
	std. dev.	7.72433 $\times {10}^{-2}$	1.14276 $\times {10}^{-1}$	3.74622 $\times {10}^{-2}$	1.95200 $\times {10}^{-1}$	1.21881 $\times {10}^{-1}$
	best	1.29753 $\times {10}^{-2}$	2.38645 $\times {10}^{-2}$	7.14960 $\times {10}^{-2}$	9.19578 $\times {10}^{-2}$	2.74557 $\times {10}^{-1}$
	worst	3.40175 $\times {10}^{-1}$	3.80079 $\times {10}^{-1}$	1.99074 $\times {10}^{-1}$	8.06625 $\times {10}^{-1}$	7.41139 $\times {10}^{-1}$
20	mean	7.53001 $\times {10}^{-2}$	1.25018 $\times {10}^{-1}$	1.25229 $\times {10}^{-1}$	3.11545 $\times {10}^{-1}$	6.59167 $\times {10}^{-1}$
	std. dev.	9.31963 $\times {10}^{-2}$	1.21136 $\times {10}^{-1}$	3.22619 $\times {10}^{-2}$	2.06710 $\times {10}^{-1}$	2.66077 $\times {10}^{-1}$
	best	2.46229 $\times {10}^{-2}$	1.34936 $\times {10}^{-2}$	5.50465 $\times {10}^{-2}$	8.49606 $\times {10}^{-2}$	2.01427 $\times {10}^{-1}$
	worst	3.47971 $\times {10}^{-1}$	3.60454 $\times {10}^{-1}$	1.80171 $\times {10}^{-1}$	8.14597 $\times {10}^{-1}$	6.21821 $\times {10}^{-1}$
30	mean	1.99517 $\times {10}^{-1}$	1.07584 $\times {10}^{-1}$	1.19432 $\times {10}^{-1}$	3.16707 $\times {10}^{-1}$	8.10926 $\times {10}^{-1}$
	std. dev.	1.42793 $\times {10}^{-1}$	9.60862 $\times {10}^{-2}$	3.23971 $\times {10}^{-2}$	1.66360 $\times {10}^{-1}$	2.33804 $\times {10}^{-1}$
	best	2.00044 $\times {10}^{-2}$	2.13355 $\times {10}^{-2}$	7.04566 $\times {10}^{-2}$	5.27357 $\times {10}^{-2}$	4.77002 $\times {10}^{-1}$
	worst	3.45483 $\times {10}^{-1}$	3.79904 $\times {10}^{-1}$	1.73480 $\times {10}^{-1}$	7.35598 $\times {10}^{-1}$	6.53800 $\times {10}^{-1}$

.

Based on the MSE results shown in Table A2, the FNNHTVS training algorithm displayed the best mean values for all cases except $H_n=30$. For $H_n=30$, the proposed training algorithm was ranked third.FNNHTVS had the smallest values for $H_n=6,9,10,12,20,30$. For other cases, it was most often ranked third. Its performance was competitive with that of the other compared algorithms in terms of its robust operation. In this problem, it was observed that the FNNVS algorithm exhibited the worst standard deviation value. In terms of the MSE values, FNNHTVS was ranked first in 7 of 11 FNN structures with $H_n=7,8,9,10,12,15,30$.

The pairwise comparisons are presented in

Table 6. WSR test results for the iris classification problem.

Method	Winner (FNNHTVS/Method 2)	Equal
FNNHTVS vs. FNNVS	9/1	1
FNNHTVS vs. FNNPSO	10/1	0
FNNHTVS vs. FNNPSOGSA	11/0	0
FNNHTVS vs. FNNGSA	11/0	0

. As a result of comparing FNNHTVS and FNNVS, FNNHTVS won in nine cases and lost in one case. The lost $H_n$ value was determined to be 30.The two compared algorithms were not able to outperform each other for $H_n=4$. In addition, the FNNHTVS algorithm lost to the FNNPSO algorithm for $H_n=30$.

3.3. Wine Recognition Problem

These data are the result of a chemical analysis of wines grown in the same region in Italy and produced from three different types of grapes [37]. Within the scope of the analysis, the amounts of 13 components found in wine types were recorded. Therefore, the dataset consists of 13 features. Wine types are divided into three classes according to these inputs. The dataset contains 178 samples. In the wine recognition dataset, there are 59 data samples for the first class, 71 for the second class and 48 for the third class. For the wine recognition problem, a 13-$H_n$-3 FNN structure is used. This structure has a total of $17H_n+3$ parameters, $16H_n$ weights and $H_n+3$ biases, and the initial parameter range is taken as [−50, 50]. The statistical results obtained after 30 independent runs are presented in

Table A3. Statistical results (MSE) in the wine recognition problem.

Hidden Nodes	Parameters	FNNHTVS	FNNVS	FNNPSO	FNNPSOGSA	FNNGSA
4	mean	1.94378 $\times {10}^{-2}$	1.33566 $\times {10}^{-1}$	1.69763 $\times {10}^{-1}$	1.65540 $\times {10}^{-1}$	4.61577 $\times {10}^{-1}$
	std. dev.	1.28846 $\times {10}^{-2}$	1.03747 $\times {10}^{-1}$	6.20304 $\times {10}^{-2}$	4.76383 $\times {10}^{-2}$	7.87211 $\times {10}^{-2}$
	best	2.73226 $\times {10}^{-5}$	1.67689 $\times {10}^{-2}$	8.40273 $\times {10}^{-2}$	8.71521 $\times {10}^{-2}$	2.62065 $\times {10}^{-1}$
	worst	5.78721 $\times {10}^{-2}$	4.12618 $\times {10}^{-1}$	3.45308 $\times {10}^{-1}$	2.74366 $\times {10}^{-1}$	5.76831 $\times {10}^{-1}$
5	mean	1.18868 $\times {10}^{-2}$	1.51479 $\times {10}^{-1}$	1.49253 $\times {10}^{-1}$	1.51786 $\times {10}^{-1}$	4.30638 $\times {10}^{-1}$
	std. dev.	6.50078 $\times {10}^{-3}$	1.13153 $\times {10}^{-1}$	3.75319 $\times {10}^{-2}$	4.80765 $\times {10}^{-2}$	4.73713 $\times {10}^{-2}$
	best	4.08510 $\times {10}^{-9}$	2.13987 $\times {10}^{-2}$	6.83132 $\times {10}^{-2}$	6.21954 $\times {10}^{-2}$	3.49284 $\times {10}^{-1}$
	worst	2.80562 $\times {10}^{-2}$	4.49282 $\times {10}^{-1}$	2.24855 $\times {10}^{-1}$	2.95900 $\times {10}^{-1}$	5.23850 $\times {10}^{-1}$
6	mean	1.46168 $\times {10}^{-2}$	1.58700 $\times {10}^{-1}$	1.30736 $\times {10}^{-1}$	1.36542 $\times {10}^{-1}$	4.10015 $\times {10}^{-1}$
	std. dev.	8.51222 $\times {10}^{-3}$	1.21009 $\times {10}^{-1}$	4.24100 $\times {10}^{-2}$	3.87619 $\times {10}^{-2}$	6.41373 $\times {10}^{-2}$
	best	1.58983 $\times {10}^{-8}$	8.33856 $\times {10}^{-3}$	4.28716 $\times {10}^{-2}$	6.82425 $\times {10}^{-2}$	2.59599 $\times {10}^{-1}$
	worst	2.80899 $\times {10}^{-2}$	4.71227 $\times {10}^{-1}$	2.29977 $\times {10}^{-1}$	2.34809 $\times {10}^{-1}$	5.50772 $\times {10}^{-1}$
7	mean	1.00025 $\times {10}^{-2}$	1.33992 $\times {10}^{-1}$	1.30839 $\times {10}^{-1}$	1.38147 $\times {10}^{-1}$	4.19058 $\times {10}^{-1}$
	std. dev.	7.39580 $\times {10}^{-3}$	1.12228 $\times {10}^{-1}$	4.53306 $\times {10}^{-2}$	4.37701 $\times {10}^{-2}$	5.85164 $\times {10}^{-2}$
	best	1.66979 $\times {10}^{-11}$	1.72366 $\times {10}^{-2}$	6.74613 $\times {10}^{-2}$	7.40539 $\times {10}^{-2}$	2.59164 $\times {10}^{-1}$
	worst	2.24719 $\times {10}^{-2}$	4.18049 $\times {10}^{-1}$	2.24881 $\times {10}^{-1}$	2.73788 $\times {10}^{-1}$	5.32682 $\times {10}^{-1}$
8	mean	4.20276 $\times {10}^{-3}$	1.96334 $\times {10}^{-1}$	1.22014 $\times {10}^{-1}$	1.40364 $\times {10}^{-1}$	4.11930 $\times {10}^{-1}$
	std. dev.	4.49867 $\times {10}^{-3}$	1.15859 $\times {10}^{-1}$	3.97087 $\times {10}^{-2}$	5.15103 $\times {10}^{-2}$	5.71548 $\times {10}^{-2}$
	best	4.04669 $\times {10}^{-14}$	6.40472 $\times {10}^{-2}$	7.04695 $\times {10}^{-2}$	7.81411 $\times {10}^{-2}$	3.08821 $\times {10}^{-1}$
	worst	1.21787 $\times {10}^{-2}$	4.66286 $\times {10}^{-1}$	2.49579 $\times {10}^{-1}$	2.96196 $\times {10}^{-1}$	5.38821 $\times {10}^{-1}$
9	mean	2.99296 $\times {10}^{-3}$	1.63649 $\times {10}^{-1}$	1.12534 $\times {10}^{-1}$	1.16186 $\times {10}^{-1}$	3.79386 $\times {10}^{-1}$
	std. dev.	4.25057 $\times {10}^{-3}$	1.20513 $\times {10}^{-1}$	3.08861 $\times {10}^{-2}$	3.59304 $\times {10}^{-2}$	6.44611 $\times {10}^{-2}$
	best	2.47483 $\times {10}^{-13}$	3.36361 $\times {10}^{-2}$	4.83561 $\times {10}^{-2}$	4.40372 $\times {10}^{-2}$	2.34178 $\times {10}^{-1}$
	worst	1.12372 $\times {10}^{-2}$	5.22962 $\times {10}^{-1}$	1.71889 $\times {10}^{-1}$	2.15566 $\times {10}^{-1}$	4.71524 $\times {10}^{-1}$
10	mean	2.25964 $\times {10}^{-3}$	1.83726 $\times {10}^{-1}$	1.03152 $\times {10}^{-1}$	1.26672 $\times {10}^{-1}$	3.92796 $\times {10}^{-1}$
	std. dev.	3.55391 $\times {10}^{-3}$	1.24834 $\times {10}^{-1}$	4.53593 $\times {10}^{-2}$	3.83574 $\times {10}^{-2}$	6.46792 $\times {10}^{-2}$
	best	4.29008 $\times {10}^{-16}$	4.16381 $\times {10}^{-2}$	5.09519 $\times {10}^{-2}$	6.44176 $\times {10}^{-2}$	2.62258 $\times {10}^{-1}$
	worst	1.12508 $\times {10}^{-2}$	4.76774 $\times {10}^{-1}$	2.06652 $\times {10}^{-1}$	2.32131 $\times {10}^{-1}$	5.06974 $\times {10}^{-1}$
12	mean	2.13262 $\times {10}^{-3}$	1.95910 $\times {10}^{-1}$	1.04961 $\times {10}^{-1}$	1.40683 $\times {10}^{-1}$	3.70757 $\times {10}^{-1}$
	std. dev.	3.30063 $\times {10}^{-3}$	1.16776 $\times {10}^{-1}$	2.62366 $\times {10}^{-2}$	5.58901 $\times {10}^{-2}$	5.98702 $\times {10}^{-2}$
	best	8.17277 $\times {10}^{-16}$	4.49510 $\times {10}^{-2}$	5.94720 $\times {10}^{-2}$	6.41966 $\times {10}^{-2}$	2.25114 $\times {10}^{-1}$
	worst	1.12401 $\times {10}^{-2}$	4.44748 $\times {10}^{-1}$	1.52097 $\times {10}^{-1}$	3.04571 $\times {10}^{-1}$	4.95184 $\times {10}^{-1}$
15	mean	1.87675 $\times {10}^{-3}$	1.63760 $\times {10}^{-1}$	8.06230 $\times {10}^{-2}$	1.52749 $\times {10}^{-1}$	3.61846 $\times {10}^{-1}$
	std. dev.	3.19606 $\times {10}^{-3}$	1.02461 $\times {10}^{-1}$	1.85006 $\times {10}^{-2}$	9.47948 $\times {10}^{-2}$	8.58804 $\times {10}^{-2}$
	best	1.48717 $\times {10}^{-16}$	4.51008 $\times {10}^{-2}$	4.98801 $\times {10}^{-2}$	7.40337 $\times {10}^{-2}$	2.03846 $\times {10}^{-1}$
	worst	1.12360 $\times {10}^{-2}$	4.49571 $\times {10}^{-1}$	1.18488 $\times {10}^{-1}$	4.52816 $\times {10}^{-1}$	5.57647 $\times {10}^{-1}$
20	mean	3.76476 $\times {10}^{-4}$	1.59726 $\times {10}^{-1}$	7.97941 $\times {10}^{-2}$	1.42076 $\times {10}^{-1}$	3.76138 $\times {10}^{-1}$
	std. dev.	1.42621 $\times {10}^{-3}$	9.16628 $\times {10}^{-2}$	4.62824 $\times {10}^{-2}$	6.59667 $\times {10}^{-2}$	1.11050 $\times {10}^{-1}$
	best	2.01689 $\times {10}^{-23}$	5.06107 $\times {10}^{-2}$	4.01168 $\times {10}^{-2}$	8.13262 $\times {10}^{-2}$	2.30746 $\times {10}^{-1}$
	worst	5.62821 $\times {10}^{-3}$	4.21890 $\times {10}^{-1}$	2.99984 $\times {10}^{-1}$	3.89202 $\times {10}^{-1}$	7.29064 $\times {10}^{-1}$
30	mean	7.44123 $\times {10}^{-22}$	1.91386 $\times {10}^{-1}$	6.83553 $\times {10}^{-2}$	2.25535 $\times {10}^{-1}$	3.70193 $\times {10}^{-1}$
	std. dev.	3.29131 $\times {10}^{-21}$	9.77874 $\times {10}^{-2}$	5.69610 $\times {10}^{-2}$	1.94427 $\times {10}^{-1}$	1.79224 $\times {10}^{-1}$
	best	1.48816 $\times {10}^{-56}$	4.49687 $\times {10}^{-2}$	2.95764 $\times {10}^{-2}$	6.16741 $\times {10}^{-2}$	1.71175 $\times {10}^{-1}$
	worst	1.75528 $\times {10}^{-20}$	4.15806 $\times {10}^{-1}$	3.58487 $\times {10}^{-1}$	7.66503 $\times {10}^{-1}$	9.70961 $\times {10}^{-1}$

.

For all $H_n$ values, the FNNHTVS training algorithm achieved the best statistical values and showed superior performance. The FNNVS training algorithm was also a follower of the proposed algorithm in terms of performance. The WSR test results presented in

Table 7. WSR test results for the wine recognition problem.

Method	Winner (FNNHTVS/Method 2)	Equal
FNNHTVS vs. FNNVS	11/0	0
FNNHTVS vs. FNNPSO	11/0	0
FNNHTVS vs. FNNPSOGSA	11/0	0
FNNHTVS vs. FNNGSA	11/0	0

support the claim that the proposed algorithm outperformed the other compared algorithms.

3.4. Seed Classification Problem

This dataset, which can be used in performance evaluations of classification and cluster analysis algorithms, includes the results of the classification of three different wheat seeds. The dataset consists of seven inputs and three classes [39]. The dataset contains 210 samples, 70 for each class. The first class is classified as Kama, the second class is Rosa and the third class is Canadian. For this problem, a 7-$H_n$-3 FNNstructure is used. This structure has a total of $11H_n+3$ parameters, $10H_n$ weights and $H_n+3$ biases, and the initial parameter range is taken as [−50, 50]. The statistical results obtained after 30 independent runs are demonstrated in

Table A4. Statistical results (MSE) in the seed classification problem.

Hidden Nodes	Parameters	FNNHTVS	FNNVS	FNNPSO	FNNPSOGSA	FNNGSA
4	mean	1.10847 $\times {10}^{-1}$	1.93308 $\times {10}^{-1}$	2.01770 $\times {10}^{-1}$	1.86016 $\times {10}^{-1}$	4.07302 $\times {10}^{-1}$
	std. dev.	1.60338 $\times {10}^{-2}$	1.18671 $\times {10}^{-1}$	4.48398 $\times {10}^{-2}$	3.99702 $\times {10}^{-2}$	3.94880 $\times {10}^{-2}$
	best	8.31405 $\times {10}^{-2}$	7.88864 $\times {10}^{-2}$	1.03638 $\times {10}^{-1}$	1.22263 $\times {10}^{-1}$	3.45558 $\times {10}^{-1}$
	worst	1.42066 $\times {10}^{-1}$	3.94932 $\times {10}^{-1}$	2.67137 $\times {10}^{-1}$	2.90377 $\times {10}^{-1}$	5.33628 $\times {10}^{-1}$
5	mean	1.01320 $\times {10}^{-1}$	2.07712 $\times {10}^{-1}$	1.81230 $\times {10}^{-1}$	1.73523 $\times {10}^{-1}$	4.09495 $\times {10}^{-1}$
	std. dev.	1.59207 $\times {10}^{-2}$	1.22636 $\times {10}^{-1}$	4.81278 $\times {10}^{-2}$	3.93328 $\times {10}^{-2}$	5.45169 $\times {10}^{-2}$
	best	7.20300 $\times {10}^{-2}$	5.82352 $\times {10}^{-2}$	1.14616 $\times {10}^{-1}$	1.09551 $\times {10}^{-1}$	3.07478 $\times {10}^{-1}$
	worst	1.49775 $\times {10}^{-1}$	3.99913 $\times {10}^{-1}$	3.01366 $\times {10}^{-1}$	2.72784 $\times {10}^{-1}$	5.48163 $\times {10}^{-1}$
6	mean	9.36586 $\times {10}^{-2}$	1.66201 $\times {10}^{-1}$	1.65856 $\times {10}^{-1}$	1.71439 $\times {10}^{-1}$	3.94956 $\times {10}^{-1}$
	std. dev.	1.08448 $\times {10}^{-2}$	1.01809 $\times {10}^{-1}$	3.00229 $\times {10}^{-2}$	5.10343 $\times {10}^{-2}$	4.75287 $\times {10}^{-2}$
	best	7.62807 $\times {10}^{-2}$	5.50233 $\times {10}^{-2}$	8.90293 $\times {10}^{-2}$	1.23650 $\times {10}^{-1}$	3.08159 $\times {10}^{-1}$
	worst	1.16848 $\times {10}^{-1}$	4.01519 $\times {10}^{-1}$	2.18201 $\times {10}^{-1}$	4.04750 $\times {10}^{-1}$	5.14334 $\times {10}^{-1}$
7	mean	8.91858 $\times {10}^{-2}$	1.67699 $\times {10}^{-1}$	1.55629 $\times {10}^{-1}$	1.67584 $\times {10}^{-1}$	3.79103 $\times {10}^{-1}$
	std. dev.	1.24263 $\times {10}^{-2}$	1.05127 $\times {10}^{-1}$	2.79420 $\times {10}^{-2}$	4.43436 $\times {10}^{-2}$	5.27818 $\times {10}^{-2}$
	best	6.09093 $\times {10}^{-2}$	5.65455 $\times {10}^{-2}$	9.05990 $\times {10}^{-2}$	1.14863 $\times {10}^{-1}$	2.94214 $\times {10}^{-1}$
	worst	1.22892 $\times {10}^{-1}$	3.96726 $\times {10}^{-1}$	1.96405 $\times {10}^{-1}$	3.50567 $\times {10}^{-1}$	5.37897 $\times {10}^{-1}$
8	mean	9.18808 $\times {10}^{-2}$	1.78536 $\times {10}^{-1}$	1.53213 $\times {10}^{-1}$	1.51797 $\times {10}^{-1}$	3.70108 $\times {10}^{-1}$
	std. dev.	1.26865 $\times {10}^{-2}$	1.14783 $\times {10}^{-1}$	2.21297 $\times {10}^{-2}$	2.61319 $\times {10}^{-2}$	4.10571 $\times {10}^{-2}$
	best	6.11826 $\times {10}^{-2}$	5.72383 $\times {10}^{-2}$	1.03633 $\times {10}^{-1}$	1.13652 $\times {10}^{-1}$	2.90596 $\times {10}^{-1}$
	worst	1.09795 $\times {10}^{-1}$	3.95296 $\times {10}^{-1}$	2.01519 $\times {10}^{-1}$	2.40753 $\times {10}^{-1}$	4.33540 $\times {10}^{-1}$
9	mean	8.58805 $\times {10}^{-2}$	1.79317 $\times {10}^{-1}$	1.56830 $\times {10}^{-1}$	1.60936 $\times {10}^{-1}$	3.53928 $\times {10}^{-1}$
	std. dev.	6.93764 $\times {10}^{-3}$	1.05511 $\times {10}^{-1}$	3.33739 $\times {10}^{-2}$	2.82649 $\times {10}^{-2}$	6.73966 $\times {10}^{-2}$
	best	7.43803 $\times {10}^{-2}$	7.50251 $\times {10}^{-2}$	1.02041 $\times {10}^{-1}$	1.14364 $\times {10}^{-1}$	1.70935 $\times {10}^{-1}$
	worst	1.00271 $\times {10}^{-1}$	4.48567 $\times {10}^{-1}$	2.21748 $\times {10}^{-1}$	2.38765 $\times {10}^{-1}$	4.84104 $\times {10}^{-1}$
10	mean	8.58320 $\times {10}^{-2}$	1.39362 $\times {10}^{-1}$	1.49952 $\times {10}^{-1}$	1.67026 $\times {10}^{-1}$	3.71755 $\times {10}^{-1}$
	std. dev.	1.13822 $\times {10}^{-2}$	6.70709 $\times {10}^{-2}$	2.61584 $\times {10}^{-2}$	4.86881 $\times {10}^{-2}$	6.20682 $\times {10}^{-2}$
	best	5.55225 $\times {10}^{-2}$	6.66799 $\times {10}^{-2}$	1.02490 $\times {10}^{-1}$	1.20878 $\times {10}^{-1}$	2.68974 $\times {10}^{-1}$
	worst	1.07147 $\times {10}^{-1}$	3.86976 $\times {10}^{-1}$	2.28392 $\times {10}^{-1}$	3.69949 $\times {10}^{-1}$	5.88624 $\times {10}^{-1}$
12	mean	7.69695 $\times {10}^{-2}$	1.72056 $\times {10}^{-1}$	1.31553 $\times {10}^{-1}$	1.77467 $\times {10}^{-1}$	3.52560 $\times {10}^{-1}$
	std. dev.	8.74892 $\times {10}^{-3}$	8.54826 $\times {10}^{-2}$	2.64553 $\times {10}^{-2}$	9.03774 $\times {10}^{-2}$	7.04322 $\times {10}^{-2}$
	best	6.23604 $\times {10}^{-2}$	7.98103 $\times {10}^{-2}$	9.64765 $\times {10}^{-2}$	1.03684 $\times {10}^{-1}$	2.45908 $\times {10}^{-1}$
	worst	9.13157 $\times {10}^{-2}$	3.90539 $\times {10}^{-1}$	2.11989 $\times {10}^{-1}$	4.09575 $\times {10}^{-1}$	5.68246 $\times {10}^{-1}$
15	mean	7.42578 $\times {10}^{-2}$	1.70866 $\times {10}^{-1}$	1.25953 $\times {10}^{-1}$	1.48903 $\times {10}^{-1}$	3.71133 $\times {10}^{-1}$
	std. dev.	8.84990 $\times {10}^{-3}$	9.90537 $\times {10}^{-2}$	2.27677 $\times {10}^{-2}$	5.71889 $\times {10}^{-2}$	8.20515 $\times {10}^{-2}$
	best	6.19053 $\times {10}^{-2}$	8.61301 $\times {10}^{-2}$	8.98466 $\times {10}^{-2}$	9.61550 $\times {10}^{-2}$	2.50839 $\times {10}^{-1}$
	worst	1.00014 $\times {10}^{-1}$	4.08016 $\times {10}^{-1}$	1.67739 $\times {10}^{-1}$	4.04558 $\times {10}^{-1}$	5.57981 $\times {10}^{-1}$
20	mean	7.64729 $\times {10}^{-2}$	1.66421 $\times {10}^{-1}$	1.16223 $\times {10}^{-1}$	2.21587 $\times {10}^{-1}$	4.08647 $\times {10}^{-1}$
	std. dev.	8.87865 $\times {10}^{-3}$	9.34156 $\times {10}^{-2}$	1.98504 $\times {10}^{-2}$	1.70788 $\times {10}^{-1}$	1.91840 $\times {10}^{-1}$
	best	5.76150 $\times {10}^{-2}$	6.33511 $\times {10}^{-2}$	7.22399 $\times {10}^{-2}$	1.01568 $\times {10}^{-1}$	2.04698 $\times {10}^{-1}$
	worst	9.27819 $\times {10}^{-2}$	4.85761 $\times {10}^{-1}$	1.83814 $\times {10}^{-1}$	7.32398 $\times {10}^{-1}$	9.24442 $\times {10}^{-1}$
30	mean	7.45181 $\times {10}^{-2}$	1.99154 $\times {10}^{-1}$	1.20852 $\times {10}^{-1}$	3.32331 $\times {10}^{-1}$	8.27154 $\times {10}^{-1}$
	std. dev.	8.67849 $\times {10}^{-3}$	1.01211 $\times {10}^{-1}$	5.55258 $\times {10}^{-2}$	2.83933 $\times {10}^{-1}$	2.21347 $\times {10}^{-1}$
	best	5.78553 $\times {10}^{-2}$	9.16919 $\times {10}^{-2}$	7.69777 $\times {10}^{-2}$	8.98902 $\times {10}^{-2}$	4.31377 $\times {10}^{-1}$
	worst	9.52382 $\times {10}^{-2}$	3.80953 $\times {10}^{-1}$	3.95592 $\times {10}^{-1}$	8.36375 $\times {10}^{-1}$	8.74650 $\times {10}^{-1}$

.

In terms of all statistical parameters shown in Table A4, the FNNHTVS training algorithm outperformed the other algorithms. In the WSR test, it outperformed all the compared algorithms. The WSR test results are presented in

Table 8. WSR test results for the seed classification problem.

Method	Winner (FNNHTVS/Method 2)	Equal
FNNHTVS vs. FNNVS	11/0	0
FNNHTVS vs. FNNPSO	11/0	0
FNNHTVS vs. FNNPSOGSA	11/0	0
FNNHTVS vs. FNNGSA	11/0	0

.

4. Performance Evaluation in Fault Classification

Short circuit fault classification is one of the important issues that are studied in order to more accurately intervene in response to faults occurring in transmission lines. Furthermore, some fault location algorithms need to know the fault class. This situation increases the importance of fault classification. For fault classification, various classification properties are obtained at first. Then, using these features and different artificial intelligence techniques, fault types are classified.

In this section of our study, short-circuit faults occurring on a 735 kV, 60 Hz, 100 km transmission line was modeled as frequency-dependent with the help of Matlab/Simulink. Classification data were produced by introducing short circuit faults into the model, which is shown in Figure 4. Classification was carried out with the FNNHTVS algorithm, the validity of which has been shown in the previous section. The performance of the FNNHTVS algorithm in fault classification was compared not only with the FNNVS, FNNPSO, FNNPSOGSA and FNNGSA algorithms, but also with other classifiers (SVM, KNN, FNN with LM and NB).

The selected classification features need to be specific and consistent for each fault type. In this study, post-fault one-cycle line currents and the zero sequence component of the line currents were taken as the input data. In each fault condition, the three-phase currents and the zero component were reduced by means of a certain method. In this reduction method, the highest peak value of the three phase currents was found in any fault, then each line current and zero component were divided by this peak value and the signals were scaled. The transmission line model studied here was a frequency-dependent model. Three-phase current signals and the zero sequence component for one cycle post-fault were sampled with a sampling frequency of 20 kHz and recorded. Measurements were made from the sending side of the transmission line. The root mean square (RMS) values of these recorded signals were calculated. The dataset was created using different fault resistance values, fault locations, fault types and fault inception angles. Single line to ground (SLG), line to line (LL), line to line to ground (LLG) and three-phase symmetric ground (LLLG) faults were generated in each phase.A random fault resistance value was chosen between 0.1 and 150 ohms. The fault inception angles (FIA) were determined as 0, 30, 45, 90 150 or 270. The fault location was chosen as 10, 20, 30, 50, 60, 80 or 90 km. A total of 250 data were created, 175 of which were training data and 75 were test data. The proposed algorithm and all other algorithms for comparison were run 30 different times. In each independent run, training and test samples were randomly selected from the created dataset.

Based on the formula $H_n=2I_n+1$ presented in [26,31], $H_n=9$ was used. $I_n$ is the input number. For the fault classification problem, an 4-9-4 FNN structure is used. This structure has a total of 85 parameters, 72 weights and 13 biases, and the parameter range is taken as [−50, 50]. The maximum iteration number was equal to 100 for all algorithms, which were evaluated based on the mean, standard deviation and best and worst MSE and accuracy values. The statistical results obtained after 30 independent runs are shown in

Table 9. Statistical fault classification results for $H_n=9$.

Algorithms			MSE					Accuracy (%)
	Mean	Median	Std. Dev.	Best	Worst	Mean	Median	Std. Dev.	Best	Worst
FNNHTVS	0.00944	0.01136	0.00590	6.89125 $\times {10}^{-32}$	0.01714	99.1111	98.6666	0.99380	100	96
FNNVS	0.01489	0.01413	0.01055	1.40023 $\times {10}^{-17}$	0.04374	98.3555	98.6666	1.31918	100	94.6666
FNNPSO	0.04263	0.03521	0.02908	0.00227	0.12496	97.9555	98.6666	2.49997	100	88
FNNPSOGSA	0.05369	0.04075	0.03241	0.01066	0.12397	97.0666	98.6666	3.21846	100	85.3333
FNNGSA	0.23634	0.24117	0.03002	0.16531	0.28993	75.0285	74.8571	10.6037	94.8571	49.7142

. When Table 9 is examined, it can be observed that the FNNHTVS algorithm had a lower mean MSE and higher mean classification accuracy, compared to the other methods. A box plot graph is shown in Figure 5 and a convergence curve is depicted in Figure 6. It can be observed that the FNNHTVS algorithm had a low standard deviation and reached a lower mean MSE value in fewer iterations. The convergence curve was obtained by taking the average of 30 different runs. The FNNHTVS algorithm was compared with the methods of SVM, KNN, FNN with LM and NB. When the results shown in

Table 10. Statistical results showing the accuracy of FNNHTVS and other classifiers.

Algorithms	Mean	Median	Std. Dev.	Best	Worst
FNN (LM)	97.9111	98.6666	5.24012	100	70.6666
KNN	98.4762	98.6666	1.25486	100	94.6666
SVM	98.8571	98.6666	1.02151	100	97.3333
Naive Bayes	98.8804	98.6666	0.88012	100	97.3333
FNNHTVS	99.1111	98.6666	0.99380	100	96

are examined, it can be seen that the proposed algorithm exhibited a very competitive structure in relation to the other classifiers.

The accuracy values obtained in the fault classification studies may vary depending on the sample number, data type and the transmission line model studied. Therefore, it would be more accurate to compare the FNNHTVS algorithm with the classifiers and algorithms used in this study. The comparison of the FNNHTVS training algorithm with the studies related to fault classification in the literature could create a misleading impression due to the differences in the datasets studied. Considering this situation, some studies in the literature are presented in

Table 11. Comparative assessment.

	Malathi, V. et al., 2010 [40]	Ekici, S., 2012 [41]	Mukherjee, A. et al., 2022 [42]	FNNHTVS
Line length (km)	225	360	150	100
Frequency (Hz)	50	-	-	60
Voltage (kV)	240	380	270	735
Method	DW-SVM	MCSVM	PbC	FNNHTVS
Fault resistance (ohm)	1–200	10–1000	0–150	0–150
FIA	36–126${}^{\circ }$	-	-	0–270${}^{\circ }$
Predicted class	4	4	4	4
Class. accuracy (%)	99.11	99	98.143	99.1111

, along with their important features. In [40], the discrete wavelet (DW)-based SVM method was used. The average accuracy rate was approximately the same as for FNNHTVS. In [41], fault classification and fault location tasks were undertaken using the multiclass SVM (MCSVM) method. In [42], it was observed that the classification accuracy decreased as the fault resistance increased. In a study using the Poincare-based correlation (PbC) method, the authors stated that higher classification rates were obtained for fault resistances up to 100 and 120 ohms. Based on the results shown in Table 11, we concluded that the FNNHTVS algorithm, with a mean accuracy rate of 99.1111%, obtained successful results that are compatible with those presented in the literature.

5. Conclusions

Many heuristic optimization algorithms have been used in the training of ANNs to determine the optimal values of weights and biases due to factors such as the non-linearity of problem types and their very large dimensions. In this study, the usability of the HTVS algorithm, which has not been used for this purpose in the literature before, was examined in the training of FNNs. The HTVS algorithm was used to train FNNs and its performance was analyzed. It was compared with other methods on test problems and a short circuit fault classification problem in a transmission line. In order to compare the training performance of the algorithms, all samples of the datasets in Section 3 were used as training data and the MSE of training error was calculated. These problems were used to demonstrate the validity of the FNNHTVS algorithm. All algorithms were run 30 different times for each problem. The FNN training process was stopped when the maximum number of iterations was reached. For each optimization algorithm, the maximum number of iterations was 100, the population size was 30, and the initial candidate solution interval was [−50, 50]. As shown in Section 4, 70% of the data were used as a training set and the remaining 30% were used as a test set in the fault classification problem. The performance of the FNNHTVS algorithm was also compared with that of the SVM, KNN, FNN with LM and NB classifiers in the task of fault classification. Performance evaluations of the algorithms were undertaken by listing the results obtained from all stages separately. Based on the obtained results, we concluded that the HTVS algorithm is a viable approach in the training of FNNs for classification purposes.

The following ideas can be explored in future works:

• It may be interesting to detect fault locations using FNNHTVS;
• The HTVS algorithm could be used to train other types of ANNs; and
• The optimal structure of FNNs could be determined using HTVS, including the number of nodes and the number of hidden layers.