Modeling of Fuzzy Systems Based on the Competitive Neural Network

Abstract

This paper presents a method to dynamically model Type-1 fuzzy inference systems using a Competitive Neural Network. The aim is to exploit the potential of Competitive Neural Networks and fuzzy logic systems to generate an intelligent hybrid model with the ability to group and classify any dataset. The approach uses the Competitive Neural Network to cluster the dataset and the fuzzy model to perform the classification. It is important to note that the fuzzy inference system is generated automatically from the classes and centroids obtained with the Competitive Neural Network, namely, all the parameters of the membership functions are adapted according to the values of the input data. In the approach, two fuzzy inference systems, Sugeno and Mamdani, are proposed. Additionally, variations of these models are presented using three types of membership functions, including Trapezoidal, Triangular, and Gaussian functions. The proposed models are applied to three classification datasets: Wine, Iris, and Wisconsin Breast Cancer (WDBC). The simulations and results present higher classification accuracy when implementing the Sugeno fuzzy inference system compared to the Mamdani system, and in both models (Mamdani and Sugeno), better results are obtained when the Gaussian membership function is used.

1. Introduction

Computer sciences have developed rapidly and researchers around the world in the field on artificial intelligence [1] have applied multiple algorithms for optimization, clustering, and classification problems. One of the challenges is to extract meaningful patterns in large datasets. This is where clustering and classification, two basic concepts in machine learning, play a transformative role. As datasets grow in size and complexity, the demands on algorithms to adapt and scale also increase.

The most famous optimization algorithms are based on swarm optimization, such as Particle Swarm Optimization (PSO) [2], Fireworks Algorithm (FWA) [3], Gravitational Search Algorithm (GSA) [4], and Grey Wolf Optimizer (GWO) [5], etc. In the field of clustering, we can find the Fuzzy C-Means [6] and K-Means clustering [7] algorithms. For classification tasks, we can explore algorithms such as K-Nearest Neighbors [8] or robust artificial neural networks [9].

Artificial neural networks can be divided into two classes depending on the type of learning they use: neural networks with supervised learning and neural networks with unsupervised learning [10]. It is important to highlight this aspect because the challenge that we are going to attack in the proposed method involves the utilization of an artificial neural network with unsupervised learning, specifically, with a Competitive Neural Network (CNN), which is a technique in the area of clustering [11,12] and not in the classification area.

On the other hand, according to the state of the art method, a plethora of researchers have used Fuzzy Inference Systems, FISs, to solve different types of problems, for example, by dynamically adjusting parameters [13] in optimization algorithms, in control problems [14], as well as prediction [15] and classification tasks. These latter problems have been solved with better accuracy, especially when working with noisy datasets, thanks to the ability of fuzzy inference systems to model the uncertainty present in the information [16,17].

All methods have strengths and weaknesses; therefore, part of the main the contribution of this paper is to generate an intelligent hybrid method using the potential of the CNN to cluster and aggregate the classification task by implementing Fuzzy Inference Systems. In the approach, first the CNN is performed to obtain the clusters and centroids of the input data, after that the Type-1 FIS is automatically designed according to the output produced by the CNN. Thus, another contribution of this paper is the methodology to dynamically generate the parameters of the input and output membership functions MFs of the FIS and these can be adapted according to the dataset classification problem used. In the approach, two FIS models were considered using Sugeno and Mamdani. Furthermore, these models are generated using three types of MFs (Trapezoidal, Gaussian, and Triangular) to have more variants in the models and analyze the behavior and precision obtained. The general approach is applied to three datasets (Wine, Iris, and WDBC) to measure the performance and accuracy of these algorithms in classification tasks.

This paper is organized as follows. Section 2 explains the theoretical concepts, techniques, and components that were used to develop this work. Section 3 presents the general proposal, describing the methodology to dynamically generate the Mamdani Type-1 FIS and Sugeno Type-1 FIS with different types of MFs. Section 4 explains the experimental results obtained with the proposed methods after being applied to the three databases. Finally, Section 5 offers the conclusions based on the results and some future works.

2. Theoretical Framework

This section presents the theory of the models used to develop this work. First, important concepts of competitive neural networks are explained. Second, a summary of the theory of fuzzy sets is presented, ending with the discussion subsection.

2.1. Competitive Neural Network

Competitive Neural Networks deviate from traditional feedforward neural networks by introducing a competitive layer. This layer is often referred to as the competitive or Kohonen layer which facilitates competition among neurons. Neurons compete to become the most responsive to specific input patterns, fostering a dynamic environment where only the most relevant neurons survive and adapt [18].

Competitive Neural Networks work in unsupervised learning scenarios, where the network must identify patterns and structure within the data without explicit labels. The competitive layer fosters neuron specialization, with each neuron becoming an expert in recognizing a specific pattern or feature. Through continuous competition and adaptation, the network refines its internal representation, making it highly adept at capturing intricate relationships within the input data.

In [19] the authors describe three fundamental components of competitive learning rules:

• Activation of Neuron Set: Neurons (process units) exhibit activation or inactivity in response to specific input patterns. Distinct synaptic weight values are assigned to each neuron, contributing to individualized responsiveness.
• Limitation on Neuron Strength: A constraint is applied to the “strength” of each neuron, regulating its responsiveness within the network.
• Competitive Mechanism: Neurons compete to respond to subsets of input patterns. The outcome is designed such that only one neuron from the group is activated, promoting specialization.

Moreover, within a CNN, a binary process unit serves as a simple computational entity capable of assuming only two states: active (on) or inactive (off). For each binary process unit i, there exists an associated synaptic weights vector ($w_{i1},w_{i2},\dots w_{iN}$) to weigh the incoming values from the input vectors.

The synaptic potential is defined as follows: if $N$ signals, represented by the vector $(x_1,x_2,\dots ,x_N$), reach the processing unit i, and the synaptic weights vector of each unit is $(w_{i1},w_{i2},\dots ,w_{iN})$, then the synaptic potential is computed using Equations (1) and (2).

$$h_i=w_{i1}{x}_1+w_{i1}{x}_2+\_\dots +w_{iN}{x}_N-{\theta }_i$$

(1)

where:

$${\theta }_i=\frac{1}{2}(w_{i1}^2+w_{i2}^2+...+w_{iN}^2)$$

(2)

Defined by N input sensors, M process units, and interconnecting links between each sensor and process unit, a CNN is characterized by the association of specific values, denoted as $w_{ij}$, with each connection linking sensor $j$ to process unit $i$. Figure 1 provides a visual representation of a Competitive Neural Network (CNN).

With each input collected by the sensors, the activation of only one process unit occurs, specifically the unit with the highest synaptic potential, which is acknowledged as the winning unit.

Consequently, when denoting the state of process unit $i$ with the variable $y_i$, it adopts a value of 1 during activation and 0 otherwise. The computational dynamics of the network are obtained by Equation (3).

$$y_i=\left\{\begin{array}{c}1 if { h}_i=\underset{k}{\mathit{max}}\left\{h_1, h_{2,\dots ,}{h}_M\right\} \\ 0 otherwise\end{array}\right.$$

(3)

$$i=1, 2, 3,.., M$$

The synaptic weights are established through an unsupervised learning process. The objective is to activate the process unit whose synaptic weight vector is most similar to the input vector. The degree of similarity between the input vector $X=(x_1,x_2,\dots ,x_N)$ and the synaptic weights vector of process unit $i$, $W=(w_{i1},w_{i2},\dots ,w_{iN}$), is determined by the Euclidean distance between these vectors. This distance is given by Equation (4).

$$d\left(x,w_i\right)=‖x-w_i‖=\sqrt{{\left(x_1-w_{i1}\right)}^2+...+{\left(x_N-w_{iN}\right)}^2}$$

(4)

Hence, if $r$ represents the winning process unit, after introduction of the input pattern, the validation that the winning unit possesses the synaptic weight vector most similar to the input vector is expressed through Equation (5).

$$d\left(x,w_r\right)\le d\left(x,w_k\right), k=1, 2,\dots ,M$$

(5)

The aim of the unsupervised learning in the CNN is to autonomously identify groups or classes. To achieve this, the criterion of the minimum squares is calculated using Equation (6).

$$E\left(k\right)={\sum }_{i=1}^M{a}_{ik}{‖\mathit{x}\left(k\right)-{\mathit{w}}_i\left(k\right)‖}^2$$

(6)

In (6), $E$ denotes the error in iteration k, derived from the distance between the input pattern $x\left(k\right)$ belonging to class M and the synaptic weight vector $x_i\left(k\right)$. This is identified as the learning rule, which is denoted in Equation (7).

$${\mathit{w}}_r\left(k+1\right)={\mathit{w}}_r\left(k\right)+\eta \left(k\right)\left[\mathit{x}\left(k\right)-{\mathit{w}}_r\left(k\right)\right]$$

(7)

In this scenario, the new synaptic weights vector $w_r(k+1)$ is a linear combination of the vectors $w_r\left(k\right)$ and $x\left(k\right)$, with $\eta \left(k\right)$ representing the learning rate, as expressed in Equation (8), where $k$ is the current iteration and T is the total number of the iterations; therefore, the initial value of ${\eta }_0$ could be (0, 1). The learning rate will be decrementing; if $\eta$ reaches 0, the CNN will stop learning.

$$\eta \left(k\right)={\eta }_0\left(1-\frac{k}{T}\right)$$

(8)

In the state of the art method, we can find a large number of research works, exploiting the advantages of Competitive Neural Networks or Competitive Learning and applying them in different areas. In [9], the author provides a comprehensive overview of competitive learning-based clustering methods. In [11], a framework for different clustering methods like Fuzzy C-Means and K-Means clustering, entropy constrained vector quantization, or topological feature maps and CNN is proposed. In [20], the authors explore the exponential synchronization problem for a class of the CNN.

In other applications, the CompNet model based on Competitive Neural Networks and Learnable Gabor Kernels is presented in [21], the model is implemented for Palmprint Recognition; according to the results, the proposal achieved the lowest error rate compared to the most commonly state of the art methods. In [22], a Competitive Neural Network is used to estimate a rice-plant area; the authors demonstrate that these kinds of models are useful for the classification of the satellite data. In hybrid methods, in [23], the CNN is optimized and is applied to solve a complex problem in the field of chemical engineering; the authors proposed a novel neural network optimizer that leverages the advantages of both an improved evolutionary competitive algorithm and gradient-based backpropagation. In [24], the Fireworks Algorithm (FWA) was implemented to optimize the neurons of the CNN and improve the results of the traditional model. In order to improve the CNN, a novel classification algorithm that is based on the integration between competitive learning and the computational power of quantum computing is presented in [25]. These and other researches [26] demonstrate the potential of this area to solve a variety of problems.

2.2. Fuzzy Sets

A fuzzy set is characterized by a membership function that assigns a degree of membership to each element of the universal set. The membership function represents the uncertainty or vagueness associated with the degree to which an element belongs to a particular set [16,17].

The most popular Type-1 FISs are Mamdani [27] and Takagi–Sugeno–Kang [28]. The Mamdani FIS [27] is a model based on fuzzy logic principles that facilitates the representation and processing of uncertain or imprecise information. It has been successfully applied in several domains, due to its ability to model complex systems and handle linguistic variables in a way that humans can interpret. In this system, antecedents and consequents are represented by fuzzy membership functions.

The Takagi–Sugeno–Kang FIS, better known as Sugeno FIS, is a type of fuzzy logic system that uses linguistic rules and fuzzy logic principles to model and infer relationships between input and output variables. Introduced by Takagi and Sugeno [28], the Sugeno FIS represents rules in an IF-THEN format, where the antecedents are formed using fuzzy sets and the consequents are linear functions. This model is widely applied in areas such as control systems, decision making, and pattern recognition due to its transparency and ease of interpretation.

Researchers around the world have incorporated the fuzzy sets theory to solve classification challenges, more specifically, to classify features with a high degree of similarity or uncertainty for different datasets. In [6], the authors proposed a fuzzy clustering method using Interval Type-2 FIS combined with Possibilistic C-Means (PCM) and Fuzzy C-Means (FCM) clustering algorithms. In [29], optimization with swarm algorithms is applied to enhance an Interval Type-2 Fuzzy FIS for data classification. The proposal was tested on datasets from the UCI machine learning library and satellite image data. The results demonstrate that utilizing optimization algorithms can significantly improve accuracy in solving data classification problems. In [30], the authors introduce an unsupervised outlier detection algorithm which implements K-NN and fuzzy logic. Recognizing the limitations of unsupervised approaches in handling complex datasets, the authors provide a solution to this problem by proposing the utilization of K-NN rule and fuzzy logic for effective outlier detection. In [31], a hybrid method based on fuzzy logic and Genetic Algorithm applied to the text classification Twitter is proposed. In [32], the approach combines Fuzzy C-Means clustering and a fuzzy inference system for an audiovisual quality of experience. Experimental analyses indicate that the proposed framework outperforms methods without optimization and existing models, specifically MLP-based techniques.

2.3. Discussion

For this research, the methodologies of Competitive Neural Networks and Fuzzy Inference Systems were selected due to the great results they have produced in research carried out by scientists around the world, specifically, in research focused on data clustering and classification problems. On the one hand, unsupervised approaches have limitations under complex datasets; therefore, to solve this problem, we propose the use of fuzzy logic to generate a robust model. Furthermore, one of the limitations of fuzzy logic is how to obtain the parameters (type of inference system, type of input and output MF, the parameters of the MF, how the MFs are granulated, and generate the fuzzy rules). In this sense, the CNN helps to obtain the MF parameters from the data, how these are granulated, and generate fuzzy rules based on the input data.

To model the FIS, we selected the Mamdani and Sugeno systems; these have yielded good results in classification problems as we have previously mentioned. The choice of Triangular, Gaussian, and Trapezoidal MFs is based on the experts in the area according to previous works [24,33], where the FIS is modeled with the aforementioned MFs and after several tests, we can obtain the best accuracy results. Finally, it is important to mention that these three types of membership functions are designed with mathematical and metric functions according to their form. The detailed explanation and the obtained results will be presented in Section 3.

3. Proposed Method

This section presents the proposed methodology for dynamically designing fuzzy systems using Competitive Neural Networks; the model will be identified in the rest of the paper as CNNT1FL. For more details, the general structure is illustrated in Figure 2.

This approach is a combination between a CNN with Mamdani Type-1 FIS and Sugeno Type-1 FIS. The aim is to propose models that, based on input data from any dataset, are capable of dynamically generating a Type-1 FIS and that these can be applied to any problem adapting to research needs, namely, it could be useful for classification or prediction.

It is important to mention that for this investigation, we are going to consider the classification line. In the state of the art method, we found three investigations with some similarities, but only one with quantitative results [6] and two with qualitative results [6,33].

As we can see in Figure 2, the model, in addition to the input dataset, comprises two parts: (1) the implementation of the CNN and (2) the design of the FIS. The next two subsections explain in detail the implementation of each part to generate the general model.

3.1. Competitive Neural Network

As highlighted in the state of the art method, a CNN employs unsupervised learning, namely, the neural network learns from input data without prior knowledge of the specific targets [9,19,20].

In the proposed method (CNNT1FL), the main objective of the CNN is to generate clusters derived from the input data. In this context, the targets of the CNN represent the centroids of these clusters.

In the CNN functionality, the targets will be considered as centroids. Equation (9) illustrates the process of cluster formation; for each data point $x_i$, where $i=1, 2, 3,..N$, the distance to each centroid (targets) is calculated.

$$dist=\Vert x_i-C_n\Vert$$

(9)

where $n=1, 2,\dots ,M; N$ denotes the total input data, and $\mathrm{M}$ represents the total number of neurons in the CNN. The minimum distance among the data ${\mathrm{x}}_{\mathrm{i}}$ and the centroids signifies that the data belong to the centroid $C_n$.

After the clusters are established with their corresponding data, the subsequent step involves designing the Type-1 FIS, as detailed in the following subsection.

3.2. Design Process of the Fuzzy Inference Systems

This section provides a detailed explanation of the Type-1 FIS design process, using the clusters generated by a CNN.

The construction of the Type-1 FIS including the antecedents, consequents, and fuzzy rules depends on all the input data and the features number. Furthermore, the MF number for each input variable will be influenced by the number of clusters generated by the CNN.

In this approach, the design of the FIS is proposed using three different MFs: Trapezoidal, Gaussian, and Triangular.

3.2.1. Triangular Membership Functions

Creating a Triangular MF requires three points denoted as $a, b,$ and $c$, with the restriction that $a<b<c$, as expressed in Equation (10) and illustrated in Figure 3.

$$Triangle\left(x;a,b,c\right)=\left\{\begin{array}{c}0, x\le a.\\ \frac{x-a}{b-a}, a\le x\le b.\\ \frac{c-x}{c-b}, b\le x\le c.\\ 0, c\le x.\end{array}\right.$$

(10)

In this approach, the MF number is based on the number of clusters.

Table 1. Points for a Triangular membership function.

$\mathit{M}\mathit{e}\mathit{m}\mathit{b}\mathit{e}\mathit{r}\mathit{s}\mathit{h}\mathit{i}\mathit{p}\mathit{F}\mathit{u}\mathit{n}\mathit{c}\mathit{t}\mathit{i}\mathit{o}{\mathit{n}}_1$	$\mathit{M}\mathit{e}\mathit{m}\mathit{b}\mathit{e}\mathit{r}\mathit{s}\mathit{h}\mathit{i}\mathit{p}\mathit{F}\mathit{u}\mathit{n}\mathit{c}\mathit{t}\mathit{i}\mathit{o}{\mathit{n}}_2$…	$\mathit{M}\mathit{e}\mathit{m}\mathit{b}\mathit{e}\mathit{r}\mathit{s}\mathit{h}\mathit{i}\mathit{p}\mathit{F}\mathit{u}\mathit{n}\mathit{c}\mathit{t}\mathit{i}\mathit{o}{\mathit{n}}_{\mathit{n}}$
$a_1=Centroid{e}_1-2\ast STD\left(Cluste{r}_1\right)$	$a_2=Centroid{e}_2-2\ast STD\left(Cluste{r}_2\right)$…	$a_n=Centroid{e}_1-2\ast STD\left(Cluste{r}_n\right)$
$b_1=Centroid{e}_1$	$b_2=Centroid{e}_2$…	$b_n=Centroid{e}_n$
$c_1=Centroid{e}_1+2\ast STD\left(Cluste{r}_1\right)$	$c_2=Centroid{e}_2+2\ast STD\left(Cluste{r}_2\right)$…	$c_n=Centroid{e}_1+2\ast STD\left(Cluste{r}_n\right)$

details how to calculate the parameters to design a Triangular MF from Triangular ${MF}_1$ to Triangular ${MF}_n$. Specifically, the following steps are implemented:

• To obtain the parameter $a_n$, two standard deviations (STDs) of the $cluste{r}_n$ are calculated from the position of the $Centroi{d}_n$.
• The parameter $b_n$ is the position of the centroid of $cluste{r}_n$.
• To calculate the parameter $c_n$, two STDs of the $cluste{r}_n$ are also added from the position of the $Centroi{d}_n$ in the same cluster.

3.2.2. Gaussian Membership Functions

The parameterization of the Gaussian MF requires two parameters, $c$ and $\sigma$. The point $c$ is the mean and $\mathsf{\sigma }$ denotes the STD. The mathematical model for generating a Gaussian MF is expressed in Equation (11) and visually represented in Figure 4.

$$Gaussian\left(x;c,\sigma \right)=e^{-\frac{1}{2}\left(\frac{x-c}{\sigma }\right)}$$

(11)

Table 2. Points for a Gaussian membership function.

$\mathit{M}\mathit{e}\mathit{m}\mathit{b}\mathit{e}\mathit{r}\mathit{s}\mathit{h}\mathit{i}\mathit{p}\mathit{F}\mathit{u}\mathit{n}\mathit{c}\mathit{t}\mathit{i}\mathit{o}{\mathit{n}}_1$	$\mathit{M}\mathit{e}\mathit{m}\mathit{b}\mathit{e}\mathit{r}\mathit{s}\mathit{h}\mathit{i}\mathit{p}\mathit{F}\mathit{u}\mathit{n}\mathit{c}\mathit{t}\mathit{i}\mathit{o}{\mathit{n}}_2$…	$\mathit{M}\mathit{e}\mathit{m}\mathit{b}\mathit{e}\mathit{r}\mathit{s}\mathit{h}\mathit{i}\mathit{p}\mathit{F}\mathit{u}\mathit{n}\mathit{c}\mathit{t}\mathit{i}\mathit{o}{\mathit{n}}_{\mathit{n}}$
$c_1=Centroid\left(Cluste{r}_1\right)$	$c_2=Centroid\left(Cluste{r}_2\right)$…	$c_n=Centroid\left(Cluste{r}_n\right)$
${\sigma }_1=STD\left(Cluste{r}_1\right)$	${\sigma }_2=STD\left(Cluste{r}_2\right)$…	${\sigma }_n=STD\left(Cluste{r}_n\right)$

explains the procedure to design a Gaussian MF from Gaussian ${MF}_1$ to Gaussian ${MF}_n$. For each feature (dimension) of data, $C_n$ is the position of the $Centroi{d}_n$ of $cluste{r}_n$, and point ${\sigma }_n$ is given by the standard deviation of $cluste{r}_n.$

3.2.3. Trapezoidal Membership Functions

Trapezoidal MF comprises four points $a,b, c,$ and $d$, with the mathematical restriction that $a<b\le c<d$, as expressed in Equation (12).

$$Trapezoidal\left(x;a,b,c,d\right)=\left\{\begin{array}{c}0, x\le a.\\ \frac{x-a}{b-a}, a\le x\le b.\\ 1, b\le x\le c. \\ \frac{d-x}{d-c}, c\le x\le d.\\ 0, d\le x.\end{array}\right.$$

(12)

In this approach, we incorporate the statistical method, specifically a summary of the five numbers, to generate the graphic known as Box-and-Whisker [34]. This process enables the automatic design of a Trapezoidal MF, as illustrated in Figure 5.

Table 3. Points for a Trapezoidal MF.

$\mathit{M}\mathit{e}\mathit{m}\mathit{b}\mathit{e}\mathit{r}\mathit{s}\mathit{h}\mathit{i}\mathit{p}\mathit{F}\mathit{u}\mathit{n}\mathit{c}\mathit{t}\mathit{i}\mathit{o}{\mathit{n}}_1$	$\mathit{M}\mathit{e}\mathit{m}\mathit{b}\mathit{e}\mathit{r}\mathit{s}\mathit{h}\mathit{i}\mathit{p}\mathit{F}\mathit{u}\mathit{n}\mathit{c}\mathit{t}\mathit{i}\mathit{o}{\mathit{n}}_2$…	$\mathit{M}\mathit{e}\mathit{m}\mathit{b}\mathit{e}\mathit{r}\mathit{s}\mathit{h}\mathit{i}\mathit{p}\mathit{F}\mathit{u}\mathit{n}\mathit{c}\mathit{t}\mathit{i}\mathit{o}{\mathit{n}}_{\mathit{n}}$
$a_1=min\left(Cluste{r}_1\right)$	$a_2=min\left(Cluste{r}_2\right)$…	$a_n=min\left(Cluste{r}_n\right)$
$b_1=Quartile 1\left(Cluste{r}_1\right)$	$b_2=Quartile 1\left(Cluste{r}_2\right)$…	$b_n=Quartile 1\left(Cluste{r}_n\right)$
$c_1=Quartile 3\left(Cluste{r}_1\right)$	$c_2=Quartile 3\left(Cluste{r}_2\right)$…	$c_n=Quartile 3\left(Cluste{r}_n\right)$
$d_1=max\left(Cluste{r}_1\right)$	$d_2=max\left(Cluste{r}_2\right)$…	$d_n=max\left(Cluste{r}_n\right)$

details how the parameters $a_n, b_n, c_n,$ and $d_n$ are calculated.

3.2.4. Fuzzy Rules

The fuzzy rules for the proposed method will be automatically generated. Specifically, the rules depend on both the number of input data and the quantity of features in each dataset.

In an FIS, the knowledge base or fuzzy rules depend on the antecedents (input variables) and the partition of these variables (number of MF). It is essential for researchers to have the option of using either the total number of fuzzy rules or, in specific cases, employing fewer rules than the total possible number.

The fuzzy rule generation process for the proposed CNNT1FL method is detailed below:

• IF ($Inpu{t}_1$ is $Cluste{r}_1$) and ($Inpu{t}_2$ is $Cluste{r}_1$) and, …, ($Inpu{t}_n$ is $Cluste{r}_1$), THEN ($Output$ is $Clas{s}_1$).
• IF ($Inpu{t}_1$ is $Cluste{r}_2$) and ($Inpu{t}_2$ is $Cluste{r}_2$) and, …, ($Inpu{t}_n$ is $Cluste{r}_2$), THEN ($OutputisClas{s}_2$).
• IF ($Inpu{t}_1$ is $Cluste{r}_3$) and ($Inpu{t}_2$ is $Cluste{r}_3$) and, …, ($Inpu{t}_n$ is $Cluste{r}_3$), THEN ($Output$ is $Clas{s}_3$).

⋮

m.

IF ($Inpu{t}_1$ is $Cluste{r}_k$) and ($Inpu{t}_2$ is $Cluste{r}_k$) and, …, ($Inpu{t}_n$ is $Cluste{r}_k$), THEN ($Output$ is $Clas{s}_k$).

where $m$ denotes the maximum number of fuzzy rules, $n$ represents the number of input data, and $k$ denotes the events to evaluate.

3.2.5. Fuzzy Outputs

The consequents or output variables may vary depending on the specific problem and rules employed. For example, when applying the CNNT1FL method to a classification problem, the output will depend on the number of classes to be classified.

4. Experimental Results

This section presents the simulations and results after being applied to the proposed method: Design of Fuzzy Inference Systems based on a Competitive Neural Network denoted as CNNT1FL. The variants of the CNNT1FL model are applied to three classification datasets: Iris, Wine, and Wisconsin Breast Cancer Dataset (WDBC).

Table 4. Parameters of the datasets.

Dataset	Classes	Features	Data Number
Iris	3	4	150
Wine	3	13	178
WDBC	2	30	569

shows the characteristics of each dataset, such as the data number, features, and number of classes.

In Table 4, the “number of data” parameter is the input data for the Competitive Neural Network, the “features” parameter represents the number of partitions of the input variable of the FIS, and the “Classes” parameter represents the FIS decision or the output variables.

On the one hand, in the first part of CNNT1FL, the parameters used in the Neural Network are the following:

• Network type: Competitive Neural Network.
• Learning of Network: Unsupervised.
• Performance: MSE (Mean Square Error).
• Inputs: $N$ (data number of the dataset).
• Outputs: $M$ (Centroids).
• Epochs: 100.

On the other hand, in the second part of CNNT1FL, we have designed two types of FISs, Mamdani and Sugeno, with the aim of analyzing and comparing the performance and improving the data classification task.

It is very important to mention that all the features of each dataset are used to generate the model, where 80% of the data are used to design each FIS and 20% are used to test the performance of the created FIS.

The results obtained with Mamdani FIS and Sugeno FIS are presented in the tables below containing the mean, standard deviation, best and worst result based on 31 independent runs of the proposed method for Iris, Wine, and WDBC datasets.

4.1. Mamdani FIS

Table 5. Mamdani FIS results for the Iris dataset.

Membership Functions	Mean	STD	Best	Worst
Triangular	75.59	4.16	86.67	70
Gaussian	81.29	5.21	90	70
Trapezoidal	70.97	4.57	76.67	60

shows the results obtained with the Mamdani FIS for the Iris dataset with the Triangular, Gaussian, and Trapezoidal MFs. The best results are shown in bold.

The best results for the Iris dataset applying the Mamdani FIS were obtained using the Gaussian MF with a mean value of 81.29% and a classifier result of 90%. Figure 6 shows the results of the best classifier, correctly classifying 27 out of 30 in the test dataset.

The Iris dataset has four features: Length sepals, width sepals, length, and width petals; therefore, the Mamdani FIS has four input variables, as illustrated in Figure 7.

Figure 8 presents the output variable of the Mamdani FIS, which has three membership functions because the Iris dataset has three different classes for classification.

The three fuzzy rules used in the Mamdani FIS are the following:

• If (Length Sepals is Cluster1) and (Width Sepals is Cluster1) and (Length Petals is Cluster1) and (Width Petals is Cluster1), then (Classification is Class 1).
• If (Length Sepals is Cluster2) and (Width Sepals is Cluster2) and (Length Petals is Cluster2) and (Width Petals is Cluster2), then (Classification is Class 2).
• If (Length Sepals is Cluster3) and (Width Sepals is Cluster3) and (Length Petals is Cluster3) and (Width Petals is Cluster3), then (Classification is Class 3).

The results obtained for the Wine dataset with Mamdani FIS are described in

Table 6. Mamdani FIS results for the Wine dataset.

Membership Functions	Mean	STD	Best	Worst
Triangular	48.53	14.46	72.22	33.33
Gaussian	63.21	9.16	74.29	36.11
Trapezoidal	51.76	10.35	68.57	44.44

.

The best result obtained for the Wine dataset is using the Gaussian membership function with 74.29% based on 26 out of 35 correct classifications in the test dataset, as shown in Figure 9. Figure 10, Figure 11, Figure 12 and Figure 13 illustrate the input variables which represent the 13 features of the Wine dataset, these are divided into parts 1, 2, 3, and 4, respectively and Figure 14 presents the output variable with two classes.

Similarly, in the Iris dataset, the Wine dataset has three classes; therefore, in Mamdani FIS, three fuzzy rules are used, which are presented below:

• If (Alcohol is Cluster1) and (Malic acid is Cluster1) and (Ash is Cluster1) and (Alcalinity of ash is Cluster1) and (Magnesium is Cluster1) and (Total phenols is Cluster1) and (Flavanoids is Cluster1) and (Nonflavanoid phenols is Cluster1) and (Proanthocyanins is Cluster1) and (Color intensity is Cluster1) and (Hue is Cluster1) and (OD280/OD315 of diluted wines is Cluster1) and (Proline is Cluster1), then (Classification is Class 1).
• If (Alcohol is Cluster2) and (Malic acid is Cluster2) and (Ash is Cluster2) and (Alcalinity of ash is Cluster2) and (Magnesium is Cluster2) and (Total phenols is Cluster2) and (Flavanoids is Cluster2) and (Nonflavanoid phenols is Cluster2) and (Proanthocyanins is Cluster2) and (Color intensity is Cluster2) and (Hue is Cluster2) and (OD280/OD315 of diluted wines is Cluster2) and (Proline is Cluster2), then (Classification is Class 2).
• If (Alcohol is Cluster3) and (Malic acid is Cluster3) and (Ash is Cluster3) and (Alcalinity of ash is Cluster3) and (Magnesium is Cluster3) and (Total phenols is Cluster3) and (Flavanoids is Cluster3) and (Nonflavanoid phenols is Cluster3) and (Proanthocyanins is Cluster3) and (Color intensity is Cluster3) and (Hue is Cluster3) and (OD280/OD315 of diluted wines is Cluster3) and (Proline is Cluster3), then (Classification is Class 3).

Continuing with the Mamdani FIS,

Table 7. Mamdani FIS results for the WDBC dataset.

Membership Functions	Mean	STD	Best	Worst
Triangular	71.84	0.35	72.81	71.05
Gaussian	92.33	1.22	94.74	89.47
Trapezoidal	73.79	5.83	91.23	71.05

shows the results obtained for the WDBC dataset with three different membership functions. The best average and classifier accuracy was achieved using the Gaussian membership function with 92.33% and 94.74%, respectively.

Figure 15 presents the best classification result for the WDBC dataset, where the best model correctly classifies 98 features out of 118 in the test dataset.

The WDBC dataset has 30 features for their data and two classes for classification. Figure 16 illustrates the first of eight parts of the input variables and shows the first four features: radius_mean, texture_mean, perimeter_mean, and area_mean.

Figure 17 and Figure 18 present Parts 2 and 3 of input variables. Part 2 comprises the features: smoothness_mean, compactness_mean, concavity_mean, and concave_points_mean, and Part 3 comprises symmetry_mean, fractal_dimension_mean, radius_se, and texture_se.

Figure 19 presents the fourth part of input variables with the features: perimeter_se, area_se, smoothness_se, and compactness_se. In Figure 20, the fifth part of input variables is illustrated.

The radius_largest_worst, texture_largest_worst, perimeter_largest_worst, and area_largest_worst features are presented in Figure 21. Figure 22 shows the seventh part.

The last two input variables of the Mamdani FIS are shown in Figure 23, which are symmetry_largest_worst and fractal_dimension_largest_worst. In Figure 24, we present the output variable with its respective classes.

In the WDBC, we only use two fuzzy rules as described below:

• If (radius_mean is Cluster1) and (texture_mean is Cluster1) and (perimeter_mean is Cluster1) and (area_mean is Cluster1) and (smoothness_mean is Cluster1) and (compactness_mean is Cluster1) and (concavity_mean is Cluster1) and (concave_points_mean is Cluster1) and (symmetry_mean is Cluster1) and (fractal_dimension_mean is Cluster1) and (radius_se is Cluster1) and (texture_se is Cluster1) and (perimeter_se is Cluster1) and (area_se is Cluster1) and (smoothness_se is Cluster1) and (compactness_se is Cluster1) and (concavity_se is Cluster1) and (concave_points_se is Cluster1) and (symmetry_se is Cluster1) and (fractal_dimension_se is Cluster1) and (radius_largest_worst is Cluster1) and (texture_largest_worst is Cluster1) and (perimeter_largest_worst is Cluster1) and (area_largest_worst is Cluster1) and (smoothness_largest_worst is Cluster1) and (compactness_largest_worst is Cluster1) and (concavity_largest_worst is Cluster1) and (concave_points_largest_worst is Cluster1) and (symmetry_largest_worst is Cluster1) and (fractal_dimension_largest_worst is Cluster1), then (Classification is Class 1).
• If (radius_mean is Cluster2) and (texture_mean is Cluster2) and (perimeter_mean is Cluster2) and (area_mean is Cluster2) and (smoothness_mean is Cluster2) and (compactness_mean is Cluster2) and (concavity_mean is Cluster2) and (concave_points_mean is Cluster2) and (symmetry_mean is Cluster2) and (fractal_dimension_mean is Cluster2) and (radius_se is Cluster2) and (texture_se is Cluster2) and (perimeter_se is Cluster2) and (area_se is Cluster2) and (smoothness_se is Cluster2) and (compactness_se is Cluster2) and (concavity_se is Cluster2) and (concave_points_se is Cluster2) and (symmetry_se is Cluster2) and (fractal_dimension_se is Cluster2) and (radius_largest_worst is Cluster2) and (texture_largest_worst is Cluster2) and (perimeter_largest_worst is Cluster2) and (area_largest_worst is Cluster2) and (smoothness_largest_worst is Cluster2) and (compactness_largest_worst is Cluster2) and (concavity_largest_worst is Cluster2) and (concave_points_largest_worst is Cluster2) and (symmetry_largest_worst is Cluster2) and (fractal_dimension_largest_worst is Cluster2), then (Classification is Class 2).

4.2. Sugeno FIS

This section presents the results obtained with Sugeno FIS; the difference between Mamdani and Sugeno is the output variable. Mamdani uses output variables with membership functions and Sugeno uses a constant in the output variables.

Table 8. Sugeno FIS results for the Iris dataset.

Membership Functions	Mean	STD	Best	Worst
Triangular	73.33	4.87	80	63.33
Gaussian	84.34	5.23	89.66	75.86
Trapezoidal	68.17	4.46	80	60

shows the results for the Iris dataset with Sugeno FIS. As we can see, the best results are obtained using the Gaussian MF with an accuracy of 89.66%.

Figure 25 illustrates the results of the best classification model. Figure 26 and Figure 27 present the input and output variables, respectively, which are used in the implementation of the Sugeno FIS.

In the output variable of the Sugeno FIS, we implement a rounding rule based on ranges: range from 0 to 1.00 is Class 1, range from 1.01 to 2.00 is Class 2, and range from 2.01 to 3.00 is Class 3.

For the Wine dataset, the best result was obtained using the Triangular MF with an accuracy of 80%, and the best average was using the Gaussian MF with a value of 66.46%, as shown in

Table 9. Sugeno FIS results for the Wine dataset.

Membership Functions	Mean	STD	Best	Worst
Triangular	49.62	15.69	80	33.33
Gaussian	66.46	7.93	75	50
Trapezoidal	52.45	10.61	67	44

.

The best classification result is presented in Figure 28, achieving 80% with 28 correct classifications out of 35 in the test dataset.

Figure 29 illustrates, as an example, part of the input variables of the best Sugeno FIS using Triangular MF; in this case, only 4 out of the 13 features are presented.

It is important to mention that the fuzzy rules in the Mamdani FIS and Sugeno FIS are the same, and the output variables are the same in the Iris dataset because both datasets have three classes.

Table 10. Sugeno FIS results for the WDBC dataset.

Membership Functions	Mean	STD	Best	Worst
Triangular	73.17	4.85	91	71.05
Gaussian	92.44	0.74	94.74	92.04
Trapezoidal	93.91	0.32	96	94

shows the results for the WDBC dataset; the best result was obtained using Trapezoidal MF. The best classification result with a value of 96% is presented in Figure 30, with a classification of 109 out of 114 in the test dataset.

A part of the input variables of the best Sugeno FIS for WDBC using the Trapezoidal MF is illustrated in Figure 31, which presents a sample of 4 out of 30 features. The output variable of the Sugeno FIS is represented in Figure 32; in this case study, it comprises two classes.

4.3. Comparative Analysis

A summary of the results is presented in

Table 11. Comparative results using Mamdani FIS.

Dataset	Triangular MF		Gaussian MF		Trapezoidal MF
	Mean	Best	Mean	Best	Mean	Best
Iris	75.59	86.67	81.29	90	70.97	76.67
Wine	48.53	72.22	63.21	74.29	51.76	68.57
WDBC	71.84	72.81	92.33	94.74	73.79	91.23

and

Table 12. Comparative results using Sugeno FIS.

Dataset	Triangular MF		Gaussian MF		Trapezoidal MF
	Mean	Best	Mean	Best	Mean	Best
Iris	73.33	80	84.34	89.66	68.17	80
Wine	49.62	80	66.46	75	52.45	67
WDBC	73.17	91	92.44	94.74	93.91	96

. Table 11 describes the comparative analysis achieved by the different MFs when implementing Mamdani FIS and Table 12 presents the results of Sugeno FIS. According to these values, we can notice that in Mamdani FIS, the configuration using the Gaussian MF was better for all datasets. Additionally, in Sugeno FIS, the behavior had a slight variation; for the Iris dataset, the Gaussian MF also presents better results, but in the case of Wine, the best precision was obtained with the Triangular MF and the best average with the Gaussian MF, and for WDBC dataset, the performance was improved with the Trapezoidal MF.

Analyzing the general performance of Mamdani FIS against Sugeno FIS, we can conclude that Sugeno was more stable than Mamdani, achieving a better precision in two of the three datasets (Wine and WDBC). On the other hand, comparing the results with respect to the MFs, the use of Gaussian MF presents greater stability in most results. In Mamdani FIS, it is better in all cases, and in Sugeno FIS, it is better in the Iris and Wine datasets.

In order to evaluate the performance of the proposed CNNT1FL method versus other state of the art methods, we performed a comparison with Interval Type-2 Fuzzy C-Means (IT2FCM) [6] using only the Iris data. This analysis comparison is described in

Table 13. FIS comparison between CNNT1FL and IT2FCM for the Iris dataset.

Method	Membership Functions	Mamdani FIS	Sugeno FIS
IT2FCM	Gaussian	65.33	94.67
	Triangular	86.67	80
CNNT1FL	Gaussian	90	89.66
	Trapezoidal	76.67	80

.

The comparison between both methods, CNNTFL and IT2FCM, comprises analyzing the best percentage classification of Mamdani FIS and Sugeno FIS for the Iris database, since it is the only dataset tested by the authors.

As we can notice in Table 13, with the three variations of membership functions in Mamdani FIS, the proposed method CNNT1FL obtained better results than IT2FCM; however, for the Sugeno FIS, the results of CNNT1FL were not good compared to IT2FCM.

It is important to mention that one of the disadvantages of CNNT1FL is that it uses all features for each dataset of the databases, on the contrary, IT2FCM did not specify how many and which features are used.

5. Conclusions

In this paper, we have presented a new hybrid method based on a Competitive Neural Network and the Type-1 Fuzzy Inference Systems. The main contribution comprises automatically designing Type-1 FIS, wherein we added three variations in the design of the FIS, specifically, we made a variant in the membership functions and designed FIS with Triangular, Gaussian, and Trapezoidal membership functions, in particular, in the input variables. We also designed two types of FISs, Mamdani and Sugeno, with the three variations mentioned above.

CNNT1FL was applied in the area of data classification and the results can be summarized into three main points. (1) In the Iris dataset, the highest result was achieved by employing the Gaussian MF for both input and output variables. (2) In the Wine dataset, the best mean across all variations was obtained with Sugeno FIS using Gaussian MF; however, the best performing FIS was reached using the Triangular MF as well as Sugeno FIS. (3) The best designed FIS for the WDBC dataset is a Sugeno FIS with Trapezoidal MF.

On the other hand, in the comparison we made, we achieved an improvement in the best design of the Mamdani Type-1 FIS, i.e., the three variations obtained a better percent of correct classification than IT2FCM [6]. For the Sugeno FIS, the results were not satisfactory compared to IT2FCM; it is important to mention two things: first, we only carried out a comparison with the Iris dataset and the best correct classification percentage because the data are not available in the paper of IT2FCM [6], i.e., the mean and standard deviation are not documented to carry out a statistics test; second, we neither found quantitative nor qualitative features data and, as mentioned in the previous paragraphs, this results in a disadvantage to the proposed CNNT1FL method because we are using all the features.

In general, we can conclude that the main objective of automatically designing a Type-1 FIS was achieved; according to the results, the Gaussian membership function is the best option for variables in Type-1 FIS, and the Sugeno FIS is the best option for data classification problems. We can also conclude that similarity in data features is very important for correct data classification, for example, in the Wine dataset, the percentage of correct data classification is not high because the thirteen features have a little similarity, namely, the range of data features is highly variable.

Finally, as described in the previous paragraph, we have achieved the main objective of this work, highlighting the contribution with the combination of two powerful computing areas using artificial neural networks and fuzzy logic to develop robust methods that can improve results in complex classification problems.

In future work, we will be designing the Interval Type-2 FIS or General Type-2 FIS and applying it to similar problems with the objective of carrying out a comparison with CNNT1FL and other state of the art methods as well as improving accuracy. Also, we could use different datasets (more data and features) with highly complex problems to evaluate the performance of the proposal.