Pruning Fuzzy Neural Network Applied to the Construction of Expert Systems to Aid in the Diagnosis of the Treatment of Cryotherapy and Immunotherapy

Abstract

Human papillomavirus (HPV) infection is related to frequent cases of cervical cancer and genital condyloma in humans. Up to now, numerous methods have come into existence for the prevention and treatment of this disease. In this context, this paper aims to help predict the susceptibility of the patient to forms treatment using both cryotherapy and immunotherapy. These studies facilitate the choice of medications, which can be painful and embarrassing for patients who have warts on intimate parts. However, the use of intelligent models generates efficient results but does not allow a better interpretation of the results. To solve the problem, we present the method of a fuzzy neural network (FNN). A hybrid model capable of solving complex problems and extracting knowledge from the database will pruned through F-score techniques to perform pattern classification in the treatment of warts, and to produce a specialist system based on if/then rules, according to the experience obtained from the database collected through medical research. Finally, binary pattern-classification tests realized in the FNN and compared with other models commonly used for classification tasks capture results of greater accuracy than the current state of the art for this type of problem (84.32% for immunotherapy, and 88.64% for cryotherapy), and extract fuzzy rules from the problem database. It was found that the hybrid approach based on neural networks and fuzzy systems can be an excellent tool to aid the prediction of cryotherapy and immunotherapy treatments.

1. Introduction

Human papillomavirus infection (HPV) is a serious emerging public health problem. Popularly known as HPV, it is a virus that causes skin and mucosal lesions, also leading to the formation of genital and extra-genital warts (conjunctiva, nasal, or oral mucosa, and laryngeal) for men and women. For a woman, lack of specialized follow-up may generate not only infections but also high chances of the evolution of cervical cancer [1]. For men, a growth of the disease can result in the development of cancer of the anus and penis [2]. Regarding the treatments of warts caused by HPV, most prominent are immunotherapy and cryotherapy. Use of immunotherapy causes the patient to become allergic to a chemical is applied, creating an allergic reaction that occurs around the wart, resulting in its disappearance. If the cryotherapy method is selected, spray jets or pre-cooled probes at below-zero temperatures are applied directly in warts. The choice of treatment should be based on the number and size of warts, morphology and location of the lesions, their medical condition as to the assessment of their immunity, as well as the financial relevance of the treatment and the patient’s discomfort aspects [2]. Recent studies have investigated fuzzy rules [3] for the prediction of the efficiency and impacts of treatments in the removal of warts and infections caused by HPV virus in humans. This system acts with a knowledge base that can generate fuzzy rules to classify whether or not the patient will respond well to the applied treatment—accomplishing the classification in 83.33% of cases if a patient will react well or not to a treatment based on immunotherapy, and 80.7% in the use of cryotherapy. This type of system has several benefits: aiding doctors in selecting the correct method for the patient; greater certainty of healing the patient; reducing risks of possible scars and using unnecessary anesthesia; and ultimately, avoiding wasting time, money and clinical efforts to reduce treatment to a group of people. However, a notable disadvantage was that in the study, a decision tree (knowledge base) is used, which must be modeled every time a new factor is inserted in the evaluation of the problem. This paper proposes the use of a hybrid structure, also based on fuzzy rule systems, but which has faster and simpler responses than a decision tree. Improveing the accuracy of the HPV treatment prediction system using methods such as immunotherapy and cryotherapy in patients suffering from benign tumors, [3] proposed the current state of the art regarding warts treatments using smart models. In his work, he used a fuzzy inference system that generated results between 80 and 84% accuracy. Fuzzy neural networks (FNNs) can generate fuzzy rules and maintain a high level of accuracy for the evaluated results. They can perform efficient training, extract information from the problem data, and maintain high-accuracy results. They effectively resolve problems of various types of science such as pattern classification [4,5,6,7,8], linear regression [9], time series forecasting [10], issues in industry [11], and also resolve problems in the areas of health [12,13,14,15,16,17] and software efforts [18,19]. Even problems in the field of immunotherapy have been the subject of judgment by these models in [20].

In this paper, we will use concepts of FNNs pruned by determination of relevance techniques based on the idea of F-scores. This model uses the generation of equally spaced membership functions and training based on the extreme learning machine [21], and finally, a neural network of aggregation is responsible for providing acceptance of the treatment by the patient analyzed by the model [22]. The objective of this study is to find a classifier to identify the feasibility of using cryotherapy or immunotherapy treatments, with a high degree of accuracy, according to the characteristics of patients.

The innovations proposed in this paper are in a hybrid architecture capable of combining efficient training of neural networks and fuzzy interpretation techniques to extract the most significant number of fuzzy rules from the database used in the problem.

The goal of bringing together fuzzy rules is to make decisions more efficient and allow treatments to go to the right people.

This article will present the following sections: In the next Section 2 the concepts of general references that will anchor the research and related works in literature. In Section 3, a pruned fuzzy neural network model will be provided. In Section 4 we will present the models that will be part of the pattern classification tests to determine the acceptability of the treatments, databases used in the experiments, and the inferences obtained in the results, with emphasis on the if/then rule systems obtained by the model. In Section 5 the results obtained in the pattern classification tests for treatment using cryotherapy and immunotherapy are evaluated, and finally, Section 6 presents the conclusions of the paper and propositions of future work.

2. Literature Review

The main theoretical concepts used in this article will present, focusing mainly on the concepts and techniques of cryotherapy, immunotherapy and intelligent hybrid models.

2.1. Human Papillomavirus Concepts

There are more than 100 types of human papillomavirus (HPV). Among these genital warts are type 6 and type 11 viruses, and viruses present in up to 70% of the cases that have progressed to various cancers like cervical cancer, whereas throat or anus are types 16 and 18, respectivley [23]. Figure 1 presents forms of HPV infection.

Prevention of the virus is due to the vaccines that have been available since 2006 in the United States and Europe, and since then their safety and clinical efficacy profiles have been carefully examined by the World Health Organization (WHO) [24]. HPV is a sexually transmitted disease because its transmission is due to direct friction with the skin and most transfers occur through sexual contact. Use of condoms is necessary protection to prevent the transmission of HPV. Other forms of transference, much rarer, are by direct contact with other skin warts, sharing of undergarments or towels, and, finally, vertical transmission, that is, from the mother to the fetus, which may occur during childbirth.

Usually, the virus manifests itself in the human body between two to eight months and may be present in the body for up to 20 years without revealing itself. When not demonstrated, it can still be transmitted to another human. Factors that contribute to contamination are unprotected sex, early sexual intercourse, low immunity, lack of preventive exams, among others. However, in most cases, HPV is eliminated spontaneously from the body being asymptomatic [25]. Some standards treatments for the lesions and warts are creams, acids and increase the immunity in place with the use of substances modifying the biological response, called immunotherapy. Removal of the lesion can also be accomplished utilizing laser cauterization and the use of cryotherapy. Most of these therapies are expensive and painful.

Cryotherapy is the treatment that uses low temperatures for aesthetic and treatments on the skin. For this, spray jets or pre-cooled probes may be used. In the use of this technique against warts caused by HPV, treatment is due to the freezing of the wart using the liquid nitrogen for 10 to 20 s every two to three weeks. The freeze–thaw cycle created by this method produces the formation of water crystals inside the cells, which promotes changes in the cellular composition of warts, resulting in dehydration and rupture of the plasma membrane [26].

In this treatment, direct contact with the skin is indispensable, arriving at temperatures of $-196$ degrees Celsius. It can cause some discomfort to the patient and professional, because the locations of these warts can be in delicate areas of the body. Often these procedures are painful, can result in scarring and require the use of anesthesia. Besides, recurrence of warts may frequently occur [27]. They also count as cryotherapy the use of gels and sprays that carry camphor or menthol in their composition, causing a cooling where they are applied. Figure 2 shows the cryotherapeutic treatment in medical offices.

Among the other methods cited to combat HPV, one can find the immunotherapeutic. This type of treatment that aims to boost the immune system so that it can fight infections and other diseases like HPV and cancer.

The human immune system is the critical component for meaningful treatment responses, not just immunotherapeutic approaches. There is a growing body of evidence that patients with long-term reactions or conditions of “clinical cure” after being subjected to immunogenic mechanisms of cell death have good results in the treatment of warts. These agents target the desired area may lead to immunogenic or immune tolerant cell death, former being associated with prolonged responses and control or even cure [28].

Immunotherapy is most effective in some types of cancer, such as prostate cancer, kidney cancer, lung cancer, to name a few. In recent years it has been shown that the association of immunotherapy with traditional chemotherapy or with other associated target therapies may be even more effective than isolated immunotherapy [29]. Figure 3 presents elements used in the treatment of immunotherapy.

2.2. Artificial Intelligence, Artificial Neural Networks and Fuzzy Systems

A human being can perform activities that involve the learning that he has acquired during his life. This makes you able to learn from environmental situations to perform everyday tasks. These tasks involve various aspects such as classifying, labeling, identifying patterns, predicting, even calculating relationships between elements [30]. Smart models are also capable of performing these activities. From the beginning of the 1940s, modeling of the biological neuron in mathematical terms allowed the evolution of techniques that would be able to simulate actions linked to human thought [31]. This set of procedures evidenced in artificial neural networks, which are computational structures that simulate intelligent human characteristics in problem solving. Neural networks involve situations ranging from receiving the problem, defining the resolution structure to achieve the goals and the final results [32]. To simulate the learning capacity of the human being in the 1970s and 1980s, algorithms were developed to perform the evolution of parameters by performing the simulation of human learning ability. These performances revolve around the mathematical calculations to update values according to learning rate values. With human beings, it is no different: after he receives an answer about some of his attitudes or assessments he ponders if the expected result is ideal; if not he checks what he missed and made the necessary adjustments not to make the same mistakes [30]. Algorithms such as backpropagation and the gradient descent [33] represent very well the learning algorithms of neural networks. Parallel to the evolution of this technique, others emerged as the algorithms based on the theory of evolution called evolutionary algorithms [34], a set of methods capable of simulating characteristics of human genetics [35].

Also during this period, techniques capable of bringing to the intelligent algorithms the capacities of interpretability that the human beings have when communicating were highlighted. Solving problems, where binary nature is a fundamental resource for the kind of a problem, was elaborated in 1965 by Zadeh [36] as the concept of fuzzy sets, where the data of a problem come to be represented by concepts of functions of relevance, calculations of significance and logical rules to represent the values involved in a context. In this way, artificial intelligence can evolve and solve incredibly complex problems, many of which humans would spend a lot of time or reasoning to solve [37].

2.3. Related Works

Several researchers to assist in better diagnosis and treatment techniques treated aspects of immunotherapy and cryotherapy treatments. In the work of [38] the focus was on the use of data mining techniques to help in the treatment of cancer with immunotherapy. Already in the works of [3,20]. the main focus was the construction of expert systems using decision trees and fuzzy neural networks respectively. Recent work of [39] proposes a hybrid classifier model to aid in these treatments, as well as a model based on a gravitational search to select the best characteristics to evaluate a better treatment [40]. Other works that use in the literature the smart models highlighted in [41,42,43,44,45,46,47,48,49,50].

2.4. Fuzzy Neural Network

Intelligent models can unite their best features to build systems with high potential for problem solving in various scientific contexts. Neural networks have excellent ways of simulating human knowledge through training algorithms that can simulate learning in the acquisition of knowledge [51]. Fuzzy systems have an enormous capacity to transform a problem into characteristics capable of being interpreted by human beings, building expert systems that translate the data into knowledge acquired through fuzzy if/then rules [52]. When these two techniques come together, concepts of hybrid models called fuzzy neural networks arise [53]. For synergy of the two contexts to occur in these original structures, artificial neurons replaced by fuzzy neurons, allowing fuzzification processes of the data to provide parameters for the construction of fuzzy neurons and the latter to extract the knowledge of the training data. In addition to this technique, the defuzzification process occurs through neural network training techniques that can update parameters dynamically and efficiently. These models serve to unite the best of the two methods that act strongly in solving complex problems [53]. Fuzzy neural networks work in issues in the area of the industry [11,54,55], controls, and actuation in robots [56,57,58,59], sectors of the economy [60,61,62], pulsar detection [63] and in the prediction of process failures [64,65,66,67]. Already in health, models have highlights in different performances. As examples, work has been done towards arrhythmia detection [68], mass lesion detection [69], risk factors of metabolic syndrome [70], ovarian cancer diagnosis [71]. Hybrid models found in applications and problems in other fields, such as [72,73,74,75].

3. Pruning Fuzzy Neural Network Applied to Problems of Cryotherapy and Immunotherapy

This section will explain the main concepts involved in the construction of the hybrid model, highlighting elements present in each of its layers, techniques and training algorithms used in the model. It was initially proposed for pattern classification [22], and in this paper, it will be applied in the identification of patients suitable for the treatment of cryotherapy and immunotherapy. Figure 4 presents a pruning fuzzy neural network model where A neurons indicate fuzzy neurons generated by the ANFIS (Adaptative Network based Fuzzy Inference Systems [76]). method, which are linked to Z neurons (unineuron) and which in turn are unified in a single neuron with a linear activation function. The first two layers of this model represent the fuzzy inference system. The third layer is the representation of a neural network of aggregation that has a single neuron.

3.1. First Layer

The first layer of the fuzzy neural network used in this paper uses fuzzification concepts based on techniques of representing the input space of the input data through methods based on the ANFIS [76]. This paper discusses the creation of equally spaced membership functions for the production of fuzzy neurons of the Gaussian type in the first layer. These neurons have the synaptic weight and bias defined at random in the interval of [−1 and 1]. These Gaussian neurons measured as the degree of relevance of the neurons that exist in the second layer. ANFIS technique has a method variation called genfis1, which creates equally spaced membership functions to divide the input space into equal representation spaces. These spaces allow the determination of the logical relations of the input space, interpreted and represented by linguistic variations. By creating a number M of membership functions, genfis1 technique allows the established functions to be construed as linguistic variables such as small, medium and large, for example. This approach is useful in representing spaces within the decision space of input data, but the number of neurons in the first layer can become an exponential problem connected to the number of dimensions of the problem. For each dimension of the problem (N), it is assigned a number M of membership functions. The higher the number of dimensions, the greater the number of neurons. To solve this problem, when the relation of dimensions and membership functions generates more than 500 neurons, model randomly defines combinations for the relation dimensions and functions of membership in the limit of 500 combinations. Therefore when the number of dimensions is high, algorithm limits the complexity of neurons through a technique with random combinations [22].

3.2. Second Layer

Fuzzy logical neurons form the second layer of the model. Type III neurons receive this assignment because they can construct diffusive if/then-like rules, extracting knowledge from the data. These logical neurons perform the aggregation of neurons of the first layer using fuzzy operators called t-norm and s-norm. These are operators of fuzzy sets so that representative values constructed for the neurons. A neuron that uses the t-norm is called an andneuron since the neuron that uses the s-norm as the operator is called the orneuron. Two neurons are efficient in solving problems, but there is a logical operator that allows using at one time a t-norm and at another time a s-norm. The operator is called uninorm [77] and is the basis for the logical neuron used in this paper, proposed by [78,79] and use in [80].

In this paper the uninorm is expressed as follows [79]:

$$U(x,y)=\left\{\begin{array}{c}gT(\frac{x}{g},\frac{y}{g}),if\mathit{y}\in [0,g]\\ g+(1-g)S(\frac{x-g}{1-g},\frac{y-g}{1-g}),if\mathit{y}\in (g,1]\\ \phi (x,y),otherwise\end{array}\right.$$

(1)

and

$$\phi (x,y)=\left\{\begin{array}{cc}max(x,y)& ,\mathrm{if}g\in [0,0.5]\\ min(x,y)& ,\mathrm{if}g\in (0.5,1]\end{array}\right.,$$

(2)

where T is a t-norm, S is an s-norm and g is an identity element that varies between 0 and 1. In other words, uninorms can switch smoothly between an s-norm (if g = 0) and a t-norm (if g = 1). In this paper, reconsidered the t-norm operator the product and as s-norm operator the probabilistic sum.

The unineuron proposed in [79] performs the following operations to compute its output:

• each pair ($a_i$, $w_i$) is transformed into a single value $b_i$ = h($a_i$, $w_i$)
• calculate the unified aggregation of the transformed values U ($b_1,b_2\dots b_n$), where n is the number of inputs.

The function p (relevancy transformation) is responsible for transforming the inputs and corresponding weights into individual transformed values. This function fulfills the requirement of monotonicity in value, which means if the input value increases, the transformed value must also increase. It also meets the requirement zero importance elements should have in effect and the normality of importance of one. Finally, function p can bring consistency of effect of $w_i$. Formulation for the p function can be described as [79]:

$$p(w,a)=wa+\overline{w}g,$$

(3)

using the weighted aggregation reported above the unineuron can be written as:

$$\mathbf{z}=UNI(w;a)=U_{i=1}^{n}p(w_i,a_i).$$

(4)

These neurons can create fuzzy rules that are the basis of the fuzzy inference system. They extract the knowledge from the database used in the test to compose a vital knowledge group for various problems. This rule-base can serve as business rules for building expert systems to assist practitioners in diverse areas, including clinicians who treat diseases and need information to define appropriate treatments for patients [22].

3.3. Third Layer

In the third layer of the model, a neural network is responsible for bringing the final responses to the model. A neural network of aggregation with a single neuron (Considered also a singleton). This neuron has an activation function capable of bringing answers according to the database involved in the training of the model. As it is a database for treatments of immunotherapy and cryotherapy, model will provide solutions on the suitability or not of the procedure according to the characteristics of the analyzed person [22]. The equation responsible for the output of the model:

$$\mathbf{y}=\sum _{i=i}^{L_p}{f}_{linear}(z_i,v_i).$$

(5)

where $z_0$ = 1, $v_0$ is the bias, $z_j$ and $v_j$, j = 1, …, $l_p$ are the output of each fuzzy neuron of the second layer and its corresponding weight, respectively. The activation function, in this case, is linear, which means the right weight of the second layer weights and the fuzzy inputs.

How fuzzy neural networks perform their training linked especially to techniques that update or generate fundamental parameters for the operation of the net. In the fuzzy neural network proposed by [22], we use the concept of the extreme learning machine (ELM) proposed by [21] and actively used in the literature to generate the weights analytically through the pseudo inverse concepts of Moore–Penrose [81]. Weights and bias of the neurons of the first layer and the second layer generated randomly. Weights that connect the second layer to the neural network of aggregation created using the concepts of partial least squares. This approach is different from methods that use backpropagation [82] to perform the performance of existing standards. Huang has proved that the generation of final network weights using the pseudo-inverse can generate a processing gain of the intelligent networks [21]. A single step obtains the weights, and as it is not necessary to update the other parameters, intelligent model gains independence from the recurrent update of the network and generates answers with a high level of accuracy for the analyzed results. With the model defined in (5), we can write y as Z * v, where v is the vector of weights of the output layer, y is the vector of outputs. Z is determined to be [21]:

$$\mathbf{Z}={\left[\begin{array}{ccc}f(w_1,a_1+b_1)& \dots & f(w_m,a_1+b_l)\\ f(w_1,a_2+b_1)& \dots & f(w_m,a_2+b_l)\\ f(w_1,a_n+b_1)& \dots & f(w_m,a_n+b_l)\end{array}\right]}_{N\times l}.$$

(6)

The columns of the matrix $\mathbf{Z}$, defined in (6), correspond to the outputs of the hidden neurons of the single layer feed forward network (SLFN) with respect to the input a $={[a_1,a_2,\dots ,a_N]}_{m\times N}^T$.

The ELM implements a random initialization of the weights of the hidden layer (based on a numerical range any), $w_k$. Then, weights of the output layer are obtained through the pseudo inverse according to the expression [21]:

$$\mathbf{v}={\mathbf{Z}}^{+}\mathbf{y}.$$

(7)

The approaches that use the ANFIS [76] model to generate equally spaced membership functions have a relation of exponential neuron creation linked by the relationship between membership functions and problem dimensions. In fuzzy neural networks, same amount of neurons in the first layer is that of the second layer. To decrease this exponential relationship and facilitate pseudo-inverse calculations, in the models of [5,18,20], resampling regularization techniques are used. Results obtained in [20] were inferior to which originated the data used in the studies of cryotherapy and immunotherapy, but it should be noted that the results were obtained with a set of fuzzy rules. Work proposed in this paper differs from the work done in [20] because of the way to select the most significant neurons, mainly because the technique dealt with in this paper does not need to choose the parameters to be used in the pruning of unnecessary information, is based entirely on the nature of the data. In the model of [22], the concept of pruning is prominent of the F-scores proposed by [83] initially to prune structures trained with ELM. The F-scores technique has two particularities where its numerator indicates the discrimination between the positive and negative sets, and the denominator is the sum of the deviation within each set of resources. A higher F-score indicates that the support has more discriminative power [83]. The problem of choosing the most relevant hidden neurons is that a classic resource selection problem may occur. In this way, the F-scores metric is used to evaluate the discriminating power of the fuzzy neurons in the second layer about the classes of the patterns [83]. All fuzzy rules with an F-score below a predefined eliminator limit are considered irrelevant to solve the problem. Model used defines this threshold from the training data, without requiring validation sets and computationally intensive cross-validation procedures as used in the procedures performed in the [20] proposal. In one-step, pruning is complete, before adjusting the weights of the network output layer, giving high speed to the algorithm without loss of performance predictive.

The F-score is a simple metric, consisting of designing the input patterns for a high dimension space and then selecting the most relevant characteristics, those that contain maximum discriminatory information about the classes but useful in evaluating the discriminative power of the variables of the feature set [84].

Given the i-th feature vector (in the case of the FNN the z-vector representing unineurons) with the number of positive instances $n_{+}$, negative instances $n_{+}$ and the number of all the instances N, the F-score value of the i-th feature is defined by [83]:

$$F\left(i\right)=\frac{{({\overline{x}}_i^{(+)}-{\overline{x}}_i)}^2+{({\overline{x}}_i^{(-)}-{\overline{x}}_i)}^2}{\frac{1}{n_{+}-1}{\displaystyle \sum _{k=1}^{n_{+}}}{(x_{k,i}^{(+)}-{\overline{x}}_i^{(+)})}^2+\frac{1}{n_{-}-1}{\displaystyle \sum _{k=1}^{n_{-}}}{(x_{k,i}^{(-)}-{\overline{x}}_i^{(-)})}^2},$$

(8)

where ${\overline{x}}_i^{(+)}$, ${\overline{x}}_i^{(-)}$ and ${\overline{x}}_i$ are the mean of the positive, negative and whole samples, respectively, and is the k-th feature value in the i-faith feature vector. Therefore all the neurons of the second layer z have their calculated f-score and the new group of neurons is composed of all those that have the calculated value more significant than the average of the f-scores.

In this case, the L neurons obtained from the first layer are evaluated for relevance, and the f-scores technique evaluates 50% of the most relevant neurons through the mean values of f-score, without using parameters in a single step [83]. These pruned neurons are called in this paper by $L_p$.

The intelligent models act in the perception of the parameters involved in research through the collection of data from patients who are undergoing chemotherapy and immunotherapy treatment. In the study of [3], Use of the database helped in the first evaluations on techniques capable of generating smart systems to support the definition of treatments to treat the curls properly. At first, existing models used in literature to aid in the prediction of treatment efficacy. Many models that are available on the Weka software [85] employed to determine elements with high levels of accuracy to discover the patient profile. In this paper, we intend to compare the results of the pruned fuzzy neural network with other classifiers commonly used in the literature. Algorithm 1 presents the necessary steps for the execution of the classification of patterns on the treatments for warts. There are two parameters:

• the number of membership functions, M
• the type of fuzzy logic neuron, unineuron

Algorithm 1: Fuzzy neural network for detection of immunotherapy and cryotherapy treatments—fuzzy neural network (FNN) training.

(1) Define the number os membership functions, M. (2) Calculate M neurons for each characteristic in the first layer using ANFIS. (3) Construct L fuzzy neurons with Gaussian membership functions constructed with center and $\sigma$ values derived from ANFIS. (4) Define the weights and bias of the fuzzy neurons randomly. (5) Construct L fuzzy logical neurons with random weights and bias on the second layer of the network by welding the L fuzzy neurons of the first layer. (6) Use f-scores to define the most significant neurons to the problem ($L_p$). (7) For all K input do (7.1) Calculate the mapping $z_k\left(x_k\right)$ using logical neurons (8) Estimate the weights of the output layer using Equation (7). (9) Calculate output y using Equation (5).

4. Results

This section presents the main aspects of the tests performed with the bases of cryotherapy and immunotherapy to demonstrate the effectiveness of the hybrid tool.

4.1. Database Used in the Test

The database was provided by [45] and was used in [3] to select the best treatment for groups of patients using that database as a source of information for an intelligent model. These databases were collected during routine examinations, and medical experts in the field defined the attributes chosen for determining the efficacy of the treatment. This dataset contains information about the results of warts treatment of 90 patients on immunotherapy and cryotherapy. It is available at https://archive.ics.uci.edu/ml/datasets/Cryotherapy+Dataset and https://archive.ics.uci.edu/ml/datasets/Immunotherapy+Dataset. Its dimensions are:

• Sex (41 male (1), and 49 female (2));
• Age (minimum of 15 years and a maximum of 56 years with mean in 31.04 years and standard deviation of 12.23);
• Time (minimum of one and maximum of 12-time units with a mean of 7.23 and a standard deviation of 3.09);
• Number of warts (minimum one and maximum of 19 warts with a mean of 6.14 and a standard deviation of 4.21);
• Type (type (1) in 47 people, type (2) in 22 people and type (3) in 21 people);
• Area (minimum of 6 and maximum of 900 measurements with the mean of 95.7 and standard deviation of 136.61);
• Induration diameter (only in immunotherapy database) (minimum of two and maximum of 70 measurements with a mean of 14.3 and a standard deviation of 17.21);
• The outcome of the treatment (19 people who did not give (0) and 71 people that the treatment was effective (1) for immunotherapy and 48 has effective (1), and 42 have not been successful in the treatment of warts in cryotherapy treatment).

This data repository made available for consultation of the database and provision of appropriate solutions to deal with machine learning issues. So many researchers around the world may be working for answers with the same database.

Figure 5 and Figure 6 respectively present the data collected by the researchers in immunotherapy and cryotherapy treatments, where red bars indicate success in treatment and blue bars that procedure was not efficient. Because they represent data histograms, the y-axis represents how often they appear, the x-axis shows the evaluated characteristic that highlighted at the top of the histogram. Data were presented graphically using the Weka software [85].

4.2. Test Settings

In this paper, we used unineuron and genfis1 [22]. FNN model represents the architectural configuration used in [20], using the same parameter values used in the proposed model, in addition to a decision consensus of 0.7 and the number of bootstrap replications in 16. The model was compared with the same test criteria set out in [3], where the number of replicates is 30 times for each algorithm. To apply the same test conditions, multilayer perceptron (MLP) [86], J.48 (J48) [87] and naïve Bayes (NB) [88], zero rule (ZR) [89] and random tree (RT) [90]. All algorithms were also used in the Weka software [85]. Configurations of the models present in the weka also followed the same assumptions of the tests defined by [3].

To perform the training, 30 repetitions performed with the samples made available through the immunotherapy and cryotherapy treatments. Percentage is defined as 70% of the samples allocated for training and the remaining 30% for the test phase of the model. To avoid trends in the characteristics of each of the examples, a proposal made where all the samples destined to the training and testing of the fuzzy neural network and all intelligent models involved in the test randomly sampled. This ensures that there will be no dependencies of the data stream for the model results. All samples involved in the test normalized with mean zero and variance 1. The activation functions of the third layer neuron are of the linear type for FNN and defined by Weka in the initial configuration of the algorithms in the tool. For fuzzy neural networks using Gaussian membership functions, many membership functions (M) were defined as 3 and 5 by the 10-k-fold process (for the interval of M = (2, 3, 4, 5, 6, 7, 8, 9) seeking to maximize training accuracy).

The result of the cross-validation test evaluated the best value of accuracy, selecting the best value of M. Outputs of the model were normalized to −1 and 1 to aid the correct calculations. Factors evaluated in this paper are as follows:

$$accuracy=\frac{TP+TN}{TP+FN+TN+FP},$$

(9)

$$sensitivity=\frac{TP}{TP+FN},$$

(10)

$$specificity=\frac{TN}{TN+TP},$$

(11)

$$AUC=\frac{1}{2}(sensitivity+specificity),$$

(12)

where, $TP=$ true positive, $TN=$ true negative, $FN=$ false negative and $FP=$ false positive.

The values in parentheses are the standard deviations. Also, time collected in the tables is provided in seconds and involves the sum of the training time and testing of the models. Simulations and performed on a Core (TM) 2 Duo CPU, 2.27 GHz with 3-GB RAM.

5. Discussion

In the evaluation of the results of the immunotherapy test, the model had the highest accuracy when compared to the other elements in the analysis. It also obtained the best indices of specificity, which tells us that the model is very good at identifying people who will not be successful with the treatment. This factor is very relevant because it would prevent a person from undergoing an expensive and embarrassing treatment. Failure of the model, like that of most tests, is in sensitivity. Although the model did not reach 40% specificity, it was in the mean of the comparison with the other elements. As a result, AUC was a bit unbalanced. Another highlight is the model time. It spent little more than 1 s to build a complex set of rules. Compared with other models in the literature, their time was longer, but the benefit of making an information base for another computerized system is an excellent value for the test.

We can see in

Table 1. Results in the immunotherapy database.

Models	Accuracy	AUC	Sens.	Spec.	Time
This paper	84.32 (5.21)	0.69 (0.01)	0.41 (0.12)	0.97 (0.03)	1.11 (0.06)
FNN	81.91 (7.64)	0.65 (0.01)	0.36 (0.24)	0.94 (0.02)	17.39 (1.03)
MLP	78.02 (7.44)	0.74 (0.12)	0.60 (0.21)	0.88 (0.10)	1.18 (0.08)
J48	83.92 (4.69)	0.71 (0.03)	0.52 (0.20)	0.91 (0.03)	0.01 (0.00)
NB	76.67 (6.55)	0.69 (0.13)	0.51 (0.18)	0.87 (0.13)	0.01 (0.00)
ZR	79.13 (1.42)	0.50 (0.00)	0.50 (0.00)	0.50 (0.00)	0.01 (0.00)
RT	81.24 (7.56)	0.74 (0.10)	0.54 (0.07)	0.94 (0.06)	0.21 (0.01)

that the accuracy of the model J48 is statistically similar to that of the fuzzy neural network presenting less time for its execution. Same happens in

Table 2. Results in cryotherapy database.

Models	Accuracy	AUC	Sens.	Spec.	Time
This paper	88.64 (5.83)	0.89 (0.05)	0.93 (0.08)	0.86 (0.08)	1.04 (0.08)
FNN	85.75 (8.08)	0.85 (0.05)	0.90 (0.16)	0.80 (0.11)	22.78 (2.11)
MLP	86.17 (7.91)	0.91 (0.05)	0.92 (0.06)	0.90 (0.05)	1.05 (0.02)
J48	85.91 (6.42)	0.89 (0.05)	0.90 (0.09)	0.88 (0.02)	0.02 (0.01)
NB	85.67 (6.18)	0.95 (0.05)	0.90 (0.08)	1.00 (0.00)	0.52 (0.06)
ZR	53.64 (1.33)	0.50 (0.00)	0.50 (0.00)	0.50 (0.00)	0.05 (0.00)
RT	87.27 (7.98)	0.87 (0.01)	0.84 (0.02)	0.90 (0.01)	0.21 (0.01)

with other algorithms. However, it should be noted that even with high time compared to neural network models, decision trees and rules-based algorithms, FNN stands out because it can extract knowledge from the database, giving meaning to the answers obtained. Execution time of the algorithm was higher than the other algorithms due to the use of fuzzification techniques, which makes possible the interpretability of the results. This extraction of knowledge can assist physicians in diagnosis, since like any intelligent model; it acts dynamically in aid of diagnoses, allowing greater dynamics in the care of patients due to a solid knowledge base that can guide it in its definitions. A primary example is if the smart system informs that cryotherapy treatment is likely to not work for young clients, the doctor may be concerned to investigate other parameters involved in their research to determine whether or not to use the treatment.

For the cryotherapy tests, the model also obtained the highest average amount of accuracy, addition to maintaining a more stable model during the test, getting a lower value of standard deviation when compared to the other models. In this test, the model proposed in this paper was carried out in a more balanced way between sensitivity and specificity, is the one that had the best sensitivity index in the test. Do not get the best runtime, considered an advance for the techniques of fuzzy neural networks to construct a model of fuzzy rules compact with a time of execution very close to the other intelligent models. Known that depending on the nature of the problem, FNNs take a very long time to obtain responses with acceptable levels of response. Therefore, we can conclude that for both the immunotherapy and cryotherapy bases the proposed model can help in the construction of expert systems to assist in the treatment of warts with high indexes of accuracy compared to other models in the literature. In the comparison between fuzzy neural networks, main highlight is the difference in execution time in the comparison between pruning and regularization techniques, so there is a highlight to improve the performance of the model proposed in this paper to solve the problem.

The fuzzy neural network has very high relevance for the construction of expert systems. Can extract information that many researchers would have difficulty identifying possible patterns. The network formed by the rules generated in the ANFIS model [76] with the immunotherapy database is presented in Figure 7. It is possible to verify the connections between rules, allowing an evaluation of the complexity of the problem.

These systems generate partitions that represent knowledge about the problem. Figure 8 identifies the relationships between bases and rules and next presents examples of rules generated for the construction of if/then fuzzy rules systems.

Table 3. Literal fuzzy rules.

1. If (SEX is lndifferent) and (AGE is New) and (TIME is Righ) and (NumberWarts is Medium) and

(TYPE is Two) and (AREA is Big) and (DIAMETER is Hard) then (TREATMENT is effective)(1)

2. If (SEX is Male) and (AGE is Medium) and (TIME is Medium) and (NumberWarts is Elevated) and

(TYPE is Two) and (AREA is Medium) and (DIAMETER is Hard) then (TREATMENT is non-effective) (1)

3. If (SEX is Male) and (AGE is New) and (TIME is Little) and (NumberWarts is Elevated) and

(TYPE is Two) and (AREA is Medium) and (DIAMETER is Medium) then (TREATMENT is non-effective) (1)

4. If (SEX is Female) and (AGE is Old) and (TIME is Medium) and (NumberWarts is Few) and

(TYPE is Two) and (AREA is Big) and (DIAMETER is Small) then (TREATMENT is effective)(1)

5. If (SEX is Male) and (AGE is Old) and (TIME is Medium) and (NumberWarts is Medium) and

(TYPE is Two) and (AREA is Medium) and (DIAMETER is Small) then (TREATMENT is effective)(1))

presents fuzzy rules extracted from the problem.

It can see that the success of the treatment has a solid relationship with induration diameter. The larger the diameter and if the amount of warts stays close to 3, treatment by immunotherapy has excellent chances of being efficient. Figure 9 presents this relation graphically in space.

Another inference that can be verified within the assessed data set is that as the relationship between the type of wart and the person’s age is established as follows: when the type of wart is type 2, and the age of the person is average, chances of success in treatment are more significant than if the person is old and the type 1 wart. Figure 10 shows the graphical evaluation, where the values of the axes are based on the membership functions and not on the range of real values.

This type of approach also worked in cryotherapy treatments. However, they follow the same logic of graphical composition and composition of fuzzy rules. So many analyses done within a single answer obtained. Therefore, it becomes feasible to use this type of approach to building expert systems to identify possible success in combating warts.

6. Conclusions

After the tests of pattern classification, pruned fuzzy neural network can act in a dynamic, efficient and straightforward way in the construction of expert systems to assist in the evaluation of cryotherapy and immunotherapy treatments with high success rates. Comparison with the models presented in the analysis and the evaluation of the generated fuzzy rules allow affirming that the hybrid model is a great approach to solve these problems.

This can help doctors work with more tools to define proper treatment for their patients. This paper also facilitates the dynamics of building expert systems based on if/then rules. This type of approach done by software development teams where rules should serve as business rules for making systems that can be embedded into mobile devices or web applications. This paper has also demonstrated that hybrid systems based on neural networks and fuzzy systems can work in synergy to obtain significant results in the pattern classification and at the same time perform them with mastery the extraction of knowledge from a database.

The advantage of the proposed method is the application of a hybrid technique capable of extracting knowledge from the database and improving the classification capacity of problems related to immunotherapy and cryotherapy.

As for disadvantages, it stands out the high time, compared to traditional neural networks models. Limitations of the model related to the characteristics of the database, which should be exclusively numerical.

Future work performed to parameterize the network better, choosing other intervals and intelligent techniques for the definition of fuzzy neural network parameters. Other pruning techniques, other smart models may also be the target of this work expansion.

Finally, expect that the use of fuzzy neural networks may contribute to the creation of expert systems based on fuzzy rules connected to the database to feed the problem with new evaluations.