An Aggregation Metric Based on Partitioning and Consensus for Asymmetric Distributions in Likert Scale Responses

Abstract

A questionnaire is a basic tool for collecting information in survey research. Often, these questions are measured using a Likert scale. With multiple items on the same broad object, these codes could be summed or averaged to give an indication of each respondent’s overall positive or negative orientation towards that object. This is the basis for Likert scales. Aggregation methods have been widely used in different research areas. Most of them are mathematical methods, such as the arithmetic mean, the weighted arithmetic mean, or the OWA (Ordered Weighted Averaging) operator. The usual presentation of Likert scale derived data are Mean. This paper presents a new approach to compute an aggregate value that represents Likert scale responses as a histogram adequate to treat better than Mean with asymmetric distributions. This method generates a set of partitions using an approach based on successive division. After every division, each partition is evaluated using a consensus measure and the one with the best value is then selected. Once the process of division has finished, the aggregate value is computed using the resulting partitions. Promising results have been obtained. Experiments show that our method is appropriate for distributions with large asymmetry and is not far from the behavior of the arithmetic mean for symmetric distributions. Overall, the article sheds light on the need to consider other presentations of Likert scale derived data beyond Mean more suitable for asymmetric distributions.

1. Introduction

The process of obtaining information from an investigation in some areas is usually achieved by designing and creating a survey. A survey is a quantitative technique used to obtain primary information and the basic tool for collecting information is provided by the questionnaire. It is used in research to obtain the information that is used to reach the conclusions about the treated problem. Questionnaires have been used in different areas, such as the social sciences [1,2,3], education [4,5], market studies [6], sports [7,8], accident prevention [9,10], augmented reality [11], clinical data [12,13], etc.

The information gathered from a questionnaire is represented using two types of variables: observable variables and latent variables. An observable variable can be defined as “a concept that can be directly observed and measured”, i.e., observable variables are “the directly measured proxy variables that contain the raw data” [14]. These variables are also called indicators (and also called items or manifest variables). In contrast, a latent variable is a theoretical concept that cannot be directly observed or measured, but can be inferred from observable variables or from other latent variables that are measured in the first instance from observable variables. A variable of this type is also called a construct, composite, or factor and can be understood as something “meaningful”. A construct, then, is inferred from one or more indicators, that is, a latent variable is calculable from some observable variable or variables. Thus, a multidimensional construct is a construct where “their indicators are themselves latent constructs” [15]. A multidimensional construct establishes a relationship among various latent constructs and allow researchers to match broad predictors with broad outcomes, increasing the explained variance [16]. An indicator of a multidimensional construct is called dimension and represents one aspect, clearly defined, of the content domain of the overarching construct. Indicator or item, factor, composite, construct and multidimensional construct are the names usually used in statistical analysis applied to social, political, and behavioral sciences.

In most cases, the data gathered from a questionnaire allows for obtaining valuable information from it. Statistical analysis or other analytic techniques such as fsQCA [17,18,19] are used for this purpose. Those methods usually need a process where the information is aggregated. This aggregation is especially useful in two main situations: (1) to conflate the responses to the different items composing a question (construct) [20] or (2) to combine the answers offered by the different respondents to a concrete question item.

The use of measures of central tendency such as the arithmetic mean is common. A measure of central location tries to represent with a single value the center or location of the distribution. A subject’s score on a factor is the sum of his or her responses to the items selected to define that factor (direct scores). This practice is equivalent to assigning weights of 1 to the items if they define the factor, or 0 if they do not [21,22]. When the factors are not defined by the same number of items (as is usual), it is recommended to calculate for each subject his/her mean on each factor (sum of his/her responses to the items of the factor divided by the number of items of the factor); in this way, the scores of each subject (or group means) on the different factors can be compared with each other [23]. These measurements have the problem that they are very sensitive to extreme values, which makes them not useful in asymmetric distributions. Furthermore, the opinions collected in a survey can generate an asymmetric distribution that will not be correctly reflected with a central tendency measure. Therefore, measures of central tendency are not the most appropriate for obtaining a value that represents the information collected in a survey. Taking into account these considerations studied in previous works of the authors in the field of Social Sciences [1,2,3], it is interesting to design a method to generate groups of opinions from the surveys. Thus, instead of representing the distribution by means of a measure of central tendency, we propose to represent it with a value closer to the most representative group. To do this, the groups of opinions are detected from the survey respondents, and each of these groups are represented by a single value when the aggregated value is computed. To obtain the appropriate groups of opinion, consensus measures are used, and, to detect such groups, domain partition techniques are applied.

This paper proposes a new aggregation method for questionnaires with closed-ended questions [24] in order to measure a given phenomenon under investigation. The proposed method can treat ordered bounded domains, such as an ordinal-polytomous scale [25]. It is based on a continuous division of the input space [26,27] that groups closer answers into partitions. To establish such clusters, the concept of entropy [28,29] and a distance measure are used. In order to test the validity of our proposal, the results obtained from experimentation are compared with the arithmetic mean using two synthetic and real databases.

1.1. Contributions of the Present Paper

Multiple-item scales are widely used in making hiring decisions, assessing student, customer, and employee satisfaction, conducting needs assessments and program evaluations, and in scientific research projects. Unfortunately, those who construct these scales often have little knowledge of how to effectively develop and evaluate them. When scales are not properly developed, they typically yield data that are unsuitable for their intended purpose.

The Likert scale aims primarily at solving a technical problem that has arisen in relation to the quantitative aspects of the study of social attitudes. Likert [30], in his original research work on the development of the scales that bear his name, assumes that, for experimental purposes, attitudes are distributed fairly normally. However, he recognizes that this assumption may not be correct, so he calls for future research to determine its correctness or incorrectness by further experiment. This paper responds to that call, proposing a solution in the calculation of Likert scales when the assumption of normality is not met, that is, when the distribution is asymmetric.

We present a new approach to compute an aggregate value that represents Likert scale responses as a histogram adequate to treat with asymmetric distributions. In addition, the operation of the our method is not far from the behavior of the arithmetic mean for symmetric distributions. There are different possibilities in the literature to aggregate information into a single value, and most of them are based on measures of central tendency that are not appropriate for asymmetric distributions.

The presented method is based on the use of successive divisions to generate partitions of the input space. Furthermore, the use of consensus has been proven to be an appropriate way to demonstrate the validity of a set of opinions. For this reason, consensus has been used to measure the goodness of the candidate partitions. More concretely, the approach of Tastle [29] used to measure a Likert scale is used in this paper. Our approach proposes a measure based on consensus and suggests the distances between the candidate partitions to select one of them. In addition, the way the candidate partitions are generated is another innovative contribution.

Finally, this paper proposes the use of a parameter $\alpha$ to control the partitioning rate. It takes values in the interval $[0,1]$; if $\alpha =1$, the parameter has no effect. In this way, the behavior of the algorithm can be controlled by the final user, something that is not possible in other existing aggregation functions.

1.2. Structure

This paper is organized as follows: Section 2 exposits the background of the treated issue. Then, Section 3 provides details on our proposal to compute the aggregate value. Section 4 tests the proposed method and presents a discussion about the results achieved. Lastly, Section 5 summarizes the final conclusions and future research.

2. Background

Aggregation methods have been widely used in different research areas. Most of them are mathematical methods, such as the arithmetic mean [31] or the weighted arithmetic mean, that allows indicating a weight for each component. The Ordered Weighted Averaging) operator is another method, introduced by Yager [32]. It is widely used in applied mathematics and fuzzy logic, and it can also be used to aggregate the information obtained from a questionnaire. For instance, He et al. [33] provide new aggregation operators for Extended Comparative Linguistic Expressions with Symbolic Translation (ELICIT) information by developing novel OWA based operators, such as the Induced OWA (IOWA) operator in order to avoid the OWA operator needs of reordering its arguments because ELICIT information does not have an inherent order due to its fuzzy representation. Pons-Vives et al. [34] provide an application of OWA operators to customer classification in hotels. These authors argue that the use of the OWA operator improves the performance of the classical K-means and reduces the number of convergence iterations. An associated collection of weights $W=(w_1,w_2,\dots ,w_n)$ lying in the unit interval are needed in order to compute an OWA operator of dimension n (Equation (1)).

$$F(a_1,\dots ,a_n)=\sum _{j=1}^n{w}_j·b_j$$

(1)

where $b_j$ is the jth largest in $(a_1,a_2,\cdots a_n)$ and ${\sum }_{i=1}^n{w}_i=1$.

Another category of methods, such as the one we present in this paper, are based on consensus [20,35,36]. A recent study of the aggregation methods based on consensus is presented in [20] and, now, some relevant papers about consensus will be described in order to introduce the reader to the treated issue.

Fedrizzi and Pasi [37] present a review of fuzzy logic-based approaches to model consensus. It is applied in the context of fuzzy Group Decision Making (GDM) and two different models are synthesized so as to model a consensus under individual fuzzy preferences. In that paper, OWA operators are used to come to a consensus. Parreiras et al. [38] presented a flexible consensus scheme that allows for capturing a consistent collective opinion from a group of experts. It is based on a linguistic hierarchical model, which is advantageous both from the human viewpoint and from the operational viewpoint. A moderator can intervene in the discussion process and the authors test the approach on a hypothetical enterprise strategy planning problem. Alcantud et al. [39] presented the concept of approval consensus measure (ACM) to measure the cohesiveness that the expression of dichotomous opinions conveys. The 2012 presidential elections in the USA were selected as a real scenario to test their approach. In [40], from a set of objects, a consensus partition containing a maximum number of joined or separated pairs in such set is established. Cultural Consensus Theory (CCT) is an approach to information pooling (aggregation and data fusion) and is used for measurement and inference in the social and behavioral sciences [41]. France and Batchelder [42] described research related to CCT where a clusterwise version of continuous cultural consensus analysis called CONSCLUS was designed based on k-means. Their CCT-Means algorithm is used to fit CONSCLUS and an algorithm similar to k-means is used to compute the clusters. The centroids of the clusters are calculated using the continuous CCT procedure and the algorithm gives a good performance on relatively large datasets. They also implemented an extension of CONSCLUS for fuzzy clustering using an alternating least squares extension to the C-means algorithm. In their experimentation, they showed how CONSCLUS could be used to analyze a set of online review data. To compute the topology of a public transport network, Fiori et al. [43] used a consensus clustering density-based approach, referred to as DeCoClu (Density Consensus Clustering). They infer the geographical locations of stops with GPS data and use a consensus clustering strategy based on a new distance function to compute the relative distances between points. Experiments were conducted on real-data collections provided by a public transport company, and showed the utility of their proposal. Plaia et al. define a consensus ranking in order to assign a class label or class ranking to each node in a decision tree [44]. Zhang and Li [45] develop two consensus-based TOPSIS-Sort-B (variation of the Technique for Order of Preference by Similarity to Ideal Solution, known as TOPSIS-Sort method) algorithms to deal with multi-criteria sorting in the context of group decision-making (MCS-GDM) problems. Then, the authors define the consensus measures and devise different feedback adjustment mechanisms and consensus reaching algorithms to help experts reach consensus by considering different needs for MCS-GDM problems. Thus, TOPSIS-Sort-B is presented as an improved version of TOPSIS-Sort for sorting problems in which boundary profiles should be determined. Other variations of the TOPSIS-Sort method have been developed such as TOPSIS-Sort-C, which should be used to address problems in which it is more appropriate to determine characteristic profiles [46]. For their part, Gai et al. [47] propose a consensus-trust driven framework of bidirectional interaction for social network large-group decision-making. The proposed consensus framework is applied to a blockchain platform selection problem in the supply chain to demonstrate the effectiveness and applicability of the model.

3. Materials and Methods

A questionnaire is composed of a group of m questions, represented as $Q=\{Q_1,Q_2\cdots$$Q_m\}$, where each question $Q_i$ can be seen as a group of $m_i$ items $Q_i=\{I_1^i,I_2^i\cdots I_{m_i}^i\}$, where each $I_j^i$ takes values in a concrete domain. The Likert scale [30] is frequently used as the domain to measure the responses in a questionnaire. For this reason, the proposed method is designed to be used with this scale.

In order to contextualize our proposal, now we study a concrete example based on the first question of the questionnaire using a 5-point Likert scale such as this: “1 = strongly disagree, 2 = disagree, 3 = have no idea, 4 = agree, 5 =  strongly agree” taken from [48] (

Table 1. Example of questionnaire with two questions.

Part I—The Planning Phase	1	2	3	4	5
Was the project score clear to the team members?
Did the team members actively participate in the estimation process?
Did you feel that your opinion was heard in the planning phase?
Did you state your opinion regarding the construction of the WBS?
Part II—The Execution Phase	1	2	3	4	5
Were the project meetings held at adequate intervals?
Were progress indicator models used?
Were changes in the project adequately controlled?

1 = strongly disagree, 2 = disagree, 3 = have no idea, 4 = agree, 5 = strongly agree.

). Suppose that nine surveyed completed this question with the values shown in

Table 2. Responses and aggregate values for the two questions of the questionnaire.

Surveyed	${\mathit{I}}_1^1$	${\mathit{I}}_2^1$	${\mathit{I}}_3^1$	${\mathit{I}}_4^1$	1st Level	2nd Level
1	4	3	5	4	$4.00$	$3.19$
2	4	2	4	3	$3.25$
3	5	4	4	4	$4.25$
4	1	4	4	3	$3.25$
5	2	1	2	1	$1.50$
6	1	2	4	3	$2.50$
7	5	4	4	3	$4.00$
8	5	4	3	3	$3.75$
9	5	1	1	2	$2.25$
col. mean	$3.67$	$2.78$	$3.44$	$2.89$	$3.19$

and that the arithmetic mean is used in this example to aggregate the responses. In Table 2, there are two levels of aggregation for each question: the first one is shown in the column “1st level” and represents the response of each surveyed to an item; the second one, column “2nd level”, is the result of the aggregation process applied to the values of the first level. In addition, the arithmetic mean of each answer is shown in the last row (col. mean).

Section 3.1 shows the algorithm in detail. The proposed approach is now detailed where a successive partition of the frequency histogram of the occurrences of the responses is built. Then, the initial histogram is split into two partitions, and then these two partitions are divided again and again until they cannot be divided any more (Section 3.2). The partition condition is a measure concretely designed with this aim (Section 3.3). Once the split has finished, the final aggregate value is computed (Section 3.4). Finally, an example is given in Section 3.5.

3.1. Detailed Algorithm of the Proposed Method

Algorithm 1 details the operation of our method. It computes the histogram without taking into account the items with a frequency equal to 0 (Lines 1–2). These elements are stored in $L_{proc}$ (list of processing). The output set containing the partitions is named $L_{out}$ (Line 3). A new element is taken from $L_{proc}$ (Line 5). After that, all possible partitions composed of two sets of responses are generated (Line 9 and Section 3.2) if this is possible (Line 6). Each partition is now evaluated using an evaluation function that allows for selecting the best partition (Line 10 and Section 3.3). A parameter $\alpha$ is used to control the way the partitions are made (Line 11). This is the only parameter used, and it allows for selecting the rate of partitioning, and its behavior will be observed in the experiments. The smaller $\alpha$ is, the greater the rate of partitioning becomes. If two new partitions are obtained, they are added to $L_{proc}$, and they will be processed in the next iteration of the loop (Lines 12–13); in other cases, the original partition is generated as output (Line 15). This process is repeated until no new partition can be obtained (Line 4). Lastly, the aggregate value is computed using the partition of the input space (Line 20 and Section 3.4). Another aggregation function must be selected if the input histogram has not been divided (Line 22).

Algorithm 1 Partitioning and consensus based method

1: $hist$ = CalculatingTheHistogram($X_1\cdots X_n$) 2: $L_{proc}$ = DeletingElementsWithFrequencyZero($hist$) 3: $L_{out}=\varnothing$ {Output list with the selected partitions} 4: while$L_{proc}\ne \varnothing$do 5:     e = $L_{proc}$.pop() {e is removed from $L_{proc}$ and is processed} 6:     if length(e)$==$1 then 7:         $L_{out}$.add(e) {e cannot be divided and is added to $L_{out}$} 8:     else 9:         $L_{parts}$ = GeneratingPartitions(e) {Section 3.2} 10:       $part$ = ObtainingBestPartition($L_{parts}$) {Section 3.3} 11:       if eval($part$) >$\alpha ·$cons(e) then 12:          $L_{proc}$.add($par{t}_1$) {Adding first partition of $part$} 13:          $L_{proc}$.add($par{t}_2$) {Adding second partition of $part$} 14:       else 15:          $L_{out}$.add(e) {e is not divided and is added to $L_{out}$} 16:       end if 17:     end if 18: end while 19: if length($L_{out}$)>1 then 20:     r = ComputingOutputValue($L_{out}$) (Section 3.4) 21: else 22:     $r=$ using another aggregation function, such as the arithmetic mean, OWA, etc. 23: end if

3.2. Generation of the Partitions

The selected partition is divided into two parts: Each part contains consecutive values of the original partition and all possible combinations are computed. If the input histogram has n values, then there exist $n-1$ possible partitions using this cutting strategy. For example, let e be the histogram of the column $I_1^1$ (Table 2) taking the values $\{\{1,\frac{2}{9}\},$$\{2,\frac{1}{9}\},\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$ representing nine responses using a 5-point Likert scale where item 1 has been chosen two times, item 2 only in one response, etc. There are three possible partitions since e contains four values.

Table 3. Partitions obtained for the histogram e.

Part	Part1	Part2
1	$\left\{\{1,\frac{2}{9}\}\right\}$	$\{\{2,\frac{1}{9}\},\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$
2	$\{\{1,\frac{2}{9}\},\{2,\frac{1}{9}\}\}$	$\{\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$
3	$\{\{1,\frac{2}{9}\},\{2,\frac{1}{9}\},\{4,\frac{2}{9}\}\}$	$\{5,\frac{4}{9}\}$

shows the partition $part$ divided into two partitions, referred to as $par{t}_1$ and $par{t}_2$.

3.3. Evaluating the Partitions

Now, the equations that define our proposal will be presented. Equation (2) is used to evaluate the partition $part$, taking values in the interval $[0,1]$. A larger evaluation value indicates a better partition:

$$eval\left(part\right)=cons\left(part\right)·distance\left(part\right)$$

(2)

The consensus of each component of a partition $part$ (Equation (3)) and the distance between the partitions $par{t}_1$ and $par{t}_2$ (Equation (5)) are considered. Equation (3) takes a value in the interval $[0,1]$ since the function $Cns$ offers an output value in this interval:

$$cons\left(part\right)=\left(\frac{|par{t}_1|}{\left|part\right|}·Cns\left(par{t}_1\right)+\frac{|par{t}_2|}{\left|part\right|}·Cns\left(par{t}_2\right)\right)$$

(3)

where $part$ is the partition composed of $par{t}_1$ and $par{t}_2$, and $|par{t}_i|$ is the width of $par{t}_i$.

Different measures of consensus have been proposed previously in the literature. The approach of Tastle and Wierman [29] has been selected since it is a measure designed specifically to measure the consensus of a standard Likert scale. Furthermore, this method is valid for measuring other scales of closed-ended questions. Tastle and Wierman proposed the use of the mean and standard deviation as measures of the dispersion along with the Shannon entropy (Equation (4)). In their proposal, the consensus of a partition with a single element is $1.0$, and several examples showing its behavior can be found in the original paper:

$$Cns\left(X\right)=1+\sum _{i=1}^n{p}_i·log\left(1-\frac{|X_i-\overline{X}|}{X_{max}-X_{min}}\right)$$

(4)

where $p_i$ is the probability associated with the distribution under consideration, $\overline{X}$ is the mean of X, $X_{max}-X_{min}$ is the width of X, and $\left|\right|$ is the absolute value.

As it is indicated above, a distance measure is defined between two partitions. In addition, Equation (5) takes a value in the interval $[0,1]$, and it fulfills that $rv\left(par{t}_2\right)>rv\left(par{t}_1\right)$ since each element in $par{t}_2$ is greater than each element in $par{t}_1$:

$$distance\left(part\right)=\left(\frac{rv\left(par{t}_2\right)-rv\left(par{t}_1\right)}{\left|Likert\right|}\right)$$

(5)

where $\left|Likert\right|$ is the width of the input Likert scale, and $rv\left(par{t}_i\right)$ returns the representative value for $par{t}_i$.

To compute the representative value of a partition function, $rv$ is used (Equation (6)).

$$rv\left(part\right)=\frac{\sum fact\left(f_i\right)·norm\left(f_i\right)}{\left|part\right|}$$

(6)

where $part=(\{j,f_j\},\{j+1,f_{j+1}\},\cdots ,\{j+|part|,f_{j+\left|part\right|}\}$, $\{j,f_j\})$ is a tuple where j is a Liker scale value, $f_j$ its frequency, and $norm\left(f_i\right)$ normalizes the value $f_i\in part$.

The idea is to detect the areas of maximum frequency concentration within the partition. For this reason, a method has been designed that reduces the small frequencies and maintains the highest ones (Equation (7)) by applying a factor to each of the frequencies in the sample. The frequencies that fall in the lower third of the highest frequency ($max\left(part\right)$) are multiplied by a factor of $\frac{1}{3}$, those that fall in the second third are multiplied by a factor of $\frac{2}{3}$, and finally those that fall in the upper third are not modified, so the factor is equal to 1:

$$fact\left(f_i\right)=\left\{\begin{array}{c}\frac{1}{3},f_i<max\left(part\right)·\frac{1}{3}\hfill \\ \frac{2}{3},f_i\ge max\left(part\right)·\frac{1}{3}and{f}_i<max\left(part\right)·\frac{2}{3}\hfill \\ 1,f_i\ge max\left(part\right)·\frac{2}{3}\hfill \end{array}\right.$$

(7)

where $part=(\{1,f_1\},\{2,f_2\},\cdots ,\left\{\right|Likert|,f_{\left|Likert\right|}\left\}\right)$ and $freq=\{f_1,\cdots f_{\left|Likert\right|}\}$.

The distribution obtained is then normalized and the mean, which is the representative value of the sample, is calculated. By reducing the lowest frequencies and keeping the highest ones, the new average approaches the areas of maximum frequency concentration.

The main idea of the approach is to split only when a new partition $part$ reaches a good consensus (Equation (3)) and when the two partitions in $part$ are far enough from each other (Equation (5)). In this case, we consider the best way of obtaining a more consistent partitioning. For instance, let $part$ be the partition selected in Section 3.2 (row 2 of Table 3): $par{t}_1=\{\{1,\frac{2}{9}\},\{2,\frac{1}{9}\}\}$ and $par{t}_2=\{\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$. First of all, Equation (4) is applied to calculate the consensus values, obtaining $0.9025$ and $0.6325$, respectively. Next, Equation (3) is computed: $\frac{|par{t}_1|}{\left|part\right|}·Cns\left(par{t}_1\right)+\frac{|par{t}_2|}{\left|part\right|}·Cns\left(par{t}_2\right)=$$\frac{1}{4}·0.9025+\frac{1}{4}·0.6325=0.7675$ where $\frac{|par{t}_1|}{\left|part\right|}=\frac{|2-1|}{|5-1|}=\frac{1}{4}$ and $\frac{|par{t}_2|}{\left|part\right|}=\frac{|5-4|}{|5-1|}=\frac{1}{4}$. Equation (5) computes the distance between $par{t}_1$ and $par{t}_2$, and the result is $0.7$. Lastly, the evaluation value is calculated using Equation (2): it is equal to $0.7675·0.7=0.5373$.

3.4. Calculating the Aggregate Value

The aggregate value is computed using Equation (8) once the best partition has been selected. This operation obtains the sum of the frequencies of each partition multiplied ($sumFreq\left(par{t}_i\right)$) by a representative value from the own partition ($rv\left(par{t}_i\right)$). Then, each partition is represented by a single value (Equation (6)):

$$v=\sum _{i=1}^{m}sumFreq\left(par{t}_i\right)·rv\left(par{t}_i\right)$$

(8)

with $L_{out}=\{par{t}_1,par{t}_2,\cdots ,par{t}_m\}$, $par{t}_i$ is a partition, and $rv\left(par{t}_i\right)$ is calculated using Equation (6).

Although, in Equation (8), rv is used, it can be substituted by other measures such as the mode or the median. The justification of why using the rv is related with our aim of representing the partition with a value that considers the zone that concentrates the maximum frequencies. As example, the partition given in Section 3.3 (row 2 of Table 3) is now used in order to illustrate the operation of Equation (8). $par{t}_1$ and $par{t}_2$ are represented by the values $1.25$ and $4.75$, respectively. Then, the aggregate value is computed in the following way:

$$\begin{gathered} (sumFreq\left(par{t}_1\right)·rv\left(par{t}_1\right))+(sumFreq\left(par{t}_2\right)·rv\left(par{t}_2\right))=((\frac{2}{9}+\frac{1}{9})·1.25)+ \\ ((\frac{2}{9}+\frac{4}{9})·4.75)=\frac{3}{9}·1.25+\frac{6}{9}·4.75=3.583 \end{gathered}$$

3.5. Example of the Operation of Algorithm 1

The operation of this algorithm, step by step, will now be shown using an example. The question used has nine items, and each one of them uses a 5-point Likert scale taking, for example, the following values $\{4,5,5,4,1,1,2,5,5\}$ have been taken from the column $I_1^1$ of Table 2. The value for $\alpha$ is $0.55$. Then,

• Line 1: The histogram $hist=\{\{1,\frac{2}{9}\},\{2,\frac{1}{9}\},\{3,\frac{0}{9}\},$$\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$ is obtained using the answers.
• Line 2: $L_{proc}$ is initialized with a set containing every single element, $\{\{1,\frac{2}{9}\},$$\{2,\frac{1}{9}\},$$\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$, where the value 3 is not added because its frequency of occurrence is equal to 0.
• Line 3: The list that contains the output partitions ($L_{out}$) is initialized to ∅.
• Line 4: The main loop finishes when there are no partitions to process, i.e., $L_{proc}$ is empty.
• Line 5: An element is taken and removed from $L_{proc}$; in this case, as there is only one, then $e=\{\{1,\frac{2}{9}\},$$\{2,\frac{1}{9}\},\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$.
• Lines 6–7: If e only contains one element, it is added to the output list $L_{out}$ (Line 7) since it can not be divided. This situation does not occur in this example.
• Lines 8–9: Otherwise, all the possible partitions of e are generated and stored in $L_{parts}$, which is the list that contains all the possible partitions as shown in

  Table 4. Partitions when $e=\{\{1,\frac{2}{9}\},$$\{2,\frac{1}{9}\},\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$.

  Part.	Part1	Part2	Eval
  $P_1$	$\left\{\{1,\frac{2}{9}\}\right\}$	$\{\{2,\frac{1}{9}\},\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$	$0.526$
  $P_2$	$\{\{1,\frac{2}{9}\},\{2,\frac{1}{9}\}\}$	$\{\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}$	$0.537$
  $P_3$	$\{\{1,\frac{2}{9}\},\{2,\frac{1}{9}\},\{4,\frac{2}{9}\}\}$	$\left\{\{5,\frac{4}{9}\}\right\}$	$0.403$

  (Section 3.2).
• Line 10: The best partition $part$ is now selected using Equation (2). The column “eval” in Table 4 shows the values obtained in this case. The best partition is $P_2$ since it reaches the greatest value ($0.537$).
• Lines 11–13: If the evaluation value of $part$ (Equation (2)) is greater than the consensus value of e, then the two components of $P_2$ ($par{t}_1$ and $par{t}_2$) are added to $L_{proc}$ (Lines 12 and 13); otherwise, $part$ is not split any more and is added to $L_{out}$ (Lines 14–15). In our example, the division is achieved because $part$ obtains a greater value ($distance\left(part\right)·cons\left(part\right)$$= 0.7·0.7675 = 0.5373$$= 0.537$) than the value obtained by e ($\alpha ·Cns\left(e\right)$$= 0.55·0.4627$$= 0.2544$), i.e., $L_{proc}=\{\{\{1,\frac{2}{9}\},\{2,\frac{1}{9}\}\},\{\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}\}$. By means of the parameter $\alpha$, the user can indicate the desired rate of split. In the next iteration of the main loop, the partitions in $L_{proc}$ are processed in the same way.
• Line 19: When the loop finishes, the selected partition takes the value $L_{out} = \{$$\{\{1,\frac{2}{9}\},\{2,\frac{1}{9}\}\},$$\{\{4,\frac{2}{9}\},\{5,\frac{4}{9}\}\}\}$ and the final aggregate value is equal to $3.583$ (Line 20).

4. Results

Two tests will be performed, one using two randomly generated synthetic samples, and the other using a real database. The synthetic datasets are used to test how the presented method creates the partitions and how the parameter $\alpha$ influences the creation of the partitions (Section 4.1). In order to test our proposal in a real dataset, a 2014 survey carried out by some of the authors of this work and used in different research publications of the field of Social Sciences [1,49] is taken as input of the experiments (Section 4.2).

4.1. Study of the Obtained Partitions and the Influence of the Parameter $\alpha$ in the Creation of the Partitions

To perform the tests, two datasets formed by 300 surveys have each been randomly generated using a 7-point Likert scale and a 9-point Likert scale, respectively. Each survey of the dataset is represented by a histogram generated randomly with the same probability for each of the items. The histogram is represented in the same way as in Section 3.2.

The obtained results are studied according to the symmetry of the sample. As a measure of symmetry, the Fisher’s coefficient [50] has been used being one of the most widely used measures of symmetry. In addition, for the study of symmetry, four intervals have been defined to categorize the symmetry of a sample, these are: “symmetric” ($Q_1=\left[\right|0|\dots |0.25\left|\right)$), “little symmetric” ($Q_2=\left[\right|0.25|\dots |0.50\left|\right)$), “very little symmetric” ($Q_3=\left[\right|0.50|\dots |0.75\left|\right)$), and “not at all symmetric” ($Q_4=\left[\right|0.75|\dots |\infty \left|\right)$). As can be seen, the negative symmetry value is converted to its same value as a positive for the calculation of the interval to which it belongs. Each dataset generated consists of 75 surveys with a symmetry value belonging to each one of the four intervals ($75·4=300$); in this way, it will be possible to study the results obtained using the same number of surveys for each one of the defined intervals. In order to evaluate the results, a representative value $v_{rep}$ has been defined to be used for comparison with the value calculated by our method and with the arithmetic mean (Equation (9)).

$$v_{rep}=\frac{\sum norm\left(f_i\right)}{n_i}\forall f_i\in part|f_i\ge 0.5·f_{max}$$

(9)

where $f_{max}$ is the maximum frequency value in $part$, $n_i$ is the number of elements of $part$ that fulfils $f_i\ge 0.5·f_{max}$, and $norm\left(f_i\right)$ normalizes the value $f_i$ considering the values $f_{max}\in part$ that fulfills $f_{max}\ge 0.5$.

Thus, only the highest frequencies are considered in order to find the greatest group of opinion.

For the study of the behavior of the parameter $\alpha$, the experiment has been repeated taking $\alpha$ the values: $0.1$, $0.2$, $0.3$, $0.4$, $0.5$, $0.6$, $0.7$, $0.8$, and $0.9$. The obtained results are shown in

Table 5. Study of the parameter $\alpha$ with a 7-Likert scale.

$\mathit{\alpha }$	Me.	Adq.	Asm.	Q1	Q2	Q3	Q4
$0.9$	Our	4 ($1.33\%$)	$0.63$	1 ($1.33\%$)	0 ($0.0\%$)	0 ($0.0\%$)	3 ($4.0\%$)
	A.M.	0 ($0.0\%$)	$0.0$	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	Equal	296 ($98.67\%$)	$0.44$	74 ($98.67\%$)	75 ($100.0\%$)	75 ($100.0\%$)	72 ($96.0\%$)
$0.8$	Our	42 ($14.0\%$)	$0.56$	7 ($9.33\%$)	6 ($8.0\%$)	11 ($14.67\%$)	18 ($24.0\%$)
	A.M.	8 ($2.67\%$)	$0.24$	3 ($4.0\%$)	5 ($6.67\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	Equal	250 ($83.33\%$)	$0.43$	65 ($86.67\%$)	64 ($85.33\%$)	64 ($85.33\%$)	57 ($76.0\%$)
$0.7$	Our	143 ($47.67\%$)	$0.62$	8 ($10.67\%$)	19 ($25.33\%$)	47 ($62.67\%$)	69 ($92.0\%$)
	A.M.	50 ($16.67\%$)	$0.18$	31 ($41.33\%$)	19 ($25.33\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	Equal	107 ($35.67\%$)	$0.34$	36 ($48.0\%$)	37 ($49.33\%$)	28 ($37.33\%$)	6 ($8.0\%$)
$0.6$	Our	197 ($65.67\%$)	$0.58$	14 ($18.67\%$)	35 ($46.67\%$)	73 ($97.33\%$)	75 ($100.0\%$)
	A.M.	78 ($26.0\%$)	$0.19$	47 ($62.67\%$)	30 ($40.0\%$)	1 ($1.33\%$)	0 ($0.0\%$)
	Equal	25 ($8.33\%$)	$0.19$	14 ($18.67\%$)	10 ($13.33\%$)	1 ($1.33\%$)	0 ($0.0\%$)
$0.5$	Our	195 ($65.0\%$)	$0.57$	17 ($22.67\%$)	34 ($45.33\%$)	70 ($93.33\%$)	74 ($98.67\%$)
	A.M.	82 ($27.33\%$)	$0.2$	51 ($68.0\%$)	29 ($38.67\%$)	1 ($1.33\%$)	1 ($1.33\%$)
	Equal	23 ($7.67\%$)	$0.28$	7 ($9.33\%$)	12 ($16.0\%$)	4 ($5.33\%$)	0 ($0.0\%$)
$0.4$	Our	186 ($62.0\%$)	$0.57$	19 ($25.33\%$)	29 ($38.67\%$)	65 ($86.67\%$)	73 ($97.33\%$)
	A.M.	76 ($25.33\%$)	$0.21$	44 ($58.67\%$)	27 ($36.0\%$)	3 ($4.0\%$)	2 ($2.67\%$)
	Equal	38 ($12.67\%$)	$0.28$	12 ($16.0\%$)	19 ($25.33\%$)	7 ($9.33\%$)	0 ($0.0\%$)
$0.3$	Our	149 ($49.67\%$)	$0.62$	9 ($12.0\%$)	18 ($24.0\%$)	52 ($69.33\%$)	70 ($93.33\%$)
	A.M.	65 ($21.67\%$)	$0.26$	32 ($42.67\%$)	23 ($30.67\%$)	6 ($8.0\%$)	4 ($5.33\%$)
	Equal	86 ($28.67\%$)	$0.28$	34 ($45.33\%$)	34 ($45.33\%$)	17 ($22.67\%$)	1 ($1.33\%$)
$0.2$	Our	75 ($25.0\%$)	$0.65$	3 ($4.0\%$)	10 ($13.33\%$)	23 ($30.67\%$)	39 ($52.0\%$)
	A.M.	29 ($9.67\%$)	$0.24$	15 ($20.0\%$)	9 ($12.0\%$)	4 ($5.33\%$)	1 ($1.33\%$)
	Equal	196 ($65.33\%$)	$0.4$	57 ($76.0\%$)	56 ($74.67\%$)	48 ($64.0\%$)	35 ($46.67\%$)
$0.1$	Our	0 ($0.0\%$)	$0.0$	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	A.M.	0 ($0.0\%$)	$0.0$	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	Equal	300 ($100.0\%$)	$0.45$	75 ($100.0\%$)	75 ($100.0\%$)	75 ($100.0\%$)	75 ($100.0\%$)

and

Table 6. Study of the parameter $\alpha$ with a 9-Likert scale.

$\mathit{\alpha }$	Me.	Adq.	Asm.	Q1	Q2	Q3	Q4
$0.9$	Our	8 ($2.67\%$)	$0.48$	2 ($2.67\%$)	2 ($2.67\%$)	1 ($1.33\%$)	3 ($4.0\%$)
	A.M.	1 ($0.33\%$)	$0.17$	1 ($1.33\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	Equal	291 ($97.0\%$)	$0.45$	72 ($96.0\%$)	73 ($97.33\%$)	74 ($98.67\%$)	72 ($96.0\%$)
$0.8$	Our	47 ($15.67\%$)	$0.53$	6 ($8.0\%$)	14 ($18.67\%$)	10 ($13.33\%$)	17 ($22.67\%$)
	A.M.	6 ($2.0\%$)	$0.17$	5 ($6.67\%$)	1 ($1.33\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	Equal	247 ($82.33\%$)	$0.44$	64 ($85.33\%$)	60 ($80.0\%$)	65 ($86.67\%$)	58 ($77.33\%$)
$0.7$	Our	150 ($50.0\%$)	$0.56$	13 ($17.33\%$)	33 ($44.0\%$)	51 ($68.0\%$)	53 ($70.67\%$)
	A.M.	34 ($11.33\%$)	$0.19$	23 ($30.67\%$)	10 ($13.33\%$)	0 ($0.0\%$)	1 ($1.33\%$)
	Equal	116 ($38.67\%$)	$0.39$	39 ($52.0\%$)	32 ($42.67\%$)	24 ($32.0\%$)	21 ($28.0\%$)
$0.6$	Our	230 ($76.67\%$)	$0.54$	25 ($33.33\%$)	57 ($76.0\%$)	74 ($98.67\%$)	74 ($98.67\%$)
	A.M.	61 ($20.33\%$)	$0.17$	44 ($58.67\%$)	16 ($21.33\%$)	0 ($0.0\%$)	1 ($1.33\%$)
	Equal	9 ($3.0\%$)	$0.19$	6 ($8.0\%$)	2 ($2.67\%$)	1 ($1.33\%$)	0 ($0.0\%$)
$0.5$	Our	227 ($75.67\%$)	$0.54$	24 ($32.0\%$)	58 ($77.33\%$)	72 ($96.0\%$)	73 ($97.33\%$)
	A.M.	69 ($23.0\%$)	$0.19$	48 ($64.0\%$)	17 ($22.67\%$)	2 ($2.67\%$)	2 ($2.67\%$)
	Equal	4 ($1.33\%$)	$0.23$	3 ($4.0\%$)	0 ($0.0\%$)	1 ($1.33\%$)	0 ($0.0\%$)
$0.4$	Our	222 ($74.0\%$)	$0.53$	26 ($34.67\%$)	55 ($73.33\%$)	69 ($92.0\%$)	72 ($96.0\%$)
	A.M.	71 ($23.67\%$)	$0.21$	46 ($61.33\%$)	18 ($24.0\%$)	4 ($5.33\%$)	3 ($4.0\%$)
	Equal	7 ($2.33\%$)	$0.32$	3 ($4.0\%$)	2 ($2.67\%$)	2 ($2.67\%$)	0 ($0.0\%$)
$0.3$	Our	202 ($67.33\%$)	$0.54$	24 ($32.0\%$)	46 ($61.33\%$)	62 ($82.67\%$)	70 ($93.33\%$)
	A.M.	80 ($26.67\%$)	$0.25$	44 ($58.67\%$)	23 ($30.67\%$)	9 ($12.0\%$)	4 ($5.33\%$)
	Equal	18 ($6.0\%$)	$0.33$	7 ($9.33\%$)	6 ($8.0\%$)	4 ($5.33\%$)	1 ($1.33\%$)
$0.2$	Our	143 ($47.67\%$)	$0.55$	16 ($21.33\%$)	31 ($41.33\%$)	45 ($60.0\%$)	51 ($68.0\%$)
	A.M.	77 ($25.67\%$)	$0.31$	36 ($48.0\%$)	19 ($25.33\%$)	13 ($17.33\%$)	9 ($12.0\%$)
	Equal	80 ($26.67\%$)	$0.42$	23 ($30.67\%$)	25 ($33.33\%$)	17 ($22.67\%$)	15 ($20.0\%$)
$0.1$	Our	0 ($0.0\%$)	$0.0$	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	A.M.	0 ($0.0\%$)	$0.0$	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	Equal	300 ($100.0\%$)	$0.45$	75 ($100.0\%$)	75 ($100.0\%$)	75 ($100.0\%$)	75 ($100.0\%$)

, respectively. In these tables, the two first columns show the $\alpha$ value and the used method (Me): our approach, the arithmetic mean, and a third field indicating when both methods obtain the same results. The third column named adequateness (Adq) indicates the percentage of times a method obtains the nearest value to the representative one, the average of the symmetry measures for which each method is the best (Asm), and the detailed percentages for each one of the symmetry values, columns $Q_1$, $Q_2$, $Q_3$, and $Q_4$. For instance, in Table 5, if $\alpha =0.6$, our proposal is the best for 197 samples, $65.67\%$, and the arithmetic mean is the best for 78 samples ($26.0\%$). In addition, the same aggregated value for both methods is obtained in 25 samples ($8.33\%$). The average of the symmetry measures where our method is the best is $0.58$ and, for the arithmetic mean, it is $0.19$. The last four columns show the number of surveys where the proposed method is the best for each of the defined intervals. For example, taking $Q_2$, our proposal is the most adequate 35 times ($46.67\%$); on 30 occasions ($40.0\%$), the arithmetic mean has been more suitable, and 10 times ($13.33\%$) the same result has been obtained.

Discussion

Next, the results obtained with the variation of $\alpha$, the values of asymmetry obtained, and the results for each interval of asymmetry are provided in detail.

On the basis of the results obtained, it can be concluded that the higher values of $\alpha$ obtain the worst results, Col Adq. of Table 5 and Table 6, and, as $\alpha$ drops in value, results improve. For $\alpha <0.5$, the results start to get worse and, with $\alpha =0.1$, the output value is the same as the one obtained with the arithmetic mean for every sample. The best results are obtained for $\alpha =0.6$, a $65.67\%$ with a 7-point Likert scale and a $76.67\%$ with a 9-point Likert scale. Results are better when using a 9-point Likert scale because there are more items and more combinations to perform the partitioning (Section 3.2).

The proposed method is the best with a high average asymmetry value (Table 5 and Table 6, col. $Asm$); more concretely, it reaches values between $0.56$ and $0.65$ for the 7-Likert scale test and values from $0.48$ to $0.56$ for the 9-Likert scale test, labeled as very little symmetry in both cases. The measures of asymmetry obtained by the arithmetic mean are much lower than this: the intervals $[0.18,0.26]$ and $[0.17,0.31]$ for 7-Likert and 9-Likert scale tests, respectively. In addition, the study of the intervals $Q_1\cdots Q_4$ corroborates these results since the adequateness of our proposal occurs when $0.2\le \alpha \le 0.7$ in the intervals with more asymmetry $Q_3$ and $Q_4$. In particular, when $\alpha =0.5$ and $\alpha =0.6$, in both tests, results greater than $95\%$ are obtained. For the $Q_2$ interval, our proposal is the best except for a 7-point Likert scale being $\alpha =0.3$. This shows that our method is suitable for asymmetric distributions (labeled as “little symmetric”, “very little symmetric”, and “not at all symmetric”), and it presents a similar behavior with respect to the arithmetic mean for symmetric distributions.

Table 7. Number of partitions using a 7-Likert scale.

	Number of Partitions (7-Points)
	0	1	2	3	4	5	6
0.9	296	4	0	0	0	0	0
0.8	250	50	0	0	0	0	0
0.7	106	192	2	0	0	0	0
0.6	9	249	42	0	0	0	0
0.5	0	165	117	18	0	0	0
0.4	0	76	111	111	2	0	0
0.3	0	5	39	131	125	0	0
0.2	0	0	0	1	79	205	15
0.1	0	0	0	0	0	0	300

and

Table 8. Number of partitions using a 9-Likert scale.

	Number of Partitions (9-Points)
	0	1	2	3	4	5	6	7	8
0.9	291	9	0	0	0	0	0	0	0
0.8	247	53	0	0	0	0	0	0	0
0.7	115	184	1	0	0	0	0	0	0
0.6	6	258	36	0	0	0	0	0	0
0.5	0	192	82	25	1	0	0	0	0
0.4	0	109	105	61	25	0	0	0	0
0.3	0	7	66	122	66	38	1	0	0
0.2	0	0	0	2	66	129	77	24	2
0.1	0	0	0	0	0	0	0	0	300

show the obtained results with respect to the partitioning rate. The rows indicate the $\alpha$ values and the cols the number of partitions, i.e., 0 indicates no division in the histogram, 1 indicates one division of the histogram, etc. With respect to the parameter $\alpha$, as it was expected (Line 11 in Algorithm 1), a greater value of $\alpha$ implies less partitioning. More concretely, when $\alpha =1$, there is no high rate of partitioning, but as $\alpha$ goes down, the rate of partitioning goes up (Table 7 and Table 8). When $\alpha \le 0.5$, partitions are made for all samples for both of the Likert scales tested. For very low $\alpha$, the partitioning ratio is very large, performing almost the maximum number of possible partitions.

In brief, on the basis of the experimentation carried out, an $\alpha$ value in the interval $[0.5\dots 0.6]$ could be a good value for the $\alpha$ parameter based on the test results. The tests show that this parameter can be used to choose the rate of partitioning. In addition, it is shown that our method is appropriate for distributions with “little symmetric”, “very little symmetric”, and “not at all symmetric”. Once the experimentation with synthetic data has been carried out, the next step is to use real data.

4.2. Real Dataset

In the field of Social Sciences, it is very typical to aggregate the value of several items. For example, in [1,49], the study conducted used a primary database that was acquired from a survey entitled “Spain Survey of Training and Dynamic Capabilities of the Firm” (STraDyCaF). This survey was both administrated and distributed between the months of May and December of 2014 using the web based tool, LimeSurvey Version 2.05+. This Web application is open source, and its specialty is the creation, distribution of surveys, and the population management too. This tool sent an e-mail to each one of the survey respondents (Senior Executives) containing a personalized link. The representative sample for the survey was Spanish companies all over the nation with a number of employees greater than 50.

Table 9. Technical data sheet. (^a) Source: DIRCE 2015 (Central Business Register, CBR or DIRCE in Spanish, on 1 January 2014).

Population scope (universe)	Spanish companies with 50 or more employees, in any sector except public administration, agricultural sector and activities of households and extraterritorial organizations and bodies
Geographical scope	All the national territory/ Spanish national territory
Sampling unit/ Unit of analysis	Firm
Population census ${}^{\mathrm{a}}$	22013
Effect size/ Statistical power	0.2301 (small)/ 0.8001 (Post-Hoc analysis, t test correlation point biserial model, one tail, $\alpha$ error probability of 0.05)
Sample size/ response rate	112 valid surveys/ 7.18
Sampling procedure	Simple random sampling without replacement
Confidence level	95%; z = 1.96; $p=q=0.50$; $\alpha =0.05$
Sampling error	$9.24\%$
Key respondents	Senior Executives
Date of fieldwork/data collected	Between May and December 2014

summarizes the technical details of the survey.

In [49], four variables were used to conduct the analysis with the method fsQCA: Training (CONDF), Organizational performance (DORG), Absorptive capacity (ACAP) and Innovation capacity (INN).

Table 10. Information of the variables used in [49].

Variable	Likert Scale	Dimensions No. and Their Name	Items No.
ACAP	7	4: acquisition, assimilation, transformation & exploitation	$14:3+4+4+3$
CONDF	7	1: Training	5
DORG	7	2: Economic performance & satisfaction performance.	$9:5+4$
INN	7	2: product innovation & process innovation	$9:5+4$

shows the characteristics of each one of the variables; more concretely, it shows the Likert scale selected, the 7-point because it is the most used in questionnaires [51,52,53], the number of underlying dimensions of the dependent variables, and the number of items. A greater level of detail of these data can be found in [1,49].

As it can be observed, it is needed to compute the representative value for each variable that is an aggregate value from a large number of items (Last col in Table 10). For instance, for variables DORG and INN, it is needed to compute the average of nine items. This fact drives us to consider a method to obtain the groups of opinion (consensus) in the items to be aggregated. This value would be closer to the group with the greatest frequency than measures of central tendency.

In this test, the aggregate value for each variable used in STraDyCaF has been computed, and a similar test as the one detailed in Section 4.1 has been completed. The values for $\alpha$ are $0.5$ and $0.6$ (the values that reach the best results in the tests detailed in Section 4.1).

Table 11. Symmetry distribution of the STraDyCaF dataset.

Var.	${\mathit{Q}}_1$	${\mathit{Q}}_2$	${\mathit{Q}}_3$	${\mathit{Q}}_4$
ACAP	96	15	1	0
CONDF	101	11	0	0
DORG	103	4	2	3
INN	104	8	0	0
Total	404 (90%)	38 (8%)	3 (1%)	3 (1%)

shows the interval symmetry distribution of the samples for each STraDyCaF dataset variable. As it can be observed, many samples have a symmetry catalogued as symmetric ($90\%$). The results are shown in

Table 12. Obtained results to variable DORG of the STraDyCaF dataset.

Var.	$\mathit{\alpha }$	Me.	N.V.	A.M.	${\mathit{Q}}_1$	${\mathit{Q}}_2$	${\mathit{Q}}_3$	${\mathit{Q}}_4$
ACAP	$0.5$	Our	28 ($25.0\%$)	$0.21$	18 ($18.75\%$)	9 ($60.0\%$)	1 ($100.0\%$)	0 ($0.0\%$)
		A.M.	40 ($35.71\%$)	$0.12$	34 ($35.42\%$)	6 ($40.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		Equal	44 ($39.29\%$)	$0.07$	44 ($45.83\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	$0.6$	Our	27 ($24.11\%$)	$0.21$	17 ($17.71\%$)	9 ($60.0\%$)	1 ($100.0\%$)	0 ($0.0\%$)
		A.M.	41 ($36.61\%$)	$0.12$	35 ($36.46\%$)	6 ($40.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		Equal	44 ($39.29\%$)	$0.07$	44 ($45.83\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
CONDF	$0.5$	Our	5 ($4.46\%$)	$0.12$	5 ($4.95\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		A.M.	7 ($6.25\%$)	$0.15$	6 ($5.94\%$)	1 ($9.09\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		Equal	100 ($89.29\%$)	$0.11$	90 ($89.11\%$)	10 ($90.91\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	$0.6$	Our	7 ($6.25\%$)	$0.12$	7 ($6.93\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		A.M.	15 ($13.39\%$)	$0.2$	8 ($7.92\%$)	7 ($63.64\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		Equal	90 ($80.36\%$)	$0.1$	86 ($85.15\%$)	4 ($36.36\%$)	0 ($0.0\%$)	0 ($0.0\%$)
DORG	$0.5$	Our	14 ($12.5\%$)	$0.47$	7 ($6.8\%$)	2 ($50.0\%$)	2 ($100.0\%$)	3 ($100.0\%$)
		A.M.	28 ($25.0\%$)	$0.11$	26 ($25.24\%$)	2 ($50.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		Equal	70 ($62.5\%$)	$0.06$	70 ($67.96\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	$0.6$	Our	14 ($12.5\%$)	$0.47$	7 ($6.8\%$)	2 ($50.0\%$)	2 ($100.0\%$)	3 ($100.0\%$)
		A.M.	28 ($25.0\%$)	$0.11$	26 ($25.24\%$)	2 ($50.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		Equal	70 ($62.5\%$)	$0.06$	70 ($67.96\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
INN	$0.5$	Our	9 ($8.04\%$)	$0.27$	3 ($2.88\%$)	6 ($75.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		A.M.	29 ($25.89\%$)	$0.14$	27 ($25.96\%$)	2 ($25.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		Equal	74 ($66.07\%$)	$0.06$	74 ($71.15\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
	$0.6$	Our	9 ($8.04\%$)	$0.27$	3 ($2.88\%$)	6 ($75.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		A.M.	30 ($26.79\%$)	$0.14$	28 ($26.92\%$)	2 ($25.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		Equal	73 ($65.18\%$)	$0.06$	73 ($70.19\%$)	0 ($0.0\%$)	0 ($0.0\%$)	0 ($0.0\%$)
TOTAL		Our	113 ($12.61\%$)	$0.27$	67 ($8.30\%$)	34 ($44.75\%$)	6 ($100.0\%$)	6 ($100.0\%$)
		A.M.	218 ($24.33\%$)	$0.14$	190 ($23.51\%$)	28 ($36.84\%$)	0 ($0.0\%$)	0 ($0.0\%$)
		Equal	565 ($63.06\%$)	$0.07$	551 ($68.19\%$)	14 ($18.42\%$)	0 ($0.0\%$)	0 ($0.0\%$)

for the variables ACAP, CONDF, DORG, and INN, respectively. The last column shows the average of all the tests.

The first thing that is observed is that, in many cases ($63\%$), our method obtains the same aggregated value as the arithmetic mean because the samples are quite symmetrical in $90\%$ of the cases (Table 12). Our method surpasses the arithmetic mean in all samples with an asymmetry belonging to the intervals $Q_3$ and $Q_4$ (very little symmetric and not at all symmetric). For samples belonging to the $Q_2$ interval, our method exceeds the arithmetic mean in most of the samples ($44.75\%$ vs $36.84\%$). For the samples falling in the $Q_1$ interval, the value obtained is identical for both methods ($68.19\%$). In summary, it is observed that our method is appropriate for distributions with little asymmetry, very little asymmetry, or without symmetry, and is not far from the behavior of the arithmetic mean for symmetric distributions. It can be said that the experiments using the actual survey show a great similarity with respect to those detailed in Section 4.1.

5. Conclusions

A new method of aggregating the collected information which is more suitable for asymmetric distributions has been presented. This method generates a set of partitions using an approach based on successive divisions. It forms groups of closer answers represented as partitions using a measure based on consensus and a distance measure between the possible partitions. The method allows for selecting the rate of partitioning by means of a parameter $\alpha$; several experiments have been completed to test the behavior of such a parameter.

The structure of the algorithm allows for changing the evaluation function to consider other criteria for making the division of the input partition. The evaluation function calculates the distance between the possible partitions using a consensus measure and a distance measure between a unique representing value of each partition.

Experimentation aims to test how the method generates the partitions and the influence of the parameter $\alpha$ on the creation of these partitions on synthetic and real datasets. Promising results have been obtained, verifying that the aggregate value obtained by our approach is appropriate for asymmetric distributions obtaining similar results to the arithmetic mean for symmetric distributions.

As future work, we intend to investigate other evaluation functions of the partitions in order to obtain an added value that improves that obtained by the current method. In addition, the study of alternative consensus measures is another important research line to be explored.