We can then rank the results for the criteria importance (from more to less important) as Cr3 ≻ Cr2 ≻ Cr5 ≻ Cr1 ≻ Cr4 ≻ Cr7 ≻ Cr8 ≻ Cr6. By interpreting the ranked results, we can see that the most important factors are related to the phone’s performance (operational memory (Cr3), memory for storing documents (Cr2), processor sum of frequency (Cr5), price of the device (CR1), and battery capacity (CR4) for the longer operation of the phone). The three criteria describing the phone’s cameras are of lesser importance, i.e., the rear primary camera (Cr7) ≻ second rear camera (Cr8) ≻ front camera (Cr6).
3.1. Examination of the Stability Dependence of the Centroidous Method on the Distance Measure
If a mathematical model or method is resistant to changing its parameters, it can be used in practice [
2]. Statistical modelling with a sequence of random numbers from a given distribution is used to test the model [
42]. The stability of the Centroidous method is examined in this subsection in relation to the chosen distance.
Euclidean (3), which is conventionally used in the K-means clustering method, as well as Manhattan (6) and Chebyshev (7), distances are studied.
The formula for calculating the Manhattan distance is the sum of the absolute difference of the criteria vectors to the group centre of gravity
, as follows:
The Chebyshev distance determines the maximum values of the vectors by calculating the absolute difference between the criteria’s vectors and the group’s centre of gravity, as follows:
The stability of the method is investigated in the following way:
Step 1. Data processing: Dataset is read. The data are normalised using Formula (1), resulting in matrix .
Step 2. Using Formula (2), the centre of gravity, , of the group of criteria is calculated.
Step 3. Distances from the group centre to each criterion are calculated. The formulas Manhattan (6), Euclidean (3), and Chebyshev (7) are used to determine the distance. The result is the following three distance vectors: , , and .
Step 4. Next, applying Formulas (4) through (5) based on the obtained distances , , and , the weights of the criterion, , are determined. The following three vectors of criteria weights are created as a result: , , and .
Step 5. The stage of creating new matrices (, ) and calculating the weights of the criteria, . The matrix numbers change slightly, increasing or decreasing by q%, thus creating a new matrix .
Using a statistical modelling technique, quasi-random numbers for matrices are produced from a uniform distribution in the interval [] for each number. The stability is checked with a equal 5% и 10%.
Step 6. The actions outlined in stages 2 through 4 are then carried out in a repeated loop (). The previously obtained weight results are combined with new ones into matrices , , and .
The matrices will have records, and the first column will have the initial weights determined from the original data. The columns will match the weights of the created matrices.
Step 7. A quality assessment of the Centroidous method using Euclidean, Manhattan, and Chebyshev distance calculations is carried out. The estimate is based on the obtained values of the weights , , and , where ; . MRE, RRM-BR, and RRM-AR metrics are used in the model’s quality assessment process.
The stability of the proposed Centroidous method is tested on real data that describe mobile phones (
Table 1 and
Table 2). The calculations of method stability testing were performed in Python (version 3.10.12) using the Google Research Collaboratory.
The results of the criteria
,
, and
of weights established by the Centroidous method using Euclidean, Manhattan, and Chebyshev distances (Step 3) are presented in
Table 6. The smallest value of standard deviation (based on the entire population) of criterion weights was noted in the case of the Manhattan distance—0.019. The greatest difference between the criterion weights is observed when using the Euclidean (0.04) and Chebyshev (0.053) distances.
The weight of the Cr3 criterion is most significant when using the Euclidean and Manhattan distances. The ranked results of the weights for criteria Cr4 and Cr5 differ when these distances are used. Since the weights of the criteria Cr1, Cr2, Cr4, and Cr5 differ only by hundredths, it can be argued that the results for the Euclidean and Manhattan distances are very similar. In contrast, when using the Chebyshev distance, the results are significantly different; Cr3 ranks third, while Cr5 ranks first. When comparing the ranked results of the Chebyshev and Euclidean distances, the criteria Cr4 and Cr7 match.
After receiving the initial weights of the criteria (
Table 6), new matrices were created in a cycle s times, changing the initial matrix by
q% (steps 5 and 6).
Table 7 provides the results of the mean relative error (MRE) metric for 10 repetitions of the testing of stability of the Centroidous method using the Euclidean distance, with q = 5% and s = 100. The smaller the value of the MRE, the more reliable and stable the method is. Intervals of min-max values for all ten repetitions will be indicated in the future.
Table 8 provides MRE intervals when checking the stability of the Centroidous method for 100 iterations (q = 5%) using Euclidean, Manhattan, and Chebyshev distances. A comparison of the mean and maximum MRE values shows that the smallest error is when using the Manhattan distance, although the difference between the former and the Euclidean is negligible. The maximum MRE values when using the Chebyshev distance are almost twice as high as those for other distance measures.
We note that with a larger number of iterations, s = 10,000 (q = 5%), the MRE error interval narrows (
Table 9). The trend remained similar to
Table 8. The use of the Euclidean and Manhattan distances ensures a more stable behaviour of the method. The use of the Chebyshev distance significantly increases the MRE.
Next, the data change interval was increased to 10% (
Table 10 and
Table 11). During the comparison of the mean MRE values at s = 100 and q = 10% (
Table 10), with changes of 5% (
Table 8), the error increased by 49–55% when using the Euclidean distance, 51–52% when using the Manhattan distance, and 51–56% when using the Chebyshev distance. A comparison of the results of
Table 9 and
Table 11, at s = 10,000, shows that the average MRE error values increased by 51% across all distance measures.
From
Table 10 and
Table 11, we can see the difference in using the Manhattan and Euclidean distances in the Centroidous method more clearly. As the iterations increase, the error interval narrows. The average MRE error is smaller when using the Manhattan distance; this results in a more stable behaviour of the method.
After summarising the results of stability of the Centroidous method using different values of MRE distance, it can be noted that the Chebyshev distance showed high values of errors. During the testing of stability with a 5% q interval of change, the results of errors for the Euclidean and Manhattan distances are similar. After the data change interval q is increased to 10%, the lowest MRE was observed using the Manhattan distance.
Table 12 and
Table 13 use other metrics of the method’s quality. Higher values of the RRM-BR and RRM-AR metrics indicate better method stability. The RRM-BR metric records the repetition of the rank of the best criterion obtained from the initial data matrix (
Table 2).
According to the RRM-BR metrics (
Table 12), the method is more stable when using the Manhattan distance. A high stability rate is also observed when using the Euclidean distance, above 91%. The use of the Chebyshev distance showed poor results. The stability values of the method are higher with a smaller change in data q. As the verification iterations s increase to 10,000, the stability interval of the method changes by up to 3%.
The RRM-AR metric records the repetition of ranks of all criteria obtained from the initial data matrix. At q = 5%, the best stability result of the method is observed when using the Chebyshev distance. The stability of the Centroidous method when using the Euclidean and Manhattan distances is similar. At q = 10%, the method is more stable when using the Euclidean and Manhattan distances.
Based on the RRM-BR metric values, the Centroidous method is more stable when using the Manhattan and Euclidean distances. During the assessment of stability by repetition of all ranks of criteria (RRM-AR metric), the Chebyshev distance showed the best value at q = 5%. At q = 10%, the method is more stable when using the Euclidean and Manhattan distances.
3.2. Comparison of Centroidous with Other Methods for Calculating Objective Weights
In this study, Centroidous is compared with the following methods: entropy, criteria importance through intercriteria correlation (CRITIC), standard deviation (SD), mean, and the method based on the removal effects of criteria (MEREC).
The entropy method was proposed by Channon in 1948 within the framework of the information theory and is also used to determine objective criterion weights. The method evaluates the structure of the data array and its heterogeneity [
16]. According to information theory, the lower the information entropy of a criterion, the greater the amount of information the criterion represents, i.e., the greater the weight this criterion has. For the initial data array, sum normalisation is used. This normalisation does not provide the ability to convert individual negative numbers to positive values. The use of logarithm ln in the entropy method for negative numbers and zero was not determined. Therefore, the entropy method is limited in handling negative numbers and zeros in the original dataset.
The CRITIC method determines the weights of the criteria by analysing the contrast intensity and the conflicting character of the evaluation criteria [
3]. Accordingly, the standard deviation and the correlation between the criteria are calculated. The method uses min–max normalisation for the initial data array.
The SD method determines the weights of criteria based on their standard deviations [
3]. In order to obtain criterion weights, the values of criteria deviations are divided by the sum of these deviations.
In this study, the mean method determines the weights of the criteria based on their mean values. In order to obtain criterion weights, the average values of the criteria are divided by the sums of these values.
The MEREC method determines the importance of a criterion by temporarily excluding it and analysing changes in the results. The criterion that has a greater impact when excluded is determined as having a greater significance. The MEREC method uses a data transformation (like the SAW method) that determines the best value to be 1. If there are negative values in the maximising criterion or there are zeros in the initial dataset, this will limit the use of the MEREC method due to the uncertain values of the logarithm ln.
In order to compare methods for the determination of objective weights, a mobile phone dataset was used (
Table 1). The following criterion weight values were obtained with the use of the entropy, Centroidous, CRITIC, SD, mean, and MEREC methods (
Table 14). A graphic representation of the weights is presented in
Figure 2.
The determined weights of the criteria vary significantly (
Figure 2). Entropy assigned the highest weight to the Cr8 criterion (second rear camera), and MEREC identified this criterion as the second most important. In other methods, such as Centroidous, CRITIC, and mean, this criterion turned out to be second to last in importance, and the SD method identified Cr8 as the criterion with the least weight. Centroidous gave the Cr3 criterion (operational memory) the highest weight, the mean method identified it as the second most important, and the CRITIC and SD methods identified it as the fifth most important. The CRITIC and SD methods assigned the highest weight to the Cr6 criterion (front camera). The mean method assigned the greatest weight to the Cr4 criterion (battery capacity), while MEREC assigned it the least weight. MEREC assigned the highest weight to the Cr1 criterion (price). The SD method has the same values of criterion weights (C1 and C2).
The standard deviation of criterion weights (
Figure 3) shows the spread of values. The widest spread of criterion weight values is found in the entropy, CRITIC, and MEREC methods. The SD method has the smallest spread, and the weights differ little from each other (
Table 14).
Correlation values can indicate the similarity of the algorithms of the methods used.
Table 15 shows a high correlation between the MEREC and entropy methods (0.708), as well as CRITIC and SD methods (0.879). There is also a weak correlation of 0.29 between the mean and Centroidous methods.
Next, the weights of the criteria (
Table 14) were used to determine the best alternative, i.e., a mobile phone, by calculating with the use of the simple additive weighting (SAW) [
10] method. The values of the SAW method are presented in
Table 16, the ranked results of the evaluations are in
Table 17.
The best alternative, A1, is clearly determined using the weights obtained by all methods for the determination of objective weights (
Table 17). This was due to the fact that the maximising criteria evaluations of alternative A1 itself dominate over other alternatives. The entropy, CRITIC, SD, and mean methods placed the A5 alternative in second place. The ranked results of the Centroidous method are similar to those of the mean method.
In this example, each method showed its uniqueness. The ranked results completely matched only for the CRITIC and SD methods, the weights of which showed a high correlation.
Three sets of data were artificially generated (
Table 18) for a more thorough study of the behaviour of the methods. They reflect different problematic issues identified during data analysis. A linear relationship and high correlation were identified among criteria C1, C2, C3, and C4 (
Table 19) in the first generated dataset (
Table 18).
Next, the weights of the criteria are established (
Table 20) using the entropy, Centroidous, CRITIC, SD, mean, and MEREC methods with the use of artificially generated data-1. The entropy and MEREC methods assign the highest weight to criterion C5, the Centroidous and SD methods assign the highest weight to criterion C3, and the CRITIC and mean assign the highest weight to criterion C6 (
Figure 4). All criterion weights are clearly defined; there are no zero-weight criteria.
In order to trace the dependence of all weights of the criteria, we will analyse the correlation of the weights presented in
Table 21. A high correlation was found between the results of the entropy and MEREC methods—0.894. The average correlation was between entropy and CRITIC (0.656), Centroidous and SD (0.687), and CRITIC and mean (0.503).
The entropy, MEREC, and Centroidous methods had the largest standard deviation of criterion weights. The mean and SD methods had the smallest deviation between the weights (
Figure 5).
The second artificially generated dataset, data-2, has slight differences in the means of the alternatives (
Table 22). The maximum difference between the means of the alternatives is 1.43.
Table 23 shows the weights of the criteria established using the entropy, Centroidous, CRITIC, SD, mean, and MEREC methods using data array-2. The highest weight was assigned to criterion C3 using the entropy, CRITIC, and MEREC methods, and the Centroidous and mean methods assigned the highest weight to criterion C7. The mean and SD methods have the same criterion values (C1 and C2, C5 and C6).
Figure 6 clearly shows the dominance of criterion C4. The entropy and MEREC methods indicate this more accurately. The remaining criterion values are less prominent.
There is a strong correlation between the values of the criteria of the entropy and MEREC methods—0.871—and the CRITIC and SD methods—0.748. The correlation of the results between the Centroidous and SD methods is average (0.35), and the correlation between the CRITIC and entropy methods is weak (0.146) (see
Table 24).
A high value of the standard deviation of criterion weights is observed in the entropy, MEREC, and Centroidous methods (
Figure 7). These methods clearly define the weights of the criteria. The weights obtained using the SD method differ little from each other.
The third artificially generated data array, data-3, has large differences in the means of the alternatives (
Table 25). The maximum difference between the means of the alternatives is 672.43.
Table 26 shows the weights of the criteria established using the entropy, Centroidous, CRITIC, SD, mean, and MEREC methods using the data-3 array. The greatest weight is assigned to the C1 criterion by the entropy and MEREC methods. The other methods determined the greatest weight based on different criteria, as follows: Centroidous—C6, CRITIC—C4, SD—C5, mean—C2. All weights are defined precisely; there is no repetition of the values of the weights of criteria.
C1 and C5 criteria, determined using the entropy and MEREC methods, stand out in
Figure 8. The outline of the mean method is noticeable, as well. A large standard deviation is observed in the entropy and MEREC methods (
Figure 9). The other methods distributed the importance between the criteria fairly evenly when a difference in the means of the initial data (
Table 25) was large and differences in the standard deviations were not large (
Figure 9).
The analysis of correlations of weights for all criteria showed a strong dependence between the entropy and MEREC methods (0.992) and the Centroidous and mean methods (0.731). The average correlation was identified between the CRITIC and mean methods (0.438) and the SD and MEREC methods (0.399) (see
Table 27).
The entropy, Centroidous, CRITIC, SD, mean, and MEREC methods have shown their uniqueness when determining the criterion weights. Criterion weights are clearly identified in all datasets. A high correlation between the results of the entropy and MEREC methods (0.71, 0.84, 0.87, 0.99) was revealed in all examples. A high correlation was also observed for the CRITIC and SD methods (0.88, 0.75), and an average correlation was observed for the entropy and CRITIC methods (0.66, 0.15), CRITIC and mean methods (0.44), entropy and SD methods (0.5, 0.48), and SD and MEREC methods (0.4). There were high and average correlations between the results of the Centroidous and mean methods (0.731, 0.29) and the Centroidous and SD methods (0.69, 0.35).
The largest deviations in criterion weights are observed in the entropy, MEREC, Centroidous, and CRITIC methods. Weights with the same values may appear in the mean and SD methods in cases in which the average values of the initial data alternatives differ little from each other.