Next Article in Journal
A Channel-Wise Spatial-Temporal Aggregation Network for Action Recognition
Previous Article in Journal
An Extended Reweighted 1 Minimization Algorithm for Image Restoration
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A New Hybrid AHP and Dempster—Shafer Theory of Evidence Method for Project Risk Assessment Problem

by
Saad Muslet Albogami
,
Mohd Khairol Anuar Bin Mohd Ariffin
*,
Eris Elianddy Bin Supeni
and
Kamarul Arifin Ahmad
Department of Mechanical and Manufacturing Engineering, University Putra Malaysia, Serdang, SL 43400, Malaysia
*
Author to whom correspondence should be addressed.
Mathematics 2021, 9(24), 3225; https://doi.org/10.3390/math9243225
Submission received: 15 September 2021 / Revised: 4 November 2021 / Accepted: 11 November 2021 / Published: 14 December 2021

Abstract

:
In this paper, a new hybrid AHP and Dempster—Shafer theory of evidence is presented for solving the problem of choosing the best project among a list of available alternatives while uncertain risk factors are taken into account. The aim is to minimize overall risks. For this purpose, a three-phase framework is proposed. In the first phase, quantitative research was conducted to identify the risk factors that can influence a project. Then, a hybrid PCA-agglomerative unsupervised machine learning algorithm is proposed to classify the projects in terms of Properties, Operational and Technological, Financial, and Strategic risk factors. In the third step, a hybrid AHP and Dempster—Shafer theory of evidence is presented to select the best alternative with the lowest level of overall risks. As a result, four groups of risk factors, including Properties, Operational and Technological, Financial, and Strategic risk factors, are considered. Afterward, using an L2^4 Taguchi method, several experiments with various dimensions have been designed which are then solved by the proposed algorithm. The outcomes are then analyzed using the Validating Index, Reduced Risk Indicator, and Solving Time. The findings indicated that, compared to classic AHP, the results of the proposed hybrid method were different in most cases due to uncertainty of risk factors. It was observed that the method could be safely used for selecting project problems in real industries.

1. Introduction

The manufacturing industry plays a crucial role in the economy of each country [1]. Developed countries mostly have better industries and, therefore, investments in potential opportunities in the industries (manufacturing projects), accordingly. For example, Aбдикeeв et al. [2] reported that in 2017, 30%, 28%, 20%, and 13.5% of the annual gross domestic product (GDP) of China, South Korea, Germany, and Russia, respectively, belong to the manufacturing industry. Of this, a significant value belongs to construction projects for developing new industries or expanding available manufacturing firms. For instance, the value and number of net cross-border Mergers and Acquisitions (M&As) and announced greenfield global foreign direct investment (FDI) projects, 2008–2017 [3].
Given the rivalry of today, choosing appropriate decisions plays a crucial role in the success of a manufacturing firm. In most cases, choosing inappropriately will impose detrimental effects on a company or cause project failure. Risks are considered an inseparable part of a project and thus should not be ignored. Each year, many projects fail due to the harm that they impose on themselves. Risks attributed to projects can have various sources, but all have the same goal: project failure. Figure 1 depicts the correlations between risks associated with a project and the amount at stake throughout the lifecycle of a project. Figure 1 indicates the level of risks associated with each stage of the lifecycle of a project [4]. The level of the risks at the earlier phases of a project is significantly higher than in the ending phases. Such a fact can reveal the importance of risk management in project selection. Carvalho and Rabechini Junior [5] also mentioned a significant correlation between the level of risk management taken by a project team and project success.
In the project management body of knowledge (PMBOK), which is considered the essential guideline in the project management field, risk management is considered one of the nine areas of project management.
Therefore, minimizing the risks of a project is a vital need. Subsequently, the more attention paid to risk identification, the less risk will be faced during the lifecycle of a project. For this purpose, and as will be shown in the literature review, efforts have been made during the last two decades to propose various decision-making methods to investigate different risk management problems. Of these, a noticeable share belongs to the project selection problem. In addition, the shortcomings of current research methods will be investigated. In the literature review, the problem will be explained in more detail.
The main question is whether deterministic decision-making methods can satisfy all risk management needs in project selecting problems. Later, this question will be answered.
Considering the uncertainty in the decision science discipline challenges the use of classic methods. Many references have used evidence theory as a powerful method to consider the beliefs of different experts. Tacnet et al. [6] stated that experts have different opinions about factors due to imperfect information provided by more or less heterogeneous, reliable, and conflicting sources, which will affect the decision-making process. Kazimieras Zavadskas et al. also argued that people might have different meanings based on their knowledge, experience, and preference. Evidence theory is an effective way to consider such issues in the decision-making process by combining multi-criteria decision-making methods such as the Analytical Hierarchy Process (AHP) and the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) [7]. Within the literature, the hybrid AHP and evidence theory have been used many times to provide an adequate base for choosing the best alternative, while the beliefs of experts are essential [8,9,10,11]. While considering the term uncertainty, each cell in the AHP will not have a specific value necessarily, and instead, a range of values (confidence intervals) must be estimated by considering confidence levels (1–α). Therefore, an optimum solution will not necessarily exist and can be changed by considering different confidence intervals.
Using the facts mentioned above, there is ample evidence to consider uncertainty in selecting the best project among the available alternatives to minimize the risks of the project in the executing phase.
Therefore, in this research, a new decision-making model will be proposed to select the best alternative while uncertainties exist for the level of risk factors. The aim will be to minimize the risks associated with the capabilities of the company, such as available money, human resource skills, machinery, documentation systems, and quality control level.
This study can help industrial business owners to select appropriate projects according to their capabilities and strengths. Choosing the wrong project for construction can have detrimental effects on a business and impose irreparable financial harm. In each county, many failed projects can be found that remain unconstructed for years or even decades. The same phenomenon happens in the industry. A quick search in each industrial zone, a series of unsuccessful projects are left alone simply because the managers believe that the projects will not be successful and make a profit, even if they are successfully working in the future. This may also simply be because they made a wrong decision about choosing projects and calculating the required budget for completing them, due to not paying enough attention to selecting appropriate projects according to the money and resource availability.
In this research, a new method will be proposed to involve uncertainty of the available evidence in the process of selecting the best alternative among a list of available proposals of manufacturing projects, which will minimize the level of the risk factors associated with a project by utilizing scheduling and line balancing risk assessment.

2. Literature Review

More than one attribute is usually considered in multi-attribute decision-making methods in project scheduling (MADM) methods. The most preeminent advantage of MADMs, compared to multi-objective decision-making methods (MODM), is that there are easier to be understood by managers in actual practice. Moreover, their outcomes are more tangible and can easily be applied directly in real manufacturing systems. Methods like the AHP and TOPSIS are among the most critical MADMs. MADM methods can usually be hybrid with other methods to enhance their functionality. Several MADM references are investigated in this section, and essential questions regarding MADM in risk management will be answered. MADM methods are regularly used in project management problems. Kuo and Lu [12] proposed a fuzzy MCDM method for assessing risk factors associated with metropolitan construction projects. The proposed method used a multiple fuzzy attributes direct rating scheme to measure the occurrence probability of risk factors. Marcelino-Sádaba et al. [13] proposed a risk management method for project management in small manufacturing systems. The method is then applied to 72 companies located in Spain. Their method was based on checklists and other simple tools to measure indicators and outline the corrective actions.
Leu and Chang [14] argued that although many papers used classic methods while safety factors are considered, most of those methods could ineffectively address the correlation between dependencies of safety factors and occupational accidents. Hence, they proposed a new safety risk-assessment model based on Bayesian networks and fault tree concepts. Their outcomes show that the proposed method can effectively address the safety management of a project. Hossen et al. [15] used a construction schedule delay risk assessment method for nuclear power plants where a hybrid AHP and relative importance index and schedule delay risk are considered.
Kim [16] developed a new model for risk management, which is designed based on Bayesian rules. Their model used the pre-project cost risk assessment and actual performance data to portray a range of possible project costs at a pre-defined confidence level. Esmaeili et al., (2015) [17] presented an attribute-based risk identification and analysis method to improve available construction safety management methods, which help schedulers and designers realize, identify, and then model safety risk independently of specific areas activities or building components. Their method showed high performance when it was used to identify and measure several injuries and fatalities in construction resulting from a finite number of hazardous attributes of the work environment. In the same year, Esmaeili et al., (2015) [18] claimed that identifying and measuring risks associated with virtual construction work environments (which usually cause work injuries) is vital for pre-construction safety management. They proposed a two-stage model based on a principal component analysis of safety attributes and leading principal components that used potential predictors in a generalized linear model. The outcomes of applying their model indicated that giving identifiable characteristics of planned work could successfully forecast the probability of a safety incident.
Islam et al. [19] focused on using an analytical network process (ANP) on project risk management and argued that fuzzy ANP has limited functionality while incorporating new information into the risk structure. Therefore, they proposed a new fuzzy Bayesian belief network (FBBN) and showed its superiority to supersede the existing fuzzy ANPs. Valipour et al. [20] focused on occupational accidents reported in metropolitan excavations in significant cities. To investigate this case, they proposed new criteria for risk assessment by using adopted step-wise weight assessment ratio analysis (SWARA) and complex proportional assessment (COPRAS) methods. Then, using a field study, they found that construction safety, unfavorable geological conditions, shortage of managerial experience, preliminary emergency plan, and subsidence of ground are the most significant risks in excavation projects. Williams [21] declared that systematicity causes difficulties in evaluating risk levels of complex projects. They also mentioned that one important activity after identifying risk is to pursue its casual chain. They outlined the steps of analyzing the systemic nature of risk and how owners and constructors can fully understand the consequences of their actions.
  • Do the relations between human factors influence the risk levels in networks?
Fabricius and Büttgen [22] argued that the overconfidence of the manager directs them to project failure in many cases. Therefore, they used risk assessment to measure overall anticipated project success and how overconfidence will influence such assessments. Using data from 204 project managers, they outlined a standardized, case-based survey and proved that overconfidence reduces risk awareness among project managers, leading them to assess risks more optimistically and with more positive conclusions about anticipated project success. C. M. Wang et al. [23] addressed a new method for construction project managers to perceive risk. They investigated how the behavioral factors of project managers such as extraversion, agreeableness, and conscientiousness influence risk propensity and their implications to see if they differ in perceiving risk. They found that extraversion, agreeableness, and conscientiousness impose detrimental effects on risk perception.
Chemweno et al. [24] focused on implementing appropriate risk management on the health level of assets. They showed that choosing the proper risk management approach would positively affect maintenance decision-making by identifying, analyzing, evaluating, and mitigating equipment failures. For this purpose, they offered a new risk assessment using generic selection criteria for the failure mode and effects analysis (FMEA), fault tree analysis (FTA), and Bayesian networks (BN). In their method, the available criteria were prioritized using ANP.
Kokangül et al. [25] dealt with solving the health and safety problem in workspaces. For this purpose, they proposed a new risk assessment method, which relies on the Fine–Kinney method and AHP for a large-scale manufacturing company. Then the correlation between the Fine–Kinney risk assessment method and AHP was examined.
Identifying risks and evaluating them is a vital step in the early stage of a project. Yet et al. [26] addressed a hybrid dynamic Bayesian network modeling framework for analyzıng risk scenarios and budget policies in agriculture projects, where both uncertainty and variability of risk and economic factors were taken into account. Continuing, Yang et al. [27] argued that many risks associated with research and development (R&D) projects make them too sophisticated when standard methods are used to examine the performance of a project. Therefore, they proposed a predictive evaluation framework where a belief rule-based system with random subspaces was used in order to assess risks in R&D projects. They applied their model in a number of projects in China and showed that while using their method prominent results with prediction accuracy were achieved.
When considering more than one objective function is essential, MODM models in project management methods promise ways to overcome the difficulties of super-complex project management problems. Many project management problems are Np-hard by their nature, which means normal optimizers like LINGO and GAMS cannot quickly solve them. The complexity of such problems will be increased when more than one objective function is taken into consideration. This fact matches with the real circumstances of projects. In real projects, managers usually consider more than one objective at the same time while selecting a project. For example, they may want to determine which project will have more profit, have fewer costs, and impose fewer risks during the execution phase. Some of the most important references that used MODM techniques in the project solving problem will be reviewed.
Ansarifar et al. [28] focused on rapid responding (time) and cost of services objectives to find the optimum location for ambulance stations and helicopter ambulances. For this purpose, they proposed a new heuristic method to solve the developed multi-objective mode l.

2.1. Risk Management and Its Importance in Project Management

The Cambridge Dictionary defined the term risk as “the possibility of something bad happening” (https://dictionary.cambridge.org/dictionary/english/risk, accessed on 30 October 2020). This means that projects (similar to other industrial sectors) can suffer from the risks if risk factors are neglected.
Risk management is an essential part of PMBOK, which is an important reference in project management worldwide. In this reference, risk assessment is divided into five main parts: Risk Identification, Risk Evaluation, Risk Analysis, Risk Planning, and Risk Response.
Y. Zhang and Fan [29] proposed a model to integrate project cost, schedule, and quality to choose the risk response strategy selection problem. They declared that by finding the optimum solution, the most appropriate risk response strategies could be taken.
  • How to outline risks associated with a project?
Failure to pay enough attention to the risk factors of a project will impost negative effects on time, cost and quality of the project, states Fabricius and Büttgen [22]. It is worth knowing that risk identification has a significant impact on the future strategies of a company. Kliestik et al. [30] believed that the beliefs of top management and the economic environment are two major points that play significant roles in shaping the organizational social norm. Abd El-Karim et al. [31] focused on the added value of risk assessment, risk strategy, and plan analysis to the construction industry in Egypt. They aimed to identify and measure the effect of the factors that negatively influence time and cost contingency. Liu et al. [32] argued that incorrect investment decisions are the main root of many losses for the investors of a project. Although using quantitative risk assessment, which project owners frequently apply, can ameliorate such problems, classic risk assessment methods usually ignore assessing the effects of risk events, such as product sales falling short of expectations. Therefore, they proposed a modified version of the quantitative risk assessment model, enabling managers to outline the direct correlations between risk events and other decision variables in investing in a project. In this research we will use the risks that are classified by [33].
However, it is essential to know that the risk identification level depends on the nature of the project. While for some projects, it is essential to use complex techniques to understand the risks and the correlations between them, for others, more straightforward methods are more useful. For example, Bowers and Khorakian [34] stated that innovative projects are riskier to succeed due to their nature. Current risk management methods might have too stern a look for innovative projects, which may damage the creativity in an innovative project accordingly. To overcome such a barrier, a new framework was offered to use the generic innovation process in the risk management process of a project to outline a stage-gate innovation process model to provide an effective interface for incorporating project risk concepts.
  • Does the size of a company have an impact on choosing a risk management approach?
A critical point about risk management is the contribution of the size of a company with risks and risk assessment methods. Brustbauer [35] investigated risk management in small and medium-sized enterprises using a field study. Their outcomes indicated that using an active or a passive risk management approach has influenced the choosing of an offensive or defensive strategy for the studied cases, respectively. Besides, when the size of the firm came into consideration, the affiliation of the sector and the ownership structure would also influence the implementation of risk management.
  • What is the role of the complexity of the activities in a company in risk identification?
Fang et al. [36] stated that complexity usually causes barriers in identifying and assessing risks associated with a project. To overcome such difficulty, they used an important measuring technique in project risk management. The complex project risk network models and provides complementary analysis results, which are used to measure the interactions of risks. Tao et al. [37] stated that location and congestion of activities must be considered during the project schedule.

2.2. Risk Assessment Methods in Project Management

An in-depth review of the opted research studies showed that the most important methods are mathematical modeling, MADM, MODM, field study and statistical analysis, heuristic and meta-heuristics, and reviewing case studies (regardless of their priority). In addition, several critical research studies are shown where risk management is the main aim.
Gutjahr [38] used a branch and bound searching algorithm for a multi-objective scheduling method while minimizing project time, and costs are considered the main objectives of the model. Wu et al. [39] provided an in-depth review of tools and methods used by researchers for business intelligence risk management.
Risk can be defined as a measurable part of uncertainty (Dziadosz and Rejment, 2015), discuss [40], by considering the occurrence and severity of the damage. However, uncertainty is defined as “a situation, in which something is not known, or something that is not known or certain” (https://dictionary.cambridge.org/dictionary/english/uncertainty, accessed on 11 August 2020). Uncertainty can increase the harms of risk or increase its occurrence likelihood.

2.3. Uncertainty and Evidence Theory in Project Management

When risk management (including risk identification and risk assessments) comes into mind, one major shortcoming is that researchers consider instant values in their calculations in most cases. While in the real world, the risk factors and their identifiers can change for many reasons. For example, the chance of lack of money in a period (occurrence rate) could be entirely different from another period due to economic conditions. For example, Davari and Demeulemeester [41] dealt with the proactive and reactive resource-constrained project scheduling problem with stochastic activity durations. Grabovy and Orlov [42] developed a risk management method for considering uncertainty factors at all stages of implementing an investment construction project in Russia using a cross-border index for calculating an investment construction project. Besides, the level of intensity of a machine breakdown (severity) may vary. While oil leakage is a minor failure in most cases, it may cause harmful damage to an engine at another time. For instance, Nasrabadi and Mirzazadeh [43] focused on uncertain conditions and the time value of money. In such cases, using constant values for risk identification and assessment is a flawed strategy and can cause uselessness, and in some cases it can even mislead the decision makers. Such a drawback is even serious for projects in the industry.
In most cases, stubble cues that show a risk level will remain at the same level until the end of the project. In fact, in most cases, risks can emerge and become exacerbated in a short period of a project, and then they can be ameliorated by risk response programs (or worsen if they are left alone). For this purpose, we focus on the MADM and MODM techniques in project management considering the uncertainty.
Dempster introduced the theory of considering uncertainty in the probability of the decision-making process in the 1960s. Then, in 1976, Shafer published A Mathematical Theory of Evidence (Shafer, 1976). Their theory was developed to consider the uncertainty of mathematics arrays and followed fundamental but functional mathematics principles. Other scientists frequently apply their methods in various fields, including engineering, management, and the humanities. In addition, several important types of research that adopted the Dempster—Shafer theory of evidence will be investigated.
Using the Dempster—Shafer theory of evidence can overcome the dilemma between exact and probabilistic methods in expert systems. Zadeh (1986) [44] stated that the Dempster—Shafer theory of evidence has been widely used in AI for considering uncertainty in expert systems.
Tang [45] addressed a fuzzy soft set approach based on grey relational analysis and the Dempster—Shafer theory of evidence. The Dempster—Shafer theory of evidence was used to integrate the available alternatives into one collective alternative to choose the best alternatives. Hatefi et al. [46] used the Dempster—Shafer theory of evidence to develop a new model for assessing risk factors in a project associated with the environment. Their method was applied to an oil company in Iran, and the outcomes were compared with those achieved by conventional risk assessment and the fuzzy inference system methods, which showed the superiority of the proposed model in uncertain conditions of a project. Li et al. [47] discussed that most of the previously worked methods based on the fuzzy soft sets were based on different kinds of level soft sets, making them too sophisticated to investigate by decision makers. Therefore, they proposed a new fuzzy soft sets approach to combine grey relational analysis with the Dempster—Shafer theory of evidence in medical diagnosis problems. In their method, the Dempster—Shafer rule of evidence was used to aggregate the available alternatives into a collective alternative to select the best alternative.
J. Wang et al. [48] enhanced the functionality of the fuzzy soft set-based decision-making method by combining ambiguity measure and the Dempster—Shafer theory of evidence, which yielded less uncertainty and increased the choice decision level accordingly.
Ballent et al. [49] believed that the Dempster—Shafer theory of evidence could provide a basis for considering various expert beliefs where structural vulnerability and damage are examined, which results in subjective assessments. Muriana and Vizzini [50] stated that quantitative risk assessment is an efficient tool for fast decision-making. At the same time, progress variances from what was targeted before have adverse effects on a project risk profile. Thus, corrective and preventive actions must be defined based on the risk index to balance the risks. Niazi et al. [51] discussed that many software organizations do not pay enough attention to project management and risk assessment before starting global software development. For this purpose, they proposed a two-step approach to identify and analyze the 19 risks associated with global software development from the client and vendor points of view. Pan et al. [52] proposed new hybrid interval-valued fuzzy sets and improved the Dempster—Shafer evidence theory, as well as fuzzy Bayesian networks, for risk assessment and risk analyzing for sophisticated uncertain conditions. They showed that the proposed method could help reduce the likelihood of potential failure occurrence and ameliorate the risk magnitudes while a failure happened. Qazi et al. [53] addressed a new method for assessing risks by considering project complexity simultaneously. They found there is interdependency available between complexity drivers, risks, and objectives and their method was also able to make priority between complexity drivers, risks, and strategies.
Sangaiah et al. [54] proposed a hybrid approach for the risk assessment of software projects, including fuzzy Decision-Making Trial and Evaluation Laboratory, fuzzy MCDM, and MADM. Their method could provide more effective results compared to classic methods. Suresh and Dillibabu [55] focused on the risk assessment of software projects using a hybrid fuzzy-based machine learning mechanism that worked based on an adaptive neuro-fuzzy inference system-based multi-criteria decision-making and intuitionistic fuzzy-based TODIM (an acronym in Portuguese for interactive multi-criteria decision-making) approaches. Tonmoy et al. [56] dealt with coastal risks identification and evaluation in Australia. They found that informing and consulting stakeholders has positive impacts on planning for risk management. Zou et al. [57] stated that multi-disciplinary collaboration in risk management is necessary to achieve more success.
In most of the classic risk assessment methods, risks were usually analyzed separately. However, Y. Zhang [58] stated that correlations between risk factors of a project can influence project performance. Therefore, they proposed a new method for measuring risks interdependently, followed by an optimization model for selecting the best risk response strategies. Zavadskas et al. [59] proposed an MADM method for risk evaluation which worked based on the TOPSIS grey and COPRAS methods. Their main aim was to consider the goals of stakeholders along with other construction process efficiency and real estate value factors. After reviewing these papers, the following findings were achieved:
  • Considering risk factors in project management is vital, and during the last two decades scientists have focused on minimizing the risk factors associated with a project.
  • Uncertainty in occurrence probability and intensity should not be ignored and will impose detrimental effects on a project. Scientists have considered various aspects of uncertainty in their research.
  • When dealing with uncertainty in risk identification and assessments come into consideration, the Dempster—Shafer evidence theory provides a promising way to express and model the uncertainty.
  • In order to consider multiple attributes in evidence theory (when more than one attribute has to be addressed), evidence theory shows flexibility in combining with other decision-making methods. The hybrid methods have superiority compared to the standard decision-making methods.
  • In this research, considering the compatibility of AHP in choosing the best project and also the outstanding features of evidence theory in addressing uncertainty, a hybrid AHP evidence theory will be proposed to address the problem of selecting the best industrial project among the available alternatives in order to minimize the production risks associate to a project.
  • The outcomes of the comprehensive research completed in this section, using a hybrid AHP evidence theory for selecting industrial projects to minimize production risks, have not been addressed before.

3. Research Methodology

3.1. Flowchart of the Proposed Framework

Figure 2 shows the flowchart of the research methodology in more detail.
According to the research flowchart in the next section, the effective risk factors that can influence project selection in an uncertain environment will be identified first by quantitative research (Phase 1). Then, an unsupervised machine learning method will be applied (Phase 2) to filter the alternatives before entering the next phase. This section aims to classify the alternatives into different clusters so the top managers of the company can focus on excellent options more effectively. In the next phase (Phase 3), a hybrid AHP and Dempster—Shafer theory of evidence is presented to select the best alternative with the lowest level of overall risks. The method is designed so that the project with a lower total risk factor range will have more chance to be selected. The performance of the outcomes of the method will then be evaluated by using some metrics.
The advantages of novelties of the proposed method are provided in Section 3.5 after explaining the method in detail.

Methodology for Each Phase of the Proposed Method

The proposed method consisted of three steps where, in the first step, quantitative research was conducted to identify the risk factors that can influence a project. Then, a hybrid PCA-agglomerative unsupervised machine learning algorithm is proposed to classify the projects in terms of Properties, Operational and Technological, Financial, and Strategic risk factors. Then, in the third step, a hybrid AHP and Dempster—Shafer theory of evidence is presented to select the best alternative with the lowest level of overall risks. Figure 3 indicates the steps of the proposed method for choosing the best project.

3.2. Identify the Effective Risk Factors (Phase 1)

Kral et al. [60] used a questionnaire to figure out the experience of the manager about the criteria, which can influence the optimization of the project portfolio. A similar approach will be carried out in this research by finding the significant risk factors that can influence the success of a project. For this purpose, “Project Success” can be defined as the dependent variable. According to the questions of the research, the risk factors that can impose detrimental effects on the success of a project are categorized into the four main sections:
  • Properties Risk Factors (Infrastructure, Machinery, Human Resource)
  • Technology and Operational Risk Factors (Scheduling, Technology, Operational Risk, Management Systems)
  • Financial Risk Factors (Evaluating Projects, Profit and Costs, Money Value)
  • Strategic Risk Factor (Competition, Market Share, Marketing, Customer Satisfaction)
The main aim of this research is to find out if the above risk categories can influence the success of a project. If so, to what extent?
Therefore, the following variables are to be addressed in this research:
  • Dependent Variable: Project Success
  • Independent Variables: Table 1 indicates the independent variables according to their risk factor categories.
Since in this research one aim is to track the influence of a variable throughout the life cycle of a project, each of the above variables will be asked in three phases:
  • Before selecting a project
  • During the execution
  • After finishing the project
Using this strategy, finding the correlations between the independent variables and dependent variables can show us if a project is selected correctly or not. Moreover, and more importantly, do companies pay enough attention to such risk factors?
Table 2 shows the statistical analysis for the data that will be used in the next section:

3.3. Classify the Project Candidates Using a Hybrid PCA-Agglomerative Method (Phase 2)

3.3.1. Input Data

Table 3 shows the summary of the data gathered from the statistical society:

3.3.2. Libraries

In this research, Python is used to code the machine learning algorithm. For this purpose, the following libraries were used:
  • NumPy: NumPy is a Python library for generating and working with homogeneous multi-dimensional arrays. It is also used for applying basic mathematical formulas. These arrays are tables of elements (usually numbers) of the same type and are indexed by a few positive integers. In NumPy, dimensions are known as axes. The number of axes is called the rank.
  • Pandas: Pandas is the second library that will be used in this research. The main aim of using Pandas is to develop the data frameworks. With Pandas, it is possible to import data with different file types, such as CSV and XLSX.
    Pandas is also an excellent library to work with matrixes and perform various functions such as adding a row, deleting a column, multiplying two matrixes, etc.
  • Matplotlib: Matplotlib is used for drawing various plots, including histograms, box charts, bar charts, and scatter charts.
  • Seaborn: Seaborn is a library that contains powerful formulas for statistical analysis.
  • Scipy: Scipy is widely used for various purposes. However, in this research, Scipy will be applied for calculating the correlations between factors and optimization purposes.
  • scikit-learn: scikit-learn is an essential library for this research. This library will be used for applying supervised and unsupervised machine learning methods.

3.3.3. Selecting Features

In this section, the Ward algorithm is used for clustering datasets in an agglomerative way. However, before using the proposed method, a PCA must be used to reduce the size of the features to prepare them to be used by WARD:
  • X = ConsumersData.iloc[:,[5, 6, 7, 8]].values
Noted that 5, 6, 7 and 8 present the column number of dataset that will be used as features.
array 4.00 ,   3.26 ,   3.44 ,   3.08 , 3.78 ,   3.93 ,   4.00 ,   3.7 , 3.44 ,   3.60 ,   4.11 ,   3.20 , 2.78 ,   2.53 ,   1.89 ,   3.70 , 3.56 ,   3.27 ,   2.89 ,   3.50
p c a = P C A n _ c o m p o n e n t s = 2
Y = p c a . f i t _ t r a n s f o r m Y
w 2 . f i t Y
The outcomes will be shown as following:
array 0.92492625 ,   0.56134014 , 0.05083249 ,   0.01913977 , 0.5594823 ,   0.04059535 , 2.35135557 ,   0.75448191 , 1.2143782 ,   0.00885093
Afterward, the Ward method is applied for the clustering algorithm while the different number of clusters is taken into consideration (k = 2, 3, and 4):
w 2 = c l u s t e r . A g g l o m e r a t i v e C l u s t e r i n g n _ c l u s t e r s = 2 , l i n k a g e = w a r d . f i t X
w 3 = c l u s t e r . A g g l o m e r a t i v e C l u s t e r i n g n _ c l u s t e r s = 3 , l i n k a g e = w a r d . f i t X
w 4 = c l u s t e r . A g g l o m e r a t i v e C l u s t e r i n g n _ c l u s t e r s = 4 , l i n k a g e = w a r d . f i t X
w 5 = c l u s t e r . A g g l o m e r a t i v e C l u s t e r i n g n _ c l u s t e r s = 5 , l i n k a g e = w a r d . f i t X

3.3.4. Determining the Appropriate Number of Clusters

One way to help estimate the appropriate number of clusters (K) in unsupervised machine learning methods is to use a dendrogram. A dendrogram is an agglomerative method for clustering data. In a dendrogram, the correct number of clusters can be estimated by looking at long vertical lines. However, it should be noted that a dendrogram can be considered as a guideline, and the correct number of clusters should be estimated based on the scores observed after solving the unsupervised machine learning algorithm (such as Silhouette and Calinski–Harabasz).
A horizontal cut in Figure 4 where the vertical lines have long distances shows that the correct number of clusters could be two or three. The correct value for the number of clusters will be outlined using Silhouette and Calinski–Harabasz metrics.
After clustering the dataset using the proposed methods, the outcomes are outlined in Figure 5. It shows the clustering scatter chart based on the Ward method and a predefined number of clusters. As shown, while the number of clusters is considered three, the machine-learning algorithm could specify the clusters more precisely. In contrast, while the number of clusters was considered 4, the algorithm could not specify the border of clusters precisely.
Then, the score of the PCA-Ward method is measured with the different cluster numbers and shown in Figure 6, Figure 7, Figure 8, Figure 9, Figure 10 and Figure 11.
Therefore, according to the findings of the proposed PCA-Ward method, projects can be clustered into two main groups while Properties, Technologic and Operational, Financial, and Strategic risk factors are taken into consideration. Such a clustering approach will facilitate the process of pre-selecting the alternatives for the next stage.

3.4. Choosing the Best Alternative Using Multi-Attribute Decision-Making Method (Phase 3)

AHP is a decision problem divided into different levels of objectives, criteria, and sub-criteria to choose the best alternative amongst those available. In this process, different options are involved in decision-making, and it is possible to analyze the sensitivity of the criteria and sub-criteria. A sensitivity analysis based on the AHP method is a way to rank alternatives in terms of the pre-defined criteria. The decision maker can also weight criteria. However, one major shortcoming of classic AHP is that the values are considered constant in this method, and therefore, it cannot reflect the uncertainty of the responses of the experts in a selecting problem. Besides, the Dempster—Shafer theory of evidence is a robust method for considering the point of view of experts when uncertainty must be taken into consideration.
In the Dempster—Shafer theory of evidence, the level of belief of individuals in expressing their opinions is used. For example, not all survey participants necessarily answer questions with 100% certainty. In the real world, it is normal to answer a question with a level of uncertainty (α %). As a result, the belief rate of a question could be (1–α)%). Therefore, in this method, the degree of belief of individuals in answering each question plays a key role and is considered a function of belief. The belief function can be defined as a mathematical function, a range of values (for example, between 0 and 100), and even a quantitative or qualitative table.
For example, an expert can be given a range for risk (i.e., score (1–5)) with a confidence rate of 30%, and therefore, the system considers (1–9) with a confidence rate of 70% automatically. They can also give a single score for risk as well. For example, they can give 5 with a confidence rate (0.6). In this case, the algorithm considered (1–9) 0.5 automatically.
Thus, the Dempster—Shafer theory of evidence, which is often used as a method based on the degree of belief of individuals, is based on two principles: first, obtaining the degrees of belief of participants for possible answers to each of the questions, and second (the Dempster rule) to combine such degrees of belief when they are based on independent evidence.
Therefore, considering the aim of this research, which is choosing the best project amongst the available alternatives to minimize the risks, combining this method with evidence theory (that uses possibility, belief, and uncertainty functions), a new method based on machine learning is used where the alternatives will be filtered before they enter into the hybrid AHP and Dempster—Shafer theory of evidence method. Then, the best alternative will be selected considering the beliefs of the experts in terms of the mentioned risk group factors.
The solutions of the proposed hybrid AHP and Dempster—Shafer theory of evidence can be represented as follows:
  • Total Risk Matrix of Project (Number of Contract options, 2, Number of Projects)
When the first index of the above matrix indicates several available contract options, the second index is used for showing the upper and lower level of risk of each alternative (upper risk level, lower risk level), and the third index is used to show the number of alternatives.
The above matrix will indicate the risk levels of each alternative. Therefore, using the statistical probability method (as shown below), the best alternative, which contains the lowest risk domain, can be elected using the following formula:
  • AI = […]
The (Alternative Index) AI matrix shows the amount of 1/μ_i that is the index for showing the mean of the total risk factor values of an alternative. Greater values of AI are preferred. Afterwards, the best alternatives can be detected and represented.

3.5. Choosing an Alternative with the Lowest Risk Domain

After calculating the total value for risk factors as a domain (with upper and lower limits) for each alternative, it is time to select the alternative with the lowest total risk factor. However, choosing the best alternative in this research is not as easy as selecting the project with the lowest value, because here the total risk values are not exact numbers and it is not possible to easily select the minimum value.
To overcome such a problem, two main factors must be taken into account:
  • Mean of the total risk factor domain ( μ i )
  • The length of the total risk factor domain ( d i )
It is evident that an alternative with the lowest μ i is more desired because the related project achieved the lowest risk values (Figure 12). However, while the μ i for two alternatives are the same, the project with a smaller d i is preferred because, generally, it has a lower risk than the other option (Figure 13).
In order to develop statistical formulas (Equations (8)–(10)) for measuring the high confidence level (99.7%), the normal distribution function is used. It is in line with the central limit theorem, which was used in this research to gather data, and the mean of variables will follow the normal distribution function if the amount of gathered data is more than 30. As a result, the following formulas based on statistical quality control of normal distribution function will be developed:
U i μ i + 3 σ i
L i μ i 3 σ i
U i L i = 6 σ i
The reason for considering 6σ for measuring distance between the upper limit and lower limit is that control rules take advantage of the normal curve in which 99.73% of the data will be within plus or minus three standard deviations from the average.
Therefore, the CV index ( 6 σ i μ i ) can be considered as an appropriate index for comparing the total risk domains while the two alternatives have equal means ( μ i = μ i ).
C V = 6 σ i μ i
In statistics, σ i μ i is usually used instead of the 6 σ i μ i .
Therefore, all possible conditions must be taken into account.
(1)
When two alternatives have different total risk factor means ( μ i μ j ) and different total risk factor domain lengths ( d i d j ), as shown in Figure 14.
1 μ i 1 μ j       &     σ i μ i σ j μ j
A I i A I j       &     C V i C V j
Result: the project with the lower μ i will be selected (more significant = 1 μ i ).
(2)
When two alternatives have different total risk factor means ( μ i μ j ) but equal total risk factor domain lengths ( d i = d j ), as shown in Figure 15.
1 μ i 1 μ j       &     σ i μ i = σ j μ j
A I i A I j       &     C V i = C V j
Result: the project with the lower μ i will be selected (more significant A I = 1 μ ).
(3)
When two alternatives have equal total risk factor means ( μ i = μ j ) but different total risk factor domains ( d i d j ), as shown in Figure 16.
1 μ i = 1 μ j       &     σ i μ i σ j μ j
A I i = A I j       &     C V i C V j
Result: the projects have the same AI; therefore, the project with the lower CV is preferred.
(4)
When two alternatives have equal total risk factor means ( μ i = μ j ) and equal total risk factor domains ( d i = d j ), as shown in Figure 17.
1 μ i = 1 μ j       &     σ i μ i = σ j μ j
A I i = A I j       &     C V i = C V j
Result: projects have the same AI and CV. Both alternatives can be chosen.

3.6. Advantages and Novelties of Using the Proposed Hybrid Evidence Theory and AHP

The proposed hybrid evidence theory and AHP is a promising method for minimizing the uncertainty in project selection problems while negative factors such as job tardiness, work in process, bottleneck machines, and over-allocated machines are taken into consideration. In addition, compared to the mathematical modeling, the outcomes are more understandable for project managers in real industries.
Compared to classic AHP, the proposed hybrid AHP and Dempster—Shafer theory of evidence has many prominent features. Table 4 outlines the features of both mentioned methods:

3.7. Novelties and Innovation of the Proposed Method

In the following, to clarify the novelties of the proposed method, the most relevant and recent similar methods that have used the evidence theory and AHP will be presented and compared with the proposed method in this research (Table 5).
Although some shreds of evidence showed the evidence theory and AHP were used before for other problems, in this research a new version of evidence theory and AHP is proposed to filter the unsuitable alternatives using an unsupervised machine learning algorithm before selecting the best alternative. Moreover, the proposed algorithm is designed to track and compare the level of each risk factor group in different phases of a project, including before execution, during execution, and after finishing. Such an approach will make a base for a project selection portfolio of a company.

4. Results and Discussion

4.1. Verifying the Proposed Algorithm (Solving Experiments Gathered from the Literature)

In this section, several case studies will be solved to verify the functionality of the proposed algorithm in different ways conditions. For this purpose, an L2^4 Taguchi method is used to design experiments (DOE) using Minitab 18.0. The reason for choosing this type is that a lower limit and an upper limit are considered (2^4).
The experiments are designed to consider various conditions that potentially surround a company while choosing the best project. To design the experiments, the following levels for each factor of DOE are taken into account (Table 6).
The case studies are designed in a way that various ranges of parameters is taken into account. Therefore, the case studies are divided into three main categories used by many researchers in the literature review [62].
Table 7 shows the case studies that the proposed algorithm must perform. As shown, the domains of each of the case studies have been selected according to Table 7 to cover each scale (small, medium, large, and very large) entirely.
In this section, each of the case studies will be solved by the proposed algorithm in Matlab. The outcomes of the case studies are shown in Table 8.
However, in order to see the steps of the proposed algorithm, in reality, case study number two in Table 8 will be explained in detail.

4.2. Solving a Case Study and Explaining the Outcomes in Detail

In this section, the third case study in Table 8 will be explained in detail regarding the mechanism of the algorithm functionality.
Suppose a company has to select the best option between the available two alternatives. One is to set up a new production line, and the other is to set up a new laboratory, which can also provide outdoor services.
  • Number_of_alternatives = 2
There are two managers in this company that must determine which alternative is to be carried out in the future. However, the quote of the company share for the first manager is two times more than the other manager, and therefore his vote will value two times more than the other.
  • Number_of_experts = 2
  • Weight_of_experts = [0.3, 0.7]
In order to choose the best option, managers decided to consider three risk factors: financial, operational, and property. From the point of view of the manager at this time, the financial risk factor is more important than the other factors, and the operational risk factors are more important than the property risk factors. Therefore, they decided to set the following values between the risk factors:
  • Number_of_risks = 3
  • Weight_of_risks = [0.5, 0.3, 0.2]
Regardless of the project title, the company has two options for financing it. One is to pay the expenditures directly and the other is to obtain a bank loan. However, each contract option will influence the level of the risks.
  • Number_of_contract_options = 2
Afterward, the managers are asked to fill out a questionnaire to set values about each risk factor and their uncertainty about their solutions. The following matrix shows the first three experts opinions (Table 9):
The following results are obtained after solving the case study using the proposed hybrid AHP and theory of evidence.
Step (1) Calculating the “expected_value_for_risk” matrix using the opinions of the experts (Table 10):
The way for calculating the first element of the above matrix will be explained:
In order to calculate the above matrix, the lower and upper values for each risk factor must be calculated. Therefore, using the “for” loop, the idea for each expert will be gained. For example, for the first risk factor, the results will be as follows (Table 11):
Using the same strategy, the rest of the elements of the “expected value for risk” will be calculated (Table 12).
Step (2) Continuing, the mean will be calculated for each of the risks in each contract option (Table 13):
Step (3) Normalize the upper and lower risk factor values (Table 14):
Step (4) Calculating the average of the normalized risk matrix:
Then, using the following formulas, the average of the normalized lower and upper-risk factors will be calculated (Table 15).
Step (5) Calculating the total risk matrix:
The total lower and upper risk matrix will be calculated (Table 16).
At this point, the total lower and upper values for each alternative (using a specific contract option) are calculated. For example, while the first alternative is assumed to be carried out by the second contract option, the total risk domain will be [0.0366, 1.8211].
Step (6): Calculating the AI matrix and choosing the best alternative:
Now, and in the last step, the alternative with the lowest risk point must be selected. However, since the risk point is not an exact value but a domain, this means selecting the project with the lowest risk domain. To solve this problem, two factors must be taken into consideration:
  • The mean of a risk factor domain ( μ i ) where the index A I = 1 μ i is used for it as described in Section 3.
  • If two or more projects have the same μ i , then the length of the risk factor domain where σ i is used for it (using the C V = σ i μ i formula) as described in Section 3.
Therefore using the following formulas, the AI index will be calculated for each alternative.
Mu = 2.3394 ,   2.6542 , 1.3566 ,   1.5035
A I = 0.4275 ,   0.3768 , 0.7371 ,   0.6651
Then, the best option will be the alternative with the highest AI index value, which in this case study is the second project when the first contract option is selected (0.7371).
  • Best_alternative = 2
  • Best_contract option = 1

4.3. Measuring the Performance of the Proposed Algorithm

In order to assess the performance of the proposed method, several indicators are defined as shown below:
  • The ability to solve all problem types
  • The ability to choose projects with the lowest uncertainty
  • The solving time
  • Comparing the hybrid AHP and Dempster—Shafer theory of evidence with classic AHP
In addition, in the second part of this section, the outcomes of problems solved using the hybrid AHP and Dempster—Shafer theory of evidence will be compared with classic AHP to show the superiority of the proposed method in solving the problems while uncertainty exists.

4.3.1. The Ability to Solve All Problem Types

The results of 24 experiments solved by the proposed hybrid method showed that the algorithm could solve all experiments (100%) and show the best alternative with the average of the lowest risk factors.
V a l i d a t i n g . I n d e x = N u m b e r   o f   S o l v e d   C a s e s N u m b e r   o f   D e s i g n e d   C a s e s . 100 = 24 24   . 100 = 100 %
Therefore, the algorithm can be used in real project selection time by industries.

4.3.2. The Ability to Choose Projects with the Lowest Uncertainty

The outcomes of all solved case studies are revised again. In each case, the AI matrix is presented in Table 17, and the lowest risk factor reported by the proposed algorithm is double-checked. In all studied cases, the solving algorithm can find and report the project with the lowest uncertainty (highest AI).
R e d u c e d   R i s k   I n d i c a t o r = M a x   A I M i n   A I M a x   A I . 100
Reduced risk indicator (RRI) shows how much percentage using the proposed algorithm helps select the alternative with the lowest risk factor.
As shown in Figure 18, the algorithm can choose the alternative with the lowest risk value in all cases.
As shown, the solving algorithm solved the small case studies in less than 0.055 s, medium-scale cases in less than 0.09 s, and large-scale cases in less than 0.56 s (Figure 19). These results are noticeable and mean that the algorithm can be used safely in actual practice.

5. Conclusions and Recommendations

This research focused on the uncertainty in the industrial project selection problem. In the real environment, several risk factors threaten the success of the project. However, such risk factors are not constant and may take various values depending on the environment of the project. Therefore, classic decision-making methods may fail to correctly report the actual risk factors value and select the best project among the alternatives. In this research, many risk factors that influence project success are extracted using the Delphi method. The findings showed that the risks could be divided into four main risk clusters: Properties risk factors; Technology and Operational risk factors; Financial risk factors; and Strategic risk factors. In each of the risk factor clusters, several variables are defined. Each variable is asked in three phases of a project: before selecting a project, during execution of the project, and after completing the project.
The aim was to track the status of a variable in the life cycle of a project. After asking the opinion of the responder for each question, their belief rate was also asked to clarify the uncertainty of the risk factors. The statistical analysis is then carried out to specify the statistical description of the variable, find out the correlations between the variables, and determine their values in project success (as the dependent variable).
A new hybrid AHP and Dempster—Shafer theory of evidence is proposed, based on the uncertainty level of the risk factors. The proposed method could determine the total risk level range of each alternative, and then report the best alternative with the lowest total risk level range. Next, a Taguchi Method (L2^4) is designed for the experiments. The proposed method is used to solve 24 experiments where the condition of the experiments was different from one experiment to another.
The performance of the proposed algorithm is then evaluated using four indicators. The proposed method could solve all small, medium, and large-scale experiments (validating index). Moreover, it could find and report the project with the lowest total risk range in all cases. In order to check the performance of the proposed method in choosing projects with the lowest total risk factor, the maximum and minimum risk factors for available alternatives of each case study are compared (reduced risk indicator). The outcomes showed that the proposed hybrid method could select projects with the lowest total risk factor of up to 90.53% for small-scale studied cases, up to 94.45% for medium-scale studied cases, and up to 19.61% for large-scale studied cases. The proposed method solved the small-scale problems in [0.036, 0.054] s, medium-scale problems in [0.033, 0.088] s, and large-scale problems in [0.062, 0.557] s, depending on the nature of the project (processing time).
It is recommended to develop a Java application for the proposed method in this research, which could be completed by computer science researchers or manufacturing engineering researchers familiar with programming languages. It is also suggested to use different MADAM methods such as VICOR and TOPSIS to compare the functionality of the proposed method in this research with them.

Author Contributions

Conceptualization, M.K.A.B.M.A. and K.A.A.; methodology, S.M.A.; software, S.M.A.; validation, K.A.A. and M.K.A.B.M.A.; formal analysis, K.A.A.; investigation, M.K.A.B.M.A.; resources, E.E.B.S.; data curation, M.K.A.B.M.A.; writing—original draft preparation, S.M.A.; writing—review and editing, S.M.A. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Delgoshaei, A.; Ariffin, M.K.M.; Bin Baharudin, B.H.T.; Leman, Z. A new method for decreasing cell-load variation in dynamic cellular manufacturing systems. Int. J. Ind. Eng. Comput. 2016, 7, 83–110. [Google Scholar] [CrossRef]
  2. Abdikeev, N.M.; Bogachev, Y.S.; Bekulova, S.R. Investment potential of the manufacturing industry. Financ. Theory Pract. 2019, 23, 24–42. [Google Scholar] [CrossRef] [Green Version]
  3. Dašić, B.; Trklja, R.; Savić, M. Comparative analysis of inflows and sectoral distribution of the foreign direct investments flows. Min. Met. Eng. Bor 2019, 2019, 31–40. [Google Scholar] [CrossRef] [Green Version]
  4. Petrova, N.; Rogleva, N.K.; Fustik, V. Managing Renewable Energy Projects Including Risk Analysis. MEST J. 2019, 7, 71–79. [Google Scholar] [CrossRef]
  5. de Carvalho, M.M.; Rabechini, R., Jr. Impact of risk management on project performance: The importance of soft skills. Int. J. Prod. Res. 2015, 53, 321–340. [Google Scholar] [CrossRef]
  6. Tacnet, J.-M.; Dezert, J.; Batton-Hubert, M. AHP and Uncertainty Theories for Decision Making Using the ER-MCDA Methodology. In Proceedings of the International Symposium on Analytic Hierarchy/Network Process, Naples, Italy, 15–18 June 2011. [Google Scholar]
  7. Kazimieras Zavadskas, E.; Antucheviciene, J.; Chatterjee, P. Multiple-Criteria Decision-Making (MCDM) Techniques for Business Processes Information Management; Multidisciplinary Digital Publishing Institute: Basel, Switzerland, 2019. [Google Scholar]
  8. Su, X.; Mahadevan, S.; Xu, P.; Deng, Y. Dependence Assessment in Human Reliability Analysis Using Evidence Theory and AHP. Risk Anal. 2015, 35, 1296–1316. [Google Scholar] [CrossRef] [PubMed]
  9. Ruan, Z.; Li, C.; Wu, A.; Wang, Y. A New Risk Assessment Model for Underground Mine Water Inrush Based on AHP and D–S Evidence Theory. Mine Water Environ. 2019, 38, 488–496. [Google Scholar] [CrossRef]
  10. Simei, L.; Jianlin, Z.; Hao, S.; Liming, L. Security risk assessment model based on ahp/ds evidence theory. In Proceedings of the 2009 International Forum on Information Technology and Applications, Chengdu, China, 15–17 May 2009. [Google Scholar]
  11. Zhang, X.; Deng, Y.; Chan, F.T.; Adamatzky, A.; Mahadevan, S. Supplier selection based on evidence theory and analytic network process. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf. 2016, 230, 562–573. [Google Scholar] [CrossRef]
  12. Kuo, Y.-C.; Lu, S.-T. Using fuzzy multiple criteria decision making approach to enhance risk assessment for metropolitan construction projects. Int. J. Proj. Manag. 2013, 31, 602–614. [Google Scholar] [CrossRef]
  13. Marcelino-Sádaba, S.; Pérez-Ezcurdia, A.; Lazcano, A.M.E.; Villanueva, P. Project risk management methodology for small firms. Int. J. Proj. Manag. 2014, 32, 327–340. [Google Scholar] [CrossRef]
  14. Leu, S.-S.; Chang, C.-M. Bayesian-network-based safety risk assessment for steel construction projects. Accid. Anal. Prev. 2013, 54, 122–133. [Google Scholar] [CrossRef] [PubMed]
  15. Hossen, M.M.; Kang, S.; Kim, J. Construction schedule delay risk assessment by using combined AHP-RII methodology for an international NPP project. Nucl. Eng. Technol. 2015, 47, 362–379. [Google Scholar] [CrossRef] [Green Version]
  16. Kim, B.-C. Integrating Risk Assessment and Actual Performance for Probabilistic Project Cost Forecasting: A Second Moment Bayesian Model. IEEE Trans. Eng. Manag. 2015, 62, 158–170. [Google Scholar] [CrossRef]
  17. Esmaeili, B.; Hallowell, M.R.; Rajagopalan, B. Attribute-Based Safety Risk Assessment. I: Analysis at the Fundamental Level. J. Constr. Eng. Manag. 2015, 141, 04015021. [Google Scholar] [CrossRef]
  18. Esmaeili, B.; Hallowell, M.R.; Rajagopalan, B. Attribute-Based Safety Risk Assessment. II: Predicting Safety Outcomes Using Generalized Linear Models. J. Constr. Eng. Manag. 2015, 141, 04015022. [Google Scholar] [CrossRef]
  19. Islam, M.S.; Nepal, M.; Skitmore, M.; Attarzadeh, M. Current research trends and application areas of fuzzy and hybrid methods to the risk assessment of construction projects. Adv. Eng. Inform. 2017, 33, 112–131. [Google Scholar] [CrossRef]
  20. Valipour, A.; Yahaya, N.; Noor, N.M.; Antuchevičienė, J.; Tamošaitienė, J. Hybrid SWARA-COPRAS method for risk assessment in deep foundation excavation project: An iranian case study. J. Civ. Eng. Manag. 2017, 23, 524–532. [Google Scholar] [CrossRef] [Green Version]
  21. Williams, T. The Nature of Risk in Complex Projects. Proj. Manag. J. 2017, 48, 55–66. [Google Scholar] [CrossRef]
  22. Fabricius, G.; Büttgen, M. Project managers’ overconfidence: How is risk reflected in anticipated project success? Bus. Res. 2015, 8, 239–263. [Google Scholar] [CrossRef] [Green Version]
  23. Wang, C.M.; Xu, B.B.; Zhang, S.; Chen, Y.Q. Influence of personality and risk propensity on risk perception of Chinese construction project managers. Int. J. Proj. Manag. 2016, 34, 1294–1304. [Google Scholar] [CrossRef]
  24. Chemweno, P.; Pintelon, L.; Van Horenbeek, A.; Muchiri, P. Development of a risk assessment selection methodology for asset maintenance decision making: An analytic network process (ANP) approach. Int. J. Prod. Econ. 2015, 170, 663–676. [Google Scholar] [CrossRef]
  25. Kokangül, A.; Polat, U.; Dağsuyu, C. A new approximation for risk assessment using the AHP and Fine Kinney methodologies. Saf. Sci. 2017, 91, 24–32. [Google Scholar] [CrossRef]
  26. Yet, B.; Constantinou, A.; Fenton, N.; Neil, M.; Luedeling, E.; Shepherd, K. A Bayesian network framework for project cost, benefit and risk analysis with an agricultural development case study. Expert Syst. Appl. 2016, 60, 141–155. [Google Scholar] [CrossRef]
  27. Yang, Y.; Wang, J.; Wang, G.; Chen, Y.-W. Research and development project risk assessment using a belief rule-based system with random subspaces. Knowl.-Based Syst. 2019, 178, 51–60. [Google Scholar] [CrossRef]
  28. Ansarifar, J.; Tavakkoli-Moghaddam, R.; Akhavizadegan, F.; Amin, S.H. Multi-objective integrated planning and scheduling model for operating rooms under uncertainty. Proc. Inst. Mech. Eng. Part H J. Eng. Med. 2018, 232, 930–948. [Google Scholar] [CrossRef]
  29. Zhang, Y.; Fan, Z.-P. An optimization method for selecting project risk response strategies. Int. J. Proj. Manag. 2014, 32, 412–422. [Google Scholar] [CrossRef]
  30. Kliestik, T.; Belas, J.; Valaskova, K.; Nica, E.; Durana, P. Earnings management in V4 countries: The evidence of earnings smoothing and inflating. Econ. Res.-Ekon. Istraž. 2021, 34, 1452–1470. [Google Scholar] [CrossRef]
  31. Abd El-Karim, M.S.B.A.; Mosa El Nawawy, O.A.; Abdel-Alim, A.M. Identification and assessment of risk factors affecting construction projects. HBRC J. 2017, 13, 202–216. [Google Scholar] [CrossRef]
  32. Liu, J.; Jin, F.; Xie, Q.; Skitmore, M. Improving risk assessment in financial feasibility of international engineering projects: A risk driver perspective. Int. J. Proj. Manag. 2017, 35, 204–211. [Google Scholar] [CrossRef] [Green Version]
  33. Delgoshaei, A.; Ariffin, M.K.A.M.; Baharudin, B.T.H.T. Pre-emptive resource-constrained multimode project scheduling using genetic algorithm: A dynamic forward approach. J. Ind. Eng. Manag. 2016, 9, 732–785. [Google Scholar] [CrossRef] [Green Version]
  34. Bowers, J.; Khorakian, A. Integrating risk management in the innovation project. Eur. J. Innov. Manag. 2014, 17, 25–40. [Google Scholar] [CrossRef] [Green Version]
  35. Brustbauer, J. Enterprise risk management in SMEs: Towards a structural model. Int. Small Bus. J. Res. Entrep. 2014, 34, 70–85. [Google Scholar] [CrossRef]
  36. Fang, C.; Marle, F.; Xie, M. Applying Importance Measures to Risk Analysis in Engineering Project Using a Risk Network Model. IEEE Syst. J. 2017, 11, 1548–1556. [Google Scholar] [CrossRef]
  37. Tao, S.; Wu, C.; Sheng, Z.; Wang, X. Space-Time Repetitive Project Scheduling Considering Location and Congestion. J. Comput. Civ. Eng. 2018, 32, 04018017. [Google Scholar] [CrossRef]
  38. Gutjahr, W. Bi-Objective Multi-Mode Project Scheduling Under Risk Aversion. Eur. J. Oper. Res. 2015, 246, 421–434. [Google Scholar] [CrossRef]
  39. Wu, D.D.; Chen, S.-H.; Olson, D.L. Business intelligence in risk management: Some recent progresses. Inf. Sci. 2014, 256, 1–7. [Google Scholar] [CrossRef]
  40. Dziadosz, A.; Rejment, M. Risk Analysis in Construction Project—Chosen Methods. Procedia Eng. 2015, 122, 258–265. [Google Scholar] [CrossRef] [Green Version]
  41. Davari, M.; Demeulemeester, E. The proactive and reactive resource-constrained project scheduling problem. J. Sched. 2019, 22, 211–237. [Google Scholar] [CrossRef]
  42. Grabovy, P.; Orlov, A. The Overall Risk Assessment and Management: Implementation of Foreign Investment Construction Megaprojects by Russian Development Companies. Procedia Eng. 2016, 153, 195–202. [Google Scholar] [CrossRef] [Green Version]
  43. Nasrabadi, M.; Mirzazadeh, A. The inventory system management under uncertain conditions and time value of money. Int. J. Supply Oper. Manag. 2016, 3, 1192–1214. [Google Scholar]
  44. Zadeh, L.A. A simple view of the Dempster—Shafer theory of evidence and its implication for the rule of combination. AI Mag. 1986, 7, 85. [Google Scholar]
  45. Tang, H. A novel fuzzy soft set approach in decision making based on grey relational analysis and Dempster—Shafer theory of evidence. Appl. Soft Comput. 2015, 31, 317–325. [Google Scholar] [CrossRef]
  46. Hatefi, S.M.; Basiri, M.E.; Tamošaitienė, J. An Evidential Model for Environmental Risk Assessment in Projects Using Dempster—Shafer Theory of Evidence. Sustainability 2019, 11, 6329. [Google Scholar] [CrossRef] [Green Version]
  47. Li, Z.; Wen, G.; Xie, N. An approach to fuzzy soft sets in decision making based on grey relational analysis and Dempster—Shafer theory of evidence: An application in medical diagnosis. Artif. Intell. Med. 2015, 64, 161–171. [Google Scholar] [CrossRef]
  48. Wang, J.; Hu, Y.; Xiao, F.; Deng, X.; Deng, Y. A novel method to use fuzzy soft sets in decision making based on ambiguity measure and Dempster—Shafer theory of evidence: An application in medical diagnosis. Artif. Intell. Med. 2016, 69, 1–11. [Google Scholar] [CrossRef] [PubMed]
  49. Ballent, W.; Corotis, R.B.; Torres-Machi, C. Representing uncertainty in natural hazard risk assessment with Dempster Shafer (Evidence) theory. Sustain. Resilient Infrastruct. 2019, 4, 137–151. [Google Scholar] [CrossRef]
  50. Muriana, C.; Vizzini, G. Project risk management: A deterministic quantitative technique for assessment and mitigation. Int. J. Proj. Manag. 2017, 35, 320–340. [Google Scholar] [CrossRef]
  51. Niazi, M.; Mahmood, S.; Alshayeb, M.; Riaz, M.R.; Faisal, K.; Cerpa, N.; Khan, S.U.; Richardson, I. Challenges of project management in global software development: A client-vendor analysis. Inf. Softw. Technol. 2016, 80, 1–19. [Google Scholar] [CrossRef]
  52. Pan, Y.; Zhang, L.; Li, Z.; Ding, L. Improved Fuzzy Bayesian Network-Based Risk Analysis With Interval-Valued Fuzzy Sets and D–S Evidence Theory. IEEE Trans. Fuzzy Syst. 2020, 28, 2063–2077. [Google Scholar] [CrossRef]
  53. Qazi, A.; Quigley, J.; Dickson, A.; Kirytopoulos, K. Project Complexity and Risk Management (ProCRiM): Towards modelling project complexity driven risk paths in construction projects. Int. J. Proj. Manag. 2016, 34, 1183–1198. [Google Scholar] [CrossRef] [Green Version]
  54. Sangaiah, A.K.; Samuel, O.W.; Li, X.; Abdel-Basset, M.; Wang, H. Towards an efficient risk assessment in software projects–Fuzzy reinforcement paradigm. Comput. Electr. Eng. 2018, 71, 833–846. [Google Scholar] [CrossRef]
  55. Suresh, K.; Dillibabu, R. A novel fuzzy mechanism for risk assessment in software projects. Soft Comput. 2019, 24, 1683–1705. [Google Scholar] [CrossRef]
  56. Tonmoy, F.; Wainwright, D.; Verdon-Kidd, D.C.; Rissik, D. An investigation of coastal climate change risk assessment practice in Australia. Environ. Sci. Policy 2018, 80, 9–20. [Google Scholar] [CrossRef] [Green Version]
  57. Zou, Y.; Kiviniemi, A.; Jones, S.W. A review of risk management through BIM and BIM-related technologies. Saf. Sci. 2017, 97, 88–98. [Google Scholar] [CrossRef]
  58. Zhang, Y. Selecting risk response strategies considering project risk interdependence. Int. J. Proj. Manag. 2016, 34, 819–830. [Google Scholar] [CrossRef] [Green Version]
  59. Zavadskas, E.K.; Turskis, Z.; Tamošaitiene, J. Risk assessment of construction projects. J. Civ. Eng. Manag. 2010, 16, 33–46. [Google Scholar] [CrossRef] [Green Version]
  60. Kral, P.; Valjaskova, V.; Janoskova, K. Quantitative approach to project portfolio management: Proposal for Slovak companies. Oecon. Copernic. 2019, 10, 797–814. [Google Scholar] [CrossRef] [Green Version]
  61. Chen, L.; Deng, X. A Modified Method for Evaluating Sustainable Transport Solutions Based on AHP and Dempster—Shafer Evidence Theory. Appl. Sci. 2018, 8, 563. [Google Scholar] [CrossRef] [Green Version]
  62. Delgoshaei, A.; Gomes, C. A multi-layer perceptron for scheduling cellular manufacturing systems in the presence of unreliable machines and uncertain cost. Appl. Soft Comput. 2016, 49, 27–55. [Google Scholar] [CrossRef]
Figure 1. Correlations between the level of risks associated with a project and the amount at stake (Reference: Petrova et al. [4]).
Figure 1. Correlations between the level of risks associated with a project and the amount at stake (Reference: Petrova et al. [4]).
Mathematics 09 03225 g001
Figure 2. Flowchart of the research methodology.
Figure 2. Flowchart of the research methodology.
Mathematics 09 03225 g002
Figure 3. Each phase of the proposed method.
Figure 3. Each phase of the proposed method.
Mathematics 09 03225 g003
Figure 4. A dendrogram of clustering risk factors.
Figure 4. A dendrogram of clustering risk factors.
Mathematics 09 03225 g004
Figure 5. The agglomerative clustering algorithm results while the different numbers of clusters are taken into consideration.
Figure 5. The agglomerative clustering algorithm results while the different numbers of clusters are taken into consideration.
Mathematics 09 03225 g005
Figure 6. Silhouette score graph for the proposed PCA-Ward while k = 2.
Figure 6. Silhouette score graph for the proposed PCA-Ward while k = 2.
Mathematics 09 03225 g006
Figure 7. Calinski–Harabasz score graph for the proposed PCA-Ward while k = 2.
Figure 7. Calinski–Harabasz score graph for the proposed PCA-Ward while k = 2.
Mathematics 09 03225 g007
Figure 8. Silhouette score graph for the proposed PCA-Ward while k = 3.
Figure 8. Silhouette score graph for the proposed PCA-Ward while k = 3.
Mathematics 09 03225 g008
Figure 9. Calinski–Harabasz score graph for the proposed PCA-Ward while k = 3.
Figure 9. Calinski–Harabasz score graph for the proposed PCA-Ward while k = 3.
Mathematics 09 03225 g009
Figure 10. Silhouette score graph for the proposed PCA-Ward while k = 4.
Figure 10. Silhouette score graph for the proposed PCA-Ward while k = 4.
Mathematics 09 03225 g010
Figure 11. Calinski–Harabasz score graph for the proposed PCA-Ward while k = 4.
Figure 11. Calinski–Harabasz score graph for the proposed PCA-Ward while k = 4.
Mathematics 09 03225 g011
Figure 12. The project with the lowest μ i is preferred.
Figure 12. The project with the lowest μ i is preferred.
Mathematics 09 03225 g012
Figure 13. When μ i is equal for two alternatives, the alternative with lower σ i is preferred.
Figure 13. When μ i is equal for two alternatives, the alternative with lower σ i is preferred.
Mathematics 09 03225 g013
Figure 14. Possible condition one: different means and different domain lengths.
Figure 14. Possible condition one: different means and different domain lengths.
Mathematics 09 03225 g014
Figure 15. Possible condition one: different means and similar domain length.
Figure 15. Possible condition one: different means and similar domain length.
Mathematics 09 03225 g015
Figure 16. Possible condition one: similar means and different domain lengths.
Figure 16. Possible condition one: similar means and different domain lengths.
Mathematics 09 03225 g016
Figure 17. Possible condition one: similar means and similar domain lengths.
Figure 17. Possible condition one: similar means and similar domain lengths.
Mathematics 09 03225 g017
Figure 18. The results of the RRI% index for the solved case studies.
Figure 18. The results of the RRI% index for the solved case studies.
Mathematics 09 03225 g018
Figure 19. The results of solving time for the studied cases.
Figure 19. The results of solving time for the studied cases.
Mathematics 09 03225 g019
Table 1. Risk factor categories and related independent variables.
Table 1. Risk factor categories and related independent variables.
Risk Factor CategoryNo.Variable (Risk Factor)Questions
Before Project SelectionDuring ExecutionAfter Project
Properties1InfrastructuresQ1Q4Q7
2MachineryQ2Q5Q8
3Human ResourceQ3Q6Q9
Technologic and Operational4Project Scheduling PerformanceQ10Q15Q20
5Project Schedule (Time and Cost)Q11Q16Q21
6Required TechnologyQ12Q17Q22
7Risk Management Q13Q18Q23
8Management SystemsQ14Q19Q24
Financial9Tender Process PerformanceQ25Q28Q31
10Costs and Profits EstimationQ26Q29Q32
11Money ValueQ27Q30Q33
Strategic12Rivals’ StrategiesQ34Q38Q42
13Market Share PredictionQ35Q39Q43
14Business PlanQ36Q40Q44
15Customer Satisfaction Q37Q41Q45
Table 2. Results of the Statistical Analysis of Risk factor Categories.
Table 2. Results of the Statistical Analysis of Risk factor Categories.
Associate RiskProperties Risk GroupTechnologic and Operational Risk GroupFinancial Risk GroupStrategic Risk Group
count128.000000128.00000128.000000128.000000128.000000
mean0.8437503.934933.8793753.8865863.889258
std.0.8365420.599400.6293810.6574350.647047
min0.0000001.750001.9330001.8890002.250000
25%0.0000003.556003.4670003.4440003.396000
50%1.0000004.000004.0000004.0000004.000000
75%1.0000004.333004.3330004.3607504.354000
max4.0000005.000005.0000005.0000005.000000
Table 3. Dataset summary that was used for the hybrid PCA-agglomerative method.
Table 3. Dataset summary that was used for the hybrid PCA-agglomerative method.
HRIncomeSuccessAssociate RiskProperties Risk GroupTechnologic and Operational Risk GroupFinancial Risk GroupStrategic Risk Group
0Large Scale Large Scale Excellent (100%)04.0003.2673.4443.083
1Large Scale Medium Scale Very Good (75–100%)13.7783.9334.0003.750
2Medium Scale Medium Scale Very Good (75–100%)13.4443.6004.1113.250
3Small Scale Small ScaleAcceptable (50–75%)22.7782.5331.8893.750
4Medium Scale Medium Scale Very Good (75–100%)14.5564.0673.5563.917
127Small Scale Small ScaleVery Good (75–100%)13.5563.2672.889Very Good (75–100%)
Table 4. Comparing classic AHP to the proposed hybrid AHP and Dempster—Shafer theory of evidence.
Table 4. Comparing classic AHP to the proposed hybrid AHP and Dempster—Shafer theory of evidence.
CriteriaAHPHybrid AHP and Theory of Evidence
Uncertainty×🗸
Multi-Expert’s Opinion×🗸
Weight of Risks🗸🗸
OutcomeSingle PointRisk Domain
Speed of Solving FastModerate
Weight of Experts’ Opinions×🗸
Table 5. The results of comparing the proposed method with similar methods in the literature.
Table 5. The results of comparing the proposed method with similar methods in the literature.
The Proposed MethodSu et al.
[8]
Chen and Deng [61]Ruan et al.
[9]
Simei et al. [10]X. Zhang et al. [11]
Uncertainty🗸🗸🗸🗸🗸🗸
ScopeProject SelectionHuman Reliability AnalysisSustainable TransportationUnderground Mine Water InrushSecurity Risk AssessmentSupplier Selection
Multi-Phase/Single PhaseMulti-PhaseSingle-PhaseSingle-PhaseSingle-PhaseSingle-PhaseSingle-Phase
Classify Alternatives with Machine Learning MethodHybrid PCA-Agglomerative Method🗶🗶🗶🗶🗶
Grouped Risk Factors🗸🗶🗸🗶🗶🗶
Including Risk Levels🗸🗸🗸🗸🗸🗸
Table 6. Experiments for verification section.
Table 6. Experiments for verification section.
FactorsL1: Small ScaleL2: Medium ScaleL3: Large Scale
Number of Risk Factors1–34–56–13
Number of Experts2–56–1011–30
Number of Alternatives1–34–56–13
Number of Contract Options1–22–34–5
Table 7. The Designed Case Studies (Taguchi L2^4).
Table 7. The Designed Case Studies (Taguchi L2^4).
No.ScaleNumber of ExpertsNumber of Risk FactorsNumber of AlternativesNumber of Contract Options
1Small2121
22132
32322
42331
55122
65131
75321
85332
9Medium6442
106453
116543
126552
1310443
1410452
1510542
1610553
17Large11664
18116105
19111365
201113105
2130665
22306104
23301365
243013105
Table 8. Results of solving numerical experiments using the proposed hybrid AHP and theory of evidence algorithm.
Table 8. Results of solving numerical experiments using the proposed hybrid AHP and theory of evidence algorithm.
No.ScaleNumber of ExpertsNumber of Risk FactorsNumber of AlternativesNumber of Contract OptionsSolving TimeBest AlternativeBest Contract OptionMinimum Observed Risk DomainObserved AI
1Small21210.03611[0.7622, 1.3121]0.9642
221320.04122[0.0447, 1.0356]1.8514
323220.05421[0.1675, 2.5457]0.7371
423310.04121[0.5039, 3.4899]0.5008
551220.03721[0.1605, 1.6070]1.1315
651310.04731[0.4416, 2.2646]0.7391
753210.03911[0.3602, 3.1599]0.5682
853320.05422[0.1753, 1.5428]1.1641
9Medium64420.03441[0.2607, 1.3144]1.2698
1064530.08451[0.1204, 1.1896]1.5267
1165430.08813[0.1021, 1.4210]1.3353
1265520.04931[0.1982, 1.8828]0.9610
13104430.0443
4
3
3
[0.0331, 3.3072]
[0.0331, 3.3072]
0.5987
0.5987
14104520.03312[0.0585, 4.4972]0.4390
15105420.03932[0.0652, 4.1237]0.4775
16105530.08213[0.0464, 2.5759]0.7627
17Large116640.18634[0.0509, 1.3212]1.4577
181161050.55771[0.0366, 1.1134]1.7391
191113650.14415[0.0365, 1.2240]1.5867
2011131050.2001
8
5
5
[0.0365, 1.2240]
[0.0365, 1.2240]
1.5867
1.5867
21306650.06222[0.0327, 1.2286]1.5857
223061040.0834[0.0393, 1.5342]1.2710
233013650.10914[0.0320, 1.2514]1.5584
2430131050.19110
10
4
5
[0.0319, 1.2293]
[0.0319, 1.2293]
1.5858
1.5858
Table 9. Expert Opinion Total Matrix.
Table 9. Expert Opinion Total Matrix.
Expert_Total_Matrix
(Expert 1)
Expert_Total_Matrix
(Expert 2)
Expert_Total_Matrix
(Expert 3)
4, 3, 0.12, 4, 0.17, 8, 0.9
8, 8, 0.28, 8, 0.25, 5, 0.9
Table 10. Expected value for risks for the first risk.
Table 10. Expected value for risks for the first risk.
Expected_Value_for_Risk 1Expected_Value_for_Risk 2
3.6667  3.30001.2333  5.7333
1.1667  5.76671.6000  5.6000
3.6667  4.50000.8667  5.3333
Table 11. Upper and lower limit of expected value for risks.
Table 11. Upper and lower limit of expected value for risks.
The Lower Limit of Expected Value for RiskThe Upper Limit of Expected Value for Risk
6.4000  0  04.5000  0  0
11.0000  0  09.9000  0  0
Table 12. Expected value for risks.
Table 12. Expected value for risks.
The Expected Value for Risk (:,:,1)The Expected Value for Risk (:,:,2)
3.6667  3.30001.2333  5.7333
1.1667  5.76671.6000  5.6000
3.6667  4.50000.8667  5.3333
Table 13. Risk associate matrix.
Table 13. Risk associate matrix.
RiskLower Risk Associate MatrixUpper Risk Associate Matrix
Risk 10.9000  0.5756
0.3364  0.2151
1.1111  2.9730
1.7374  4.6486
Risk 20.2023  0.2083
0.2775  0.2857
4.9429  3.6042
4.8000  3.5000
Risk 30.8148  0.6875
0.1926  0.1625
1.2273  5.1923
1.4545  6.1538
Table 14. Normalized risk factor matrix.
Table 14. Normalized risk factor matrix.
Normalized_Upper_Risk_
Associate_Matrix
Normalized_Upper_Risk_
Associate_Matrix
10.3026  7.29521.2182  6.1086
10.0048  7.08431.4439  7.2398
Table 15. Averaged normalized risk factor.
Table 15. Averaged normalized risk factor.
RiskAveraged Normalized Lower Risk Associate MatrixAveraged Normalized Upper Risk Associate Matrix
Risk 10.1957
0.0732
2.3293
3.6422
Risk 20.0250
0.0343
8.7989
8.5446
Risk 30.1822
0.0431
3.6634
4.3418
Table 16. Total risk matrix.
Table 16. Total risk matrix.
Total Lower Risk MatrixTotal Upper Risk Matrix
0.0979  0.0075  0.03641.1647  2.6397  0.7327
0.0366  0.0103  0.00861.8211  2.5634  0.8684
Table 17. The results of RRI% for studies while solved by the proposed method.
Table 17. The results of RRI% for studies while solved by the proposed method.
RowReported AI Value by the AlgorithmRRI%RowReported AI Value by the AlgorithmRRI%
10.964290.53%130.59875.66%
21.851428.92%140.439058.79%
30.737148.88%150.477512.13%
40.500870.01%160.762720.57%
51.131548.02%171.457719.61%
60.739148.63%181.739112.31%
70.56823.41%191.58679.67%
81.164137.38%201.58679.67%
91.269844.70%211.58574.48%
101.526725.27%221.271010.24%
111.335394.45%231.55841.76%
120.961036.08%241.58583.86%
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Albogami, S.M.; Ariffin, M.K.A.B.M.; Supeni, E.E.B.; Ahmad, K.A. A New Hybrid AHP and Dempster—Shafer Theory of Evidence Method for Project Risk Assessment Problem. Mathematics 2021, 9, 3225. https://doi.org/10.3390/math9243225

AMA Style

Albogami SM, Ariffin MKABM, Supeni EEB, Ahmad KA. A New Hybrid AHP and Dempster—Shafer Theory of Evidence Method for Project Risk Assessment Problem. Mathematics. 2021; 9(24):3225. https://doi.org/10.3390/math9243225

Chicago/Turabian Style

Albogami, Saad Muslet, Mohd Khairol Anuar Bin Mohd Ariffin, Eris Elianddy Bin Supeni, and Kamarul Arifin Ahmad. 2021. "A New Hybrid AHP and Dempster—Shafer Theory of Evidence Method for Project Risk Assessment Problem" Mathematics 9, no. 24: 3225. https://doi.org/10.3390/math9243225

APA Style

Albogami, S. M., Ariffin, M. K. A. B. M., Supeni, E. E. B., & Ahmad, K. A. (2021). A New Hybrid AHP and Dempster—Shafer Theory of Evidence Method for Project Risk Assessment Problem. Mathematics, 9(24), 3225. https://doi.org/10.3390/math9243225

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop