Success Factors in Management of IT Service Projects: Regression, Confirmatory Factor Analysis, and Structural Equation Models

Abstract

Although there have been some studies on the success factors for IT software projects, there is still a lack of coherent research on the success factors for IT service projects. Therefore, this study aimed to identify and understand the factors and their relationships that contribute to the success of IT service projects. For this purpose, multivariate regressions and structural equation models (SEMs) were developed and analyzed. The regression models included six project management success criteria used as dependent variables (quality of the delivered product, scope realization and requirements, timeliness of delivery, delivery within budget, customer satisfaction, and provider satisfaction) and four independent variables (agile techniques and change management, organization and people, stakeholders and risk analysis, work environment), which had been identified through exploratory factor analysis. The results showed that not all success factors were relevant to all success criteria, and there were differences in their importance. An additional series of exploratory and confirmatory factor analyses along with appropriate statistical measures were employed to evaluate the quality of these four factors. The SEM approach was based on five latent constructs with a total of twenty components. The study suggests that investing in improving people’s knowledge and skills, using agile methodologies, creating a supportive work environment, and involving stakeholders in regular risk analysis are important for project management success. The results also suggest that the success factors for IT service projects depend on both traditional and agile approaches. The study extensively compared its findings with similar research and discussed common issues and differences in both the model structures and methodologies applied. The investigation utilized mathematical methods and techniques that are not commonly applied in the field of project management success modeling. The comprehensive methodology that was applied may be helpful to other researchers who are interested in this topic.

1. Introduction

The factors that influence project management success or failure are also of great interest to stakeholders in the information technology (IT) sector. One of the first systematic studies on this topic in the IT industry was conducted by the Standish Group. They published their report on software project failures as early as 1995 [1] and continued providing similar publications later on. Additional research on project management success in the IT sector emerged in subsequent years. For instance, the research of White & Fortune [2] or the studies of Besner & Hobbs [3,4] in the early years of the twenty-first century. Recently, an extensive review of project management success factors in three IT-related areas was carried out [5]: IT software manufacturing, IT services, and IT systems. Our literature review of over 150 papers resulted in distinguishing 45 potential factors, which constituted the basis for the investigation of project management success components. By applying the exploratory factor analysis methodology to the questionnaire results collected from 155 respondents, the model for the IT service project management success was developed and carefully formally evaluated. It included four main dimensions: (1) agile techniques and change management, (2) organization and people, (3) stakeholders and risk analysis, and (4) work environment. The results were compared with analogous studies found in the literature. The variables and corresponding questions used in the model are given, for convenience, in

Table A1. A set of questions obtained by means of exploratory factor analysis, published in [5].

No.	Question
F1_Q24	The project manager underwent training in agile methodology
F1_Q25	The work in progress was limited and bottlenecks removed for faster throughput
F1_Q30	The project focused on the work that was delivered (outcomes) instead of how busy people were (utilization) to increase the throughput and flow
F1_Q31	The change request process was used in the project (i.e., recording, planning, documenting, testing, accepting, categorizing, assessing, authorizing, implementing, and reviewing in a controlled manner)
F1_Q32	Throughout the project, the right amount of documentation was maintained, not too focused on producing elaborate documentation as milestones but not ignoring documentation altogether either
F2_Q01	The project received strong executive support (by the Board of Directors or CEO, CFO, CIO, etc.), which influenced the decision-making
F2_Q03	In the project, a hierarchal culture that has clear divisions of responsibility and authority was employed
F2_Q08	The selected project team members had high technical competence and expertise (problem-solving, subject matter)
F2_Q09	Project team members had great motivation and were committed to executing the project in the best possible way
F3_Q34	From the customer’s point of view, the most important features/outcomes were delivered first in the project
F3_Q41	In the project, risk analysis was evaluated at each change
F3_Q42	In the project, risk analysis was evaluated at control points
F3_Q44	The impact of stakeholders on the project was analyzed
F4_Q06	All team members worked in the same location for ease of communication and casual, constant contact
F4_Q07	The project team worked in a facility with a work environment like one of these: an open space, communal area, ample wall spaces for postings, etc.
F4_Q38	In the project, no multiple, independent teams were working together

of Appendix A.

In this paper, we aim to better understand project management success in the field of IT services by extending the previous analysis to include the following aspects, which constitute new contributions:

−

We examined how the six criteria of measuring project management success [6,7,8,9], that is, (1) quality, (2) scope, (3) time, (4) cost, (5) customer satisfaction, and (6) provider satisfaction, relate to the four project management success dimensions obtained in the exploratory factor analysis. For this purpose, we performed a series of stepwise regression analyses that resulted in multivariate models showing which of these dimensions significantly influenced the measurement criteria and to what extent. These results are presented in Section 4.1.

−

We conducted a confirmatory factor analysis to evaluate the quality of the structure identified in the previous paper [5] using the exploratory factor analysis approach. Additionally, we validated the structure by performing a series of ten exploratory and confirmatory factor analyses on two random samples, dividing the initial data into two groups. The outcomes of this analysis are provided in Section 4.2.

−

In our study, we also gathered information on the respondents’ perception of project management success, which was not included in the analyses published by Zaleski & Michalski [5]. We found the relationships between the obtained success dimensions interesting, and thus, we employed structural equation modeling procedures to examine them. To find the best possible model that fits all the obtained questionnaire results, we developed a series of SEM analyses. We started with the orthogonal structure of the identified dimensions and the latent variable representing the overall perception of project management success, which was based on four questions. Next, we used the model specification search procedure available in the IBM SPSS Amos (version 28) software to identify the best overall model that involved all the gathered data. The entire procedure, along with the proposal of the final model that fits the data well and is logically interpretable, is provided in Section 4.3.

−

The additional contribution of this paper is related to the presented methodological approach for examining the relationships between different aspects of project management success. To the best of our knowledge, such a procedure has not been presented in any research that involved SEM in the field of project management and there are very few similar approaches in other areas.

A brief scientific literature review of SEM-based investigations in the examined context is provided in Section 2.

2. Literature Review

As we are extending our previous research mainly by including further SEM-based analysis, this section will focus on studies that have employed this approach in the context of project management success. In the next two subsections, we briefly review recently published papers in this regard.

2.1. Project Management Success and SEM

The SEM approach has increasingly been employed in project management studies in recent years. It allows researchers to assess and understand the strength and significance of the relationships among factors and identify the most important drivers of project management success. Much research in this field has been conducted in the construction industry. For example, Shi and colleagues [10] studied the interrelationships among critical success factors in infrastructure projects involving public–private partnerships in China. They used a literature review and expert interviews to identify vital issues for improving project performance and sustainability. Associations between these factors were then examined using SEM. The success aspects of public–private partnership projects were also investigated in Saudi Arabia by Almeile et al. [11].

Another example of construction project management practices in developing countries is the study conducted by Banihashemi et al. [12]. They employed SEM to investigate critical success factors related to the environmental, social, and economic bases of sustainability. Different research on the success concept in the context of sustainable construction projects in Thailand was published by Krajangsri and Pongpeng [13]. They documented a strong influence of sustainable infrastructure assessments on project management success using SEM. In the same industry, Watfa et al. [14] employed SEM to study the effect of sustainability management on project success in the United Arab Emirates. They elaborated a comprehensive theoretical framework in this regard. Gunduz et al. [15] developed an SEM to examine and assess the importance of potential risks in Qatar’s public and private construction projects. Kineber et al. [16] explored aspects of cloud computing, which notably supports achieving sustainable construction success, in relation to construction activities in Nigeria. They conducted an interesting study including SEM analysis. Also, critical success indicators related to value management and their impact on the sustainability of building projects in Egypt were analyzed by SEM [17]. Charles et al. [18] examined the success factors of post-disaster rebuilding projects in Caribbean islands. Their SEM results suggest that safety and satisfaction are the most important factors from an end user’s perspective. The SEM methodology was also applied by Unegbu et. al. [19] to analyze construction project performance measures and management practices in Nigeria.

There have been works not directly associated with the construction industry. Project management success from the perspective of many different types of business and governmental organizations was examined by Yazici [20]. By employing, among others, the SEM methodology, they showed that corporate sustainability and its integration with project management have strategic significance in perceiving organizational success.

The financial and non-financial aspects of renewable energy project success were subject to examination by Maqbool et al. [21]. They conducted the questionnaire-based research in small- and medium-sized companies in Pakistan. The SEM analysis led them to suggest that the success of projects in this area depends on both aspects and that there is a considerable and positive relationship between them. Recently, another study linked with the renewable energy topic was published by Hussain et al. [22], in which they modeled and analyzed the role of government support, organizational innovativeness, and community participation in project success in this area using SEM.

2.2. IT-Related Project Management Success and SEM

The use of the SEM methodology in studies related to IT project management has not been widely spread. Some insights were provided in more general studies where a variety of industries were included, and the IT sector was one of them. For example, Irfan et al. [23] modeled and analyzed the causal relationship between project management maturity and project success in Pakistan. Overall, the presented results may be informative; however, one should be cautious in drawing conclusions about IT-related projects since they accounted for only about 5% of the total sample size.

Among the rare SEM-based studies fully devoted to the IT project management success issue, is the work of Komal et al. [24]. They focused their effort on the project scope creep phenomenon and its influence on effectively and efficiently achieving project goals in small- and medium-sized software organizations in Pakistan. Taking advantage of the thoroughly performed systematic literature review and interviews, they identified crucial scope creep aspects responsible for software project failures. This qualitative analysis was the basis for the conceptual framework, which was further validated and analyzed in the questionnaire study, administered to 250 practitioners.

Tam et al. [25] examined the success factors of ongoing software development projects. They presented an SEM model with the following constructs: customer involvement, personal characteristics, societal culture, and team capability. The approach was developed and tested based on results from 216 surveys gathered on a seven-point Likert scale. The results showed that all the examined latent variables contributed considerably to explaining the project’s success, however, personal characteristics and societal culture affected this construct only indirectly. The authors additionally examined if the training and learning variable moderated the impact of customer involvement and team capability on the project’s success. Only the former factor was significantly influenced by this moderator.

An interesting way of employing the SEM approach was presented in the research published by Fakhkhari et al. [26]. The authors assessed factors influencing the information communication technology project management success by determining the frequency of success factors occurring in publications. This extensive systematic literature review was the basis for performing the SEM analysis of the derived conceptual model.

Malik and Khan [27] focused on project management success strategy development for a specific type of information technology solution, that is, the implementation of enterprise resource planning software. Their results from the exploratory factor analysis and SEM were used for developing such an implementation strategy, which was validated in one of the large telecom organizations in Pakistan.

Amid the latest publications involving the use of SEM techniques for the analysis of project management success regarding software development, there is the work of Hamid et al. [28]. They investigated a substantial number of senior developers and project managers (339) from software companies in an underdeveloped country. The authors identified four dimensions (planning, human resources, time, and cost estimation) and examined how they affect software success in this context. Another recent study on critical success factors related to information technology or information system projects was conducted by Yohannes and Mauritsius [29]. Their conceptual model, based on a systematic literature review, included five general dimensions (project management, effective organization communication, project team capability, methodology, and documentation). The survey-based validation of the model showed that the first and last dimensions did not influence project success.

3. Methods

The basis of our considerations in this paper is the survey-based study presented by Zaleski and Michalski [5]. In the previous work, an exploratory factor analysis was employed to determine the conceptual structure of IT services project management success dimensions. In the present paper, we elaborate further on this issue and extend the analysis by applying additional methods and including new, unpublished data. The main goal is to develop and examine models that will allow us to identify key relationships between variables and better understand the investigated issue. For this purpose, we conducted an examination of the identified project success factor dimensions combined with new survey data within the framework of structural equation modeling. It involved linear multivariate regression analyses with stepwise variable selection, confirmatory factor analysis, and the development of models with both the path analysis and dimensions’ structure. For convenience, in the following subsections, we briefly describe the questionnaire development along with the sample and data collection, which are given in detail in [5]. Next, we briefly describe the modeling techniques applied in this study.

3.1. Questionnaire Development

A web-based questionnaire was developed for IT service projects that incorporated both traditional and agile approaches based on an extensive literature review. The first version of the survey was evaluated by subject matter experts in survey creation and project management to ensure correctness, structure, and logical consistency. In a preliminary study [30], the questionnaire was tested, and improvements were made according to the feedback from 15 IT project managers. The changes included reordering sections, adding a new section on success perception, extending a risk management question, and correcting minor grammatical and stylistic errors in 18 questions. As a result, the total number of potential success factors used as input to exploratory factor analysis increased from 44 to 45.

3.2. Sample and Data Collection

The survey was anonymously conducted from February to June 2019 in a large international company operating worldwide in the IT service area. The data analyzed consisted of 155 fully completed questionnaires collected from project managers who were asked about recently completed projects. The questionnaire comprised five sections, covering factors that could potentially influence project management success, respondents’ perception of success, success criteria, project information, and comments. The sample was approximately balanced in terms of gender, with 57% being male, and had significant IT sector experience, with a mean of over 10 years (SD = 8.1). The average experience as project managers in the IT sector was over seven years (SD = 6.2), with nearly 70% having more than four years of experience.

3.3. Modeling

The gathered data were analyzed using Tibco Statistica 13.3, IBM Statistical Package for the Social Sciences (SPSS Statistics, version 28), and IBM SPSS Amos (version 28). In all the graphically presented models, explicit (independent) variables are illustrated in rectangles, while implicit (latent, dependent) variables in ellipses, and errors for variables in circles. Relationships, along with regression weights, are represented by arrows with one arrowhead, and covariances are indicated with arrows having arrowheads in two directions.

To ensure clarity in our analyses and discussions without unnecessary data clutter, we did not explicitly state hypotheses regarding R-squares, regression model parameters, or model significance. Unless otherwise specified, we hypothesize that these parameters are equal to zero and provide the appropriate probability values for the corresponding statistics. If not otherwise stated, the classical cut-off significance level was employed, that is, α = 0.05. A similar approach is used in confirmatory factor analysis and structural equation models to check if path parameters and covariances are significantly different from zero.

In our approach, we do not pose explicit hypotheses about the relationship structure in the model, as is common in many papers involving SEMs. This is an informed decision since this study primarily aims to identify and understand the factors and their relationships contributing to the success of IT service projects. Furthermore, considering that the models presented in this article constitute a direct extension of our previous study involving exploratory factor analysis, the current paper is also mainly explorative in nature.

3.3.1. Multivariate Regressions

The purpose of conducting the regression analysis was to verify the relationships between the key success factors obtained in the exploratory factor analysis and six project management success measures, which were also assessed by study subjects in the questionnaire. Since there were multiple independent and dependent variables, a general multivariate linear regression model [31,32] was used for each individual project management success measure. The mathematical form of the model is as follows:

$$Y=b_0+b_1{X}_1+b_2{X}_2+\cdots +b_n{X}_n+ϵ,$$

where Y—dependent variable; X—independent variable; b—regression coefficient; and $ϵ$—error term.

We applied a number of stepwise regression methods [33] to identify meaningful independent variables to be included in consecutive models. Moreover, only variables with regression coefficients significantly different from zero (p < 0.05) were taken into account. Standardized beta regression coefficients were used to evaluate the extent to which the independent variable influenced the dependent variable [34].

3.3.2. Confirmatory Factor Analysis (CFA)

The purpose of this study was to verify and validate the factor structure obtained in the exploratory factor analysis, as a continuation and extension of our previous research. To achieve this goal, we employed confirmatory factor analysis [35,36,37]. To ensure the validity and quality of fit between the observed data and the hypothesized model, we randomly split the entire data sample into two groups. One group underwent exploratory factor analysis while the other underwent confirmatory factor analysis. We repeated this process 10 times and used the results to assess the model’s validity and quality. For the exploratory factor analysis, we used composite reliability, Cronbach’s alpha, and its standardized version, as well as the average variance extracted measures. The same assessment criteria were employed for the confirmatory factor analysis models as for the SEM and they are described in the following subsection.

3.3.3. Structural Equation Modeling

SEM is a statistically based approach aimed at analyzing multiple variates and their interdependencies. One of the main advantages of this methodology is the possibility to investigate multifactorial constructs with many variables, which can be either directly observable or hidden—also called latent [38,39]. While SEM as a general framework allows for modeling classic approaches such as those mentioned in previous subsections, multivariate regressions, or confirmatory factor analysis, it is usually associated with creating models of complex causal structures involving some kind of path analysis [40,41,42,43]. This technique has found widespread application across various disciplines, particularly in psychological, social, or econometric research, as it can account for measurement errors in complex multivariable systems. SEM has been successfully used to evaluate complex relationships among various factors in the IT-related context. By providing insights into the key drivers of success, SEM can help project managers and researchers develop more effective strategies for managing projects and improving project outcomes.

Apart from data collection, the following stages can be distinguished in structural equation modeling [44,45]:

(1)

Model specification, which should be based on the theory and results of previous research. In this step, the necessary variables both dependent and independent along with their relations are defined. In our case, the model specification was built upon the results of exploratory factor analysis, which was based on a comprehensive literature review.

(2)

Model identification aims at finding the most parsimonious structure that reflects the links observed in the gathered data as best as possible. Only the most significant variables and relations should be introduced.

(3)

Parameter estimation, which consists of calculating values of the model parameters and accompanying errors. The most commonly used technique here is the maximum likelihood method, which is robust to change in measurement scale. Such a method of parameter estimation was used both in our confirmatory and path analysis (SEM).

(4)

Testing, which generally involves checking the quality of the model fit to the empirical data. Many techniques are available for this purpose. In the current study, we report typical absolute fit indices, such as chi-square test ${\chi }^2$ [38] and scaled ${\chi }^2$ [46]. We include frequently used Steiger’s root mean square error of approximation (RMSEA) measure [47], as well. Since these indicators are sensitive to sample size, therefore, we also present indices related to the extreme models (saturated and independent), that is, incremental fit index (IFI) [48] and comparative fit index (CFI) [49]. While modeling any phenomenon, researchers strive to include as few parameters as possible while at the same maximizing time reconstructing properties of the proposed model. For assessing and finding the most parsimonious proposals, we employed mainly Akaike information criterion (AIC) [50], Browne-Cudeck criterion (BCC) [51], and Bayes information criterion (BIC) [52]. The strength of model parameters was evaluated by standardized estimates for path coefficients, and the applied bootstrap procedure for 500 samples [53] allowed for verifying their statistical significance. The latent variable of overall project management success perception, based on four question variables was additionally assessed by composite reliability, Cronbach’s alpha and its standardized version, as well as the average variance extracted measures.

(5)

Modifications of the initial model are often necessary, for example, in the face of either insufficient values of fit indices or statistically insignificant coefficients. This was also the case in this study. We tested and analyzed many models and provided both formal statistical and substantive-based justification for our choices. For this purpose, we have taken advantage of the model specification search functionality of the Amos software [54] and additional qualitative analysis.

4. Modeling Results

4.1. Multivariate Regression Models

The main goal of the regression analysis is to determine which of the dimensions of project management success identified in the exploratory factor analysis can be used to model various project management success assessment measures. In other words, what is their importance in explaining the subjective evaluation of different project management success criteria? Another question is the extent to which the project management success criteria depend on the established success factors dimensions.

To achieve this goal, a series of stepwise regression analyses were performed, resulting in models that relate project management success with the four dimensions of project management success factors identified by the exploratory factor analysis. These dimensions were treated as the initial set of independent variables:

• F1: Agile techniques and change management (AgileChange);
• F2: Organization and people              (OrgPeople);
• F3: Stakeholders and risk analysis        (StakeRisk);
• F4: Work environment               (WorkEnv),

which led to the construction of this general linear regression formula:

$$Y=b_4\cdot AgileChange+b_3\cdot OrgPeople+b_2\cdot StakeRisk+b_1\cdot WorkEnv+b_0.$$

(1)

Values $b_{0-4}$ are regression coefficients, and the dependent variable $Y$ denotes the project management success measure. Based on the literature analysis, the following specific project management success criteria were assessed by project managers in the research questionnaire:

• Quality of the delivered product ($Quality$);
• Scope realization and requirements ($Scope$);
• Timeliness of delivery ($Time$);
• Delivery within budget ($Cost$);
• Customer satisfaction, measured by a satisfaction survey completed by the customer-side project manager; the results were made available to the project manager by the provider side ($SatCust$);
• Provider satisfaction, assessed by the provider’s project manager ($SatProv$).

Thus, the set of dependent variables used in the regression analyses can be specified as:

$$Y=\{Quality,Scope,Time,Cost,SatCust,SatProv\}$$

(2)

To find the best possible regression models for these variables, we took advantage of all the following stepwise regression methods [33]:

−

Forward—starting from zero model, step by step adding variables until the best form of model is achieved;

−

Backward—starting from the full model and analogously reducing variables in subsequent steps;

−

Using Mallow’s indicator, which refers to the estimation of least squares methods and adjusted R² that takes into account the amount of variance measured by independent variables affecting dependent variables.

The results of these stepwise analyses allowed us to identify models with decent statistical characteristics. The best regression models along with their parameters are summarized in

Table 1. Summary of the stepwise linear regression analyses results for six project management success measures used as dependent variables: $Y=\{Quality,Scope,Time,Cost,SatCust,SatProv\}$ and four identified success factor dimensions {AgileChange, $OrgPeople$, $StakeRisk$, $WorkEnv$} used as independent variables.

1.	$\mathit{Q}\mathit{u}\mathit{a}\mathit{l}\mathit{i}\mathit{t}\mathit{y}=$	$0.446\cdot AgileChange$	$+0.189\cdot OrgPeople$	$+0.353\cdot StakeRisk$		$+5.439$
	$F_{(3,151)}={19.8}^1$ $p<{0.001}^2$ $R^2={0.282}^3$	$t={5.73}^4$, $p<0.001$ $\beta ={0.395}^5\left(0.069\right)$ 6	$t=2.43$, $p=0.016$ $\beta =0.168\left(0.069\right)$	$t=4.54$, $p<0.001$ $\beta =0.313\left(0.069\right)$		$$\begin{gathered} t=70.1, \\ p<0.001 \end{gathered}$$
2.	$\mathit{S}\mathit{c}\mathit{o}\mathit{p}\mathit{e}=$			$0.199\cdot StakeRisk$		$+5.581$
	$F_{(1,153)}=2.63$ $p=0.107$ $R^2=0.017$			$t=1.62$, $p<0.107$ $\beta =0.130\left(0.080\right)$		$$\begin{gathered} t=45.6, \\ p<0.001 \end{gathered}$$
3.	$\mathit{T}\mathit{i}\mathit{m}\mathit{e}=$	$0.271\cdot AgileChange$	$+0.421\cdot OrgPeople$	$+0.255\cdot StakeRisk$	$+0.452\cdot WorkEnv$	$+4.735$
	$F_{(4,150)}=8.88$ $p<0.001$ $R^2=0.192$	$t=2.24$, $p=0.027$ $\beta =0.164\left(0.073\right)$	$t=3.48$, $p=0.001$ $\beta =0.255\left(0.073\right)$	$t=2.11$, $p=0.037$ $\beta =0.155\left(0.073\right)$	$t=3.74$, $p<0.001$ $\beta =0.275\left(0.073\right)$	$$\begin{gathered} t=39.3, \\ p<0.001 \end{gathered}$$
4.	$\mathit{C}\mathit{o}\mathit{s}\mathit{t}=$	$0.229\cdot AgileChange$			$+0.255\cdot WorkEnv$	$+5.632$
	$F_{(2,152)}=6.20$ $p=0.003$ $R^2=0.076$	$t=2.36$, $p=0.020$ $\beta =0.184\left(0.078\right)$			$t=2.62$, $p=0.010$ $\beta =0.204\left(0.078\right)$	$$\begin{gathered} t=58.0, \\ p<0.001 \end{gathered}$$
5.	$\mathit{S}\mathit{a}\mathit{t}\mathit{C}\mathit{u}\mathit{s}\mathit{t}=$	$0.406\cdot AgileChange$	$+0.445\cdot OrgPeople$	$+0.401\cdot StakeRisk$		$+5.510$
	$F_{(3,151)}=29.4$ $p<0.001$ $R^2=0.$368	$t=5.27$, $p<0.001$ $\beta =0.341\left(0.065\right)$	$t=5.77$, $p<0.001$ $\beta =0.373\left(0.065\right)$	$t=5.20$, $p<0.001$ $\beta =0.336\left(0.065\right)$		$$\begin{gathered} t=71.7, \\ p<0.001 \end{gathered}$$
6.	$\mathit{S}\mathit{a}\mathit{t}\mathit{P}\mathit{r}\mathit{o}\mathit{v}=$	$0.458\cdot AgileChange$	$+0.529\cdot OrgPeople$	$+0.415\cdot StakeRisk$		$+5.258$
	$F_{(3,151)}=40.8$ $p<0.001$ $R^2=0.$448	$t=6.23$, $p<0.001$ $\beta =0.377\left(0.060\right)$	$t=7.19$, $p<0.001$ $\beta =0.435\left(0.060\right)$	$t=5.65$, $p<0.001$ $\beta =0.341\left(0.060\right)$		$$\begin{gathered} t=71.7, \\ p<0.001 \end{gathered}$$

¹ F statistic for given degrees of freedom, ² probability level for a statistic, ³ coefficient of explained variance of a dependent variable by all independent variables in the model, ⁴ Student’s t-statistic for a coefficient, ⁵ standardized regression coefficient for a variable, ⁶ standard errors in brackets.

.

• Y = Quality. The stepwise procedure identified a statistically significant linear model (p < 0.001) of a moderate value of R². According to the Student’s t-statistics, three out of four independent variables included in the regression and the intercept were meaningfully different from zero. Interestingly, the study subjects’ opinions did not take into account the $WorkEnv$ factor when evaluating the quality of the delivered product. Conversely, all the remaining variables were associated with this success criterion. The standardized $\beta$ coefficients indicated that the most influential factor was $AgileChange$ (agile techniques and change management), followed by $StakeRisk$ (stakeholders and risk analysis). In contrast, the $OrgPeople$ (organization and people) factor had the smallest impact.
• Y = $Scope$. The most challenging task was to find a decent regression for the scope and requirements dependent variable. The best model, in terms of formal statistical measures, was the linear regression with $StakeRisk$ as the only independent variable. The intercept was statistically different from zero, and the probability levels of both the Student’s t-statistic for $StakeRisk$ and the F-statistic for the whole model were slightly higher than the more relaxed limit of 0.1. The model suggests that if any factor was related to the subjective assessment of the project management success Scope criterion, it was probably the $StakeRisk$ variable. However, given the small value of R², one should be very cautious in interpreting this outcome, and further research is required in this aspect.
• Y = $Time$. We found that the best model for completing the project on time involved all of the considered independent variables. The moderate R² was statistically different from zero, as shown by the F-statistics value. Similarly, Student’s t-statistics confirmed that the parameters for all variables, along with the intercept, were also statistically relevant. The beta values suggested that the most influential factors were $WorkEnv$ and $OrgPeople$, while $AgileChange$ and $StakeRisk$ were somewhat less important. However, the comparable values of standardized regression coefficients indicated that all of these factors contributed substantially to the assessment of the timeliness aspect of project management success.
• Y = $Cost$. The proposed regression model for delivering the project within a budget includes only two variables:$WorkEnv$ and $AgileChange$, with similar standardized beta coefficients. From the formal point of view, the model is acceptable, since all included parameters and R² are statistically significantly different from zero. However, the very small value of R² raises questions about whether the included independent variables sufficiently explain the Cost dependent variable. Additional investigations are needed to explore this issue.
• Y = $SatCust$. The regression analysis for the customer satisfaction dependent measure identified three factors that could explain it. As expected, $OrgPeople$ was the most influential. Beta coefficients for the other two variables $AgileChange$ and $StakeRisk$ were only slightly smaller. The statistical tests confirmed the good quality of this model, and the R² was decidedly larger than for the previous four success criteria.
• Y = $SatProv$. The presented regression model for provider satisfaction proved to be the best in terms of the results of formal verification, as well as the highest value of R². Similar to the customer satisfaction model, the most influential factor was $OrgPeople$, followed by $AgileChange$ and $StakeRisk$, with only somewhat smaller standardized beta coefficients.

Overall, the presented models show that all the project management success factors identified by exploratory factor analysis play a significant role in determining the six success criteria. However, only the Time criterion involved all four factors. For other success dimensions directly assessed by project managers, only a subset of these factors was included in the models. Although none of the factors were present in all regressions, $AgileChange$ and $StakeRisk$ variables appeared in five out of six models ($AgileChange$ was absent in Scope while $StakeRisk$ was not present in Cost). The $OrgPeople$ variable contributed to four cases (Quality, Time, SatCust, and SatProv), whereas $WorkEnv$ occurred only with two criteria, Time and Cost. This suggests that survey participants did not directly associate $WorkEnv$ with the other four criteria, although it could have a significant indirect impact. Such an explanation is supported by the results of the SEM described in Section 4.3.

The worst regression model, from a formal point of view, was found for the scope and requirements criterion, with only one independent variable included ($StakeRisk$), which was on the verge of significance. However, this finding can still be of substantial theoretical and practical importance. It suggests that, from the perspective of project managers in the present study, this criterion is either not associated with project management success or depends on factors that were not included in the study. Another possibility is that this dimension relates to questions that were eliminated during the exploratory factor analysis, or the Scope criterion is mediated by other factors. Nevertheless, the presented models can be generally regarded as some kind of additional validation of the exploratory factor analysis outcomes.

4.2. Confirmatory Factor Analysis Model

Assuming that the dimensions obtained in the exploratory factor analysis from the Zaleski and Michalski [5] work are orthogonal, a confirmatory factor analysis model was constructed for success factors of IT service projects. The model includes the identified four latent dimensions: F1—Agile Techniques and Change Management, F2—Organization and People, F3—Stakeholders and Risk Analysis, and F4—Work Environment, which are composed based on 16 survey questions. These questions are provided in Table A1 of Appendix A, while the model structure along with weight coefficients is demonstrated in Figure 1. All the covariances were statistically significant at p < 0.005, whereas the regression weights were statistically meaningful at p < 0.001.

The fit measures of the model are put together in

Table 2. Fit measures for the confirmatory factor analysis model of success factors in IT service projects.

${\mathbf{\chi }}^{{\displaystyle 2}}$	df	p	${\displaystyle {\chi }^2/\mathbf{d}\mathbf{f}}$	IFI	CFI	RMSEA
140	98	0.003	1.429	0.948	0.946	0.053

. The ratio of ${\mathsf{\chi }}^2/df$, equal to 1.429, is much smaller than the suggested value of 5 by Schumacker and Lomax [45] or even the more restrictive value of 2 recommended by Kline [44]. This indicates a good fit. Additionally, the indicators of IFI (0.948) and CFI (0.946), which are above the recommended value of 0.9 [55], along with an RMSEA of less than 0.1, further confirm the good quality of the model. The RMSEA parameter is only slightly higher (0.053) than the more restrictive value of 0.05 suggested by Kline [44].

To further validate the structure of the success factors in the IT service projects model, we randomly divided the entire data sample into two nearly equal groups (77 vs. 78) and performed the exploratory factor analysis on one group and confirmatory factor analysis on the other. We repeated this procedure 10 times. The reliability and validity measures for a series of exploratory factor analysis models for all 10 random subsamples are provided in

Table A2. Measures of reliability and validity for a series of exploratory factor analysis models of success factors in IT service projects, estimated by using PCA as the extraction method, followed by Varimax rotation with Kaiser normalization, for ten random subsamples.

Sample No.	Dimension	CR 1	CA 2	Std. CA 3	AVE 4
1	F1	0.744	0.741	0.742	0.378
	F2	0.856	0.831	0.832	0.598
	F3	0.830	0.784	0.787	0.555
	F4	0.781	0.668	0.673	0.546
2	F1	0.815	0.784	0.785	0.471
	F2	0.779	0.774	0.773	0.471
	F3	0.791	0.723	0.725	0.498
	F4	0.730	0.638	0.635	0.479
3	F1	0.814	0.800	0.801	0.473
	F2	0.866	0.834	0.834	0.617
	F3	0.832	0.780	0.781	0.557
	F4	0.779	0.644	0.637	0.557
4	F1	0.772	0.745	0.745	0.406
	F2	0.834	0.813	0.811	0.559
	F3	0.826	0.774	0.780	0.553
	F4	0.800	0.684	0.682	0.577
5	F1	0.832	0.794	0.795	0.499
	F2	0.834	0.825	0.823	0.560
	F3	0.845	0.789	0.788	0.581
	F4	0.759	0.618	0.621	0.515
6	F1	0.791	0.787	0.788	0.445
	F2	0.855	0.847	0.846	0.597
	F3	0.821	0.789	0.789	0.538
	F4	0.763	0.614	0.633	0.522
7	F1	0.822	0.779	0.779	0.480
	F2	0.831	0.828	0.828	0.552
	F3	0.818	0.751	0.755	0.539
	F4	0.744	0.639	0.627	0.495
8	F1	0.734	0.706	0.704	0.365
	F2	0.833	0.820	0.821	0.556
	F3	0.809	0.801	0.799	0.523
	F4	0.789	0.679	0.680	0.556
9	F1	0.835	0.805	0.805	0.506
	F2	0.813	0.796	0.796	0.522
	F3	0.837	0.789	0.793	0.566
	F4	0.777	0.647	0.644	0.539
10	F1	0.795	0.768	0.767	0.439
	F2	0.744	0.770	0.772	0.430
	F3	0.819	0.780	0.780	0.537
	F4	0.717	0.643	0.630	0.483

¹ Composite reliability, ² Cronbach’s alpha, ³ standardized Cronbach’s alpha, ⁴ average variance extracted.

of Appendix B. We used PCA as the extraction method, followed by Varimax rotation with Kaiser normalization. The measures of quality and fit for corresponding confirmatory factor analysis models are summarized in

Table 3. Fit measures of series of confirmatory factor models of success factors in IT service projects for ten random subsamples.

Sample No.	p	${\chi }^2/\mathbf{d}\mathbf{f}$	IFI	CFI	RMSEA
1	0.008	1.375	0.913	0.908	0.07
2	0.041	1.262	0.944	0.941	0.058
3	0.072	1.216	0.951	0.949	0.053
4	0.109	1.179	0.958	0.956	0.048
5	0.015	1.336	0.924	0.92	0.066
6	0.022	1.308	0.919	0.914	0.063
7	0.082	1.204	0.952	0.949	0.052
8	0.045	1.254	0.938	0.934	0.058
9	0.030	1.286	0.933	0.929	0.061
10	0.088	1.198	0.951	0.948	0.051

.

For exploratory factor analysis models, the factorial structures exhibited good quality in terms of composite reliability and validity. Similarly, the fit measures of confirmatory factor models were also satisfactory and consistent with the overall confirmatory factor analysis model characterized in Figure 1 and Table 2. All results were within acceptable limits and further confirmed the good validity of the presented four-factorial model.

4.3. Structural Equations Models

In the study described by Zaleski and Michalski [5], the questionnaire included questions related to the perception of success in the IT service projects under evaluation. To verify and further validate whether the factorial structure of project management success corresponds to the overall impression of project success assessed by direct questions, SEM was employed. Initially, we used a model based on the confirmatory factor analysis structure with assumed orthogonal dimensions. Then, we utilized the model specification search procedure available in the Amos software to identify the best overall model that considers both the factorial approach and the overall perception of the project’s success within the realm of information technology services.

4.3.1. Initial Orthogonal SEM Model

There were four questions asked of experienced project managers regarding the perception of success in IT service project management, which is under evaluation. These questions are compiled in

Table 4. A set of questions on the overall project management success perception asked of respondents in a questionnaire from the study of Zaleski and Michalski [5].

Abbreviation	Short Description	Question
SP_Q1_GenSuccess	Generally successful	Generally, I consider the project as successful
SP_Q2_WinNewProj	Helps win new projects	The implementation of the project increases the chances of obtaining new projects
SP_Q3_Achievement	Great achievement	I think the project was a great achievement
SP_Q4_FailureRev	Complete failure (reversed)	Overall, the project was a complete failure

, along with their respective abbreviations and shortened versions used throughout the paper.

The overall direct subjective opinion of project management success was rated on the seven-point Likert scale ranging from strongly disagree (1) to strongly agree (7), similar to the questions used in the exploratory factor analysis. To minimize response bias, question number four used an inverted measurement scale. Therefore, for all calculations presented in this study, the responses to this question were reversed to match the remaining three questions. Figure 2 depicts the SEM model of the overall perception of project management success dimension, including the regression weights.

The model’s fit parameters for this dimension were satisfactory and amounted to: ${\chi }^2=$ 4.862, p = 0.088, ${\chi }^2/df=$ 2.431, CFI = 0.991, IFI = 0.991, RMSEA = 0.096. All the regression weights were statistically significant at the level of p < 0.005. Additionally, a group of these questions underwent qualitative evaluation, similar to examining dimensions for exploratory factor analyses.

Table 5. A qualitative assessment of the overall subjective perception of IT service project management success.

Latent Variable	CR 1	CA 2	Std. CA 3	AVE 4
Overall Perception of PM Success	0.91	0.867	0.867	0.718

¹ Composite reliability, ² Cronbach’s alpha, ³ standardized Cronbach’s alpha, ⁴ average variance extracted.

provides the measures of reliability and validity.

We took advantage of the overall perception of PM success dimension model and the results of confirmatory factor analysis to develop a combined SEM model. Similar to the confirmatory analysis, we assumed that the dimensions from exploratory factor analysis are fully orthogonal. The result of this approach is schematically demonstrated in Figure 3.

The left part of the model represents the structure obtained through exploratory factor analysis and validated by the series of confirmatory factor analyses. The right part of the model presents the perceived success of project management. The variables constituting the latent variables here are the same as in Figure 1 and Figure 2. We estimated the model using the maximum likelihood method and the bootstrap procedure for 500 samples to provide parameter estimates [53]. The full model, along with coefficients, is provided in Figure A1 and

Table A3. Regression weights for the initial orthogonal model, which employed the bootstrap procedure.

Regression Relation			Regression Weight	Standard Error	p
F1_Q24	←	F1:AgileChange	0.712	0.506	0.004
F1_Q25	←	F1:AgileChange	0.905	0.652	0.004
F1_Q30	←	F1:AgileChange	0.761	0.531	0.004
F1_Q31	←	F1:AgileChange	0.740	0.514	0.004
F1_Q32	←	F1:AgileChange	1	1
F2_Q01	←	F2:OrgPeople	0.709	0.545	0.004
F2_Q03	←	F2:OrgPeople	0.722	0.562	0.004
F2_Q08	←	F2:OrgPeople	0.784	0.624	0.004
F2_Q09	←	F2:OrgPeople	1	1
F3_Q34	←	F3:StakeRisk	0.749	0.504	0.004
F3_Q41	←	F3:StakeRisk	0.856	0.685	0.004
F3_Q42	←	F3:StakeRisk	1	1
F3_Q44	←	F3:StakeRisk	0.892	0.708	0.004
F4_Q06	←	F4:WorkEnv	0.937	0.576	0.004
F4_Q07	←	F4:WorkEnv	1	1
F4_Q38	←	F4:WorkEnv	0.574	0.292	0.004
SP_Q1_GenSuccess	←	SuccessPer	0.884	0.764	0.004
SP_Q2_WinNewProj	←	SuccessPer	0.677	0.547	0.004
SP_Q3_Achievement	←	SuccessPer	1	1
SP_Q4_FailureRev	←	SuccessPer	0.730	0.569	0.004
F1:AgileChange	←	SuccessDim	0.773	0.556	0.004
F2:OrgPeople	←	SuccessDim	1	1
F3:StakeRisk	←	SuccessDim	0.729	0.504	0.004
F4:WorkEnv	←	SuccessDim	0.485	0.107	0.030

of Appendix C. All regression weights computed by the bootstrap procedure are statistically significant at p < 0.005. Additionally, the covariance between Dimensions-based PM Success and Overall Perception of PM Success is statistically significant at p = 0.003. The fit measures of this SEM model are put together in

Table 6. Fit measures of the SEM model combining the approach of factorial orthogonal structure to project management success with the overall perceived project management success.

${\chi }^2$	df	p	${\chi }^2/\mathbf{d}\mathbf{f}$	RMSEA	IFI	CFI	AIC	BIC
248	165	<0.001	1.503	0.0571	0.934	0.933	338.0	474.9

.

The obtained values indicate that the model exhibits a good quality. There is a strong and significant correlation between project management success factors and respondents’ subjective perception of project management success. In practice, this confirms the consistent relationship between the obtained factorial structure of project management success and the manager’s overall subjective perception of the project management success.

4.3.2. Search for the Most Appropriate SEM Model

Although the model presented in Section 4.3.1 is of decent quality, certain theoretical and practical premises suggest that different structures may also fit the gathered data well. Since the identified factors may be interconnected, we explored other possible models taking advantage of the model specification search functionality of the Amos software [54].

Considering the lowest regression coefficient (0.49) between factor four F4:Work Environment and Dimensions-based PM Success, and the fact that this dimension could possibly influence other factors, we set the relationship as optional. In addition, we included potential relationships between all four identified factors in the search for the best solutions procedure. As a result, the overall number of possible optional relationships amounted to 13 (Figure 4), resulting in a search space of 2¹³ = 8192 SEM models.

All the models were assessed according to several criteria, including Akaike information criterion (AIC) [50], Browne–Cudeck criterion (BCC) [51], Bayes information criterion (BIC) [52], ${\mathsf{\chi }}^2/df$ [45], CFI [49], IFI [48], and RMSEA [38]. An example of the search results according to the BIC criterion is shown in Figure 5, where the optimal number of model parameters is clearly visible.

We utilized all subset search methods, and the entire procedure lasted approximately 30 s. The results, which show the criteria of the best models found by the search algorithm, are presented in Figure 6.

Model number 35, denoted further in this paper as Model 35 (0), was found to be the best based on many of the employed fitting quality criteria. It had the lowest ${\chi }^2-df$, ${\chi }^2/df$, AIC, BIC, BCC, and RMSEA parameters among all the examined models, while its CFIs were the highest. Therefore, we subjected this model to further analysis. Its simplified structure showing relationships between latent variables is displayed in Figure 7.

The fitting parameters of the model are decent and are provided in

Table A4. Measures of fit for Model 35(0).

${\chi }^2$	df	p	${\chi }^2/\mathbf{d}\mathbf{f}$	RMSEA	IFI	CFI	AIC	BIC
222	162	0.001	1.370	0.0490	0.952	0.951	318.0	464.1

of Appendix D. The full model, which includes regression coefficients, their standard errors, and levels of significance, is displayed in Figure A2 and

Table A5. Regression weights for Model 35(0).

Regression Relation			Regression Weight	Standard Error	p
F1_Q24	←	F1:AgileChange	0.710	0.112	0.004
F1_Q25	←	F1:AgileChange	0.915	0.115	0.004
F1_Q30	←	F1:AgileChange	0.764	0.12	0.006
F1_Q31	←	F1:AgileChange	0.743	0.104	0.004
F1_Q32	←	F1:AgileChange	1
F2_Q01	←	F2:OrgPeople	0.703	0.087	0.007
F2_Q03	←	F2:OrgPeople	0.719	0.082	0.004
F2_Q08	←	F2:OrgPeople	0.788	0.077	0.004
F2_Q09	←	F2:OrgPeople	1
F3_Q34	←	F3:StakeRisk	0.751	0.111	0.005
F3_Q41	←	F3:StakeRisk	0.857	0.095	0.008
F3_Q42	←	F3:StakeRisk	1
F3_Q44	←	F3:StakeRisk	0.893	0.101	0.004
F4_Q06	←	F4:WorkEnv	1
F4_Q07	←	F4:WorkEnv	0.947	0.189	0.005
F4_Q38	←	F4:WorkEnv	0.580	0.141	0.008
SP_Q1_GenSuccess	←	SuccessPer	0.867	0.061	0.005
SP_Q2_WinNewProj	←	SuccessPer	0.678	0.066	0.004
SP_Q3_Achievement	←	SuccessPer	1
SP_Q4_FailureRev	←	SuccessPer	0.725	0.075	0.003
F1:AgileChange	←	SuccessDim	0.534	0.120	0.003
F2:OrgPeople	←	SuccessDim	1
F3:StakeRisk	←	SuccessDim	0.597	0.116	0.002
F1:AgileChange	←	F4:WorkEnv	0.206	0.091	0.014
F2:OrgPeople	←	F4:WorkEnv	0.428	0.117	0.003
F2:OrgPeople	←	F3:StakeRisk	–0.489	0.238	0.090
F4:WorkEnv	←	F3:StakeRisk	0.415	0.128	0.010

of Appendix D. The obtained structure differs qualitatively from the initial orthogonal model, which was derived from the exploratory and confirmatory factor analyses. First, there is no direct relationship between the fourth factor F4:Work Environment and the Dimensions-based PM Success. Secondly, the model includes relationships between the four identified dimensions of IT services project management success latent variable, which does not exist in the orthogonal approach. The covariance between Dimensions-based PM Success and Overall Perception of PM Success is statistically significant at p = 0.002, which is consistent with the initial orthogonal model.

Almost all regression weights computed by the bootstrap procedure were statistically significant at p < 0.01, except for one relationship between the second and the third factor (F2:Organization and People ← F3:Stakeholders and Risk). In this case, p amounted to 0.09 and the beta regression value was negative (−0.489). Therefore, we simplified the model by excluding this link from further analysis, making it easier to interpret. After the exclusion, there were no negative relationships, and all regression weights were statistically significant. However, we noticed that the obtained link directions may not be the best interpretable ones from the practical and theoretical points of view. Hence, we decided to further search for an SEM model that decently fits the data and allows for reasonable interpretation. For this purpose, we developed eight possible variants of directions for the three identified relationships between the four dimensions. We assumed that there is no direct link between F4:Work Environment and the Dimensions-based PM Success. All of these variants are schematically demonstrated in Figure 8.

We estimated model parameters for all these variants. The obtained fit measures are put together in

Table 7. Fit measures for all eight variants of the SEM Model 35, which depend on the directions of the dimensions’ relations. (1b, 6g, 4e, 2c).

Variants of Model 35	${\chi }^2/\mathbf{d}\mathbf{f}$	RMSEA	IFI	CFI	AIC	BIC
(1) F1←F4, F2←F4, F4←F3	1.397	0.0508	0.949	0.948	321.7	464.8
(2) F1←F4, F2←F4, F3←F4	1.409	0.0515	0.947	0.946	323.7	466.7
(3) F1←F4, F4←F2, F4←F3	1.425	0.0526	0.945	0.944	326.4	469.4
(4) F1←F4, F4←F2, F3←F4	1.406	0.0513	0.947	0.946	323.2	466.2
(5) F4←F1, F2←F4, F4←F3	1.428	0.0527	0.945	0.943	326.7	469.9
(6) F4←F1, F2←F4, F3←F4	1.402	0.0511	0.948	0.947	322.5	465.6
(7) F4←F1, F4←F2, F4←F3	1.460	0.0546	0.941	0.939	331.9	474.9
(8) F4←F1, F4←F2, F3←F4	1.448	0.0539	0.942	0.941	330.0	473.0

.

Upon closer analysis, it was revealed that in Model 35, variants number (3), (5), (7), and (8) had several regression weights that were statistically insignificant at the level of 0.05. As a result, they were considered inappropriate and excluded from further analysis. The remaining four variants were ranked according to the presented model quality criteria. When taking into account the criteria of ${\mathsf{\chi }}^2/\mathrm{d}\mathrm{f}$, RMSEA, AIC, and BIC, the order from best to worst was (1) > (6) > (4) > (2). According to IFI and CFI, the ranking was similar: (1) > (6) > (4) = (2). Since the differences in fit measures for those four variants of Model 35 were not large, and all the regression coefficients were statistically significant at p < 0.05, we examined them more thoroughly in terms of relationships between dimensions and variables. These relationships are presented in Figure A3, Figure A4, Figure A5 and Figure A6 and

Table A6. Combined total effects (direct + indirect) and significance levels (in brackets) for Model 35(1).

Total Effects	F1:Agile Change	F2:Org People	F3:Stake Risk	F4:Work Env	Success Dim
F1:Agile Change	0	0	0.102 (0.017)	0.255 (0.01)	0.839 (0.003)
F2:Org People	0	0	0.152 (0.009)	0.38 (0.004)	1.128 (0.002)
F3:Stake Risk	0	0	0	0	0.844 (0.005)
F4:Work Env	0	0	0.399 (0.009)	0	0.337 (0.009)

,

Table A7. Combined total effects (direct + indirect) and significance levels (in brackets) for Model 35(2).

Total Effects	F1:Agile Change	F2:Org People	F3:Stake Risk	F4:Work Env	Success Dim
F1:Agile Change	0	0	0	0.231 (0.013)	0.751 (0.003)
F2:Org People	0	0	0	0.360 (0.003)	1.000
F3:Stake Risk	0	0	0	0.190 (0.013)	0.775 (0.003)

,

Table A8. Combined total effects (direct + indirect) and significance levels (in brackets) for Model 35(4).

Total Effects	F1:Agile Change	F2:Org People	F3:Stake Risk	F4:Work Env	Success Dim
F1:Agile Change	0	0.097 (0.005)	0	0.257 (0.005)	0.769 (0.003)
F2:Org People	0	0	0	0	1
F3:Stake Risk	0	0.078 (0.013)	0	0.207 (0.024)	0.782 (0.003)
F4:Work Env	0	0.377 (0.005)	0	0	0.377 (0.005)

and

Table A9. Combined total effects (direct + indirect) and significance levels (in brackets) for Model 35(6).

Total Effects	F1:Agile Change	F2:Org People	F3:Stake Risk	F4:Work Env	Success Dim
F1:Agile Change	0	0	0	0.000	0.828 (0.006)
F2:Org People	0.173 (0.016)	0	0	0.371 (0.004)	1.143 (0.004)
F3:Stake Risk	0.092 (0.023)	0	0	0.198 (0.019)	0.865 (0.006)
F4:Work Env	0.465 (0.010)	0	0	0.000	0.385 (0.012)

of Appendix D.

In variant (1), the relationship between F1—Agile and Change ← F4:Work Environment was relatively weak: 0.25 (see Figure A3 of Appendix D), so we checked the model after removing this link. Such a modification worsened the model and resulted in a decrease in the weights of other relationships such as between F2:Organization and People ← F4:Work Environment from 0.38 to 0.30, F4_Q07 ← F4:Work Environment from 0.97 to 0.87, and F4_Q38 ← F4:Work Environment from 0.60 to 0.52. Other coefficients did not improve. As a result, we decided to keep this link in the model.

Variant (2) of Model 35 had a similar issue, with the links between F3:Stakeholders and Risk ← F4:Work Environment and F1—Agile and Change ← F4:Work Environment being as low as 0.19, and 0.23 (see Figure A4 of Appendix D). Removing the F3:Stakeholders and Risk ← F4:Work Environment relationship caused the model not to converge while deleting F1—Agile and Change ← F4:Work Environment resulted in a further decrease in the small link between F3:Stakeholders and Risk ← F4:Work Environment to 0.14. However, these changes did not increase weights in other relationships.

The regression weights for the F3:Stakeholders and Risk ← F4:Work Environment and F1—Agile and Change ← F4:Work Environment relationships in variant (4) of Model 35 were 0.21 and 0.26, respectively (see Figure A5 of Appendix D). These values are not satisfactory, so we checked if removing them one by one would improve the model. Without the former link, the F1—Agile and Change ← F4:Work Environment weight decreased to 0.21 and there was an improvement in the F3:Stakeholders and Risk ← Dimensions-based PM Success link from 0.70 to 0.77. Without the latter relationship, the F3:Stakeholders and Risk ← F4:Work Environment link weight decreased to 0.15 and there was a small improvement in the F1—Agile and Change ← Dimensions-based PM Success relationship from 0.67 to 0.77.

As in the previous variants, variant (6) of Model 35 includes a relationship of a relatively small value of 0.2, that is, F3:Stakeholders and Risk ← F4:Work Environment (see Figure A6 of Appendix D). However, this time, after removing this link, the decrease in regression weights was marginal. The biggest drop was merely 0.02 and was observed for F2:Organization and People ← F4:Work Environment (from 0.37 to 0.35). Meanwhile, there was a positive change in the regression weight value for the F1—Agile and Change ← Dimensions-based PM Success relationship from 0.83 to 0.85, and a significant increase for the F3:Stakeholders and Risk ← Dimensions-based PM Success link, from 0.79 to 0.85.

4.3.3. Final SEM for Success Factors of IT Services Project Management

Based on the above analysis and the examination of various versions of the consecutive variants, it appears that variant (6) of Model 35 without the smallest relationship (F3:Stakeholders and Risk ← F4:Work Environment) is relatively the best. The model is not only simpler than all the other analyzed variants but also contains no regression weights smaller than 0.35. In other variants, the smallest values oscillated close to 0.2. This model is regarded as the final in this paper and is denoted as Model 35(6 mod). We demonstrate this in Figure 9.

The simplified version of the variant (6) model exhibits similar fit quality measures as the other variants, indicating its decent quality. The values of these measures are provided in

Table 8. Fit measures of SEM Model 35 (6 mod), which integrates the project management success factorial orthogonal structure with the overall perceived project management success.

${\chi }^2$	df	p	${\chi }^2/\mathbf{d}\mathbf{f}$	RMSEA	IFI	CFI	AIC	BIC
235	164	0.0002	1.431	0.0529	0.944	0.943	326.7	466.7

. For all criteria, they were found to be better than those of the initial orthogonal model (Table 6).

All regression weights computed by the bootstrap procedure based on 500 samples are statistically significant with p < 0.01 and are given in

Table 9. SEM regression weights and their statistical significance for Model 35(6 mod).

Regression Relation			Regression Weight	Standard Error	p
F1_Q24	←	F1:AgileChange	0.711	0.111	0.004
F1_Q25	←	F1:AgileChange	0.902	0.113	0.004
F1_Q30	←	F1:AgileChange	0.756	0.119	0.006
F1_Q31	←	F1:AgileChange	0.733	0.102	0.004
F1_Q32	←	F1:AgileChange	1
F2_Q01	←	F2:OrgPeople	0.717	0.088	0.008
F2_Q03	←	F2:OrgPeople	0.722	0.084	0.004
F2_Q08	←	F2:OrgPeople	0.795	0.078	0.005
F2_Q09	←	F2:OrgPeople	1
F3_Q34	←	F3:StakeRisk	0.747	0.112	0.005
F3_Q41	←	F3:StakeRisk	0.858	0.095	0.007
F3_Q42	←	F3:StakeRisk	1
F3_Q44	←	F3:StakeRisk	0.890	0.101	0.004
F4_Q06	←	F4:WorkEnv	1
F4_Q07	←	F4:WorkEnv	0.959	0.202	0.005
F4_Q38	←	F4:WorkEnv	0.581	0.145	0.009
SP_Q1_GenSuccess	←	SuccessPer	0.874	0.061	0.006
SP_Q2_WinNewProj	←	SuccessPer	0.677	0.066	0.004
SP_Q3_Achievement	←	SuccessPer	1
SP_Q4_FailureRev	←	SuccessPer	0.727	0.075	0.003
F1:AgileChange	←	SuccessDim	0.846	0.153	0.006
F2:OrgPeople	←	SuccessDim	1
F3:StakeRisk	←	SuccessDim	0.852	0.153	0.004
F4:WorkEnv	←	F1:AgileChange	0.459	0.132	0.005
F2:OrgPeople	←	F4:WorkEnv	0.346	0.110	0.005

along with standard errors. The covariance between Dimensions-based PM Success and Overall Perception of PM Success is also statistically significant at p = 0.002.

The total effects including both direct and indirect relationships together with corresponding significance levels are provided in

Table 10. Combined total effects (direct and indirect) and significance levels (in brackets) for SEM Model 35(6 mod).

Total Effects	F1:Agile Change	F2:Org People	F3:Stake Risk	F4:Work Env	Success Dim
F1:Agile Change		0	0	0	0.846 (0.006)
F2:Org People	0.159 (0.006)	0	0	0.346 (0.005)	1.134 (0.005)
F3:Stake Risk	0	0	0	0	0.852 (0.004)
F4:Work Env	0.459 (0.005)	0	0	0	0.389 (0.003)

.

The modified variant (6) of Model 35 appears not only to be well-suited to the obtained data, but also logical and interpretable. Unlike the initial orthogonal approach derived from the exploratory and confirmatory factor analyses, the model reveals an interesting structure of latent variables. Firstly, the fourth dimension F4:Work Environment is not directly connected with Dimensions-based PM Success. However, its indirect impact (0.389) cannot be negligible. This was not obvious, but it seems reasonable as the dimension is related to setting the right conditions rather than directly influencing the process of achieving success in project management.

Secondly, the model demonstrates that the F1:Agile and Change dimension has an impact on F4:Work Environment, which in turn influences the F2:Organization and People latent variable. It is not surprising that F4:Work Environment, which is determined in this study by questions related to team location, workspace conditions, and team independence, directly affects F2:Organization and People. These features determine the way work is organized or how people perceive executive support, which can improve the motivation of the project team. For example, the close physical location of project team members to the board of directors may strengthen the relations between them and increase the speed of decision-making. As a result, this may increase the subjective assessment of executive support, positively affect the quality of collaboration between project team members, and increase the sustainability of the project management process. It should be noted that the observed relationship is one of the main principles of the agile manifesto.

The significant relationship between F1:Agile and Change and F4:Work Environment also indicates that the application of the agile philosophy to project management has a significant impact on aspects of team workspace and affects the level of autonomy of project teams. It appears that appropriate solutions in the work environment facilitate the effective application of agile-based techniques such as Kanban, pull model, or agile documentation for sustainable and successful project management in the IT services domain.

Additionally, F1:Agile and Change is strongly linked with the main Dimensions-based PM Success latent variable. This observation is supported by the total effect weights presented in Table 10. Such a path of direct and indirect effects suggests a strong dependence of the whole project management success on the F1:Agile and Change dimension. In light of this result, one could consider F1:Agile and Change as one of the most important aspects of project management that could determine if other factors translate successfully to project management success in the IT services area.

Furthermore, a considerable and significant covariance between Dimensions-based PM Success and Overall Perception of PM Success shows a high validity of the conducted research. It is additionally supported by the good validity and reliability measures of the latter overall subjective dimension (Table 5).

5. Discussion

5.1. Discussion of Multivariate Regression Models

The regression analysis performed verified the key factors that have the greatest impact on the success of IT service projects. Our regression models involved all factors that were identified by the exploratory factor analysis ($AgileChange$, $OrgPeople$, $StakeRisk$, $WorkEnv$). They appeared in different configurations and strongly varied in their contribution to explaining the dependent success criteria (Quality, Scope, Time, Cost, SatCust, SatProv). Compared to similar studies conducted by other investigators, differences were observed in both quantitative and qualitative areas.

For example, in the seminal research of Chow and Cao [56], they build regression models for four dependent variables, not including customer and provider satisfaction (SatCust, SatProv). As our results clearly show, the satisfaction issue is substantial in better understanding the project management success factors. Chow and Cao [56] considered, at first, 39 attributes that were consolidated by the use of Cronbach’s alpha coefficient to 12 dependent factors. However, they have not provided detailed results in this regard. Next, by applying the stepwise approach, they finally restricted the number of factors to six (agile software techniques, customer involvement, delivery strategy, project management, team environment, and team capability) that contributed the most significantly to explaining the variability of dependent variables. Thus, at this step, the screening procedure aimed at finding the critical success factors was performed during the selection of dependent variables to regression models. Their set of initial success factors and their characteristics were based on a literature review. In our study, all the initial four factors were included in various configurations in the models. It was probably because these factors and their specific measuring question variables were identified by an extensive literature review and then thoroughly refined during the full and extensive exploratory factor analysis process.

Stankovic et al. [57], inspired by the Chow and Cao [56] study, performed similar research on former Yugoslavia IT companies. As dependent variables, they used the same four success criteria as Chow and Cao [56] without two variables regarding satisfaction that were included in our analysis. Stankovic et al. [57] started their analysis with the same set of initial success factors as Chow and Cao [56] but performed a more comprehensive refining process in the form of exploratory factor analysis. The applied scree plot suggested 12 factors, however their components seemed to be significantly interrelated. Therefore, for creating regression models, they decided to use dependent variables identified by Chow and Cao [56]. Three out of four developed regressions were not statistically significant, therefore analyzing even significant factors within these models is inconclusive. The last model for the Cost criterion was statistically significant with four (project definition, project management, project nature, project schedule) out of twelve predictors having regression coefficients statistically different than zero. Interestingly, none of those variables were the same as in the corresponding regression provided by Chow and Cao [56]. Moreover, Stankovic et al. [57] neither applied any stepwise procedures nor created the model with only those four significant predictors. There is a substantial chance that removing eight dependent variables from the model would make the whole model insignificant. All the presented formal statistical problems are probably related to an extremely small sample size. This study’s findings are based on barely 23 fully completed surveys, which is much smaller than in the Chow and Cao [56] work (n = 109) and decidedly less than in our present study (n = 155).

Another attempt to verify the list of success factors identified by Chow and Cao, [56] was performed by Brown [58]. He replicated the multivariate regression analysis on, underrepresented in previous studies, IT companies operating in the United States of America (USA) that were mainly involved in large and complex agile projects. The three dependent and twelve independent variables were the same as in the study of Chow and Cao [56], however, Brown [58] put more emphasis on the formal side of the modeling than was the case in previous studies. The sample size was also bigger (n = 127) than in earlier investigations and included participants from 16 states in the USA. He found six factors that considerably influenced success criteria, that is delivery strategy, management commitment, project definition, project nature, project schedule, and project type.

Similar research was conducted by Stanberry [59], which involved the results from 132 practitioners located around the world but working for USA-based, large global software companies. She showed that a different set of five (delivery strategy, project definition, project management, project nature, team capability) out of twelve factors originally specified by Chow and Cao [56] have a significant effect on the four success criteria: Quality, Scope, Time, Cost. Although the formal side of constructing the models was on a decent level, Stanberry [59] did not apply any stepwise procedures of variable selections, and all inferences are based on regression coefficients and their statistical importance. Thus, just like in the work of Stankovic et al. [57], it is uncertain whether the presented models would have the same meaningful statistical qualities after removing insignificant factors from the equation. For example, excluding one or more variables may cause the remaining variables to be insignificant due to intercorrelations between them. Moreover, the whole model can turn out to be statistically irrelevant. Therefore, after determining that a given factor is not statistically different than zero, the whole regression should be re-estimated without it. In this study, we adhered to this recommendation and employed various types of stepwise selection techniques to avoid such problems.

From a substantive point of view, when comparing our factors and their components with the results obtained by Chow and Cao [56], Stankovic et al. [57], Brown [58], and Stanberry [59], we can observe some similarities. The key factor in our study agile techniques and change management ($AgileChange$), partly aligns with their agile software engineering techniques and project management process factors.

It is strange that, unlike in the initial work of Chow and Cao [56] and our current study, agile software engineering techniques did not significantly contribute to any of the dependent success criteria in studies reported by Stankovic et al. [57], Brown [58], and Stanberry [59]. This fact is puzzling, given that in all of these studies, IT projects were realized using at least some components of the agile approach. In our present research, the $AgileChange$ variable, which is directly associated with agile software engineering techniques from the studies described above, is the most common factor in our models. It appears in five regressions and is absent only for the Scope dependent variable.

Our organization and people ($OrgPeople$) factor, on the other hand, appears to be somewhat related to the combination of team capability, organizational environment, and management commitment factors. It is worth noting that the latter two factors were not statistically significant in any regression from the Chow and Cao [56] work. In all four compared studies, only Organizational environment did not appear in any of the proposed regressions. In contrast, the $OrgPeople$ factor from our study, which directly addresses organizational issues, is included in four (Quality, Time, SatCust, SatProv) out of six regressions. Furthermore, it is an important component with some of the highest standardized beta coefficients in these models.

Furthermore, the work environment ($WorkEnv$) identified in this research can be considered equivalent to two factors pointed out by Chow and Cao [56], that is team environment and organizational environment. However, the importance of the latter factor was not found to be significant in any of the discussed studies.

The customer involvement factor, which was only found to be meaningful by Chow and Cao [56] appears to be reflected in part by our stakeholders and risk analysis (StakeRisk). The StakeRisk factor is an important component in five of our six models, and clearly has a significant impact, especially on the satisfaction-related criteria. However, it is unfortunate that neither customer nor provider satisfaction was investigated by any of the papers cited.

Despite the general similarities noted, multivariate regressions presented in the current investigation differ decidedly in their ability to explain the consecutive success criteria, both in terms of the significance and level of influence of the identified success components. These differences can likely be attributed to the selection of different questions for building the factors, the specific sample used in our study, which focused on IT service projects, and distinct methodological approaches.

5.2. Discussion of SEMs

As was noted in the literature review (Section 2.1) and modeling (Section 3.3.3) sections, SEM has gained popularity in various fields, since it enables researchers to examine complex models and relationships between variables. However, the use of SEM in studies related to IT project management has not been widely spread, despite its potential benefits. The reviewed papers differ decidedly from our study in their goals and detailed methodological aspects. Thus, it is difficult to make direct comparisons, however, we did our best to discuss at least some similar aspects of the findings.

Irfan et al. [23] examined how the concept of project management maturity relates to project success in Pakistani companies, also including the IT sector. The project success structure in their paper comprised five factors: future potential, organizational benefits, project efficiency, project impact, and stakeholder satisfaction. It appears that their organizational benefits factor probably shares properties with our $OrgPeople$ factor. Although the importance of stakeholders was manifested in the $StakeRisk$ factor in our study, stakeholder satisfaction was treated as two different success criteria in our case, regarding customer (SatCust) and provider satisfaction (SatProv) separately. These were used as dependent variables in multiple regression analyses. The authors defined project success by the latent variables in the simplest possible way with only direct relationships and did not consider and verify other possible model configurations. Moreover, the project management maturity construct contains factors that are at least partially covered in our project success structure. The incorporation of some aspects of project management maturity directly into the project success concept seems to be reasonable in light of their significant indirect influence on the concept of project success in the work by Irfan et al. [23]. These meaningful factors were generally related to strictly management issues, namely, knowledge transfer and process management, and project management awareness. In our study, some similar aspects are included in AgileChange and OrgPeople factors. The project management maturity concept, which included the Continuous improvement latent variable, is also closely associated with these two factors in our study. However, its indirect impact on project success was irrelevant in the work by Irfan et al. [23], which could have been a consequence of investigating companies not only from the IT industry.

The study by Komal et al. [24] focused on assessing the influence of the scope-creeping phenomenon on software project management success. Although their goal was different from ours, their SEM analysis contained the structure of the project success concept. According to the approach used by Komal et al. [24], it includes three latent variables named: technology, organization, and human, which were based on 8, 4, and 5 measuring items, respectively. The model also shares some similarities with our proposal. Clearly, organization and human correspond directly to the $OrgPeople$ variable in this research SEM. The standardized beta coefficients, although statistically significantly different than zero, reflect a relatively small influence of these factors on project success. Their biggest absolute value amounted merely to 0.2, which is much smaller than the smallest value of relationship strength in our model (0.35). Probably if the authors had been tempted to check other structures with different latent variables and take into account possible indirect dependencies, these indicators could have been much better. Komal et al. [24] have also examined correlations between success variables with five success criteria, namely: schedule, budget, quality, and customer satisfaction. This analysis is similar to our multivariate regression. Their criteria correspond directly to our Time, Cost, Quality, and SatCust, respectively. However, they did not include any equivalent to our SatProv and Scope criteria. It should also be noted that the multivariate regression approach seems to be more suitable for such analyses. Although it is more complex, it provides casual information, which helps in better understanding the relationships and, thus, facilitates drawing explanatory conclusions.

Tam et al. [25] dealt directly with identifying success factors related to ongoing agile software development projects. They developed a four-factorial SEM model for this purpose, which included the following latent variables: customer involvement, personal characteristics, societal culture, and team capability. These constructs are consistent with our findings to a substantial degree. Customer involvement is included within StakeRisk, our OrgPeople covers team capability, and their societal culture is related to some degree with WorkEnv. Personal characteristics, in turn, are partially present in the OrgPeople construct from the current research. The authors also used the training and learning variable as a moderator to examine its possible role in modifying customer involvement and team capability factors. In the current study, we did not include any moderator in our model; however, this could be an interesting idea for designing further investigations. In the Tam et al. [25] SEM analysis, statistically significant beta coefficients were relatively high, with value levels generally comparable to the results of this study. Akin to our approach, this research includes the analysis of indirect influences, which positively distinguishes it from other studies. However, a considerable number of high cross-loadings show a rather weak level of discriminant validity. This could have some impact on obtaining two irrelevant direct relationships: personal characteristics and societal culture on project success. The authors neither presented and analyzed the model without those insignificant connections nor tried to search for some other, maybe better, structures.

Success issues of IT projects aimed at implementing ERP systems were subject to analysis by Malik and Khan [27]. Their SEM conceptual framework included seven factors identified by exploratory factor analysis: top management commitment, project management, change management, business process re-engineering, education and training, and vendor management. They also used six variables (questions) that determined the success construct. These generally matched our success criteria and corresponded to Time (S1 and S2 variables), Cost (S3 variable), Scope (S4 and S5 variables), and SatCust (S6 variable). The exploratory factor analysis statistical characteristics are appropriate and the whole procedure resulted in a good exploratory model. However, in the SEM approach, the authors included two latent variables (education and training, and vendor management) for which the beta coefficients were statistically irrelevant (p > 0.05). After obtaining such results, it would have been better to exclude these factors from the final proposal and verify if then, the remaining model is still valid. The importance of the business process re-engineering factor can probably be attributed to the specific IT area (ERP systems) examined in this research. The three remaining significant factors are partly consistent with our results. Top management commitment corresponds directly to a question from our OrgPeople factor. Project management is a broad concept that includes several aspects from AgileChange and OrgPeople, whereas change management is partly included in the AgileChange variable from the current study.

The success of project management In a qualitatively different IT-related area was examined by Fakhkhari et al. [26]. They focused on information communication technology for development (ICT4D). Unlike the present study, which is based on exploratory factor analysis, their conceptual model structure was derived from a literature review. The SEM input data were based on the frequency of items appearing in appropriate papers. They included 33 indicators and categorized them into five latent variables: leadership and governance, ICT4D project success, project management, quality management, and foundation establishment. According to Fakhkhari et al. [26], project and quality management, just as in the previously described work, can be associated with our AgileChange and OrgPeople variables. Some components of project management refer also to our $StakeRisk$. Leadership and governance partially overlaps with the OrgPeople factor, whereas foundation establishment is included in our AgileChange variable. What differs in their approach from the previously discussed ones is the inclusion of indirect relationships, which are also considered in this paper. Such a conceptual framework probably contributed to obtaining higher values of beta coefficients. However, in contrast to our study, they did not report any attempts to search for better structural configurations.

The study by Hamid et al. [28] concentrated on factors that influence software project success. The SEM analysis included four constructs: planning, human resource, estimation of time, and estimation of cost. These concepts were used to explain the project success hidden variable, which was based on seven items. Their estimation of time and cost latent variables correspond to our directly measured success criteria variables: Time, and Cost, respectively, which we used in multivariate regressions. The human resource factor partly reflects OrgPeople in the present study and the planning construct is probably somewhat related to our AgileChange factor. Unfortunately, Hamid et al. [28] neither provided item descriptions nor their original wording, so it is difficult to determine what they covered in this factor. The overall model quality parameters were decent, and the beta coefficients were statistically different than zero. However, the strengths of the relationships between the examined constructs were rather small, with the largest absolute value being 0.1, whereas in our proposal, the lowest coefficient was 0.35. The authors did not pursue better models that allow for indirect relationships.

Critical success factors of broadly understood information technology projects were also investigated by Yohannes and Mauritsius [29]. Their SEM conceptual framework was based on the frequency of occurrence of specific success factors in the literature. They arbitrarily selected five of them: leadership/project management, effective organization communication, project team capability/competence, methodology, tools, and techniques, and project documentation. Indicators for those concepts were established by referring to previously published papers. Substantial similarities can be found between these constructs and the proposal of this paper. Project team capability/competence seems to be totally covered by our $OrgPeople$ construct. Likewise, effective organization communication is entirely consistent with WorkEnv, which has not been observed in any other analyzed research involving SEM. As in previously discussed studies, leadership/project management is related to $AgileChange$ and $OrgPeople$ to a certain extent. Furthermore, our $AgileChange$ includes some aspects of methodology, tools, techniques, and project documentation factors. We could not identify any meaningful relations with our $StakeRisk$ construct. Yohannes and Mauritsius [29] specified the IT project success latent variable using seven components that generally corresponded to the success criteria used in the present investigation as independent variables. The first two items were related to Time, the next two to Cost, while variables five and six were associated with SatCust. The last item directly asked if the project was successful. Although the basic quality parameters for the examined constructs were on a good level, there were multiple and very high cross-loadings present in the final model. It seems that the presented construct components are highly correlated with each other, casting serious doubts on the discriminant validity of the proposed structure. This problem could have contributed to insignificant beta coefficients for two out of five latent variables’ relationships with the success construct: leadership/project management and project documentation. The authors did not re-estimate their model without these factors either.

There are also similarities with our proposal in modeling project success in other areas beyond IT-related investigations. For instance, Unegbu et al. [19] examined project success in the context of project management practices in the construction industry. They presented a highly complex conceptual framework with 11 latent variables defined by a total of 72 components and 19 relationships between them. Such a sophisticated model may be difficult to follow and therefore less informative and explanatory. One may have doubts whether it provides a better understanding of the phenomenon, especially if the authors neither provide standardized coefficients nor test if they are statistically significant. Given that some of the relationship coefficients were quite small, they could have been excluded from the model. Four of their latent variables (Quality, Scope, Time, Cost, and customer satisfaction) were directly equivalent to five out of six success criteria that were directly assessed by our participants. Their stakeholder, risk, and procurement management factors corresponded to our $StakeRisk$ latent variable, while human resource and communication management were partly related to $OrgPeople$ and $AgileChange$. Unlike in previous investigations, the authors made some effort to improve the model. However, some of the SEM quality parameters were far worse than in our study. For example, after modification, their CFI was only 0.765 compared to 0.943 from the best model presented in this research. Generally, decent models require values bigger than 0.9 for this parameter.

In the discussed SEM-based studies from the IT industry, surprisingly little attention is devoted to aspects that are covered in our $StakeRisk$ construct. However, these issues appear to be more present in SEM-based studies on project success in different sectors, such as in the work of Almeile et al. [11]. They examined the success issue in public–private partnership projects (PPP). Their SEM model included three latent variables: critical success components, PPP project success construct, and two political and economic variables used as a moderating factor. The critical success components construct contained twelve items, among which there were five components (a strong and good private consortium, appropriate risk allocation and risk-sharing, commitment and responsibility of project parties, open and constant communication among stakeholders, understand and respect the main PPP parties and each other’s goals) related to our $StakeRisk$ latent variable.

Very few of the discussed studies considered project success model variants that involved indirect links. The consideration of them could have probably better explained the gathered data. We also did not find any IT-related research on project management success that systematically explored the space of possible model structures using formal optimization criteria. This is surprising since, as we demonstrated in the present study, today’s computer systems supporting structural equation modeling provide such a possibility.

In the proposed methodological framework, the modeling procedure is explorative in nature as we seek models that best fit the data while remaining reasonable. Therefore, we do not explicitly specify hypotheses about the entire SEM structure. As demonstrated in the Modeling Results Section, several qualitatively different SEM models can formally meet multiple statistical criteria. In such a situation, providing fixed hypotheses might be misleading and could lead to an excessive simplification of the modeled phenomenon. Imagine a scenario where a specific hypothesis is accepted by one model while rejected by another. Given that both models are formally correct, there is no clear answer to whether the hypothesis was rejected or not. It is up to the researcher or practitioner to decide which of the possible approaches is better, considering factors such as common sense or results obtained in previous studies.

Instead of formulating a specific hypothesis for every relationship, we focused on a more flexible and open-ended approach. This iterative and data-driven analysis allowed for in-depth exploration guided by observed patterns rather than predetermined hypotheses. In the present article, SEMs are used to aid the understanding of intricate interplays between construct relationships without rigid adherence to specific hypotheses. The applied methodological framework involved model modifications based on fit indices and parameter significance tests, allowing us to refine or even change models to better reflect observed data.

5.3. Limitations and Future Research

The investigation presented in this study has limitations that need to be considered while drawing conclusions. However, these shortcomings open up opportunities for future research. While the findings on IT service-related project management success are general since they are based on multiple projects from all around the world, they have restricted generalizability due to the non-random sample selection. However, given the increasing tendency for information technology professionals to become more closed-off regarding scientific research and publishing results, it is worth noting that our research is based on a large and homogeneous sample of IT service project managers. It is noteworthy that similar international studies in this field seldom have samples of comparable size. Naturally, future investigations should try to validate the presented results on other populations and involve as many relevant participants as possible.

The obtained results may have been influenced by the diverse types of projects in which the interviewed managers were involved. Additionally, the research sample’s characteristics may hold significance. Factors like project management experience, age, and mastery of different project management techniques, may have had a notable effect on the findings. It is also possible that the project managers’ work environment, such as the organizational culture or internal practices, may have influenced the final models presented in this study. Prospective research could possibly better control these variables or include them in the experimental setup as, for instance, moderators.

In this study, the success criteria were directly measured by asking questions about them. However, in some discussed examinations, researchers treated them as complex constructs with multiple components. Therefore, extending our research by incorporating such latent variables into the SEM models would be interesting. Another possible direction of prospect investigation would be to incorporate a long-term perspective, especially in the context of criteria such as customer and provider satisfaction. The present study only examines the short-term perception of project management success. Moreover, the additional inclusion of other, more qualitative research methods would also help to delve deeper into the examined aspect. It is worth considering using different methods to retrieve the relative importance of the critical success factors, such as those based on pairwise comparisons, e.g., the analytic hierarchy process [60].

Considering that the data collection occurred in 2019 and the transformative impact of the COVID-19 pandemic on the global landscape, a legitimate question arises regarding the relevance of presented models in contemporary circumstances. In essence, the COVID-19 pandemic catalyzed a reevaluation of many traditional work practices in multiple sectors including project management practices. One could observe the accelerated adoption of agile approaches or the newfound emphasis on flexibility and digital tools. However, the COVID-19 pandemic had a limited impact on project management within the IT sector.

Firstly, the IT industry was relatively well-prepared for the transition to remote work due to its existing foundation in digital collaboration tools and methodologies. Many IT projects, inherently digital in nature, continued their operations, facilitated by the virtual aspects of coding, development, and testing.

Moreover, the prevalent use of cloud-based infrastructure in the IT sector played a crucial role in maintaining accessibility and scalability. This ensured that teams could continue their development and testing activities without significant disruption. The adaptability of agile methodologies, commonly employed in IT project management, also played a significant role. These methodologies, designed to be flexible and responsive to changing circumstances, allowed teams to adjust project priorities and timelines in response to the evolving situation.

While acknowledging the above argumentation, individual experiences varied, and some IT projects did face challenges, especially in areas where physical presence was traditionally considered crucial. However, the overall impact of the pandemic on project management in the IT sector was mitigated to a certain extent, thanks to the industry’s preparedness, the digital nature of projects, and inherent adaptability to remote work practices.

Although the influence of a pandemic may not be as significant as in other industries, there have likely been considerable changes in project management processes. Therefore, conducting follow-up research on post-pandemic project management practices in IT services would be exceptionally interesting. The models presented in this study constitute a solid basis and advanced methodological framework for future studies and comparisons with the new circumstances in project management.

6. Conclusions

This study aimed to model and extend our understanding of the success construct and its factors in the management of IT service projects. A broad review of earlier research in all IT areas [5] led to the collection of possible candidates for project management success components. Based on these candidates, an initial construction of dimensions was developed and refined through an exploratory factor analysis. These dimensions included (1) $AgileChange$—agile techniques and change management, (2) $OrgPeople$—organization and people, (3) $StakeRisk$—stakeholders and risk analysis, and (4) $WorkEnv$—work environment. These results were additionally verified and validated in the current paper by repeatedly applying exploratory and confirmatory factor analyses to the data randomly split into two groups.

The present investigation further expands our knowledge in this area by building and analyzing formal causal models for the concept of success. In addition to the previous results, this paper includes six typical criteria for measuring success, as well as a newly added latent variable that captures the overall perception of project management success. We determined and formally verified a number of relations between the dimensions identified by exploratory factor analysis and these additional variables. By employing strict methodological approaches, we were able to search for optimal models in the form of both multivariate regressions and SEMs. Consequently, we managed to provide and analyze a well-validated conceptual structure of the project management success concept. We used stepwise variable selection methods for these regressions and the model specification search in the SEM framework to gain a better understanding of the complex relationships between various factors that influence project management success aspects.

The multivariate regression analysis clearly showed that, according to the study subjects, not all identified success factors influenced all the investigated success criteria. Additionally, there were noticeable differences in the strength of their relevance. These findings provide a better comprehension of how specific project success aspects are related to each other and perceived by project managers. Comparisons with previous studies reveal significant discrepancies with our results, which can be attributed, to some degree, to the methodological and formal shortcomings of these preceding works.

Our final conceptual framework is relatively simple in structure compared with other studies that involve the SEM technique. However, considering the parsimony paradigm, the quality measures of our proposal, and the clear and reasonable explanation, this should be perceived as a major advantage. Unlike many earlier studies in the IT-related context, we examined not only direct but also indirect relationships between success constructs.

From a substantive point of view, the structure of the obtained SEM latent variables and their relationships in this study suggest a need to invest in improving the knowledge and skills of employees. In this respect, the presented model appears to reflect the specificity of IT-related projects by emphasizing the importance of various agile aspects. The results support the use of agile methodologies in practice, such as reducing work-in-progress, focusing on results, creating agile documentation, or executing the most important features from the customer’s point of view first. Interestingly, the success factors seem to rely on a combination of traditional techniques and agile methodologies, rather than solely on agile approaches.

Team members, who are not only motivated materially, but also by their identification with the organization, significantly increase the chances of success of individual projects. Such identification with the organization can be viewed as an investment that will certainly pay off. Another important aspect that was highlighted in the presented SEM models is the need to create an environment in the organization that is friendly and attuned to the requirements of the workers. Along with solid support from senior management, this can greatly influence the success of the project management. The strong relationship between stakeholders and risk analysis suggests the necessity of performing regular risk analyses in which every stakeholder is involved. This should be completed both in the case of a project change and at its checkpoints. Such a risk analysis allows for the systematic monitoring and control of the project management process, and the introduction of appropriate actions, including project adjustments, when necessary.

Our findings were compared with analogous studies available in the international literature. Detailed analyses showed that the model of success factors obtained for IT service projects is not fully consistent with the approaches proposed for other areas of IT. Despite some similarities, the differences are significant and concern the number and characteristics of the factors, their components, and the methodological approaches applied.

The models presented here deal with project management success factors in IT services, but similar relations are highly probable in other IT-related products. Therefore, analogous research and analysis could be beneficial in these areas. There is currently little research devoted to SEM modeling of the success aspects of IT-related project management in the scientific literature. Thus, from the methodological point of view, the findings and methods applied in this study can be helpful to other investigators who wish to explore this subject.