Next Article in Journal
Temperature Dependence of the Elastic Modulus of (Ni0.6Nb0.4)1−xZrx Membranes: Effects of Thermal Treatments and Hydrogenation
Previous Article in Journal
Experimental Study on Dry Torrefaction of Beech Wood and Miscanthus
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Project Management Maturity Analysis in the Serbian Energy Sector

by
Marko M. Mihic
*,
Dejan C. Petrovic
,
Vladimir Lj. Obradovic
and
Aleksandar M. Vuckovic
Faculty of Organizational Sciences, University of Belgrade, Jove Ilica 154, 11000 Belgrade, Serbia
*
Author to whom correspondence should be addressed.
Energies 2015, 8(5), 3924-3943; https://doi.org/10.3390/en8053924
Submission received: 4 March 2015 / Revised: 8 April 2015 / Accepted: 20 April 2015 / Published: 6 May 2015

Abstract

:
In this paper we will present an analysis of the current state of practice in project management in the energy sector in Serbia, wherein the mentioned analysis is based on the results of original research on project maturity in this field. This paper includes a literature review about the current situation in the Serbian energy system, the basic concepts and significance of the research of project maturity in organizations, the basic concepts related to the projects in the energy sector, as well as the general state of project management in Serbia. After this, an analysis of the results of the research is presented, which is structured according to the elements of project maturity included in the research, and the methods of analysis. The analysis of the results includes a general analysis as well as statistical and cluster analysis. At the end, based on the analysis results, recommendations for improvements in the area of project management in the energy sector in Serbia are presented.

1. Introduction

Energy sector projects are the backbone of development in all transition countries, and Serbia is no exception to this trend. These projects have the potential to create an environment fostering efficient use of energy potentials and a faster economic growth. In addition to being of great significance for a particular country, they may serve as an instrument of economic and political cooperation among different countries. Further development of the energy sector is one of the key pillars of the National Sustainable Development Strategy of the Republic of Serbia; one that keeps receiving a significant amount of investments [1].
Serbia faces numerous problems in the energy sector. Great dependence on the importation of oil and natural gas, which constitute as much as 15.1% of the overall national import volume [2], obsolete electrical and energy capacities [3], energy efficiency problems in almost all energy consumption sectors [4], represent only some of the many problems affecting the country’s development. In an attempt to solve these problems, the government presented the Draft Energy Sector Development Strategy of the Republic of Serbia for the period by 2025 with projections by 2035 [5]. This document stresses the main priorities for the development of the Serbian energy sector: modernization of existing and construction of new energy production and distribution capacities, a more intensive exploitation of renewable energy sources and improvements in energy efficiency [5]. In order to achieve the strategic goals aimed at fostering energy development, the government predicted investments amounting to more than EUR 14 bn to be used for implementing energy projects by 2030 [5]. Given the overall workload anticipated over the mentioned period, the need to establish an efficient system of project management in energy sector is an emerging imperative.

2. Project Management Maturity Models and Energy Projects

The maturity model that served as the precursor to all later maturity models was the Quality Management Maturity Grid, conceived by Crosby [6]. The principle of testing and presentation of maturity remained more or less the same in the later models, with addition that areas of application of maturity models expanded to almost all processes in an organization [7]. With the increasing importance of project management in achieving strategic organizational goals, there was a need to assess project management maturity [8]. Kerzner [9] defines project management maturity as the implementation of a standard methodology and accompanying processes such that there is a high likelihood of repeated successes. Models of project maturity identify the strengths and weaknesses of the organization regarding the practice of project management [10]. Data collected by analyzing project maturity are used for benchmarking [10], as well as for monitoring the development of project management practice over time [11]. According to Kwak and Ibbs [12], project management maturity is a well-defined level of sophistication that assesses an organization’s current project management practices and processes.
When it comes to the relation between organizational project maturity and the project performance, the research conducted by Yazici [13] points out that organizations at a higher project maturity level achieve higher levels of project success. In the literature, there are several project maturity models among which CMM, PMMM, OPM3, PRINCE2, EFQM, IPMA project excellence award model, etc. may be mentioned [14]. All previously mentioned models are characterized with different, predefined project maturity levels [14]. According to Brooks and Clark [15], the level of project maturity can be determined by analyzing the key elements of project management practice in an organization. On the other hand, Patanakul et al. [16] suggested the determination of project maturity level by using the analysis of the use of certain project management tools and techniques in the organization. However, the models that have basically an analysis of the elements of project management practice are prevalent in the literature, wherein said elements are defined in accordance with the knowledge areas defined by the Project Management Institute [17,18]. Nevertheless, these elements can also be defined in accordance to the requirements of each individual research [19,20].
Enhancing project management maturity represents an effort aimed at establishing and improving a systematic approach to project management, where project success does not focus on controlling the end results, but rather on enabling an efficient and effective functioning of the system. Therefore, it is very difficult to give a single, comprehensive grade when evaluating project maturity, especially since poorer results in one area cannot be compensated with better results in another. Actually, for the optimal functioning of a system, it is necessary to achieve maximal functionality for each individual project element. In addition, if we group organizations according to project maturity into predefined categories, we may make a mistake. Namely, characteristics of project management maturity in an organization could deviate from those predicted by the model. When analyzing project maturity in several organizations, cluster analysis is useful for defining groups of organizations with similar project management practices [21]. The characteristics of these groups can deviate from the characteristics of categories predicted by already existing models. Therefore, it would be more prudent and efficient to analyze project maturity of each organization or each individual group of organizations, than to try and fit them into an already existing pattern. Project maturity model presented in this paper, aims to use all benefits and eliminate all disadvantages of the existing methods for analyzing project management practices.
In many organizations in Serbia, project management is regarded as a tool for more efficient problem solving and a way to enhance project team results, communication between team members, relationship with stakeholders, development of professional skills in employees and modernization of business operations in general [22]. Nevertheless, generally speaking, project management practice in Serbia is far from being satisfactory. From a methodological point of view, leading organizations in Serbia do not use contemporary project management tools and techniques [23]. In addition, from a cultural point of view and having in mind the value of Project Management Deployment Index, project management in Serbia is far behind the practices used in other Western countries [24]. The political and cultural heritage as well as a poorly developed economy are often regarded as the main causes for such a state in Serbian project management [22,24].
In general terms, an energy project is a unique effort that supports a program mission with defined start and end points, undertaken to create a product, facility, or system with interdependent activities planned to meet a common objective/mission. Projects include planning and execution of construction, renovation, modification, environmental restoration or decontamination and decommissioning efforts, as well as large capital equipment or technology development activities. Tasks that do not include the above elements, such as basic research, grants, operations and maintenance of facilities are not considered projects [25]. According to [26,27], most important characteristics of energy projects are:
  • Substantial financial investments required for project implementation.
  • Numerous threats which are hard to forecast due to long project duration.
  • Complex implementation and hundreds (or thousands) of project activities.
  • Non-standardized technology of project implementation.
  • Many stakeholders who may have conflicting interests.
  • Workload may differ during the project.
  • Exceeding budget limits or deadlines which comes as a result of the absence of contingency reserves.
Regarding the situation of accelerated depletion of non-renewable energy sources, almost all countries are facing challenges concerning how to meet the needs for development, while achieving energy sustainability [28]. Organizations in the energy sector operate in an environment prone to frequent fluctuations with significant risk exposure [3,29]. The mentioned very dynamic environment, beside the regular operations of energy companies, also affects the project management in this field.
All organizations included in this research are partially or fully project-oriented organizations. Previously in this paper we mentioned that the investments in energy projects would amount to EUR 14 bn by 2030. In addition, strategic goals in energy sector represent one of the main pillars of Serbia’s future development and economic growth. Since Serbia belongs to the group of underdeveloped countries in Europe, rational spending and consistent achievement of strategic energy goals are imperative. The establishment of project maturity level in energy sector in Serbia would facilitate the identification of weaknesses and introduction of enhancements aimed at improving project management in this sector.

3. Overview of the Research

Empirical research of the project management maturity in Serbian energy sector presented in this paper was conducted in 2013. The sample included 75 respondents from 75 organizations. A questionnaire (presented in the Appendix) was built around key elements of project maturity, which are pertinent to energy projects, not necessarily adhering to PMI knowledge areas. For each element of project maturity, there were five questions defined. The project maturity elements included in the research are: program/project management, program/project planning, benefits from the program/project, stakeholders, risk management, quality management, communications management, financial management, IT support [27]. Some of these elements are derived from the PMI knowledge areas, but elements related to project management organization, benefits management and IT support are included in this research due to contemporary trends in project management. Namely, strategic orientation in project management, which is a dominant paradigm in project management in recent years [30], has a primary objective in achieving project benefits rather than achieving project goals related to limited cost, time and quality [31]. In this way, it is necessary to pay significant attention to processes and procedures related to benefits management. Also, project management organization and its various forms can be considered as a link between project itself and organization and, thus, has to be examined [32]. Progress in information technologies has created a various opportunities for efficient project management by application of suitable software tools [33]. Having this in mind the level of IT support to project management in an organization can’t be avoided in any research on project management practice. There is also an additional group of questions that are related to strategic aspect of project management in Serbian energy sector. The statistical analysis of the data has been implemented by using the SPSS ver.16.0 software. The research included organizations and projects of different size and importance for Serbian economy.

4. Results and Discussion

4.1. Discussion by Project Maturity Elements

The results of the analysis of the entire questionnaire, which were extracted by calculating the average value of the results of all questions, are presented in Figure 1.
Figure 1. The values of all answers from the questionnaire relating to project maturity in Serbian energy sector, in%.
Figure 1. The values of all answers from the questionnaire relating to project maturity in Serbian energy sector, in%.
Energies 08 03924 g001
The analysis of all results shows that there is plenty of space for the improvement of project management in Serbian energy sector. The number of positive answers about project management cannot be regarded as satisfactory, and neither can the percentage of negative answers (nearly 20%), as it should have been much lower. The high occurrence of answers labeled “partially” indicates that the need for project management is recognized in the organizations, but that they require certain improvements. These improvements primarily relate to education and certification of project managers in this area, in order for the contemporary project management principles to be adequately implemented. Successful implementation of energy projects requires complete and consistent implementation of project management concept. Total or partial lack of implementation of project management principles leads to the absence of planned benefits, inability to meet deadlines or budget limitation and other setbacks. Therefore, given the goals defined in the national development strategies, more attention should be paid to the implementation of contemporary project management methods and techniques in energy sector in Serbia [27].
Analysis of gathered results relating to various aspects of project maturity and individual questions has shed the light on some of the obvious problems and positive examples in the area of managing energy projects in Serbia.
The first group of questions is related to project/program management and aims to determine the level of existence of clearly defined roles and responsibilities of project managers and project team members. Results of these questions can be considered satisfactory. However, when it comes to existence and functionality of project management office (PMO), there were recorded fairly low results (42.7% of respondents gave a positive answer and 32% negative). The main functions of PMO are: communications management; stakeholder management; risk management, project coordination and monitoring as well as application of software tools in order to ensure efficient project management [14]. According to this, level of functionality of PMO largely contributes to results related to aforementioned functions. It is clear from this that energy companies in Serbia need to fully establish project management offices that would strive to efficient implementation of the organizational strategy [27]. Results of dependency analysis indicate that a PMO is essential in organizations dealing with multiple projects characterized with a long duration, a large budget and a vast number of participants (Table 1 and Table 2).
Table 1. Analysis of dependencies between variables using Chi-squared test.
Table 1. Analysis of dependencies between variables using Chi-squared test.
Variable 2 Number of Staff HiredDuration of the ProjectSize of the Project BudgetSuccess of the Project
Variable 1
Project management office in placeValue71.1858.3497.0516.12
Significance0.017 *0.048 *0.045 *0.041 *
Basic planning methods and techniques implementedValue59.0154.5279.5116.06
Significance0.033 *0.0930.3690.042 *
Defined business benefits on the project levelValue51.2735.5660.1823.18
Significance0.3470.7480.9080.003 *
Stakeholder interests defined and alignedValue44.1361.3780.8629.18
Significance0.6320.027 *0.3300.000 *
Risks defined for each level of activityValue62.2242.6593.648.55
Significance0.0810.4430.0830.382
Project quality plan in placeValue54.9241.4984.7820.27
Significance0.029 *0.4930.2300.009 *
Internal communication efficiency measuredValue54.3258.34100.914.98
Significance0.046 *0.048 *0.029 *0.049 *
Monitored use of resources on program levelValue39.0735.5549.1222.24
Significance0.8170.7490.9930.004 *
Project staff uses project planning and monitoring softwareValue70.3451.9177.386.14
Significance0.019 *0.1410.4340.632
*: The variables with statistically significant dependencies (significance lower than 0.05).
Table 2. Analysis of dependencies between variables using Pearson correlation coefficient.
Table 2. Analysis of dependencies between variables using Pearson correlation coefficient.
Variable 2Number of Staff HiredDuration of the ProjectSize of Project BudgetSuccess of the Project
Variable 1
Project management office in placeValue0.298−0.1530.2480.088
Significance0.009 *0.1890.032*0.454
Basic planning methods and techniques implementedValue0.2290.1840.2230.295
Significance0.049 *0.1140.0540.010 *
Defined business benefits on the project levelValue−0.270−0.122−0.1120.457
Significance0.8200.2960.3380.000 *
Stakeholder interests defined and alignedValue0.1120.0340.1050.321
Significance0.3370.7690.3700.005 *
Risks defined for each level of activityValue0.5700.0330.0830.196
Significance0.6280.7800.4790.092
Project quality plan in placeValue0.343−0.0510.1660.407
Significance0.003 *0.6630.1540.000 *
Internal communication efficiency measuredValue0.269−0.1350.2600.231
Significance0.020 *0.2490.024 *0.047 *
Monitored use of resources on program levelValue0.2410.350−0.0190.399
Significance0.1370.1120.8720.000 *
Project staff uses project planning and monitoring softwareValue0.2940.0820.0940.221
Significance0.011 *0.4860.4220.057
*: The variables with statistically significant dependencies (significance lower than 0.05).
Analysis of the segment of project planning revealed that usage of project planning methods and techniques (schedule analysis, work breakdown structure diagrams—WBS, critical events, etc.) is fairly low. These methods form the basis for application of planning methods in other project management processes such as assessment of project duration, risk analysis, human resources, budget, etc. [27]. For instance, the research has shown similarity between this segment and a segment of risk analysis. By breaking down the project to the level of individual activities, it becomes possible to identify potential risks at every activity, which makes a WBS diagram a basis for construction of the Risk Breakdown Structure (RBS) diagram [34]. It can be concluded that there is a need for consistent use of project planning methods, which can contribute to better results in other project management maturity segments. This conclusion was confirmed by dependency analysis (Table 1 and Table 2). Some discrepancies have been identified between the processes of project planning and its implementation. Over 90% of the organizations included in this research partially or fully monitor the course of projects in terms of costs and resource use, but there is no efficient monitoring system implemented in the majority of these organizations. In order to make monitoring efficient, it is important to clearly define and implement a project monitoring system, which should be supported by appropriate software tools. Poor results in these segments stem from underdeveloped PMO, like it was mentioned before.
When talking about identification and alignment of benefits that project or program should deliver, the research results can be regarded as positive (68% respondents gave positive answer and only 6.7% negative). However, results related to other aspects of benefits management are not that good as previous and deficiencies are visible in areas of planning, monitoring and assuring the benefits. Benefits that energy projects deliver are fairly easy to identify. Some of these benefits are national energy balance improvement, increased of national energy security, decreased CO2 emissions, etc. [35,36,37]. On the other hand, when it comes to the next phases of the benefits management process, the problems appear due to complexity of energy projects (numerous activities and risks, long duration, many participants, etc.). Poor results in this area also can be regarded as a consequence of a low use of project planning methods [27]. According to dependency analysis, benefits management has a crucial role in achieving project success in the energy sector (Table 1 and Table 2). In addition, a t-test showed that organizations in this sector in Serbia pay more attention to benefits management on the level of entire project/program (Table 3). In order to improve this segment, it is necessary to enhance the usage of benefits management tools on the level of individual activities.
Table 3. Comparison of averages between the two levels of project management in Serbian energy companies [27].
Table 3. Comparison of averages between the two levels of project management in Serbian energy companies [27].
Level of Project ManagementAverageVarianceMinimumMaximumCronbach’s Alpha
Program/project2.3920.1281.8272.6930.547
Activities2.2640.0881.8402.6270.574
Risk management was found to be a one of the weakest segments of project maturity in the energy sector of Serbia. Research results indicate that risk analysis is taken more seriously only at the entire project level, because the budget and duration of a project are often preset due to a need to satisfy different and conflicting stakeholder interests [27]. According to results of dependency analysis, respondents don’t think that the extent of use of risk management methods depends of project size (Table 1 and Table 2). Also, the same analysis showed that risk management has no effect on project success. Nevertheless, detailed project planning as well as efficient implementation monitoring can contribute to timely identification of potential threats and implementation of adequate risk management measures [27]. In order to improve risk management practice, it is necessary to enhance the usage of risk analysis methods as well as improve project planning in general [38]. Likewise, it is important to improve the efficiency of implementation monitoring by establishing an appropriate system supported by adequate software tools [27].
In terms of quality, answers to individual questions are largely similar; therefore, the state of this aspect of project maturity can be generalized. On the whole, although the results for aspect of quality management could have been much better, they still do not fall within the critical category. Quality plan stems from the results of planning of other project elements (resources, risks, schedule, key participants, etc.) [17]. Therefore, the results of project quality management processes should be regarded in the context of results relating to the planning of other project management aspects [27]. In terms of the number of positive answers, the results related to making and implementation of quality plans are quite similar to the results of the group of questions related to resource use and risk identification. However, planning processes within other aspects of project maturity (stakeholder identification, definition and alignment of their interests, use of project planning methods, the development of benefits plan) don’t show such similarity like the previous. The results about external quality audits indicate that 33% of organizations from Serbian energy sector don’t conduct any external quality audits, which has to be improved. Problems with project quality within energy companies in Serbia would largely be able to be resolved by implementation and certification of quality management system ISO 9000 [27]. Analysis of dependencies has confirmed that quality management is of critical importance for project success; therefore, greater attention should be paid to this aspect of project maturity (Table 1 and Table 2).
In terms of project communication structures, the recorded results can be regarded as positive. Like in the case of other capital projects, the positive results in communications management are the consequence of the necessity to establish stable communications structure designed to create connections between numerous participants in the same project [39]. The results should still be interpreted with caution, since the communications plan partially neglects the aspect of communication with stakeholders—very important, if not key participants in the energy projects. Given the high financial value and significance of energy projects, as well as the nature of their stakeholders (representatives of local, state and international institutions, owners of capital, non-governmental organizations involved in environmental protection, general public, etc.), whose interests are often conflicted, this type of projects requires project managers to be involved in politics more than any other type of projects. Therefore, neglecting the aspect of communication with stakeholders in energy projects in Serbia cannot be excused in any way, and it often has insurmountable negative impact on the success of a project. This finding was confirmed by analysis of dependencies. The analysis also confirmed the existence of statistically relevant dependencies between the duration of a project and stakeholder management. This can be interpreted as a need to avoid risks inherent to long-term planning and long implementation periods. One of the causes of poor results in this area is the problem of underdeveloped project management offices in many of the organizations included in this research. When it comes to project communication, it has been noted that the measuring of communication efficiency is insufficiently developed, thus impeding the efforts focused on benchmarking and implementing measures aimed at improving this aspect of project maturity. Still, the analysis of dependencies shows that the frequency of measuring communications efficiency increases proportionally to the size of the project, leading to the conclusion that more attention should be paid to improving this category in smaller projects (Table 1 and Table 2).
Examination of the IT support aspect of project management in the overall research of project maturity in the Serbian energy sector yielded exceptionally low results. Even though the majority of organizations covered by this research have unified information systems, the use of project management software tools is at an extremely low level. Over the course of time, software tools have become sophisticated and comprehensive to such an extent that they can be used in virtually all phases of the project/program or portfolio lifecycle [40]. The use of these tools can significantly contribute to the efficiency of planning and monitoring of implementation of complex and large-scale projects/programs, such as those in the energy sector. The underlying causes of such poor results should be sought in the lack of skills and training of project staff for the use of software tools such as MS Project, Primavera, CA Clarity, etc. Analysis of dependencies has confirmed that the respondents do not maintain that the use of information technology tools can significantly contribute to the success of a project. This is a direct consequence of the low awareness about the capabilities of an adequate IT support. This analysis also established that the use of the mentioned tools depends on the number of participants in a project. The higher the number of participants, the more complex the project, so the use of project management software becomes very important. The implementation of appropriate software tools and training of project staff can significantly improve this aspect of project maturity.
Results regarding the strategic aspect of project management in Serbian energy sector indicate some inconsistency between a recognized need for strategic management in organizations and implementation of its procedures. To be precise, 61.3% of surveyed organizations are fully aware of strategic management and they have recognized the need for it. Also, among surveyed organizations, 72% of them have a strategic plan in place. In addition, results related to centralized resource coordination in organization can be considered as good (62.7% of positive answers). On the other hand, less positive results were recorded in the areas of implementation of strategic management phases as well as functionality of the department for planning and (strategic) analysis. Organizations being part of Serbian energy sector need to further strengthen the links between strategic management and project management in order to achieve organizational and national strategic goals. One of the key measures that can strengthen aforementioned link is the development of strategic plans that will consolidate the efforts of functional units and project staff [27].

4.2. Statistical Analysis

Analysis of the obtained results should be complemented with the results of the statistical testing relating to the existence of statistically relevant dependencies between certain categories in the conducted research and statistical differences in maturity between strategic and operational management of observed projects. The results of statistically relevant dependencies between certain categories are presented in Table 1 and Table 2. These tables show the results of examination of dependencies between categories pertaining to one of the two extremes: the best or the weakest result, and the categories that could affect the research results.
The analysis used two tests for determining the existence and strength of dependencies: Chi-squared test presented in Table 1 and Pearson’s correlation coefficient presented in Table 2. The Chi-squared test was used to determine any dependencies between variables. In these cases it is possible to analyze the intensity of the dependencies by comparing the value of the Pearson coefficient with the extreme values (−1 and 1) and use this as a basis for establishing the level of dependence between categories.
Research conducted by Andersen and Jessen [41], indicates that there is a difference in project maturity between three levels of project organization, i.e., firstly, the level of individual project management, secondly, the level of program management, and finally, the level of portfolio management. In order to find out if there is a statistically significant difference in project maturity between a level of project management and a level of individual project activities, in our research has been identified two groups of question, one related to the level of project management and second related to the level of individual project activities (Table 3). The comparison of means has shown that project management practice in energy projects in Serbia is more mature on the level of project as a whole than on the level of individual activities. A slight caution should be applied when discussing about the results, because the value of Cronbach’s alpha coefficient indicates a low internal consistency of the gathered data.
Table 3 features the comparison of averages derived from the answers to the two groups of questions relating to managing an entire project and to managing individual project activities in Serbian energy sector. The ordinal numbers of questions from the questionnaire that correspond to the said levels are:
  • Program/project: 1, 11, 21, 36, 42;
  • Activities: 2, 12, 22, 37, 43.
According to the value of paired t-test which equals 4.143, with statistical significance at 0, it can be concluded that there is a strong statistically significant difference between project maturity on the level of project management and the level of individual project activities [27].

4.3. Cluster Analysis

This paper features a cluster analysis aimed at identifying groups of organizations in Serbian energy sector with similar characteristics in terms of project management maturity. In this way, it is possible to determine the causes that affect project maturity and recognize consequences generated by certain characteristics of project maturity. Cluster analysis was performed on RapidMiner 5.2 and Orange Python 2.7. After calculating the maximum value of Silhouette coefficient for Euclidean and Manhattan metrics, the authors established the optimal number of clusters to be 2. After that, the authors cross-referenced the data that relate to project characteristics, organizational strategic management and project success. Cluster centers are presented in Figure 2, while the comparison of clusters according to their strategic management characteristics is presented in Table 4.
Figure 2. Distribution of cluster centers for each of the questions in the questionnaire.
Figure 2. Distribution of cluster centers for each of the questions in the questionnaire.
Energies 08 03924 g002
Figure 2 presents cluster centers for each question relating to project maturity. Namely, cluster center curves represent values closest to organizations belonging to a certain cluster.
Table 4. Comparison of clusters in terms of organizational strategic management.
Table 4. Comparison of clusters in terms of organizational strategic management.
QuestionsOffered AnswersCluster 1Cluster 2
Key success factors for implemented program/projectAdhering to the deadlines21.62%18.42%
Achievement of goals16.22%31.58%
Customer satisfaction32.43%15.79%
Exceeding the budget5.41%13.16%
Manager satisfaction2.70%0%
Employees satisfaction0%0%
Quality of project results21.62%21.05%
Organization has a strategic planNo10.81%44.74%
Yes89.19%55.26%
The main phases of strategic management are definedNo0%23.68%
Partially40.54%60.53%
Yes59.46%15.79%
Resources necessary for implementation are providedNo24.32%50%
Yes75.68%50%
Organization has a plan and analysis sectorNo43.24%63.16%
Yes56.76%36.84%
Management acknowledges the need for strategic managementNo0%18.42%
Partially21.62%36.84%
Yes78.38%44.74%
The results from this table should be complimented with data on project success for each cluster, as well as with data relating to the evaluation of compatibility between project goals and organizational strategy. To be precise, project success in the case of companies from cluster one was graded with 3.97, while the success of cluster 2 has an average grade of 3.1. In addition, the harmonization of goals and strategy in cluster 1 was graded with 3.95, while the same value for cluster 2 equals 3.29.
After performing a cluster analysis of energy companies in Serbia, we can clearly identify two clusters composed of companies with similar project management characteristics. The basic difference between the two clusters is the level of project maturity (reflected in the number of positive answers from the questionnaire, Figure 2). Cluster 1, incorporating companies with a more advanced project maturity, has 37 companies. Cluster 2, as a cluster with weaker results, has 38 companies. Having in mind that the analysis identified two clusters obviously different in terms of project management quality, we can continue to analyze the consequences and causes of their different levels of project maturity. Namely, the average grade of project success in cluster 1 is 3.97, while project success in cluster 2 is 3.1. This confirms the hypothesis that better project maturity contributes to improved project success. On the other hand, the project maturity discrepancies between two clusters of Serbian energy companies mostly come from differences in strategic management [27]. The research has revealed numerous differences in this field between two clusters (Table 4). For example, in the case of project success indicators, organizations from cluster 1 underlined customer satisfaction as the most important indicator. Customer satisfaction is seen as one of the basic goals of strategic project management [42,43]. Most companies from cluster 2 consider project success as the project implementation within limited budget and duration and required quality. Also, differences are visible in other areas, and the most prominent are those related to quality management, coordination of activities and resources, regular monitoring of cash flow and relationship with stakeholders. Problems relating to project coordination and relationship with stakeholders are often a consequence of the fact that the company does not have a project management office, or has a dysfunctional project management office [27].
As mentioned previously in this paper, a PMO represents a function of strategic project management. Its inexistence or lack of functionality can endanger the management of complex projects or group of projects, where the team needs to manage a large number of activities and handle interests of numerous stakeholders. Quality management is an element of project management that, among other things, acknowledges clients’ demands and turns them into project requirements and goals. Fulfilling these demands contributes to clients’ satisfaction, which, as previously mentioned, is one of the main differences between the two clusters. The absence of regular cash flow monitoring and comparison against planned results represents a huge disadvantage. This disadvantage is the consequence of an organization’s focus on controlling the end results, without systematically approaching project implementation.

5. Recommendations for Improvements

Presented research results revealed many deficiencies in almost all segments of project management maturity within energy sector of Serbia. In order to remove these deficiencies and to enable faster development of Serbian energy sector, appropriate measures were proposed in Table 5.
Table 5. The most important recommendations for improving the level of project management maturity in Serbian energy sector [27].
Table 5. The most important recommendations for improving the level of project management maturity in Serbian energy sector [27].
AreaRecommendations
General
  • Planning and management in detail of individual project activities.
  • Education of project managers which need to contribute in improved use of project management methods and software tools.
  • More attention dedicated to strategic project management in organizations.
  • For organizations in cluster 2, the main project success criterion should be customer satisfaction, instead of the current focus on costs, resources and deadlines.
  • Further development of strategic project management.
Program/project management
  • Establishing of project management office in order to improve the implementation of numerous projects.
  • Establishing of systems for project progress monitoring or improving their efficiency where they already exist.
Program/project planning
  • A more intensive use of project planning methods and contemporary project management software.
  • Establishing of system for project elements coordination or improving its efficiency where it already exists.
  • It is necessary to establish a system for cost monitoring and resource coordination in organizations from cluster 2.
Benefits from the program/project
  • Definition of project benefits plans and responsibilities for their achievement.
Stakeholders
  • Improving communication with stakeholders through the definition of communication strategy and plans, as well as by measuring stakeholders’ contribution to a project over time. This especially relates to organizations from cluster 2.
Risk management
  • A more detailed definition of risk management procedures related to individual project activities.
Quality management
  • Establishing an office or putting in charge a person responsible for project quality management.
  • Implementation and certification of quality management system in organizations.
  • Definition of project quality plans and its consistent implementation during project management process.
  • Measures for providing project quality are supposed to be aimed at establishing a functional project management system, instead on focusing on controlling the end results. This recommendation is extremely important for companies from cluster 2.
Communications management
  • Definition and implementation of procedures for measuring the efficiency of project communication.
Financial management
  • Improvements relating to analysis and monitoring of cash flow during projects, especially in cluster 2.
IT support
  • More intensive use of project management software tools as well as training of project staff in that field.

6. Conclusions

One of the strategic directions of development of Serbia in the coming decades is based on significant investments in the energy sector. This fact indicates that the project management plays an important role in the effective implementation of these investments. However, generally speaking, the practice of project management in Serbia is not at a high level. The state of practice of project management can best be determined by examination of project maturity, for which numerous models were developed. In this paper research on project maturity in the energy sector in Serbia was presented, which was conducted in order to test the readiness of this sector to successfully meet the requirements of future development. The results that were obtained by the survey, among others, point to the fact that organizations in the energy sector are in need to pay greater attention to the application of methods and techniques of project management, especially at the level of individual activities. Also, there is a need to enhance the extent of the linkage between projects and strategic management. Implementation of the recommendations for improvement that are given in this paper could raise the level of project maturity in organizations of the energy sector in Serbia. This would increase the success of energy projects and also significantly contribute to the achievement of national goals in terms of energy sector development.
Research presented in this paper is the first of its kind in the Serbian energy sector, which brings some benefits in the context of revealing the current advantages and shortcomings in the energy project management in Serbia as well as in general. On the other hand, the uniqueness of the research precludes the comparison of results obtained in the Serbian energy sector with other country markets. In order to overcome this problem and complete the study, the focus of future research in this field should be on analysis the project management maturity in the energy sector of other comparable countries and regions, primarily on Balkans, but also in the European Union. In terms of methodology, it is necessary to conduct the analysis of project maturity in other industrial and non-industrial sectors by using the elements of project maturity presented in this paper and which are in accordance with contemporary trends in project management. Also, application of cluster analysis in project management maturity research, which is presented for the first time in this paper, needs further testing in order to fully prove its effectiveness.

Acknowledgments

This paper is a result of the Project No. 179081 founded by Ministry of Education and Science of the Republic of Serbia: Researching Contemporary Tendencies of Strategic Management Using Specialized Management Disciplines in Function of Competitiveness of Serbian Economy.

Author Contributions

All authors contributed equally to this work and have approved the submitted manuscript.

Appendix—Questionnaire

Research results, organized according to individual project management maturity aspects, are provided below. The first list represents the elements of project management maturity and the ordinal numbers of questions relating to relevant elements. The second list represents the questionnaire itself, featuring percentages of respondents who opted for one of the three answers.
  • Program/project management: 1–5;
  • Program/project planning: 6–10;
  • Benefits from the program/project: 11–15;
  • Stakeholders: 16–20;
  • Risk management: 21–25;
  • Quality management: 26–30;
  • Communications management: 31–35;
  • Financial management: 36–40;
  • IT support: 41–45;
  • Roles and responsibilities of managers of individual projects/subprojects are well defined and aligned. NO: 1.3%; PARTIALLY: 28%; YES: 70.7%.
  • Roles and responsibilities of the project staff are well defined and aligned. NO: 2.7%; PARTIALLY: 32%; YES: 65.3%.
  • Project management office with well-defined and aligned procedures, documents, resources and processes is in place. NO: 32%; PARTIALLY: 25.3%; YES: 42.7%.
  • The links between the program/project and the rest of the organization are well defined and aligned. NO: 4%; PARTIALLY: 45.3%; YES: 50.7%.
  • Professional and efficient mechanisms for reporting to project management office on program implementation progress are in place. NO: 21.3%; PARTIALLY: 45.3%; YES: 33.3%.
  • Key methods and techniques of project management such as WBS, milestones, scheduling analysis, etc. were used in the course of planning of individual projects. NO: 21.3%; PARTIALLY: 48%; YES: 30.7%.
  • Program/project was planned in great detail – links and interdependencies between projects and sub-projects were recognized, identified and aligned. NO: 5.3%; PARTIALLY: 45.3%; YES: 49.3%.
  • The use of resources was planned on the level of programs/projects and mechanisms for allocation of joint resources (staff, infrastructure, etc.) were defined and established. NO: 31.3%; PARTIALLY: 36%; YES: 42.7%.
  • An efficient monitoring system for all projects, or parts of programs, is in place. NO: 21.3%; PARTIALLY: 58.7%; YES: 20%.
  • There is efficient coordination between projects/activities with the aim of transferring surplus resources to the projects/activities lacking the same, thereby maximizing the efficiency of the entire program/project. NO: 33.3%; PARTIALLY: 37.3%; YES: 29.3%.
  • Business benefits that the program/project is to create are identified, defined and aligned. NO: 6.7%; PARTIALLY: 25.3%; YES: 68%.
  • Links between individual benefits (benefits from individual projects/activities) and the costs of their production are defined and aligned. NO: 16%; PARTIALLY: 44%; YES: 40%.
  • A plan aiming to demonstrate how individual business benefits will be produced and how their attainment will be measured is defined and aligned. NO: 16%; PARTIALLY: 52%; YES: 32%.
  • Responsibilities for producing and monitoring the achievement of expected business benefits are assigned and aligned. NO: 20%; PARTIALLY: 45.3%; YES: 34.7%.
  • Mechanisms for monitoring and reporting on achievement (implementation) of all business benefits are in place. NO: 17.3%; PARTIALLY: 46.7%; YES: 36%.
  • All relevant stakeholders are identified and contacted. NO: 9.3%; PARTIALLY: 40%; YES: 50.7%.
  • Particular interests and individual participation of each of the stakeholders are defined and aligned. NO: 13.3%; PARTIALLY: 45.3%; YES: 41.3%.
  • Strategy of communication with stakeholders is defined and aligned. NO: 13.3%; PARTIALLY: 48%; YES: 38.7%.
  • Strategy of communication with stakeholders is fully operational (implementable). NO: 17.3%; PARTIALLY: 42.7%; YES: 40%.
  • Stakeholders demonstrate clear understanding of program and obviously support it. NO: 14.7%; PARTIALLY: 46.7%; YES: 38.7%.
  • Potential risks to program/project are identified. NO: 16%; PARTIALLY: 42.7%; YES: 41.3%.
  • Potential risks to project activities are identified. NO: 20%; PARTIALLY: 42.7%; YES: 37.3%.
  • Probability of occurrence and impact of each identified risk is defined. NO: 38.7%; PARTIALLY: 49.3%; YES: 12%.
  • Procedures for minimizing the probability of risk occurrence are defined. NO: 34.7%; PARTIALLY: 52%; YES: 13.3%.
  • Risk response plan is defined. NO: 30.7%; PARTIALLY: 46.7%; YES: 22.7%.
  • Quality standards for all project results are defined and aligned. NO: 18.7%; PARTIALLY: 36%; YES: 45.3%.
  • Activities for quality assurance of all project results are defined and aligned. NO: 22.7%; PARTIALLY: 28%; YES: 49.3%.
  • Program/project quality assurance plan is effectively implemented. NO: 18.7%; PARTIALLY: 40%; YES: 41.3%.
  • External audit of quality level is performed regularly. NO: 33.3%; PARTIALLY: 20%; YES: 46.7%.
  • Quality performances are measured, monitored and used as a basis for continuous improvement of the program/project. NO: 22.7%; PARTIALLY: 32%; YES: 45.3%.
  • Program/project manager forwards all relevant information about the program/project. NO: 1.3%; PARTIALLY: 28%; YES: 70.7%.
  • Formal communication procedures are in place. NO: 13.3%; PARTIALLY: 36%; YES: 50.7%.
  • Formal procedures for project communication are implemented and facilitate two-way communication. NO: 12%; PARTIALLY: 36%; YES: 52%.
  • All participants in the project have all necessary information about project plans and progress of program/project implementation. NO: 8%; PARTIALLY: 41.3%; YES: 50.7%.
  • Efficiency of internal communication is measured – relevant reports are in place. NO: 30.7%; PARTIALLY: 37.3%; YES: 32%.
  • Use of resources on the level of program/projects is monitored. NO: 1.3%; PARTIALLY: 40%; YES: 58.7%.
  • Use of resources on the level of project activities is monitored. NO: 10.7%; PARTIALLY: 34.7%; YES: 54.7%.
  • Incurred costs are monitored, compared with the planned costs and new financial forecasts are developed. NO: 5.3%; PARTIALLY: 44%; YES: 50.7%.
  • Budget changes are monitored and their impact on the program/project results is defined. NO: 8%; PARTIALLY: 38.7%; YES: 53.3%.
  • Cash flows are analyzed and compared with the objectives/project results achieved to date. NO: 17.3%; PARTIALLY: 37.3%; YES: 45.3%.
  • The organization has a uniform information system in place. NO: 22.7%; PARTIALLY: 22.7%; YES: 54.7%.
  • Program/project manager uses appropriate software for planning and monitoring program/project implementation. NO: 40%; PARTIALLY: 37.3%; YES: 22.7%.
  • Project staff uses appropriate software for planning and monitoring programs/projects. NO: 37.3%; PARTIALLY: 41.3%; YES: 21.3%.
  • The software is used for managing more than one project at the same time. NO: 57.3%; PARTIALLY: 22.7%; YES: 20%.
  • Standard reports supported by project management software represent the basis for analysis of the project. NO: 49.3%; PARTIALLY: 36%; YES: 14.7%.
These results should be complemented with the results on strategic project management in the organizations included in this research. The success of the projects was given an average grade of 3.53, on a scale of 1 to 5 (1 being the lowest, and 5 being the highest score), whereas the alignment of program/project goals with organizational strategy received the score of 3.61. The key success indicators for the implemented projects/programs are as follows: achievement of goals and customer satisfaction in 24% of the cases; quality of project results in 21.3% of the cases; adhering to the deadlines in 20% of the cases; exceeding the budget in 9.35% of the cases and manager satisfaction in 1.35% organizations. The need for strategic planning was fully recognized by the management in 61.3% of the cases; partially recognized in 29.3% and not recognized in 9.3% of the cases. 72% of the surveyed organizations have strategic plans, in contrast to 28% that failed to make such plans. A department for planning and (strategic) analysis is in place in 46.7% organizations, in contrast to 53.3% of organizations that do not have such a department. 37.3% of respondents answered “yes” to the question whether their organizations have recognized and implemented key strategic management phases (situation analysis, defining the strategy, implementation and control of strategy implementation), whereas 50.7% respondents have answered “partially”, and 12% of respondents opted for “no”. The resources necessary for the implementation of company strategy are centralized in 62.7% of organizations, in contrast to 37.3% of companies where this is not the case.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Government of the Republic of Serbia. National Sustainable Development Strategy; 2005. Available online: http://www.gs.gov.rs/english/strategije-vs.html (accessed on 9 February 2015). (In Serbian)
  2. Statistical Office of the Republic of Serbia. Statistics of External Trade, Serbia, 2013; Statistical release, Number 194—Year LXIV; 2014 16 July. Available online: http://webrzs.stat.gov.rs/WebSite/repository/documents/00/01/45/00/st12g072014e.pdf (accessed on 9 February 2015).
  3. Makajic-Nikolic, D.; Jednak, S.; Benkovic, S.; Poznanic, V. Project finance risk evaluation of the electric power industry of Serbia. Energy Policy 2011, 39, 6168–6177. [Google Scholar] [CrossRef]
  4. Oka, S.; Sedmak, A.; Djurovic-Petrovic, M. Energy efficiency in Serbia—Research and development activity. Therm. Sci. 2006, 10, 5–32. [Google Scholar] [CrossRef]
  5. National Assembly of the Republic of Serbia. Draft Energy Sector Development Strategy of the Republic of Serbia for the Period by 2025 with Projections by 2030; 2013. Available online: http://www.parlament.gov.rs/upload/archive/files/lat/pdf/akta_procedura/2014/113-14Lat.pdf (accessed on 9 February 2015). (In Serbian)
  6. Crosby, P.B. Quality is free: The Art of Making Quality Certain; McGraw-Hill: New York, NY, USA, 1979. [Google Scholar]
  7. Khoshgoftar, M.; Osman, O. Comparison of maturity models. In Proceedings of the 2nd IEEE International Conference on Computer Science and Information Technology, Beijing, China, 8–11 August 2009; pp. 297–301.
  8. Cooke-Davis, T.J.; Arzymanov, A. The maturity of project management in different industries: An investigation into variation between project management models. Int. J. Proj. Manag. 2003, 21, 471–478. [Google Scholar] [CrossRef]
  9. Kerzner, H.R. Project Management: A Systems Approach to Planning, Scheduling, and Controlling; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2013. [Google Scholar]
  10. Jugdev, K.; Thomas, J. Project management maturity models: The silver bullets of competitive advantage. Proj. Manag. J. 2002, 33, 4–14. [Google Scholar]
  11. Mulally, M. Longitudinal analysis of project management maturity. Proj. Manag. J. 2006, 37, 62–73. [Google Scholar]
  12. Kwak, Y.H.; Ibbs, C.W. Project management process maturity (PM)2 model. J. Manag. Eng. 2002, 18, 150–155. [Google Scholar] [CrossRef]
  13. Yazici, H.J. The role of project management maturity and organizational culture in perceived performance. Proj. Manag. J. 2009, 40, 14–33. [Google Scholar] [CrossRef]
  14. Mihic, M. Strategic Aspects of project management maturity analysis of Serbian ICT sector. In Proceedings of the XIV International Symposium YUPMA 2010, Opening Lecture, Zlatibor, Serbia, 14–16 May 2010.
  15. Brookes, N.; Clark, R. Using maturity models to improve project management practice. In Proceedings of the POMS 20th Annual Conference, Orlando, FL, USA, 1–4 May 2009.
  16. Patanakul, P.; Iewwongcharoen, B.; Milosevic, D. An empirical study on the use of project management tools and techniques across project life-cycle and their impact on project success. J. Gen. Manag. 2010, 35, 41–65. [Google Scholar]
  17. PMI Standards Committee. A Guide to the Project Management Body of Knowledge; Project Management Institute: Newtown, PA, USA, 1996. [Google Scholar]
  18. Crawford, J.K. The Project management maturity model. Inf. Syst. Manag. 2006, 23, 50–58. [Google Scholar] [CrossRef]
  19. White, D.; Fortune, J. Current practice in project management—An empirical study. Int. J. Proj. Manag. 2002, 20, 1–11. [Google Scholar] [CrossRef]
  20. Bryde, D.J. Project management concepts, methods and application. Int. J. Oper. Prod. Manag. 2003, 23, 775–793. [Google Scholar] [CrossRef]
  21. Besner, C.; Hobbs, B. Contextualized project management practice: A cluster analysis of practices and best practices. Proj. Manag. J. 2013, 44, 17–34. [Google Scholar] [CrossRef]
  22. Cicmil, S.; Dordević, Z.; Zivanovic, S. Understanding the adoption of project management in Serbian organizations: Insights from an exploratory study. Proj. Manag. J. 2009, 40, 88–98. [Google Scholar] [CrossRef]
  23. Mihic, M. Strategic Orientation of project management in Serbia. In Proceedings of the XIII International Symposium YUPMA 2009, Zlatibor, Serbia, 6–8 June 2009.
  24. Bredillet, C.; Yatim, F.; Ruiz, P. Project management deployment: The role of cultural factors. Int. J. Proj. Manag. 2010, 28, 183–193. [Google Scholar] [CrossRef] [Green Version]
  25. United States Department of Energy. Improving Project Management in the Department of Energy; 1999. Available online: http://science.energy.gov/~/media/opa/pdf/processes-and-procedures/various/99nrc.pdf (accessed on 9 February 2015).
  26. Flyvbjerg, B.; Glenting, C.; Rønnest, A.K. Procedures for Dealing with Optimism Bias in Transport Planning: Guidance Document; 2004. Available online: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/191523/Procedures_for_dealing_with_optimism_bias_in_transport_planning.pdf (accessed on 9 February 2015).
  27. Mihic, M.; Petrovic, D.; Obradovic, V. Organizational project management maturity analysis of energy sector companies. In Proceedings of the XIV International Symposium, Zlatibor, Serbia, 6–10 June 2014; pp. 1400–1407.
  28. Mihic, M.; Petrovic, D.; Vuckovic, A. Comparative analysis of global trends in energy sustainability. Environ. Eng. Manag. J. 2014, 13, 947–960. [Google Scholar]
  29. Kaufmann, R.K. The end of cheap oil: Economic, social and political change in the US and former Soviet Union. Energies 2014, 7, 6225–6241. [Google Scholar] [CrossRef]
  30. Jugdev, K.; Muller, R. A retrospective look at our evolving understanding of project success. Proj. Manag. J. 2005, 36, 19–31. [Google Scholar]
  31. Melton, T.; Yates, L.; Iles-Smith, P. Project Benefits Management: Linking Projects to the Business; Butterworth-Heinemann: Oxford, UK, 2011. [Google Scholar]
  32. Kendall, I.G.; Rollins, S.C. Advanced Project Portfolio Management and the PMO: Multiplying ROI at Warp Speed; J. Ross Publishing, Inc.: Boca Raton, FL, USA, 2003. [Google Scholar]
  33. Raymond, L.; Bergeron, F. Project Management Information Systems: An empirical study of their impact on project managers and project success. Int. J. Proj. Manag. 2008, 26, 213–220. [Google Scholar] [CrossRef]
  34. Hillson, D.; Grimaldi, S.; Rafele, C. Managing project risks using a cross risk breakdown matrix. Risk Manag. 2006, 8, 61–76. [Google Scholar] [CrossRef]
  35. Mihic, M.; Petrovic, D.; Vuckovic, A. Possibilities of application of cost-benefit analysis to energy efficiency projects in buildings. Econ. Themes 2011, 49, 355–378. [Google Scholar]
  36. Mihic, M.; Petrovic, D.; Vuckovic, A. Energy efficiency project portfolio optimization for public buildings. Metal. Int. 2012, 17, 166–173. [Google Scholar]
  37. Mihic, M.; Vuckovic, A.; Vuckovic, M. Benefits management in energy efficiency projects in public buildings. Management 2012, 17, 57–66. [Google Scholar]
  38. Raz, T.; Michael, E. Use and benefits of tools for project risk management. Int. J. Proj. Manag. 2001, 19, 9–17. [Google Scholar] [CrossRef]
  39. Ibbs, C.W.; Kwak, Y.H. Assessing project management maturity. Proj. Manag. J. 2000, 31, 32–43. [Google Scholar]
  40. Ahlemann, F. Towards a Conceptual Reference Model for project management information systems. Int. J. Proj. Manag. 2009, 27, 19–30. [Google Scholar] [CrossRef]
  41. Andersen, E.S.; Jessen, S.A. Project maturity in organizations. Int. J. Proj. Manag. 2003, 21, 457–461. [Google Scholar] [CrossRef]
  42. Baccarini, D. The logical framework method for defining project success. Proj. Manag. J. 1999, 30, 25–32. [Google Scholar]
  43. Shenhar, A.J.; Levy, O.; Dvir, D. Mapping the dimensions of project success. Proj. Manag. J. 1997, 28, 5–13. [Google Scholar]

Share and Cite

MDPI and ACS Style

Mihic, M.M.; Petrovic, D.C.; Obradovic, V.L.; Vuckovic, A.M. Project Management Maturity Analysis in the Serbian Energy Sector. Energies 2015, 8, 3924-3943. https://doi.org/10.3390/en8053924

AMA Style

Mihic MM, Petrovic DC, Obradovic VL, Vuckovic AM. Project Management Maturity Analysis in the Serbian Energy Sector. Energies. 2015; 8(5):3924-3943. https://doi.org/10.3390/en8053924

Chicago/Turabian Style

Mihic, Marko M., Dejan C. Petrovic, Vladimir Lj. Obradovic, and Aleksandar M. Vuckovic. 2015. "Project Management Maturity Analysis in the Serbian Energy Sector" Energies 8, no. 5: 3924-3943. https://doi.org/10.3390/en8053924

APA Style

Mihic, M. M., Petrovic, D. C., Obradovic, V. L., & Vuckovic, A. M. (2015). Project Management Maturity Analysis in the Serbian Energy Sector. Energies, 8(5), 3924-3943. https://doi.org/10.3390/en8053924

Article Metrics

Back to TopTop