Bibliometric Analysis of the Application of Artificial Intelligence Techniques to the Management of Innovation Projects

Abstract

Due to their specific characteristics, innovation projects are developed in contexts with great volatility, uncertainty, complexity, and even ambiguity. Project management has needed to adopt changes to ensure success in this type of project. Artificial intelligence (AI) techniques are being used in these changing environments to increase productivity. This work collected and analyzed those areas of technological innovation project management, such as risk management, costs, and deadlines, in which the application of artificial-intelligence techniques is having the greatest impact. With this objective, a search was carried out in the Scopus database including the three areas involved, that is, artificial intelligence, project management, and research and innovation. The resulting document set was analyzed using the co-word bibliographic method. Then, the results obtained were analyzed first from a global point of view and then specifically for each of the domains that the Project Management Institute (PMI) defines in project management. Some of the findings obtained indicate that sectors such as construction, software and product development, and systems such as knowledge management or decision-support systems have studied and applied the possibilities of artificial intelligence more intensively.

1. Introduction

Artificial intelligence (AI) is a broad term, so there are numerous existing approaches and definitions. It can be defined as a discipline that tries to develop systems capable of performing certain operations that are considered typical of human intelligence, such as self-learning or reasoning [1]. It has also been defined from the general process of acquiring these faculties, as the ability of a system to correctly interpret external data, learn from said data, and use those learnings to achieve specific objectives and tasks through flexible adaptation [2]. Various artificial-intelligence techniques have been adopted in very different areas with very different objectives and have been studied globally [3]. More specifically, there are also numerous works that have analyzed the impact of artificial intelligence on project management, both from the point of view of the discipline as a whole [4] and in more specific and different aspects such as risk management [5], effort estimation [6], product development [7], or project-duration forecasting [8].

On the other hand, the main reference to define the term innovation and all related activities is the Oslo Manual [9]. According to this reference, any innovation implies the use of new knowledge or a new combination of existing knowledge. Innovation activities include all scientific, technological, organizational, financial, and commercial actions that lead to innovation. Thus, innovation projects can be developed both internally and in external collaboration. In this context, innovation activities have always been considered a specifically human field given their characteristics of creativity and novelty. However, numerous studies are highlighting the impact that artificial intelligence is generating in different aspects of the field of innovation. Some authors highlight the importance of the level of risk between projects in general and research projects [10]. For their part, other studies [11,12] review which aspects are important to consider in the transformation towards the digital organization of innovation. The application of artificial intelligence techniques in specific aspects such as the initial selection of innovation projects and decision making in innovation management has also been frequently analyzed [13,14,15].

The general objective of this study was to analyze the impact that artificial intelligence techniques have had on the management of innovation projects in recent years. Bibliometric analysis is a widely used method to explore large volumes of scientific data, which allows, among other aspects, trends in a specific scientific field to be analyzed [16]. To this end, the next section describes the research methodology used to carry out a bibliometric review. Afterwards, the results obtained are described, first from a general point of view, and then for each of the specific areas of project management. Finally, the conclusions reached are described, as well as the next steps to take for a more extensive and detailed study.

2. Materials and Methods

Techniques for bibliometric analysis can be classified into two categories, performance analysis and scientific mapping [16]. While the first of these studies the contributions of the constituents of the research, scientific mapping focuses on the relationships between them. Techniques used for scientific mapping include citation analysis, co-citation analysis, bibliographic coupling, co-word analysis, and co-authorship analysis. In this work, co-word analysis was used. This studies the actual content of the publication from the keywords, defined by the authors themselves, or by their frequent occurrence in titles, abstracts, or full texts. In this analysis, it is assumed that the words that frequently appear together have a thematic relationship and have been used in different works to preview the future of a scientific field [17,18,19,20].

Co-word analysis is the study of co-occurrences, that is, the joint appearances of two terms in a given text with the purpose of identifying the conceptual and thematic structure of a scientific domain [21]. This analysis allows for the exploration of existing or future relationships among topics in a research field by focusing on the written content of the publication itself [16]. The quality of a co-word analysis depends on several factors such as the data sources used, the appropriate selection of search terms, and the mathematical methods of subsequent analysis of the results [22].

The source used in this study was Scopus, an abstract and citation database launched in 2004 by Elsevier [23] and one of the main bibliographic reference databases [24,25]. In addition to the source used, another fundamental aspect in a bibliometric analysis is the quality of the search terms used. One of the main problems with co-word analysis is the so-called “indexer effect” [22,26], derived from the search terms used in data acquisition. In this study, author keywords were used as search terms in the database, as opposed to automatically indexed keywords, because they better describe the content of the papers [27]. Three areas overlap in this work: project management, innovation, and artificial intelligence. To select the most appropriate search terms in each field, other bibliometric studies carried out in those fields were reviewed:

• The application of artificial intelligence to different scientific fields has been the aim of numerous bibliometric studies [28,29,30,31,32,33]. Some of these studies use only “artificial intelligence” as the search term [32,33], but this generic name would not include in the results valid publications for the purpose of this work, in which the authors have been more specific when defining the part of artificial intelligence that defines their study. On the opposite side, other papers incorporate many other keywords in trying to include all the related results [29,30,34]. Delving into this same line of research, several studies analyze, among other aspects, the temporal evolution of the most frequently used keywords in this field of artificial intelligence [28,31,35,36]. As these studies show, the number of terms and their variants is very high, given the multitude of techniques (neural networks, fuzzy logic, expert systems, support vector machine, random forest, etc.) and applications (classification, prediction, optimization, pattern recognition, decision aid, etc.) included in this field. In this work, we chose to select only those keywords used most frequently and with a more general character, that is, those that, by themselves, encompass several techniques and applications. As a result, the search terms used were: “artificial intelligence”, “knowledge based”, “machine learning”, and “data mining”.
• In the field of innovation, different bibliometric studies have also been carried out with derivative terms such as “open innovation”, “innovation management”, “innovation system”, “innovation policy”, “radical innovation”, and “innovation model” [37]. In this case, the search terms “innovation”, “research”, and “development” were selected to include different types of projects.
• In relation to the project-management area, obviously the most common keyword is “project management”, and variants are not widely used, so this was the selected search term [38].

In addition to the combination of these three broad areas (artificial intelligence, project management, and innovation), the application of artificial intelligence techniques in more specific areas of project management was studied. For this, it was decided to take as a reference the seventh edition of the PMBok published by the PMI [39]. It defines a set of project performance domains to achieve the desired project outcomes.

The defined “Performance Domains” are the following:

• Stakeholders, who aim to maintain project alignment and collaborate with those affected to foster their satisfaction and positive relationships;
• Team, establishing the culture and environment necessary for its development and encouraging the leadership behaviors of the members;
• Development and Life Cycle Approach, optimizing the delivery of the project results;
• Planning, organizing, preparing, and coordinating the work throughout the entire project;
• Project Work, addressing the activities and functions associated with establishing project processes, managing physical resources, and fostering a learning environment;
• Delivery, focused on meeting the requirements, scope, and quality expectations of the expected deliverables;
• Measurement, with the aim of evaluating the performance of the projects and adopting the appropriate adjustment measures;
• Uncertainty, so that the team can manage the threats and opportunities that arise during the development of the project.

To carry out the queries, the specific keywords of each domain were selected from those used by the PMI in its definition and delimitation (

Table 1. Keywords for each project performance domain.

Stakeholders	Team	Life Cycle	Planning
stakeholders	team	life cycle	planning
suppliers	group	phase	schedule
customers	leader	predictive	duration
end users	member	adaptive	effort
regulatory bodies	individual		dependency
	motivation
	communication
	collaboration
	skill
	conflict
Work	Delivery	Measurement	Uncertainty
work	delivery	measurement	risk
resource	scope	metric	threat
bid	quality	baseline	opportunity
contract	requirement	indicator	uncertainty
cost	change	efficiency	contingency
		performance
		forecast

).

Therefore, a general search of the three overlapping areas was carried out, the results of which were analyzed globally. Subsequently, with the keywords indicated in each case, specific queries were carried out for each of the PMI domains.

For the analysis of the results, the software developed by Nees Jan van Eck and Ludo Waltman of the Center for Science and Technology Studies (CWTS) of the University of Leiden in the Netherlands was used [40]. VOSviewer allows for building, analyzing, and visualizing bibliometric networks. To do this, it works with different analysis elements (authors, organizations, countries, documents, sources/journals, keywords, cited references, cited authors, or cited sources/journals) and measurements (co-authorship, co-occurrence, citation, bibliographic coupling, or co-citation). The normalization, mapping, and clustering methods used have been described in detail by their authors in several publications [41,42,43]. This software has been used in numerous research works in very diverse disciplines [44,45,46,47,48] including project management [49].

The VOSviewer software represents the keywords in a multidimensional space, increasing the size of their presentation depending on the number of appearances, and distributes them as more or closer points depending on the relationships between them, represented in the form of links. In addition, the groups (clusters) were identified by VOSviewer through a multidimensional scaling algorithm and are colored for better visualization. In each case, those keywords that exceed a certain threshold of frequencies are represented. Depending on the number of publications resulting from each query, said threshold was adjusted in each case so that thematic groups with a certain consistency were obtained and that the number of resulting terms was not too high in a way that made it difficult to interpret the groups obtained. The terms included in each cluster are described and analyzed later in the Results section.

3. Results

As a result of the keyword search of the three reference areas in the Scopus database, a total of 970 publications was obtained, whose temporal distribution can be seen in Figure 1. Starting in 2000, a considerable number of documents have been published, reaching a maximum of 91 documents in 2008, with the minimum value in 2014 with 16 documents. This has increased again in recent years, exceeding 50 articles in 2020 and 2021. It is possible to affirm, therefore, that the application of artificial intelligence in the field of innovation project management has been an area of great interest in recent years.

The main sources with the highest number of publications, such as Lecture Notes in Computer Science and the International Journal of Project Management, are shown in

Table 2. Main sources of published documents.

Source	ISSN	Number of Documents
Lecture Notes in Computer Science	0302-9743	41
International Journal of Project Management	0263-7863	19
Procedia Computer Science	1877-0509	18
Advances in Intelligent Systems and Computing	2194-5357	15

.

Figure 2 presents a two-dimensional bibliometric map resulting from the analysis of the 970 publications obtained in the initial query. As explained in the Materials and Methods section, VOSviewer software only shows on the map those keywords that exceed a certain frequency threshold. This threshold was adjusted so that the number of resulting terms was not too high in a way that made it difficult to interpret the groups obtained. In this case, four clusters were identified, and, as expected, given the keywords used in the query, terms such as project management, artificial intelligence, and knowledge-based systems appeared highlighted, that is, with a high frequency of occurrence.

Therefore, the interest of the co-word analysis is in the rest of the terms that appear in each cluster:

• Cluster 1, colored in red in the figure, is defined by 14 keywords, mainly related to software (computer software, software design, and software engineering), risk (risk assessment and risk management), and machine learning as the most outstanding aspects;
• Cluster 2, presented in green and defined by 11 terms, is mainly related to the management of knowledge and information (knowledge acquisition, knowledge management, knowledge-based systems, and information technology);
• Cluster 3, colored in blue and defined by 9 terms, includes terms such as decision-support systems, decision making, problem solving, and product development;
• Cluster 4, in yellow in Figure 2, is made up of only 2 terms, engineering research and research projects.

The analyses carried out in the specific areas of project management defined by the PMI are described below. The number of publications related to each of these domains is reflected in Figure 3 and can provide an idea of which areas are being worked on more intensively. In this sense, Work (211), Planning (162), and Delivery (154) are the domains that have generated the most publications thus far.

Results by Domain

The results obtained in the specific queries of Scopus for each of the domains defined by the PMI are presented below. For each domain, the terms that define each of the clusters identified using the VOSviewer software are described, paying special attention to those terms that have not been used as search terms.

Figure 4 shows the clusters identified by the VOSviewer software in the results of the bibliographic search in the Stakeholders domain:

• The first cluster is related to customer satisfaction through research projects in areas such as quality control or software and its management;
• The use of information systems for the design, development, and marketing of products as part of the strategic planning of different organizations seems to be the focus of the second cluster;
• The third cluster focuses on the decision-making process and the associated systems and tools for a sustainable development process;
• The fourth cluster is related to the acquisition and management of knowledge in relation to customer requirements, sales, and competitiveness.

Other terms included in the bibliographic search such as end users, providers, and regulation are not highlighted.

Similarly, for the Team domain, the identified clusters in Figure 5 are described:

• The sustainable development of products using knowledge management is the focus of cluster 1;
• The second cluster deals with the management of human resources through advanced software tools;
• The figure of the project manager and the decision-making tools used appear in cluster 3;
• The terms engineering education and software development are highlighted in cluster 4;
• The relationships between team members and project success are studied in the last cluster.

More specific aspects of team management such as motivation, communication, collaboration, and conflict resolution, which are included among the search terms, do not appear in the results.

Figure 6 shows the three clusters identified in the Life Cycle domain:

• Perhaps due to the generality of the life cycle search term, the first cluster includes numerous terms such as problem solving, mathematical models, product development, construction industry, and sustainable development together with information and knowledge management;
• The second cluster fundamentally focuses on software design and development and also introduces other aspects such as learning systems and risk assessment;
• The last cluster deals with decision making and associated processes and systems.

Search terms used in the query such as the type of life cycle (predictive, adaptive) or its phases are not reflected.

In the case of the Planning domain, three clusters were identified, as shown in Figure 7.

• The first cluster presents very similar terms to the one identified in the Life Cycle domain, including the industrial sector of construction, product development, and knowledge management;
• The second cluster is clearly aimed at studying software design and development, especially effort estimation using various techniques;
• The third cluster focuses on support systems for decision making in aspects such as production planning and control.

In the Work domain, four clusters were identified (Figure 8):

• The first cluster, in red in the figure, is like others that appear in some previous domains with terms such as knowledge acquisition and management, problem solving, and the construction industry;
• Cost, cost–benefit analysis, product development, information management, and life cycle are the terms that make up the second cluster;
• Sustainable development together with decision-making systems defines the third cluster;
• The last cluster relates to the terms of human resource management and software design and engineering.

Searching for terms related to the Delivery domain resulted in the following three clusters (Figure 9):

• The cluster colored in red in the figure focuses on the requirements, design, and development of the software and the control and assurance of its quality;
• The terms that appear in the second cluster (knowledge management, sustainable development, and construction industry, among others) are similar to those that define groups in other previous domains;
• Decision support systems, human resources, and information management, together with product development, make up the last cluster, shown in blue in the figure.

Analysis of the bibliographic results obtained related to the Measurement domain search gave rise to the following five clusters:

• Decision making in the construction industry seems to be the area of interest of the first cluster;
• Different topics involved in the evaluation of the performance and development of projects such as knowledge management or risk evaluation give rise to the formation of the second cluster;
• The cluster colored in blue in Figure 10 is configured around forecast, relating to topics such as information management, neural networks, and quality control, in the field of industrial research;
• The fourth cluster is defined by different tools such as fuzzy logic or learning systems in the field of software design and engineering;
• Sustainable product development and energy efficiency are the terms included in the fifth cluster, in purple in Figure 10.

The last domain corresponds to “Uncertainty”, and Figure 11 shows the following four clusters:

• The different aspects related to risks (perception, analysis, and management) in the field of computer software (design, development, and management) make up the first cluster of this domain. Techniques such as Bayesian networks also appear to be prominent;
• Cluster 2 is made up of terms that are very similar to others that have already been repeated in other domains, related to information and knowledge management in the construction industry;
• The terms related to decision making in a context of uncertainty give rise to the third cluster;
• Cluster 4 is defined by a single term, “sustainable development”, connected to others that appear in cluster 2.

4. Discussion

As indicated above, one of the difficulties of this type of bibliometric analysis of co-words is the correct choice of search terms in such a way that all related publications are identified, but without conditioning the results obtained. In this sense, attention has been paid mainly to the results that were not used as search terms. As already indicated in the Methodology section, the names of specific artificial intelligence techniques (neural networks, fuzzy logic, expert systems, support vector machine, random forest, etc.) were not used in the searches. In this sense, in the results obtained, only some of them appeared as relevant in the clusters, such as neural networks, fuzzy logic (Measurement domain), and Bayesian networks (Uncertainty domain). The importance of this last technique in recent years in the field of project management and uncertainty has already been pointed out by other authors [50].

One of the novelties of this study is the division of project management into the domains defined by the PMI with the aim of identifying the main research trends in each of them. This separation made it possible to identify areas of interest that are both specific to each domain and common to many of them: for example, clusters related to the success of the project and team members in the Team domain, with customer requirements and competition in the Stakeholders domain, effort estimation in the Planning domain, costs and human resource management in the Work domain, and, as expected, the perception, analysis, and management of risks in the Uncertainty domain. On the contrary, several areas of research frequently appear in different domains.

Terms related to decision making appear in all PMI domains. In other words, the development of decision-making support tools and systems using artificial intelligence to help in the management of innovation projects is a topic of wide interest. The scientific publications retrieved show that, up to now, these systems have been developed for specific sectors, projects, or areas [51,52,53] and not from a full general project-management perspective.

Numerous specific features are studied in software projects, such as risks [54,55], performance [56], human resource management [57], quality [58,59], and effort estimation [60,61,62], as well as more general or methodological aspects [63,64].

The construction sector is usually linked to information and knowledge management for product development and sustainability. Other recent research [65,66,67,68] focusing on the impact of artificial intelligence in the construction sector reached similar conclusions, with a continued increase in the number of scientific investigations in this field.

Product-development projects and their sustainability are another recurring research topic that frequently appears linked to knowledge management or decision-support systems.

The current impact of artificial intelligence in many different fields, including science, is being highlighted in many publications [69]. Bianchini et al. [70] recently proposed considering artificial intelligence as “an emerging general method of invention” and, on this basis, deriving its policy implications.

As limitations of this study, it is possible to mention the one inherent to this type of bibliographical analysis, based on the frequency of appearance of the terms, which allows for detecting general and continuous trends over time, but which does not allow for revealing the most recent lines of research. Another limitation is the use of a single database (Scopus) due to the difficulty of combining systems with different indexing methods using keywords. The same analysis could be extended to other reference bibliographic databases such as the Web of Science (WOS) [71], which would allow the findings to be compared.

5. Conclusions

In this work, an exploratory study of the impact of artificial intelligence on the management of innovation projects was carried out. With this objective, a set of queries for the Scopus database were defined to obtain the related publications. Using the VOSviewer software, a co-word analysis was applied to the data of the documents obtained.

In addition to the study of the results as a whole, they were segmented according to the performance domains in the field of project management defined by the PMI. This division made it possible to identify the topics, techniques, and other aspects that are being worked on the most. Decision-support systems and tools and the different processes associated with knowledge management (knowledge acquisition, knowledge sharing, and knowledge management) and information frequently appear in the different domains of project management. In the same way, some specific technological sectors are also distinguished, such as:

• The field of software development, where aspects related to human resource management and teams, such as effort estimation and learning systems, as well as software design and quality, are highlighted;
• The construction sector with aspects such as risks, planning, and problem solving;
• Product development, where sustainability is also a prominent aspect that appears in several of the domains.

On the other hand, it is also interesting to point out the appearance of different artificial-intelligence techniques (Bayesian networks, fuzzy logic, neural networks, etc.) related to specific areas, as can be seen in the groups generated in the different PMI domains.

Regarding the continuity of the work, it is possible to delve into the bibliometric analysis in relation to different aspects such as the techniques used or the most studied terms, or to extend the work using other relevant databases such as the Web of Science Core Collection.