Roadmap for Implementing Business Intelligence Systems in Higher Education Institutions: Systematic Literature Review ^†

Abstract

Higher education institutions (HEIs) make decisions in several domains, namely strategic and internal management, without using systematized data that support these decisions, which may jeopardize the success of their actions or even their efficiency. Thus, HEIs must define and monitor strategies and policies essential for decision making in their various areas and levels, in which business intelligence (BI) plays a leading role. This study presents a systematic literature review (SLR) aimed at identifying and analyzing primary studies that propose a roadmap for the implementation of a BI system in HEIs. The objectives of the SLR are to identify and characterize (i) the strategic objectives that underlie decision making, activities, processes, and information in HEIs; (ii) the BI systems used in HEIs; (iii) the methods and techniques applied in the design of a BI architecture in HEIs. The results showed that there is space for developing research in this area since it was possible to identify several studies on the use of BI in HEIs, although a roadmap for its implementation was not identified, making it necessary to define a roadmap for the implementation of BI systems that can serve as a reference for HEIs.

1. Introduction

Higher education institutions (HEIs) are characterized by having different specificities in their mission and management strategies [1] and there is a clear need for HEIs to improve decision making in all areas and at all levels [2,3,4] as HEIs often make decisions without the use of any specific data or analysis [1]. The organizational culture of an HEI is an important aspect to take into account, as it influences decision making and the processes that are implemented to ensure the efficiency of the HEI’s activity, taking into account the established strategy or goals [5]. In this sense, HEIs need relevant information to monitor their performance, in accordance with the goals set out in their strategic plans [6]. However, the success of HEIs’ performance depends on identifying and measuring the main key performance indicators (KPIs) [7].

Information systems (IS) play a key role in the management of HEIs, supporting their activities and decision-making processes. IS has a class called decision support systems (DSS) which is geared towards this objective, although the degree of complexity required, with the need to involve all of the HEI’s IS, leads to the adoption of business intelligence (BI) systems [8]. BI is a process that combines data storage and collection with knowledge management, enabling HEIs to optimize their performance and decision making through the use of applications, tools, infrastructures, and practices that allow access to and analysis of information that ordinary reports are usually unable to provide [9]. In fact, BI makes it possible to create dashboards and synthesized reports that can be created in a very simple way. In this way, a BI system takes advantage of information technologies and infrastructures to use KPIs to measure the institutional performance of HEIs [10].

In this context, the exploratory work carried out by Sequeira et al. [11] offers an essential starting point for exploring this theme, presenting the methodology of a roadmap that can serve as a reference for implementing a BI system in HEIs to support decision making in their various areas and levels.

The main objective of this research is to broaden the discussion initiated by Sequeira et al. [11] by presenting a systematic literature review (SLR) to identify academic contributions that have scientific validation in relation to the main areas of interest addressed in this research, with the aim of identifying and analyzing roadmap proposals for the implementation of a BI system in HEIs, with a view to supporting decision making in their various areas and levels.

Our systematic review followed the Kitchenham and Charters [12] guidelines. We aimed to identify, appraise, and synthesize all relevant research on the implementation of BI systems in HEIs, to provide a comprehensive understanding of the methodologies, findings, and gaps in the current literature. The primary studies were selected on the basis of this methodology and read carefully. The objectives defined for the SLR are: (i) to identify and characterize the strategic objectives that underpin decision making, activities, processes, and information in HEIs; (ii) the BI systems used by HEIs; (iii) the methods and techniques applied in designing a BI architecture in HEIs.

This document is divided into four more sections that will continue to explain this research. Section 2 presents the main theoretical foundations. Section 3 refers to the methodology used, such as keywords and selection criteria. Section 4 compiles the results of this article, describing the relevant points found in the articles reviewed, and discusses the results and the points we feel should be highlighted. Finally, Section 5 reports the main conclusions and scientific contributions of this research, as well as future directions.

Motivation

The increasing complexity in the management of HEIs, driven by the demand for more effective and informed decision making, highlights the crucial importance of implementing BI systems. In this context, this study is part of the “BI@UTAD” project, which aims to develop and implement a BI system at the University of Trás-os-Montes and Alto Douro (UTAD). This effort aims to overcome the limitations identified in the existing information systems (IS), characterized by outdated and non-integrated information, time-consuming manual processes, and errors, which affect the quality and efficiency of institutional management.

The BI@UTAD project aims to address these challenges by implementing a BI solution that provides an integrated and reliable view of the information that is essential to support the decision-making process. This initiative is particularly relevant given that UTAD, like many other higher education institutions, is faced with the need to improve the quality of teaching, promote academic success, and prevent students from dropping out, as well as attract students to higher study cycles and increasing productivity in its areas of activity.

The motivation for this research, therefore, stems from the urgent need to improve decision-making processes at UTAD by adopting BI technologies to overcome the shortcomings identified in the current IS. This study is not only a response to an academic requirement or an isolated institutional need but reflects a global trend towards digitization and the intelligent use of data in higher education institutions.

Therefore, this article presents an SLR on the implementation of BI systems in HEIs, to identify proposed roadmaps for their implementation, which can serve as a reference for UTAD and other similar institutions. This work is a crucial step in understanding how BI solutions can be adapted and implemented to meet the strategic objectives of higher education institutions, thus contributing to the continuous improvement of teaching quality and institutional management.

2. Theoretical Background

This section presents an overview of the three main research domains related to this article, namely decision making in HEIs, BI systems in HEIs, and BI systems architecture.

2.1. Decision Making in HEIs

Decision making can be defined as a management process for framing a given situation in which a decision needs to be made. By defining a decision model, made up of a set of actions and evaluation criteria, it is possible to select the most appropriate course of action and even learn from the decision-making process [13].

2.1.1. Strategic Management

Identifying challenges and implementing effective management strategies can improve process and infrastructure management [14]. On the other hand, it is important for HEIs to have their strategies aligned with their resources [15]. It should also be noted that defining and prioritizing areas of activity allows HEIs to better align their resources and strategic opportunities, as well as the possibility of boosting socio-economic development and improving their curricula and teaching methods in order to achieve sustainable development [16].

Thoenig and Paradeise [17] refer to the importance of internal organizational capacities, including academic human resource management, cultural norms, and organizational governance, in strengthening strategic capacity. HEIs are, in fact, primarily responsible for the governance and management of their finances, activities, and human resources, with autonomy to decide on their organizational operations, including how decisions are made about institutional priorities, strategies, objectives, and resource allocation [1]. In this sense, HEIs seek to improve their competitiveness by implementing management strategies geared towards quality and financial sustainability, using performance measurement systems that include a wide variety of financial and non-financial measures. These measures give decision-makers a better understanding of the relationships between the various strategic objectives, as well as facilitating communication between the actions of employees and the defined objectives, and also help to allocate resources and define priorities, taking these strategic objectives into account [18]. Sawhney et al. [19] reinforce the relevance of strategic management in HEIs, suggesting the adoption of frameworks to help HEIs achieve sustainability and long-term success.

As far as the strategic level is concerned, decision making encompasses the policies, strategies, and actions that HEIs implement in a holistic way, having an impact on all HEI activity. For example, the number of students enrolled each semester affects the allocation of an HEI’s resources: budget, teaching staff, and facilities [1]. It is, therefore, important for HEIs to have a thorough understanding of their objectives and the methods they must use to achieve them successfully [4].

2.1.2. Performance and Control

Performance management procedures can improve the overall performance of HEIs by aligning individual objectives with strategic objectives [20]. Performance management becomes important in determining the organizational success of an HEI, as it facilitates the management of its resources and the measurement of its results. In addition, it allows for the analysis of its organizational objectives and breaks them down into specific benchmarks to ensure that the objectives are measurable [21]. On the other hand, measuring the performance of organizational objectives is a relevant factor in the management of HEIs, because, without these measurement systems, HEIs are unable to respond efficiently and effectively to the uncertainties that exist in the contexts in which they operate. However, HEIs’ strategic objectives are intangible and subjective, making it difficult to identify critical success factors (CSFs) and KPIs [18].

There are several types of CSFs in HEIs, which can vary depending on the context and specific objectives of the HEI. KPIs are commonly used as an important tool to measure the performance and quality of HEIs [22,23]. For Varouchas et al. [24], KPIs are measurable values that explain the effectiveness of an HEI and how it achieves its main objectives. Morais and Castro Lopes [25] indicate that KPIs consist of quantifiable values that show how HEIs achieve their institutional objectives. Tomchuk et al. [26] present the potential for HEI leaders to monitor and implement development management functions through the use of a KPI system, which can improve the management system and the motivation of teaching and management staff in higher education. KPIs should be developed according to the different factors that characterize an HEI, to ensure that they actually add value to it, so it is important that the KPIs are aligned with the institutional strategy and its respective objectives, and that they have the capacity to require reliable and accurate data that can be easily obtained and subsequently acted upon [25]. Given that KPIs are directly linked to the mission of HEIs, the level of achievement can demonstrate whether or not the HEI is aligned with its mission and strategic objectives [10].

Volchik and Maslyukova [27] argue that the use of quantitative indicators can have a negative impact on academic cultural values and that attention should be paid to promoting long-term sustainable development. Gontareva et al. [28] identify a set of management efficiency factors for controlling the provision of information and communication for the sustainable development of HEIs. On the other hand, Bisschoff et al. [7] state that the success of HEI performance depends on identifying and measuring the main KPIs.

Thus, the management of an HEI must be based on measuring its organizational performance and using relevant information to support its political orientation and organizational operation. In this context, a well-structured system, capable of correctly measuring performance, is essential if an HEI is to convert its strategy into operational objectives, as well as guide employees to achieve the proposed goals [29].

It is, therefore, essential to have access to strategic information so that it is possible to diagnose the current state of an HEI, with a view to defining a strategic orientation, according to internal and external factors, which allows the methods that support the development of a strategy to be determined [4]. Decision making based on relevant information enables HEIs to improve their performance [10,13].

Mukhtar et al. [30] emphasize the importance of information management in the development of management IS, based on an analysis of HEI needs. For Deja [5], effective information management can improve the efficiency of decision making in HEIs, consisting of a process integrated into the administrative activities of HEIs. HEI decision-makers can take advantage of information visualization, provided that all the elements needed to replicate the studies and methods are properly recorded by tools suitable for this purpose [31]. Through their IS, HEIs collect and produce a vast amount of data relating to students, teachers, academics, research processes, and financial aspects [32] in order to increase their strategic results [33].

In this sense, IT governance plays an important role in the overall performance of HEIs [34]. Julianti et al. [35] also refer to the importance of IT governance in supporting IT-based educational services and achieving better performance of academic IS, with a view to aligning strategic objectives with the implementation of IT in the HEI environment. Thus, BI is an effective strategic performance management technology, enabling users to collect, integrate, access, and analyze data to help make efficient decisions in HEIs. On the other hand, BI must consider and focus on analytical aspects to meet today’s performance management needs [21].

2.2. BI Systems in HEIs

Information and communication technologies are currently preponderant in assisting the management of HEIs, assuming an important role in improving the decision-making process [1]. The search for simple and quick transformation of data into relevant information, which often goes unnoticed, is essential for matching internal and external data of HEIs and, consequently, for effective decision making [36].

IS plays a key role in the management of HEIs, supporting their activities and decision-making processes. However, it is essential that IS are developed according to the needs of HEIs, and there must be integration of the various IS and IT, to enable better access to and processing of information. IS has a class called DSS which is geared towards this goal, namely at the tactical and strategic levels, with analytical specificities that allow for the creation of knowledge and organizational intelligence. Although the ability of DSS to support decision making is recognized, the degree of complexity required, with the need to involve all of the HEI’s IS, leads to the adoption of BI systems [8]. A BI solution consists of a data-oriented DSS that supports a set of operations, such as querying historical data, creating summary reports, executive IS, and online analytical processing (OLAP) [37].

Some Concepts

Advances in the integration of IT, particularly BI, have been decisive in the evolution of HEIs, allowing decision making based on data analysis to be a constant reality today [3,38,39]. Following on from this, BI can be considered to have the capacity to take advantage of the various pieces of information generated to provide more efficient responses [40] improving the ability to deal with the data generated and the quality of the information obtained, thus significantly supporting HEIs in their decision-making processes [6,10,41]. Despite this, HEIs need to find the best way to implement BI systems and thus maximize their benefits [38,40]. However, these benefits can only be achieved if a BI system is implemented properly [42].

There is a need to implement effective coordination by exploiting the diverse data that is produced and collected daily in HEIs so that it can be converted into knowledge and thus support the creation of institutional strategies [43,44]. By way of example, the technological advances seen in HEIs in general have created a progressive interest in using data from student behavior to provide process-oriented information aimed at improving teaching [45]. HEIs use data analysis architectures, known as BI, to obtain results that support various decision-making processes. The BI implementation process involves extracting data from different sources, processing it, and transforming it into relevant knowledge [46]. Currently, several HEIs use BI concepts to improve internal management processes, such as the administrative and operational aspects, reducing costs and optimizing processes, as well as increasing the quality of teaching itself [46,47]. It is, therefore, essential that IS and HEI strategies are fully aligned. On the other hand, the current state of HEIs shows that there may be some barriers to achieving this goal, essentially due to the existence of heterogeneous IS and applications [2].

BI plays an important role in supporting HEIs’ decision making. Although the benefits it brings are mainly indirect [41], they are wide-ranging, such as aiding financial oversight and operational performance, identifying critical areas, and providing insights for research [10,48]. In short, the successful adoption of BI can allow HEI decision-makers direct access to accurate and up-to-date information whenever they need it. On the other hand, BI can also provide a holistic view of the entire HEI, enabling faster, more accurate, and more reliable decisions to be made [3].

According to Ain et al. [38], a BI system is regularly referred to as a set of technological solutions that support decision-making processes by: (i) facilitating more aggregation, systematic integration, and management of unstructured and structured data; (ii) working with large amounts of data; (iii) providing end users with more processing power to discover new knowledge; (iv) providing analysis solutions, ad hoc queries, reports, and forecasts. For Tripathi et al. [37], BI aims to gather information from users and improve their knowledge so that they can make efficient decisions in line with the HEIs’ mission and vision.

Therefore, for a BI system to be successful, it is crucial to delve into decision theories in order to understand how the systems and the information produced can benefit the decision-making process. Despite this, traditional BI tools do not support predictive capabilities and, therefore, do not meet some of the current requirements of HEIs, as they only lead to historical reports and ad hoc queries, which relate to descriptive analysis.

Descriptive analysis provides a set of insights into the current state and its history, and is, therefore, not appropriate for a strategic level, making predictive analysis necessary, where the objectives are long-term strategy and decision making [3]. On the other hand, through predictive analysis, it is possible to identify what will arise in the future, which allows for more efficient internal and external planning [49]. However, a properly implemented BI solution supports data management and provides a constant flow of information in real time [3].

According to Scholtz et al. [3], the benefits of a BI solution are diverse, including integrated and improved information, time savings for data providers and users, more and better information, polished decision-making and business process capabilities, specific support for strategic and tactical objectives, and improved organizational performance. Despite this, there is limited use of BI in HEIs, probably due to a number of factors related to insufficient data. Thus, there are several common challenges to take into account for BI, such as data quality, complexity and cost of implementing and using the system, information technology (IT) support, and organizational alignment. In addition, technological capabilities and data quality, end-user access, and the integration of BI solutions with other systems are pertinent aspects for the successful use of BI, regardless of HEI strategies and policies [37].

Soliman et al. [50] emphasize the importance of data management and analysis in HEI decision making. In this way, data visualization should encompass the search, collection, and extraction of data, which after being processed, are analyzed and represented visually [31]. On the other hand, Khoujaet et al. [51] and Bianchi and Sousa [52] highlight the importance of IT governance in HEIs, with Khoujaet et al. [51] emphasizing the need for effective communication between IT, companies, and stakeholders.

In short, BI is an easy-to-use tool that allows information to be collected, stored, and processed [3]. However, as with IS, the BI system must be aligned with the HEI’s strategy to ensure that the strategic objectives are met as stipulated [10].

2.3. BI Systems Architecture

There are several studies that similarly present the architecture of a BI system. For Ain et al. [38], a BI system consists of a combination of tools, such as a data warehouse, OLAP, and dashboards. A data warehouse groups and analyzes accurate, clean, and detailed data from multiple sources. OLAP supports multidimensional analysis in real time and allows users to perform various operations, such as aggregation, roll-up filtering, and drill-down to obtain details. Dashboards consist of front-end applications that enable data visualization and performance management.

2.3.1. Definition of Architecture

Bessa et al. [8] argue that the structure of a traditional BI architecture is made up of the collection of internal data, from the various operational systems, or external data; the data extraction, transformation, and loading module, i.e., data integration is carried out in three stages, more commonly known as extraction, transformation and loading (ETL); data storage sources (e.g., apartment database, data warehouse, data marts); analytical tools for data processing (OLAP and data mining); and the data visual representation component, using data visualization techniques.

According to Combita Niño et al. [43] it is important that a BI architecture has a correct interaction between the four components described below: (i) system source: data collection; (ii) data acquisition: the process of ETL of data in a single repository; (iii) data warehouse: repository where the information gathered by ETL is stored; (iv) reporting and analysis tools: allow information to be manipulated and analyzed, from standard reports, ad hoc reports, dashboards, dynamic process analysis, and statistical or predictive analysis.

According to Srivastava et al. [53], a BI architecture has six stages: (i) metadata: the technical and business process for storing data in a data layer repository; (ii) data source: internal and external data sources; (iii) data management: the data manipulation process, which must include a data plan and consistent methods for accessing, organizing, managing, maintaining and planning data for analysis; (iv) data storage: data can be stored in databases; (v) data analysis: data are sent to the user for analysis; (vi) BI tools and their resources: BI tools are integrated with analytical performance to examine data from BI resources in order to achieve the desired results, such as reports, graphs, dashboards, or other ad hoc analyses.

2.3.2. Types of Architecture

Enterprise architecture (EA) is considered to be one of the main tools used by organizations to align their strategic objectives with their technological capabilities. However, EA management practices have not yet been widely used in HEIs. In recent years, reference architectures and reference models have emerged, consisting of abstract artifacts suitable for increasing the quality of EA practices and designed architectures [2].

Data presentation includes different perspectives such as reports and interactive data search, alerts, and graphical user interfaces or dashboards. In general, for Sorour et al. [10] and Boulila et al. [47], a BI architecture in HEIs comprises three main layers or components: (i) data source layer: in which data are collected from different sources; (ii) ETL process layer: in which relevant data are collected for analysis and then loaded into a data warehouse, which stores the data for analysis; (iii) data presentation layer: dashboards are used to present analysis in a summarized form, although they can be detailed to better assist decision making when reviewing objectives and monitoring HEI KPIs.

Through data mining and OLAP techniques, BI takes on the role of interfacing with these data, allowing it to be queried and information reports to be created. In other words, by integrating analytical applications with BI, the desired visualizations are provided. According to Calitz et al. [6], BI actions should focus on developing a single organizational vision, creating the desired BI infrastructure and digital transformation. It should also be noted that the operational data of HEIs is consolidated and integrated as organizational metadata, and then transferred to the data warehouse. A data warehouse can, therefore, be said to comprise one or more data marts. Data marts contain related data and make it possible to identify a particular piece of data from a large body of information [40,53]. However, BI systems are not always equipped with a data warehouse [40]. However, due to the large amount of data generated and manipulated by HEIs, the existence of a data warehouse is essential. The integration of a data warehouse with the BI system makes it possible to achieve greater flexibility in the processing and availability of data [46]. In addition, for BI to be developed effectively, it must comprise only a single reliable data source [43].

2.3.3. Visualization Tools

Visualization tools, in which dashboards stand out, consist of advanced resources that allow users to access automated results and instant visualizations, which meet the established information requirements [6]. Despite this, the use of dashboards in BI systems in HEIs has been little investigated, making it important to understand their adoption with a view to becoming an important resource for decision-making processes [54].

The processing and transfer of information flows, as well as the resulting accumulation of information, means that it is important to improve the selection and visualization processes in order to guarantee more concise information and to identify relationships implicit in the information. The simultaneous use of current and accumulated data is also essential for analyzing changes, trends, and possible modifications to the metrics used. In this sense, dashboards are the appropriate option, as they provide access to various sources of information or represent different aspects of a single data source [36]. In this way, the data presentation and monitoring layer can be considered to consist of dashboards and scorecards, which enable better management of HEI KPIs, providing decision-makers with crucial strategic information in real time, and allowing trends, patterns, and irregularities to be identified [3]. Through these data, which can be considered dynamic and interactive, a single view of the information can be achieved [36] making it possible to ascertain compliance with the long-term goals identified in the strategic planning of HEIs [3].

Given that BI architectures produce a large volume of data, it is necessary to develop data visualization techniques so that decision-makers can better analyze and understand the data [8]. These data visualization techniques use dashboards to present the data and respective analyses through reports and graphical representations [10], resulting in a very useful tool for decision-makers, as it is easier to draw conclusions and identify patterns in a visual representation than in a textual representation [8]. On the other hand, dashboards allow users to present data in different ways, from graphs, tables, widgets, maps, and ad hoc reports, so that decision-makers can flexibly and dynamically monitor KPIs [38,55].

Although HEIs still provide basic descriptive statistics [56], dashboards are considered a mandatory BI feature [36] and can be used in multiple ways by HEIs, including by teachers to provide information related to teaching [45].

3. Research Method

Snyder [57] states that a literature review is essential to keep up with the latest research and assess the collective evidence in a given area of research. In this research, an SRL was carried out using the methodology of Kitchenham and Charters [12]. The study was developed taking into account the following three phases: planning, execution, and analysis of the results. In the planning phase, a protocol is defined which specifies the research questions, keywords, inclusion, and exclusion criteria for primary studies and other topics of interest. In the execution phase, the literature search is carried out according to the defined protocol, and it is in this phase that the inclusion and exclusion of primary studies is carried out. Finally, data extraction is carried out in the results analysis phase, and the results are compared.

3.1. Research Questions

This SLR aims to summarize, clarify, and examine the implementation of a BI system for decision support in HEIs, based on a roadmap that can serve as a benchmark for HEIs, from January 2017 to May 2023 inclusive.

Table 1. Research questions.

RQ#	Research Questions	Motivation
RQ1	What are the main decision-making processes at HEIs?	Identify the main processes that underpin decision making at HEIs
RQ2	What BI solutions are used in HEIs?	Identify the BI solutions used in HEIs
RQ3	What are the reference architectures used to implement BI in HEIs?	Identify the reference architectures used to implement BI in HEIs

presents the research questions (RQ) addressed in this review process.

3.2. Search Process

The digital portals used to select relevant studies were Scopus and Web of Science, where searches were conducted by combining various keywords (e.g., “business intelligence”, “decision making”, “architecture”, “dashboards”, “HEI”) with Boolean operators (AND, OR). This approach aimed to identify an appropriate number of existing studies, considering resource limitations, that are representative of qualitative analysis. The search strategy was specifically adapted for each database to ensure the inclusion of a broad spectrum of relevant studies.

All papers published from January 2017 to May 2023 inclusive were considered, reflecting our commitment to incorporating current research pertinent to the evolving technological and educational context. The search query used for the research was, therefore, as follows:

(“business intelligence” OR “business analytics” OR “decision making”) AND (roadmap OR architecture OR dashboards OR system) AND (“higher education” OR “higher education institution” OR “HEI”).

3.3. Inclusion and Exclusion Criteria

The research questions formulated were used as guidelines throughout this review process. In addition, inclusion and exclusion criteria are defined in order to identify relevant studies for the research, which are presented in

Table 2. Inclusion and exclusion criteria.

Inclusion	Criteria
IC1	Recent studies, particularly those published since 2017, which have already been approved by the scientific community.
IC2 IC3 IC4	Studies written in English. Studies that use or refer to the implementation of BI systems for decision support in HEIs. Primary studies.
Exclusion	Criteria
EC1 EC2 EC3 EC4 EC5 EC6	Studies that do not use BI systems for decision support in HEIs. Studies that are not written in English. Studies that do not provide the full paper. Studies without an abstract. Studies published before 2017. Books, letters, notes, and patents are not included in the review.

.

Figure 1 illustrates the flow diagram of the primary study selection and analysis process. Using the inclusion and exclusion criteria detailed in Table 2, studies were filtered in several stages. Initially, the search yielded a total of 705 papers, 331 from Web of Science, 346 from Scopus, and 28 from other sources. After a preliminary review of the titles and abstracts, we identified 156 duplicate papers, reducing the selection to 549. Of these, 455 were discarded because they did not fully meet the established criteria, leaving out the least relevant studies. A more detailed evaluation of the full text of the remaining 95 papers led to the exclusion of a further 84, resulting in the final selection of 10 relevant studies. Zotero was used to organize and manage bibliographic references throughout the review.

3.4. Quality Assessment Rules

Quality assessment rules (QARs) are important to guarantee and assess the quality of primary studies and are carried out in the final stage to define the studies that will be added for review. Thus, four QARs were assigned, as shown in

Table 3. Quality assessment questions.

QAR#	Quality Questions
QAR1	Are the objectives of the study clearly defined?
QAR2	Are the methods or techniques reported objectively?
QAR3	Is the application of a BI system for decision support in HEIs clearly defined?
QAR4	Are there practical applications described in detail?

. Each criterion can be given a score from 1 to 10. Studies were only accepted if they obtained a score of 5 or more; otherwise, the study was rejected.

3.5. Data Collection and Data Analysis

Data extracted from the selected studies included author(s), year of publication, research context, methods used, key findings, and recommendations. We used a narrative synthesis approach to synthesize and discuss the evidence, identifying common themes, trends, and gaps in the literature.

4. Results and Discussions

After completing all the SLR steps, Table 4 shows the 10 primary studies selected for detailed analysis. This section presents a synthesis of the main findings of these studies, according to the objectives defined for the systematic review.

4.1. Description of Primary Studies

Table 4 shows the information on the primary studies, such as the assignment of an ID to each study, title, year of publication, and the respective reference.

A1: Using the DSR methodology proposed by Peffers et al. [58], Menolli et al. [59] present a BI-based methodology for analyzing IS courses, in particular the phenomenon of school dropouts, from different perspectives. Public data were used, which, after being analyzed and understood using BI tools, made it possible to create a dimensional model.

Table 4. Selected primary studies.

ID	Title	Year	Refs.
A1	“BI-based Methodology for Analyzing Higher Education: A Case Study of Dropout Phenomenon in Information Systems Courses”	2020	[59]
A2	“Business Intelligence Governance Framework in a University: Universidad de la Costa Case Study”	2020	[43]
A3	“Business Intelligence Usage Model for Higher Education Institutions”	2021	[60]
A4	“Comparative Frameworks for Monitoring Quality Assurance in Higher Education Institutions using Business Intelligence”	2020	[61]
A5	“Application of Business Intelligence in the Quality Management of Higher Education Institutions”	2018	[62]
A6	“Role of Business Intelligence Systems in Croatian Higher Education Quality Assurance”	2020	[63]
A7	“Implementing a Business Information System to Improve the Quality Assurance Mechanisms in a Portuguese Higher Education Institution”	2019	[25]
A8	“Analytical Data Mart for the Monitoring of University Accreditation Indicators”	2019	[64]
A9	“A Business Intelligence Framework for Sultan Qaboos University: A Case Study in the Middle East”	2017	[65]
A10	“A Big Data Architecture for Learning Analytics in Higher Education”	2017	[66]

Table 4. Selected primary studies.

ID	Title	Year	Refs.
A1	“BI-based Methodology for Analyzing Higher Education: A Case Study of Dropout Phenomenon in Information Systems Courses”	2020	[59]
A2	“Business Intelligence Governance Framework in a University: Universidad de la Costa Case Study”	2020	[43]
A3	“Business Intelligence Usage Model for Higher Education Institutions”	2021	[60]
A4	“Comparative Frameworks for Monitoring Quality Assurance in Higher Education Institutions using Business Intelligence”	2020	[61]
A5	“Application of Business Intelligence in the Quality Management of Higher Education Institutions”	2018	[62]
A6	“Role of Business Intelligence Systems in Croatian Higher Education Quality Assurance”	2020	[63]
A7	“Implementing a Business Information System to Improve the Quality Assurance Mechanisms in a Portuguese Higher Education Institution”	2019	[25]
A8	“Analytical Data Mart for the Monitoring of University Accreditation Indicators”	2019	[64]
A9	“A Business Intelligence Framework for Sultan Qaboos University: A Case Study in the Middle East”	2017	[65]
A10	“A Big Data Architecture for Learning Analytics in Higher Education”	2017	[66]

A2: Combita Niño et al. [43] propose a BI governance framework for HEIs, focusing on data analysis for strategy and forecasting. By assessing the analytical maturity of an HEI, they developed a model that encompasses everything from organizational culture to governance, without specifically detailing BI solutions or architectures. This study highlights the importance of governance for the success of BI projects in higher education.

A3: Musa et al. [60] suggest nine factors to determine the success of implementing a BI system in HEIs, organized into three dimensions: organizational, procedural, and technological. This study implemented a questionnaire research method, which collected data, BI users, and descriptive statistics to validate the data. The results obtained demonstrate the possibility for HEIs to obtain important patterns and forecasts in the formulation of strategies, knowledge creation, and decision-making processes.

A4: There are various frameworks for monitoring quality assurance in HEIs. Sorour et al. [61] identify five frameworks, which have different orientations and perspectives, although all of them support the use of data to measure the performance of HEIs, and there is consensus that BI tools, such as dashboards, can be useful for providing real-time information on the performance and quality control of HEIs.

A5: Pérez-Pérez et al. [62] demonstrate the application of BI in HEIs, focusing on administrative and academic management processes. The study relates BI to its BSC, data mining, and data warehouse tools. The results show that it was possible to establish a relationship between the tool used and the type of process, identifying three basic categories: administrative management, educational quality assurance, and academic performance.

A6: HEIs use different IS to record their activities, although these IS do not have the capacity to use the data generated to improve their organizational processes and decision making. Brecic [63] provides examples of IS that fill this gap and lead to improvements in the quality system of HEIs, obtained by analyzing the requirements and obligations that HEIs have in relation to collecting, processing, and analyzing data as part of their operations.

A7: Morais and Castro Lopes [25] describe the implementation of a BI solution in a Portuguese HEI, which aims to support its quality system and improve its future strategy, based on the area of teaching activity. The project includes several stages: mission, strategy, and process analysis, identification of KPIs, validation of KPIs by process managers, identification of the IS in use at the HEI, identification of the existence of the necessary information in these systems, definition of access profiles according to the different users of the process, and the selection and implementation of the BI system.

A8: Higher education evaluation and accreditation agencies develop evaluation processes to accredit HEIs. HEIs, therefore, need tools that enable effective academic analysis, which requires a systematic and balanced process for collecting, synthesizing, and evaluating relevant data. In order to support the accreditation process of HEIs, Ortiz and Hallo [64] present an analytical data mart.

A9: Al Rashdi and Nair [65] show a BI framework implemented in an HEI in order to collect useful information from the big data generated by the HEI. The framework was tested for the key activity: teaching and learning, and the results show that the aggregation of these activities and KPIs contributes to the overall performance of the HEI, even allowing for a better perception of the HEI’s operation.

A10: According to Matsebula and Mnkandla [66], traditional processing and analysis of structured and unstructured data, using relational database management and data storage systems, does not take advantage of the potential of the big data produced by HEIs. The lack of suitable architectures for exploiting big data in HEIs has even led to many failures to produce meaningful, accessible, and timely information for decision making. Using the DSR methodology, the study integrates various analytical frameworks to develop a more comprehensive big data architecture for learning analytics.

4.2. RQ1: What Are the Main Processes That Underpin Decision Making at HEIs?

The processes identified in the selected primary studies are shown in

Table 5. Summary of each study where the decision-making processes and KPIs, BI solution types, and BI architecture references were identified.

ID	Decision-Making Processes	BI Solution Type	BI Architecture Reference
A1	Analysis of school dropouts	Pentaho	Pentaho
A2	-	-	-
A3	-	-	-
A4	Monitoring quality assurance	-	-
A5	Academic management Administrative management	-	-
A6	Internal quality assurance and the social role of HEIs Study programs Teaching process and student support Pedagogical and institutional capacities Scientific/artistic activity	-	-
A7	Design and development Application management Enrollment management Education management Skills and career management	Qlik Sense	-
A8	Accreditation	Pentaho	Pentaho
A9	Teaching and learning Research and consulting Community service Resources and facilities	Power BI	Microsoft Business Intelligence Solution
A10	Learning analysis	-	-

. We found that most of the processes that support decision making in HEIs fall within the teaching and learning dimension.

In Primary Study A1, various descriptive statistical analyses are carried out on the phenomenon of school dropout. The results show differences in dropout rates in the dimensions of race, type of city, location, of course, type of institution, type of teaching, year of entry into the course, and period of the course.

Primary Study A5 refers to administrative and academic management processes, identifying three categories: administrative management, educational quality assurance, and academic performance.

In Primary Study A7, five processes were selected that support the teaching and learning dimension: design and development, application management, enrollment management, teaching management, and skills and careers management, and a set of KPIs were identified and associated with the respective processes.

In Primary Study A8, the accreditation process was selected in order to obtain adequate information for its rapid interpretation and management, as well as to avoid dispersing the data required for the respective accreditation.

In Primary Study A9, the teaching and learning dimension was selected, with the following processes: (1) teaching and learning; (2) research and consultancy; (3) community service; (4) resources and facilities. Under this dimension, there are 15 KPIs, with different algorithms to calculate them.

In Primary Study A10, the main objective of learning analytics is to improve learning outcomes and the overall learning process in virtual e-learning classrooms, as well as in computer-supported teaching.

4.3. RQ2: What BI Solutions Do HEIs Use?

Table 5 shows the BI solutions used in the selected primary studies. Three BI systems were mentioned in the studies analyzed, but only two primary studies use the same solution, in this case, Pentaho. These systems are mainly used for visual analysis and interactive dashboards, making it easier for education managers to access actionable insights.

In Primary Study A1, a data warehouse was created, where the original data were extracted, transformed, and loaded, and from this, a BI solution was deployed and configured, using Pentaho Business Analytics.

In Primary Study A2, a BI governance framework was designed for an HEI, consisting of four fundamental layers: the strategic layer, communication layer, process layer, and operation layer.

In Primary Study A5, the results related to the BSC showed that it focused on quality assurance, data mining focused on academic performance and data warehouses acted transversally to support information analysis.

In Primary Study A7, Qlik Sense was selected based on two factors: Gartner’s magic quadrant for BI solutions and the knowledge of the HEI’s IS development team.

In Primary Study A8, Pentaho tools were selected, in addition to PostgreSQL as the database engine. The solution enables efficient monitoring of indicators prior to HEI accreditation, reducing response time and resources in the reporting process.

In Primary Study A9, the solution consists of three main elements: an analysis tool using Microsoft Excel 2013 with Microsoft Power BI and a web interface that represents the visualization of the dashboards.

4.4. RQ3: What Are the Reference Architectures Used to Implement BI in HEIs?

This section describes the reference architectures used in the implementation of BI identified in the selected primary studies, which are shown in Table 5. The design of BI architecture at HEIs was diverse, describing customized approaches that integrate on-premise data solutions with cloud computing tools. As in the previous section, only two primary studies use the same architecture, Pentaho. Integrating data from different sources, including learning management systems and academic records, is a common challenge, overcome through the adoption of ETLs and data warehouses.

In the A1 primary study, a software architecture was established using the Pentaho Suite, and the following analytical tools were used: Mondrian, ETL: Kettle, Postgres was used for storage, while OLAP was used for web-based visualization.

In Primary Study A8, the solution was developed using the Kimball methodology and an open-source BI tool, the Pentaho Suite. The ETL processes were carried out using the Pentaho Data Integration tool, also known as Kettle.

In Primary Study A9, Microsoft’s BI architecture was used as a reference for the BI solution and is divided into three levels: (1) data level: this level is based on Microsoft’s database server (SQL server) and consists of four main elements; (2) Microsoft SharePoint provides the main content management and search, and finally, (3) the end-user reporting tools, such as Microsoft Excel and the Performance Point Dashboard.

Primary Study A10 proposes an architecture for a learning analytics framework based on big data, which has the ability to collect, pre-process, clean, analyze, and visually represent the results of the analysis of any application involving large volumes of data.

Through this study based on an SLR, it was possible to verify that HEIs use BI for various purposes. Peng et al. [67] highlight the benefits that BI confers on HEIs in terms of supporting their decision-making management, as it is clear that BI is useful for analyzing and extracting data on student learning and institutional functioning. Apraxine and Stylianou [49] and Pérez-Pérez et al. [62] emphasize that BI can improve decision-making processes in various HEI departments, such as admissions, academics, and management. On the other hand, Calitz et al. [6] demonstrate that BI is an essential component of an information framework on HEI sustainability and that HEIs should, therefore, invest in technological tools, including BI, to provide information in understandable formats that can be used by management and relevant stakeholders. Sorour et al. [10] propose a BI solution for monitoring quality assurance activities in HEIs, while Abduldaem and Gravell [54] state that the successful adoption of a BI solution improves performance and decision-making processes in HEIs. Richards et al. [68] emphasize that the effective implementation of BI is strongly related to planning and measurement, which are important elements of decision making in HEIs.

BI can be adopted in various departments of an HEI, leading to better decision-making processes and supporting strategic objectives. In order to be able to implement a BI system, it is essential to design a plan consisting of a roadmap, an architecture, and some guidelines [69]. In this sense, the architecture serves as a guideline for the development of the roadmap itself [70]. The roadmap can be used as a guide for the process of developing a strategy [69], in which case it is responsible for presenting the processes needed to implement a BI system in HEIs [71]. On the other hand, a roadmap is an established concept in knowledge management, aimed at gathering knowledge and finding solutions to problems in a structured way [72]. It is, therefore, essential to identify objectives, criteria, and alternatives in order to define a roadmap that is appropriate to the reality of an HEI.

Although there have been some studies on BI in HEIs, BI has progressed along two different paths, theoretical and practical. Most studies on BI point towards describing the advantages of using it, while there is little research on implementing BI [73]. Ain et al. [38] reinforce this position by noting in their research that previous studies do not comprehensively discuss the issues and challenges related to the adoption, use, and success of BI systems. In fact, in our research, we were unable to find any reference to the use of a roadmap for implementing BI systems in HEIs. Despite this, it is clear that the development of a BI system consists of a progressive process, in which it is possible to identify the various stages that should make up a roadmap, with a view to using BI to support the decision-making process at HEIs [74]. Morais and Castro Lopes [25] state that the process of implementing a BI solution in an HEI consists of the following phases: (i) analysis of the business processes, as well as the HEI’s mission and strategy; (ii) analysis of the pre-existing indicators for each process; (iii) review of the literature to identify and define the KPIs; (iv) validation of the KPIs by the employees responsible for each process and consequent adjustments to the list of KPIs resulting from the validation; (v) identification of the IS implemented at the HEI; (vi) analysis of the IS to identify the sources of information needed for the KPIs; (vii) definition of the users’ access profile; (viii) selection and implementation of the BI technological solution.

We can see that it is, therefore, essential to take some steps before implementing a BI system. First, it is essential to assess whether the HEI is prepared from a technical point of view, as well as from a management point of view, and it is essential to have clear support from its administration. For Jahantigh et al. [73], there are two key factors in determining the quality of BI support management: the familiarity of those responsible with IT and the relevance of reliable information. In addition, the compatibility of the systems with the BI application must be ensured [40] and the BI solution must be able to manage technological assets, people, and processes [75]. Before a BI architecture can be implemented, all the data stored by HEIs, from their most diverse sources, must be integrated into a data warehouse by applying the ETL process.

It is, therefore, essential to identify all the data sources to be analyzed, and only afterwards is the data selected to ensure that the BI process delivers reliable results. This process involves a data dictionary, which consists of a technical description for one of the HEI’s repositories, in which the data fields, origin, availability, and responsible party are recorded. Using the information gathered about the sources and the data dictionary, it will be possible to identify the variables that will guide the operation of the BI system [46]. Given the quantity and diversity of these data, it is necessary to extract the related data before proceeding with the execution of the respective query. Once the data warehouse has been developed, different servers can efficiently access its data via front-end applications [76].

Finally, the BI solution must be able to present the data in an appropriate way and at an appropriate time, for which dashboards should be used, and their integration is ensured through the data warehouse which has quality data. Another option is to integrate the data presentation applications with the data mining tool [46]. In addition to the data warehouse, there is a diverse range of tools and techniques that can support HEIs in developing BI capabilities, such as enterprise resource planning (ERP) systems, document management systems, and knowledge repositories, among others. A generic BI system must integrate data from different sources and then transfer it. The data considered are grouped into data marts, from which the data are accessed through applications that allow customized views to be created [6].

5. Conclusions and Future Directions

This article presents an SLR with the aim of characterizing the implementation of a BI system in HEIs, with a view to supporting decision making in their various areas and levels.

Firstly, an overview of decision making in HEIs, BI systems in HEIs, and BI system architecture was presented. Secondly, the research method adopted is described, following a systematic series of steps that analyze the quality of the primary studies. Thirdly, the results are demonstrated, which provide answers to the research questions of the systematic review, and finally, these results are discussed.

Eighteen different processes were identified from the primary studies selected. Only quality assurance and teaching and learning processes were mentioned twice. It is, therefore, possible to define a set of processes to support decision making in HEIs.

With regard to BI solutions, only four selected primary studies present their use, and only Pentaho is mentioned in two of them.

As far as BI reference architectures are concerned, only three selected primary studies present their use, and only Pentaho is mentioned in two of them.

In summary, eight of the primary studies selected identify decision-making processes in HEIs, while only four of the studies implement a BI solution in HEIs, and only three use a reference architecture.

With this systematic review, it was possible to obtain an overview of the main strategic objectives underpinning the decision making, activities, processes, and information of HEIs, to gain an insight into the use of the data and information that HEIs produce, as well as their use in obtaining knowledge and intelligence to support decision-making processes, with an emphasis on identifying the characteristic processes of HEIs, as well as their respective KPIs. The types of BI solutions used in HEIs, the processes they support, and the information they hold were also identified. Finally, some reference architectures for BI systems were identified.

Our research shows that it is important to note that HEIs must seek the right balance between standardization and personalization of BI and that it is important to establish a common set of practices and processes that enable the collection, organization, and storage of relevant information, which facilitates the integration of data from different sources and systems, in order to provide a comprehensive and accurate view, for which the possibility of personalizing the creation and presentation of insights is essential [6]. Overall, we can see that the adoption of BI in HEIs is a complex process that requires careful consideration of several factors, including organizational culture, top management support, and IT infrastructure. In addition to these factors, other aspects to be considered include the quality of the data available, data governance, information security, and the training of users to interpret and use the insights generated by BI. In summary, we can say that a proper approach to these aspects allows HEIs to maximize the potential of BI to gain valuable insights, make informed decisions, and improve institutional performance.

It is important to include artificial intelligence (AI) in the proposed artifact in the future. AI allows data to be analyzed, patterns to be identified, and actionable insights to be provided, and it can thus provide a number of significant benefits for improving the quality of HEIs’ decision making. However, it will be crucial to ensure ethics and data privacy when implementing AI solutions.

The literature review allows us to conclude that there is a need to define a roadmap, which can serve as a reference for implementing a BI system in HEIs, in order to facilitate data processing and the detection of trends and patterns, and thus obtain an adequate visual representation that allows HEIs to make decisions based on concrete data.