Critical Success Factors Evaluation by Multi-Criteria Decision-Making: A Strategic Information System Planning and Strategy-As-Practice Perspective

Abstract

Strategic information system planning (SISP) is a central process that enables organizations to identify the strategic alignment of their IT portfolio to achieve their business needs and objectives. The extant SISP literature has focused on theoretical and processual aspects and has left methodological ambiguity about how SISP is practiced. This paper contributes to the current knowledge by providing a mixed-methods SISP framework labeled CSF-MCDM for company-wide strategic alignment. The paper conducts a methodological synthesis, embracing an expert-based qualitative approach based on a PEST-SWOT and causal layered analysis to draw the critical success factors of a next-generation business system for an automotive company in South Korea. The derived CSF dimensions and sub-criteria are evaluated by the multi-criteria decision-making model, engaging a strategy-as-practice lens to SISP to enable an integrative analysis of IS strategy formulation, planning, and implementation. The findings reveal the relative strategic priorities of dimensions, the following core activities, and the global priorities for resource distribution planning for IS strategy of the firm. This paper argues that bringing replicability with SISP and diversifying methodological approaches within the organization is substantial. This paper also suggests that future researchers validate the suggested framework for scientific replicability and expand the SISP research stream within the entire IS/IT ecosystem.

1. Introduction

A firm’s information systems (IS) significantly affect business results in the modern management environment. The information systems substantially shape the way the organizations work, and their performance is embodied and realized through those systems [1]. The rapid penetration of information technology has resulted in considerable attention on digitized data and business intelligence [2], and the enabled digital connectivity has changed how businesses and people work, expediting the proliferation of business information systems to increase efficiency, value, and innovation opportunities [3,4]. Therefore, modern businesses actively aim at improving their performance through adequate information systems [1] as an innovative and core means to secure competitive advantage [5,6,7], and an undoubtable de facto element of success [8].

However, contrary to the expectations, the introduction of all information systems does not necessarily lead to desired performance. The information systems consist of infrastructure, data, applications, and, most importantly, the people who embrace the technology services within the organization [9,10]. IT assets, such as a newly adopted mission-critical system whose usage is often mandated, may lead to dissatisfactory results if the system design is not aligned with the strategic direction or fails to meet the requirements of its users [11,12,13,14]. Furthermore, businesses undergo unprecedented technological changes with their information systems under COVID-19 [15]. Business systems face new technological challenges, such as infrastructure that enables remote work, virtual meetings, contactless commerce, privacy protection, cybersecurity, data analytics, and data-driven decision-making processes [16]. Therefore, it becomes imperative for firms to entrench the new system in the desired organizational practices and processes, while achieving continued system usage from the employees [14,17,18] in the “New Normal,” the post-COVID-19 era.

Strategic information system planning (SISP) becomes central for any business when an organization faces an inflection point concerning its information system. Overall, SISP is a process through which an organization identifies its IT applications portfolio to achieve its organizational objectives and to help execute its business plans [7]. Changing environments enforce organizations to entail significant investments from their revenue and R&D budgets to develop strategic information systems [10,19,20]. Estimating the effectiveness of the investments has been the primary purpose of strategic planning for IS/IT decision-makers [21,22], and the proliferation of new IT technologies since the 1990s has further strengthened the value and contribution of the SISP practice in organizations [20,23]. IS long-term planning that is well-aligned with business strategy has been one of the top management concerns for decades [24].

Moreover, desired performance generated from the information systems in which all the resources are shared and interconnected with the users is crucial for firms [20]. Hence, today’s organizations include SISP as an essential process to improve their performance in designing the elements in information system development, seeking the best effectiveness and efficiency available [7]. As a result, it is understood that aligning the firm’s strategy with the core business system based on a SISP perspective becomes critical for organizations.

A firm’s management can consider various methodological approaches to identify and evaluate the priorities in developing an information system, and one of the vital issues in the SISP is to choose the best-suited method for the stakeholders [25]. In exploring the business needs for IS, various analytical frameworks and techniques may help managers find insight into maximizing organizational effectiveness. The critical success factors (CSFs) approach [26] is widely utilized in implementing the SISP process. Rockart [26] defined CSFs as the crucial areas of business activities that require constant and careful attention from top management. Identifying the CSFs is substantial to businesses [27,28]; the approach has been broadly accepted in the IS literature for a long time [29,30], and is believed to be a valid research methodology to make sense of finding latent elements for competitive advantage [27,31,32].

After analyzing the organizational goals and objectives and the stakeholders having extracted the key CSFs, prioritizing those dimensions would be an issue of significance [31]. Then, a multi-criteria decision-making (MCDM) model can be considered to evaluate the factors drawn [33]. MCDM has helped to overcome the choice problem in various research fields [33,34]. MCDM methodologies support the decision-makers in resolving problems in situations where multiple conflicting criteria exist that need coordination. It is of practical value, capable of being utilized under certain or uncertain situations, and enables stakeholders to scientifically make critical decisions, in line with quantitative and qualitative analyses [34]. Among the various MCDM methods available, AHP [35] has been chosen as a viable technique, proving its compatibility with other methods [33]. In a general sense, a synthesis of methodologies can help researchers to overcome the limitations of a single method and enable a better understanding of a phenomenon, e.g., [25,36,37,38].

Although the extant literature has dealt with SISP as a research topic, most have focused on theorizing SISP in an academic sense and focused on the literature research. There is an information asymmetry in the IS literature on how SISP can occur in reality [39,40], leaving methodological ambiguity. Despite the large pool of literature dealing with SISP, many studies are concerned with its general processual characteristics, i.e., [41], leaving the detailed procedures of how SISP is practiced out of focus, loosening the links between the SISP process and underexplored macro-level contexts. Moreover, there is a strong need for IS researchers to consider connecting SISP with largely uncertain societal factors [42]. Therefore, this study embraces a strategy-as-practice (SAP) method, e.g., [43,44,45], as a theoretical lens to explore SISP through an empirical case study. SAP allows scholars to shift their strategic focus from a mere concentration on effects to organizational performance, while enabling a more comprehensive, in-depth analysis of the real-world details of strategy formulation, planning, and implementation. Ultimately, this study seeks to find a theoretical contribution to the existing literature, providing a practical and comprehensive case of SISP and a novel framework labeled CSF-MCDM. Relatively few academic efforts have been made to present the managerial benefits of a SISP based on the integrative and practical framework as presented in this paper. With this gap in mind, to theoretically contribute to the current knowledge, the primary objective of this study is twofold. First, the study aims to discover the CSFs in SISP practices that will enhance the suitability and effectiveness of the business core system required in the post-COVID-19 era. Second, the study attempts to encapsulate the identified CSFs with a novel framework based on an MCDM model that would help businesses to acquire strategic alignment within the internal needs and existing resources to respond to the rapidly changing business environment, sustaining successful business results stemming from the newly developed information systems.

To address the theoretical gap in the body of the literature and to fulfill the research aim, this study proposes a novel mixed-method—a qualitative dimension and criteria development further enhanced by a quantitative MCDM approach—to enrich and make the existing SISP concept more applicable. Therefore, the study conducts an expert-based qualitative approach, including a comprehensive PEST and SWOT analysis and causal layered analysis (CLA) [37,46,47] to draw CSFs of a next-generation mission-critical system for an automotive company in South Korea. Next, the identified CSFs are evaluated by an MCDM method to present the prioritized relative weights and significance among dimensions and criteria for strategic decision-making.

Ultimately, the study presents the following research questions; (a) What are the corporate mission-critical system’s strategic direction and CSFs in the post-COVID-19 era? (b) What are the relative and weighted priorities of strategically derived factors based on the CSF-MCDM framework? The rest of this paper is organized as follows. First, the theoretical background required for the research framework development is discussed in Section 2. Section 3 deals with research context and methodology for academic replicability. Section 4 presents the qualitative and quantitative results, while Section 5 summarizes the findings. Lastly, the theoretical and practical implications are discussed in Section 6 and Section 7 presents the limitations and suggestions for future researchers.

2. Literature Review

2.1. Strategic Information System Planning (SISP)

Strategic information system planning (SISP) has been a vital concept and interest for information systems managers since the 1980s [20], and the advent of new technologies such as internet-based computing further promoted IS/IT strategic planning in the 1990s, expediting the value of SISP practice [20,23]. Many scholars and the relevant literature have proposed various definitions of SISP. However, it can be generally defined as a process of identifying a computer-based portfolio/applications aligned with a firm’s strategy, which ultimately create a competitive advantage or help the organizations to perform their business by realizing their objectives, e.g., [6,7,48,49,50,51].

The concept of SISP arose with the unavoidable investment pressure to develop strategic information systems [10,19,20], with requirements for the evaluation of the investments becoming the primary drivers of strategic planning for IS/IT assets [21,22,52]. Therefore, SISP has gained considerable recognition and acceptance as an essential management practice and process for improving organizational performance in various fields [7]. Previous scholars stated a that focus and emphasis on SISP could help organizations to enhance their performance, productivity, efficiency, and effectiveness [7,53,54,55].

A large body of the literature has dealt with SISP, providing slightly different definitions from each author. However, some substantial elements are commonly found at its core throughout the research stream, highlighting SISP’s significance. SISP is an integrative and continuous planning activity, a review [20,56,57,58], or an analytical, evaluating exercise [59], which integrates technological elements [56,60], such as a computer-based portfolio/applications [5,6,7,48,49]. SISP is strategic thinking, planning, or deciding a direction for desirable information management and policies [61,62,63,64], which aligns, supports, and influences the business strategy for competitive advantage [55,56,57,60,65], benefiting organizations with superior IS/IT evaluation [25]. From the above, it can be understood that the following common elements are crucial in the SISP process; (a) identification of IS/IT elements, (b) alignment with strategy, (c) decision, review, and process for long-term planning, and (d) being based on the business’ needs and requirements. SISP’s definitions in the literature are presented as follows (see

Table 1. SISP definitions in the literature.

No.	Description	Author
1	An integrative process that includes a firm’s various strategies such as IT, information management, change management, and human resources	[56]
2	A continuous planning activity, ensuring the implementation of information and communication technology (ICT) in an organization, aligning to business strategies, improving organizational process effectiveness, creating business opportunities, and contributing to an organizational competitiveness	[57]
3	A way of supporting and influencing a firm’s strategic direction that identifies value-adding information systems and integrates organizational technologies through holistic information architecture development for successful systems applications	[60]
4	An analysis or an exercise of the corporate process using the business information models with the evaluation regarding risk, needs, and organizational requirements, enabling organizations to develop IS development priorities	[59]
5	A process of deciding the direction for development and policies regarding the organization’s information use, management, and networking technologies	[62]
6	A continuous review of the need to prepare, acquire, transfer, store, retrieve, access, present, and manipulate information in all forms	[58]
7	A strategic thinking process or a mechanism that identifies the most desirable IS development through which a firm implements its long-term IT activities and policies, aligning the evolving organizational needs and strategies	[50,63,64]
8	A process that helps to develop the information systems aligned with the organization’s strategic planning, including objectives and policies	[65]
9	A process to create IS deployment plans to fulfill a firm’s strategic objectives	[55]
10	A process of identifying a computer-based portfolio/applications aligned with corporate strategy, which is capable of creating a competitive advantage or helping organizations to execute their business, realizing their business goals	[5,6,7,48,49]

Source: Author’s elaboration.

).

The existing SISP literature pool has proposed numerous methodologies to help organizations with strategic plans for information systems [20]. Often, the typical SISP process engages the following five stages; (a) strategic business planning, (b) setting the information systems’ mission and vision, (c) current information system assessment, (d) resource guidelines for the new information system, and (e) long-term strategic proposal [51]. The overall planning process should ensure that technology-related elements are well-aligned with the organization’s needs and strategy [25]. The success of SISP depends on the developers’ ability to ensure a proper alignment among the relevant components [51]. Moreover, as a critical part, the SISP process should define the planning objectives and environmental analysis that connects to the new strategy for the business systems [25,66,67].

2.2. Critical Success Factor (CSF)

Identifying critical success factors is substantial for businesses [27,28]. It is required for the top management to identify the performance factors and priorities in the information systems development strategy to stay competitive. Rockart [26] defines the critical success factors (CSFs) as the limited number of activity areas that must receive continuous and persistent attention from management and may ensure the organization’s successful competitive performance when satisfactory. The author argues that CSFs are substantial performance factors that would bring a competitive advantage to firms. Identifying CSFs allows management to determine the direction of the business focus, develop adequate measurements, and decide the scope of the required business data [26]. Daniel [68] first suggested the CSF concept, which was then further developed by Rockart [26], adding its value to business practices. Usually, the CSF is identified by four major factors; industry, environmental, strategic, and temporal factors.

Choosing an appropriate methodology is one of the vital issues for IS project managers before entering the SISP activities [25]. In implementing SISP to develop the information systems, researchers can consider various methodological options, such as the competitive forces model, value chain analysis, or the scenario planning method [25], to provide a long-term and integrative perspective. However, the CSF method is one of the most widely utilized methods for the SISP process. In particular, the CSF approach has been broadly accepted and utilized as a methodology in the IS/IT literature, has proved itself legitimate for a long time [29,30], and remains valid for the sense-making of a problem based in identifying potential factors for business success [27,31,69]. The CSF method has practical value and influence that enables project managers to integrate sustainability exploration into projects [32,70].

Thus, this study considers the CSF approach a legitimate way to induce critical dimensions to be prioritized in the SISP process by the MCDM-based approach. Additionally, it is known that a synthesis of methodologies can help researchers overcome the limitations of a single method, e.g., [25,36,37,38]. Methodological synthesis is also applicable in the SISP process to provide IT managers with information to carefully plan, review, and align information system development with their business strategy. As a result, after drawing the CSFs, this study tries to combine an MCDM method to evaluate the factors to provide more validity and theoretical robustness.

2.3. Multi-Criteria Decision-Making (MCDM)

Multi-criteria decision-making (MCDM) is a line of research methodology that enables decision-makers to resolve complex problems with multiple conflicting criteria that need to be prioritized based on evaluation values, e.g., [33,34]. It is a powerful and practical tool that can be applied to decision situations where both certainties and uncertainties prevail, and can be incorporated with other quantitative and qualitative methods to provide more scientific rigor [34]. Despite the differences in choosing the methodologies, in the recent literature, there have been efforts to embrace the MCDM approach to prioritize CSFs in other domains [71,72,73,74].

In general, a decision-making process based on MCDM engages the following three stages [75,76,77]; (a) structure the decision problem, (b) choose the best MCDM model, and (c) review the final result with prioritization pointing at preferable alternative orders, decided by the weighted scores. The literature has proposed multiple MCDM methods surrounding the complexity in decision problems. AHP (analytic hierarchy process) [78,79], the ELECTRE (Elimination Et Choix Traduisant la Realité) method [80,81] and its variants (e.g., ELECTRE I, II, III, IV, and Tri), PROMETHEE (preference ranking organization method for enrichment evaluation) [82,83] and its variations, TOPSIS (technique for order of preference by similarity to ideal solution) proposed by Hwang and Yoon [84], and the WASPAS (weighted aggregated sum product assessment) method suggested by Zavadskas et al. [85] are all among the prominent ways that are applicable to the solving of decision problems.

However, despite multiple decision-making methods, there is no single best approach applicable to all decision-making situations [34,86]. It is true that regarding its flexibility for applying it as a stand-alone method or as a part of a combined research strategy, e.g., [35,87], AHP has been the most applied in the relevant research stream, e.g., [34,88,89,90]. Furthermore, among various MCDM methods available in the literature, AHP [35,78,79] has been chosen as a principal MCDM technique, proving its compatibility with other methods [33]. Thus, based on prior discussions, this study evaluates the CSFs discovered throughout the SISP process based on a synthesis with an MCDM method, the AHP.

3. Methodology

3.1. Research Context

Company A, a local affiliate of the world’s largest carmaker that sells over 10 million cars [91] and is an automotive distributor based in South Korea, was established in 2000. It handles two brands (luxury and non-luxury) with a 5.4% market share in the passenger car sector [92]. The company partners with 16 dealers while operating 51 showrooms and 56 service facilities (24 showrooms and 25 service centers for the non-luxury brand, and 27 showrooms and 31 service centers for the luxury brand). The company’s nationwide network hires approximately 1,500 employees, accounting for about 6.8% of all automotive retail sector employment [92].

Based on the 3S concept (sales, service, and spare parts), the company has tried to establish an integrative information system to support the nationwide dealer network. Unlike other global brands in the market, the company has developed and maintained its own locally developed mission-critical system (DMS—dealer management system). This system, connecting the global headquarters network and the local dealers, has undergone two full-scale development stages in 2006 and 2009. However, under the pressure of the rapid environmental changes, the company management team decided to develop the next-generation information system. The DMS processes real-time transactions of critical data such as vehicle production, logistics, sales, service, and detailed information of over 20,000 parts, and provides a comprehensive central system that enables the distributor and the dealers to share and utilize the vast customer information database, connecting other sub-systems, e.g., accounting and human resources. As such, the core system becomes one of the most crucial IS/IT assets of the company, providing substantial strategic values. Further, it can be considered appropriate that this study’s research questions aim to identify the critical success factors of the aforementioned corporate information system as part of the SISP perspective and present an analysis framework to derive the strategic priorities with empirical examples. Therefore, this study decides to utilize the case of Company A in the planning process for strategic information system development, assessing the critical success factors based on the analytical framework explained in the next section.

3.2. Evaluation Framework and Analytical Approach

This section presents CSF-MCDM, a generalized framework for evaluating the critical success factors in the strategic information system planning process, and a detailed research approach for replicability. As discussed in the previous Section 2, the SISP process reflects the firm’s strategic business planning. It requires clarification of the information systems’ mission/vision analysis based on the current system assessment to propose a long-term strategic proposal that would lead to the organization’s competitive advantage. The CSF-MCDM framework presented in this study streamlines the overall process to induce the dimensions and their subsequent criteria for CSFs and assess them based on the MCDM’s proposed weighted priorities of items, aligned with the firm’s strategic needs.

Researchers may embrace mixed methods for social sciences [93], and extant studies have adopted a mixed-methods design that integrates quantitative and qualitative methodology in a single study [94,95], presenting the best of both worlds [93]. Hence, this study first induces the CSFs of the company’s information systems based on the qualitative approach, using methods such as causal layered analysis [47,96], which will be introduced in Section 4, and moves on to the quantitative MCDM method to analyze the derived CSFs to suggest implications for information systems development based on a SISP perspective. The detailed processes of the CSF-MCDM framework presented in this study are;

• Define scope (IS mission and vision). In this stage, the mission and vision of the firm’s mission-critical system are identified. Caralli [31] suggested the following five-step method for deriving CSFs; (a) define scope, (b) collect data, (c) analyze data, (d) derive CSFs, and (e) analyze CSFs. Therefore, the organizational mission and vision should be clarified and reconfirmed among the stakeholders before the IS mission and vision are defined for the rest of the strategic planning process. The researcher and six business analysts in charge of the firm’s information systems participated in this step. This analysis included; (a) defining the company’s mission, domain, core value, vision, and strategy, and (b) aligning the mission-critical system’s mission and vision with the firm’s strategic direction. The business analysts checked the purpose, background, and history of its mission-critical system development while formulating the mission and vision from the IS standpoint, and used this as the primary data for the following process.
• Define scope (PEST-SWOT). PEST-SWOT analysis was performed based on the derived IS mission and vision, reviewing its current status. The six analysts divided the PEST aspects (political, economic, social, and technological) as internal and external factors, combining them with the positive (+) and negative (−) factors of the SWOT (strength, weakness, opportunities, and threat) framework to stimulate emergent strategic thoughts. The results were shared with the 23 experts who attended the following CLA process to draw CSF dimensions and sub-criteria.
• Collect data based on CLA. This stage applied CLA methodology [96] to derive the qualitative data and expert opinions surrounding the firm’s mission-critical system. To this end, 23 experts with various IS/IT backgrounds participated in the session. CSFs can be derived by qualitative approaches such as document review or discourses based on interviews with management personnel or specific stakeholders asking about the barriers or hurdles to the organizational objectives [97]. CLA, which effectively enables stakeholders to search for the hidden drivers and assumptions for a particular surface/superficial issue, was considered a way to collect primary data through a workshop setting. This session consisted of the following five steps; (a) environmental scan, which is a preliminary session for the following four steps, (b) litany, (c) system causes, (d) worldview, and (e) myth/metaphor layer.
• Analyze data. This step focused on finding components from the qualitative data collected in the previous session. The researcher grouped the similar drivers found in the CLA’s second step (system causes), based on the consensus of participants, and coded the essential ingredients (hidden thoughts and ideas) to build sub-criteria in the next stage.
• Derive CSFs. A dimension-reducing process led to finalizing the CSF dimensions and sub-criteria from the analyzed qualitative information. The participants linked the relevant dimensions (level 1 hierarchy) with sub-criteria (level 2 hierarchy), finishing the preparation for the evaluation and prioritization of CSFs in the next stage, which is essential in the SISP process.
• Evaluate CSFs. The next stage carried out an MCDM quantitative evaluation of the CSF dimensions and sub-criteria derived from qualitatively collected data. AHP, a widely accepted decision-making MCDM methodology, was applied, as described in Section 2 of this study. The final dimensions and criteria were sent to the participants by email to be evaluated by the pairwise scoring. Lastly, after the recollection of the respondents’ data, the AHP-based weighted values were created.
• Strategic proposal. Finally, this stage synthesized the qualitative approach and quantitative evaluation results, presenting the study’s findings with theoretical implications and practical suggestions based on long-term strategic information system planning. The detailed research framework is presented as follows (see Figure 1).

4. Results

4.1. Extracting Critical Success Factors

4.1.1. Mission and Vision Analysis

The analytical procedure took place in April 2021. The researcher has over 20 years of experience in the sector and has served as the company’s information service manager. The company has six business analysts responsible for various areas of the firm’s mission-critical system. The company’s existing strategic direction, including its mission and vision, was confirmed;

• Business domain: Regional distributor supporting sales, service, and spare parts;
• Mission: Producing happiness for all customers, employees, and stakeholders;
• Vision: Mobility for all and the most respected brand;
• Core value: Always customer-first and customer delight;
• Strategy: Maximize profit based on managed customer and vehicle lifecycle.

To this end, the firm’s following IS mission and vision were reconfirmed.

• IS Mission: To design, provide, manage, and maintain the mission-critical system that can foster a business environment where secure collection, storage, use, and transaction of the data for customers, dealers, and the headquarters takes place and that flexibly meets the needs of the customers and the system users to maximize customer delight and the productivity of the fieldwork teams.
• IS Vision: Become the top-level information systems provider in the town, who would go beyond the competition with the most advanced, personalized, foresighted, flexible, adaptable state-of-the-art technology, which would generate strategic advantage based on the continuous improvement and respect for people, which is the company’s most substantial philosophical value.

4.1.2. PEST-SWOT Analysis

As a next step, the company’s business analysts attempted a PEST-SWOT analysis regarding the current situation. PEST is often used to analyze and objectify situations affected by the political, economic, social, and technological macro environments in the areas of interest [98], while SWOT analysis addresses the internal strengths and weaknesses with the external threats and opportunities of organizations that lead to tactics to achieve the strategic goals [25]. SWOT can also be utilized as an analytic technique based on and combined with internal and external environmental analysis [99] to develop logical strategic alternatives. Previous researchers strived to synthesize environmental analytical frameworks such as PEST or STEEP (socio-cultural, technological, economic, environmental/ecological, and political) to map the environmental factors with the organizational internal and external aspects derived from SWOT, e.g., [100,101,102]. The best illustration of integrating PEST and SWOT is to divide internal and external PEST factors into positive (strengths and opportunities) and negative factors (weaknesses and threats) to maintain the logical consistency of the analysis. As a result, this study conducted an integrative analysis to enrich the explanatory power of the AS-IS situation of Company A’s information system. Five positive internal aspects were identified in PEST-Strength, while nine negativities were found in PEST-Weakness. Moreover, the PEST-Opportunity quadrant pointed at ten positive aspects, while the PEST-Threat area identified seven negativities. The summarized result of the PEST-SWOT framework is presented below (see

Table 2. PEST-SWOT analysis result.

	SWOT
PEST (Inner)	Strength (+)	Weakness (−)
Political	(PS1) Risk management system and accumulated experience that has been developed in line with the gradually strengthening local regulation, such as the Private Information Protection Act (PIPA)	(PW1) Absence of a dedicated organization to keep up with domestic legal compliance and compliance pressure, limitations in management support (PW2) Higher compliance standards applied to foreign companies, resulting in a lower effectiveness of information systems
Economic	(ES1) Low IS/IT applications development cost stemmed from taking advantage of the mature and competitive local ecosystem	(EW1) High applications/infrastructure management cost due to high complexity system architecture (EW2) Lagging in productivity improvement based on the introduction of high value-added applications due to too conservative IS/IT policies that heavily focus on cost-performance analysis
Social	(SS1) Internal IS/IT support system that responds to the organization’s internal needs for remote work (i.e., work from home and distance meetings) and flexible working system	(SW1) Strong demand for efficient and fast decision-making structure and work support system by millennials and Gen Z employees
Technological	(TS1) Immediate technical support available based on longstanding partnerships with a local vendor network (TS2) Locally developed and customized dealer management system that best suits the existing needs for the current operation and working scope	(TW1) Severe dependence on existing vendors, leading to low capabilities in developing the new third-party vendors and weakness in diversification (TW2) Limitations of on-premise-based core system (TW3) Difficult system maintenance due to complex infrastructure design (TW4) Limitations in expansion due to closed system architecture
PEST (Outer)	Opportunity (+)	Threat (−)
Political	(PO1) Cross-border privacy and compliance regulations standardization efforts (PO2) A national initiative to support data utilization and information exchange between industries (i.e., the MyData project led by the government looks at the integration of cross-sector big data)	(PT1) Accelerating central control efforts on IS/IT strategic assets by global headquarters to minimize risk (PT2) Difficulties in ensuring the security of cross-border movement of personal information, simultaneously complying with regulations in multiple countries, and building infrastructure for connected car services
Economic	(EO1) Satisfying customer needs and raising satisfaction by introducing new services (EO2) Possible commercialization/sales opportunities for developed applications within the local industry or throughout the global network	(ET1) Delayed decision-making in IS/IT investment due to increased economic uncertainty
Social	(SO1) Increased usage of front-end services based on the enhanced customers’ digital literacy (SO2) Social receptivity to the growing demand for direct-to-customer service	(ST1) Increased pressure to respond to rapidly changing customer needs since the new normal in the COVID-19 era (i.e., contactless sales and service procedures or protocols) (ST2) Forced to minimize human error resulting from the existing process (replacing employees with technology)
Technological	(TO1) Introduction of online payment and direct sales system (TO2) Adoption of distributed ledger (i.e., blockchain) technology for payment, transaction of data, and information connectivity (TO3) Productivity improvement or business process automation (i.e., RPA) (TO4) Reinforcing data analytics through Big Data + ML/AI	(TT1) Pressure to replace the existing mission-critical system with ready-made applications, solutions, or infrastructure (e.g., SaaS, PaaS, and IaaS) for further efficiency (TT2) Rapid disposal/necessity to review the existing mid-/long-term IS/IT projects and initiatives

).

4.1.3. CLA Process and Results

Participants: The mission–vision and PEST-SWOT analysis results were used as primary data for the qualitative data collection process through CLA, which is the following process. The choice of research participants and the sharing of available up-to-date information is critical to the qualitative research’s effectiveness because the stakeholders’ higher engagement means more valuable insights and foresight [103]. This study invited 23 IS/IT experts to conduct a CLA workshop for CSF derivation. All of the experts were directly/indirectly involved in the operation of the company’s information system and were judged to have sufficient knowledge to proceed with the discussion. The detailed information for the expert group is presented as follows (see

Table 3. Descriptive statistics of participants.

Category	Levels	Counts	%
Expertise	Developer	6	26.1%
	IT Maintenance	7	30.4%
	IT Planning/management	6	26.1%
	Infrastructure/network	3	13.0%
	Executive/Supervisor	1	4.3%
Working experience (current position)	Under 3 years	8	34.8%
	10~15 years	7	30.4%
	3~5 years	5	21.7%
	5~10 years	2	8.7%
	15~20 years	1	4.3%
Working experience (career)	Under 3 years	6	26.1%
	10~15 years	6	26.1%
	3~5 years	6	26.1%
	Over 20 years	2	8.7%
	5~10 years	2	8.7%
	15~20 years	1	4.3%

n = 23, Mean (current position) = 7.1, Mean (career) = 9.0.

).

Causal layered analysis: The CLA process conducted for this study aimed to identify the latent dimensions and sub-criteria related to CSFs, and the qualitative methods are regarded as the most appropriate technique for collecting rich and detailed data from experts [104]. CLA is an approach that generates foresight to effectively reshape the future [47]. It is one of the newest [105] and most well-known future theories and methodologies [96]; it is a structured method to look through the surface-level issues and dig deep down to the understanding/misunderstanding that may define/constrain the surface issues [105]. CLA classifies the different concerns and views of the stakeholders to inspect multiple strategic options, identifying driving factors and people’s worldviews that affect the surface issue [37]. CLA encourages discourses visualizing the unseen causalities of the variables, leading to a more robust strategy based on a better understanding of stakeholders, and creating a different, desirable future state [46].

The CLA process consists of four layers; litany, system causes, worldview, and myth/metaphor [37]. In the litany layer, stakeholders share a superficial description of reality that best describes the current issue (the official description or the topic). The system causes layer lets participants discuss and identify what drivers cause the problem. Worldview explores and summarizes various stakeholders’ hidden thoughts and views on reality. Eventually, the myth/metaphor layer deals with hidden myths behind the worldviews. Usually, a new litany (redesigned reality) and the subsequent layers can be discussed to create a new future state. However, this paper only focuses on the analytical process, including the four layers mentioned above, because scenario building is not the scope of this paper nor the interest. The detailed CLA process is presented in the following illustration (see Figure 2).

Environmental scan: Beginning with an explicit picture of the topics is essential in qualitative research [103]. Before commencing the CLA workshop, previously collected data (IS mission, vision, and PEST-SWOT results) were delivered to the participants through email as an initial understanding effort of the environmental factors affecting the mission-critical system strategy. Before participating in the CLA workshop, the experts were asked to contemplate the issues.

Litany: The litany layer in the CLA process discusses the visible surface issues that arise as imminent problems for organizations and seeks a consensus in facing the issue among the stakeholders. Various stakeholders’ perspectives on Company A’s information system formulated in the environmental scan stage were reviewed. The workshop facilitator again displayed the prevailing issues found previously and led the group with questioning skills, letting the experts brainstorm and respond to the topics based on their experiences and knowledge. The participants revealed that the firm’s information system is under severe pressure to change to immediately provide long-term plans for a next-generation business system with a strategic roadmap to ensure the change. Significantly, the whole group agreed that to strive with the mission and vision that the company holds, strategically aligned success factors should be drawn and reviewed based on the internal/external elements that the organization had induced with management effort to secure a future competitive advantage with information service.

System causes: After reaching consensus in the litany layer, the drivers for the surface issues were examined and explored, while the experts tried to review the complex dynamics regarding IS/IT tasks. As a result, the expert group identified nine factors as presented below.

• Agility in business applications. Required business systems are not developed nor provided promptly. The experts identified the significance of providing field-support applications to adapt to rapid change.
• Performance-enhancing UX. A business application that is not user-friendly is degrading business efficiency. It is also vital to provide a flow-generating environment through a usability strategy.
• Technical complexity. More and more unheard-of technologies are applied and integrated with the existing information systems, which requires extraordinary effort in understanding and using the technology.
• Resource management. Managing workforce (HR) and internal resources efficiently from an integrative management perspective is essential and should be considered in the system design.
• Shifting way of work. Due to social distancing measures during the COVID-19, diverse operational options such as remote work and supporting contactless customer services have become indispensable IS responsibilities.
• Increasing cyber threats/incidents. Under the ever-spreading digital connectivity driven by the pandemic, illegal/unauthorized access efforts, intrusions, and cyberattacks on vital corporate IS/IT assets have increased.
• Privacy protection issues. A very high level of personal information protection policies in major EU countries/advanced economies is requested, and meeting these requirements in the SISP process becomes fundamental.
• Demand for data-driven DB and interface. As the need to interconnect numerous existing applications, sub-systems, and databases intensifies and the demands for data analysis inside and outside the enterprise are constantly increasing, the inherent flexibility to utilize data must be secured.
• Integrative view for business insights. The ability to sense meaningful signals in real-time interactive data and transform them into business intelligence is becoming a competitive advantage for modern enterprises.

Worldview: In this session, participants discussed the invisible structures behind the drivers, revealing their impacts on imminent surface issues. Often, the industry’s common assumptions and views about “how the world is or should be” emerge throughout the session [96]. Finally, the experts presented fourteen critical views behind the system causes layer.

• IS Myopia. IS/IT teams are obsessed with request-based development, focusing only on operational improvement, not being transformational.
• Bureaucracy, top-down, and big-bang. It is difficult to respond to urgent IS/IT issues due to the bureaucratic budget allocation and management that does not allow for exceptions. Only budgets for predictable/concrete investment plans such as infrastructure are reflected. In addition, top-down decision-making and big-bang-type project operation may limit flexible operation.
• Lack of user orientation. The firm’s weak business analyst capabilities limit the complete reflection of real-world business processes and needs towards the mission-critical system.
• No look back. Lack of PDCA (plan-do-check-act) cycle matters. In reality, “check” and “act” might be of more significance than plan and do. Regular system evaluation and performance review should follow suit.
• Difficulties in system maintenance. Because each pillar of information service requires clients to make rapid decisions on adopting and applying new technologies, it is almost impossible to respond to technical needs with internal resources alone. Strategic partnerships, links with vendors, or external ecosystems should be prioritized.
• Path-dependency on a legacy system. Decision-makers (client-side) often fail to realize modeling real-world business operations within the business system due to the inertia within the organization, leading to lagged support and inefficiency.
• Shifting to the new normal. Due to uncontrollable and unpredictable socio-economic events, such as COVID-19, customers continue to request new channels (i.e., digitized omnichannel service) and contactless sales/service while the employees need to work remotely.
• Less human processes. It is deemed that the introduction of business process automation becomes substantial to deal with workforce unavailability. Further, organizations are expected to increase system dependence with simplified standard work procedures to improve organizational effectiveness, reducing human errors.
• Inborn deficiencies. Legacy systems were not inherently designed to keep up with the severe cyber threats, attacks, and penetration efforts that we experience today, nor the high standards of privacy protection requested by government authorities. Security aspects should be considered together with applications development or infrastructure configuration.
• Phantom menace. Organizations need extended internal capabilities for organized responses to the existing threats. Flexible and convenient system usability that responds to users’ needs is critical. However, internal capabilities to remove invisible threats must be stressed.
• Loose control. There is a tendency to downplay the importance of policy management on the client side. Efforts for risk management, such as establishing, controlling, and reconfirming system access control based on periodic inspection, are still insufficient.
• Where is what? Organizations should be able to gain the information whenever they need it in the desired form. In addition, it is necessary to have an active data platform structure to collect, converge, classify, and analyze endogenous and exogenous data generated throughout the business process. Many organizations are transforming themselves into data companies. Building a standardized interface to respond to national initiatives such as “myData”, a project that allows industries to share personal data to create added value, will become increasingly critical.
• Machine intelligence for human intelligence. Systems that use artificial intelligence and machine learning in crucial decision-making processes will increase. Artificial intelligence will become essential in almost every process along the value chain to maximize organizational effectiveness, and flexible business intelligence becomes paramount to all members.
• Customers, out of nowhere. The growing effort to trace and leverage the digital footprint of every step of the customer journey requires redesigning the existing information systems, focusing on the front-end of the business, and requesting the review towards the overhaul or reconfiguration of the database.

Myths/metaphors: Lastly, the bottom layer of CLA was examined, revealing the myths/metaphors unseen behind the worldviews. The following four relevant images/descriptions appeared in this layer.

• IS? Information status-quo (company inertia). One may wonder if there is a strong will for the next generation of the mission-critical system within the client organization. Management talks about the critical necessity for the IS development strategy, but the executives are always reluctant and conservative on any changes relevant to long-term investment. Hence, only a mere operational transformation is achieved, following the competitors.
• Sometime later (low priority). Even after deciding to proceed with SISP, its priority eventually lowers because many stakeholders do not see IS/IT planning as a project with business impact and priority. As a result, an isomorphism occurs, seeking to level themselves only at the competition capabilities level.
• A perfect storm (digital connectivity). The importance of IS/IT has become more substantial than ever. Organizations that stand still and satisfy themselves with the status quo will gradually fall behind. The need for long-term strategic planning of information systems will come to the forefront.
• A frog in the well. A company’s information system is no longer just a mere “system,” but is instead everything that connects the organization to external systems based on connectivity. Technology such as blockchain or distributed ledger will gradually strengthen belief over data connectivity. To survive, businesses should focus more on the “ecosystem” perspective rather than the “just a system” view.

All the variables, surface issues, drivers, hidden reasons, and unrevealed stakeholders’ thoughts surrounding the information systems in the entire CLA process are summarized below (see

Table 4. Summary of CLA results.

	Litany	System Causes	Worldview	Myth/Metaphor
Causality	Surface issues	← drivers	← hidden reasons	← unrevealed thoughts
Factor	Imminent long-term planning	(SC1) Agility in business applications	(WV1) Information Service “Myopia”	IS? Information status-quo (company inertia)
	Strategic roadmap required	(SC2) Performance enhancing UX	(WV2) Bureaucracy, top-down, and big-bang	Sometime later (low priority)
		(SC3) Technical complexity	(WV3) Lack of user-orientation	A perfect storm (digital connectivity)
		(SC4) Resource management	(WV4) No look back	A frog in the well
		(SC5) Shifting way of work	(WV5) Difficulties in system maintenance
		(SC6) Increasing cyber threats/incidents	(WV6) Path-dependency on a legacy system
		(SC7) Privacy protection issues	(WV7) Shifting to the new normal
		(SC8) Demand for data-driven DB and interface	(WV8) Less human processes
		(SC9) Integrative view for business insights	(WV9) Inborn deficiencies
			(WV10) Phantom menace
			(WV11) Loose control
			(WV12) Where is what?
			(WV13) Machine intelligence for human intelligence
			(WV14) Customers, out of nowhere

).

4.1.4. Finalizing CSF Dimensions and Sub-Criteria

While deciding CSFs in a strategic planning process, it is advised that organizations limit their focus areas to a small number, ideally less than 10 [106]. Four CSF dimensions were induced based on the nine variables derived from the system causes (SC1~SC9) (see

Table 5. CSFs derived from key issues of CLA (dimensions and sub-criteria).

No.	CSF-Dimension	CLA Code	No.	CSF-Criteria	CLA Code
D1	Applications development	SC 1, 2	C1	Development-oriented decision-making and authority	WV1
			C2	Enhanced development resources and budget	WV2
			C3	Early detection and instant support for system requirements	WV3
			C4	Periodic review of the application development roadmap	WV4
D2	Business operation	SC 3, 4, 5	C5	Outsourcing for integrated system operation and management	WV5
			C6	As-a-service expansion (cloud-based)	WV6
			C7	System support for contactless, distant work	WV7
			C8	Extended business process automation	WV8
D3	Compliance and cybersecurity	SC 6, 7	C9	Security involvement in initial designs (DevSecOps)	WV9
			C10	Enhanced expertise and CISO organization	WV10
			C11	Reinforcement of internal controls and policies	WV11
D4	Data-driven flexibility	SC 8, 9	C12	Standardized data provision platform	WV12
			C13	Data-driven decision-making (AI/ML), business intelligence	WV13
			C14	Digital leads, integrated database	WV14

Dimensions: derived by reducing the drivers that appeared in system causes layer. Criteria: derived by matching the hidden reasons that appeared in worldview layer.

). After closing the CLA session, the researcher, facilitator, and the 23 experts compressed the similarly grouped variables to reduce the dimensions based on the discussion. As a result, four dimensions (D1~D4) emerged; applications development (from SC1 and 2), business operation (SC 3, 4, and 5), compliance and cybersecurity (SC 6 and 7), and lastly, data-driven flexibility (SC 8 and 9). The participants labeled the strategy as the “ABCD” of information systems based on these four dimensions.

After that, the participants agreed to match relevant sub-criteria derived from 14 findings of the worldview layer (WV1~WV14) under each dimension. Consequently, the statements were refined to become representative criteria. The following table shows the matching results (see Table 5). Finally, the CSF dimensions and sub-criteria were defined and neatly arranged for the relative weighted priority analyses based on the MCDM method in the next section (see Figure 3).

4.2. Evaluating CSFs Using MCDM

AHP, one of the most broadly used MCDM methods, was adopted to evaluate and prioritize the derived CSF dimensions. AHP was initially designed and proposed by Saaty [78] to propose a reasonable solution to a complicated problem for decision-making, and has been widely applied in a variety of research fields [107]. It provides a systematic process to comprise factors (e.g., logic, experience, and knowledge) with a sense of optimized result as a decision-making method [34]. AHP simplifies a multi-criteria problem into a hierarchy with levels. “Hierarchy” represents a complex problem in a multi-level structure where often the top level means a goal or an objective having sub-levels down to the last level of the alternatives [79], making a complex problem visible and structured into a form of hierarchy with more clarity and visibility. Typically, AHP goes through the following four steps; (a) modeling a hierarchy, (b) conducting a pairwise comparison among the dimensions or sub-level criteria, (c) summarizing the evaluation results, and (d) synthesizing the priorities derived from the normalized evaluation matrix [108].

There are two indices that a researcher may want to consider in interpreting the AHP result; consensus and consistency ratio (CR). Goepel [109] suggested the idea of group consensus levels to present an estimate of the agreement on the decided priorities among evaluators; low (50% to 65%), moderate (65% to 75%), and high (75% to 85%). Researchers must pay attention to values lower than 50% because it means that there is no consensus within the group and the diversity of judgment is high. AHP accepts logical inconsistencies in judgments evaluated by the CR indicator. Theoretically, there is no problem when the “consensus” is zero because it simply means that the population is divided into two opposing groups. However, as a golden rule in AHP, CR should be equal to or smaller than 10% [108] because the CR indicator relates to the confidence level of the analytical results [110]. CR should be strictly distinguished from a consensus because CR looks at the subjective evaluation consistency of participants. CR is calculated by dividing CI (consistency index) by RI (random consistency index). One may need to change the survey participants or consider increasing the number of evaluators to produce a sufficient sample size when CR is not acceptable.

4.2.1. Dimension (Level 1) Analysis

This paper checked the consensus level and CR ratios in all dimensions and sub-criteria comparisons. As for the analytical tool, AHP-OS (AHP online system) by Goepel [111] was used. A total of 23 experts who participated in the previous CLA workshop received an email to conduct a pairwise comparison of all dimensions and sub-criteria and were asked to provide their opinion using the standard AHP 9-point linear scale (the chosen factor is more critical than the other; 1 = even, 3 = slightly, 5 = moderately, 7 = significantly, and 9 = absolutely).

As for level 1 priorities (CR = 1.9%), compliance and cyber security (D3) appeared to be the dimension of top priority (34.2%), followed by business operation (D2, 26.6%), data-driven flexibility (D4, 22.2%), and applications development (D1, 17.0%). The final consolidated decision matrix and priorities for the CSF dimensions (level 1 hierarchy) are presented (see

Table 6. Consolidated decision matrix for main CSF dimensions.

Category	D1	D2	D3	D4
Applications development (D1)	1.000	0.823	0.498	0.587
Business operation (D2)	1.215	1.000	0.923	1.296
Compliance and cybersecurity (D3)	2.010	1.083	1.000	1.855
Data-driven flexibility (D4)	1.703	0.772	0.539	1.000

n = 23 experts.

and

Table 7. Consolidated priorities for main CSF dimensions.

Category	Priority	Rank
Applications development (D1)	0.170	4
Business operation (D2)	0.266	2
Compliance and cybersecurity (D3)	0.342	1
Data-driven flexibility (D4)	0.222	3

CR (consistency ratio): 0.019.

).

4.2.2. Sub-Criteria (Level 2) Analysis

Regarding level 2 priorities, the weights distributed among D1 (CR = 2%) are; early detection and instant support for system requirements (C3, 28.2%), periodic review of the application development roadmap (C4, 27.3%), development-oriented decision-making and authority (C1, 25.3%), and enhanced development resources and budget (C2, 19.3%). The consolidated priorities for D2 (CR = 1.3%) are; outsourcing for integrated system operation and management (C5, 28.6%), as-a-service expansion (cloud-based) (C6, 27.5%), extended business process automation (C8, 24.6%), and system support for contactless, distant work (C7, 19.4%). D3 (CR = 0.0%) priorities appeared to be in the order of; enhanced expertise and CISO organization (C10, 47.2%), security involvement in initial designs (DevSecOps) (C9, 26.5%), and reinforcement of internal controls and policies (C11, 26.3%). Lastly, priorities for D4 (CR = 0.9%) were; data-driven decision-making (AI/ML), business intelligence (C13, 52.3%), digital leads, integrated database (C14, 31.8%), and standardized data provision platform (C12, 15.9%). AHP consensus for all four dimensions (D1~D4) was 65%, which is moderate. The final consolidated decision matrices and consolidated priorities for CSF criteria (level 2 hierarchy) are presented in the following tables (see

Table 8. Consolidated decision matrix for D1 sub-criteria.

Category	C1	C2	C3	C4
Development-oriented decision-making and authority (C1)	1.000	1.704	0.814	0.783
Enhanced development resources and budget (C2)	0.587	1.000	0.900	0.684
Early detection and instant support for system requirements (C3)	1.228	1.111	1.000	1.230
Periodic review of the application development roadmap (C4)	1.276	1.462	0.813	1.000

n = 23 experts.

,

Table 9. Consolidated priorities for D1 sub-criteria.

Category	Priority	Rank
Development-oriented decision-making and authority (C1)	0.253	3
Enhanced development resources and budget (C2)	0.193	4
Early detection and instant support for system requirements (C3)	0.282	1
Periodic review of the application development roadmap (C4)	0.273	2

CR (consistency ratio): 0.020.

,

Table 10. Consolidated decision matrix for D2 sub-criteria.

Category	C5	C6	C7	C8
Outsourcing for integrated system operation and management (C5)	1.000	1.233	1.564	0.915
As-a-service expansion (cloud-based) (C6)	0.811	1.000	1.584	1.203
System support for contactless, distant work (C7)	0.639	0.631	1.000	0.941
Extended business process automation (C8)	1.093	0.831	1.063	1.000

n = 23 experts.

,

Table 11. Consolidated priorities for D2 sub-criteria.

Category	Priority	Rank
Outsourcing for integrated system operation and management (C5)	0.286	1
As-a-service expansion (cloud-based) (C6)	0.275	2
System support for contactless, distant work (C7)	0.194	4
Extended business process automation (C8)	0.246	3

CR (consistency ratio): 0.013.

,

Table 12. Consolidated decision matrix for D3 sub-criteria.

Category	C9	C10	C11
Security involvement in initial designs (DevSecOps) (C9)	1.000	0.572	0.989
Enhanced expertise and CISO organization (C10)	1.749	1.000	1.828
Reinforcement of internal controls and policies (C11)	1.011	0.547	1.000

n = 23 experts.

,

Table 13. Consolidated priorities for D3 sub-criteria.

Category	Priority	Rank
Security involvement in initial designs (DevSecOps) (C9)	0.265	2
Enhanced expertise and CISO organization (C10)	0.472	1
Reinforcement of internal controls and policies (C11)	0.263	3

CR (consistency ratio): 0.000

,

Table 14. Consolidated decision matrix for D4 sub-criteria.

Category	C12	C13	C14
Standardized data provision platform (C12)	1.000	0.278	0.550
Data-driven decision-making (AI/ML), business intelligence (C13)	3.596	1.000	1.501
Digital leads, integrated database (C14)	1.819	0.666	1.000

n = 23 experts.

and

Table 15. Consolidated priorities for D4 sub-criteria.

Category	Priority	Rank
Standardized data provision platform (C12)	0.159	3
Data-driven decision-making (AI/ML), business intelligence (C13)	0.523	1
Digital leads, integrated database (C14)	0.318	2

CR (consistency ratio): 0.009.

).

4.2.3. Global Priorities

Lastly, the global priorities of all sub-criteria are provided (see

Table 16. CSF hierarchy for dimensions and sub-criteria.

Level 0	Level 1	Priority	Level 2	Priority	Global Priority
CSFs	Applications development	0.170	Development-oriented decision-making and authority	0.253	0.043
			Enhanced development resources and budget	0.193	0.033
			Early detection and instant support for system requirements	0.282	0.048
			Periodic review of the application development roadmap	0.273	0.046
	Business operation	0.266	Outsourcing for integrated system operation and management	0.286	0.076
			As-a-service expansion (cloud-based)	0.275	0.073
			System support for contactless, distant work	0.194	0.051
			Extended business process automation	0.246	0.065
	Compliance and cybersecurity	0.342	Security involvement in initial designs (DevSecOps)	0.265	0.091
			Enhanced expertise and CISO organization	0.472	0.161
			Reinforcement of internal controls and policies	0.263	0.090
	Data-driven flexibility	0.222	Standardized data provision platform	0.159	0.035
			Data-driven decision-making (AI/ML), business intelligence	0.523	0.116
			Digital leads, integrated database	0.318	0.071

). The top five criteria identified as contributing to the upper level of critical success factors, the so-called “ABCD” of strategic information systems planning, are as follows;

• Enhanced expertise and CISO organization (16.1%);
• Data-driven decision-making (AI/ML), business intelligence (11.6%);
• Security involvement in initial designs (DevSecOps) (9.1%);
• Reinforcement of internal controls and policies (7.6%);
• Outsourcing for integrated system operation and management (7.3%).

5. Discussions

This paper aimed to discover the CSFs of a company business system based on SISP and present a novel evaluation framework by MCDM for company-wide strategic alignment, answering the two primary research questions. Among various factor-developing techniques that range from the competitive forces model to the scenario planning and MCDM methods such as ELECTRE [80,81], PROMETHEE [82,83], TOPSIS [84], and WASPAS [85], this study presented a synthesized mixed-method framework based on CSF and AHP to enrich and make the existing SISP concept more applicable. Based on the CSF-MCDM framework, the researcher followed six steps, combining the qualitative approach to derive CSFs that incorporated PEST-SWOT and CLA procedures and the quantitative approach to prioritize and evaluate the main pillars of IS/IT strategy. The results and findings of the paper are as follows.

First, the structured SISP process based on the framework presented in this study completed the IS strategy formulation based on the insider/stakeholders’ perspectives. The qualitative process reflected diverse opinions, views, assumptions, and environmental drivers to draw the main four dimensions of CSFs that consist of future IS strategy. After internal business analysts thoroughly reviewed the firm’s strategy pillars (mission, domain, core value, vision, and business strategy), environmental factors combined with strengths and weaknesses (PEST-SWOT) were analyzed to be proposed as fundamental data for the other qualitative process (CLA). A total of 23 IS/IT professionals then discussed the importance of imminent IS planning and the strategic roadmap, providing qualitative data for inducing relevant dimensions and sub-criteria that make up a CSF hierarchy. As a result, new IS strategic pillars such as applications development, business operation, compliance and cybersecurity, and data-driven flexibility were derived.

Second, the analysis of layer one presented relative strategic priorities among dimensions based on the MCDM method. The quantitative process revealed that among the CSF dimensions, compliance and cybersecurity take the top priority in preparing the next-generation mission-critical system. Understandably, businesses have faced life-or-death situations with risk management of their information systems since COVID-19 [15]. Further, it was also found that issues with efficiencies and effectiveness in business operation and data-driven flexibility come before applications development in terms of strategic priority.

Third, the sub-criteria (level 2) findings indicated the core activities to support the respective CSF dimensions drawn for IS strategy. Early detection and instant support for system requirements (C3 for D1), outsourcing for integrated system operation and management (C5 for D2), enhanced expertise and CISO organization (C10 for D3), and data-driven decision-making (AI/ML) and business intelligence (C13 for D4) were chosen as the result of the weighted distribution. As a supporting layer that makes up the higher layers connecting to the IS strategy and the firm’s business strategy, the priorities decided in this layer seemed to bring more alignment with the organizational direction.

Lastly, the global priorities analysis results pointed at the optimized resource distribution planning in the overall IS strategy. The top five sub-criteria found to have relative weights were; (a) three items in D3: security involvement in initial designs (DevSecOps), enhanced expertise and CISO organization, and reinforcement of internal controls and policies, (b) one item in D1: outsourcing for integrated system operation and management, and (c) data-driven decision-making (AI/ML) and business intelligence. The result shows that D3 takes approximately 60% of the weight among the four dimensions of CSFs. It also hints at the experts’ concern that enhancing compliance and risk management against the cyber threats are the commonly set priorities in modern IS/IT implementations.

6. Conclusions

This paper has significant theoretical implications, as follows. First, this paper contributes to the current knowledge regarding SISP by proposing a novel mixed-method research framework consisting of a qualitative dimension and sub-criteria development enhanced by a quantitative MCDM to provide rich context and more applicability. To this end, the study conducted a PEST and SWOT analysis (endogenous view) and an expert-based CLA (exogenous view) [37,46] to draw CSFs of a next-generation information system, followed by evaluation by AHP to present prioritized relative weights and significance for strategic decision-making. The recent literature dealing with CSFs evaluated by MCDM includes studies based on (a) fuzzy interpretive structural modeling (ISM) combined with matrix impact of cross-multiplication applied to classification (MICMAC) analysis [74], (b) expert Delphi and Additive Ratio Assessment (ARAS) method [73], (c) literature reviewed factor selection and DEMATEL (decision making trial and evaluation laboratory) approach [72], and (d) policy framework derived by fuzzy DEMATEL method [71]. However, despite their unique contributions to the relevant knowledge, none considered strategic alignment with higher-level company-wide strategy, neglecting the details of the strategic alignment process. In light of the above, the value and contribution of this paper that further distinguishes itself from the other research are rich, integrative, and comprehensive strategy processes. Further, prior studies focused on the “evaluating and analyzing” of CSFs, weighing on the statistical procedures of MCDM rather than “reflecting and inferencing” rich contexts and unseen causalities to build CSFs characterized by qualitative methodologies as presented in this study. No prior research that attempted to synthesize CLA and MCDM methods for SISP is found in the literature. Further, this paper enriches the existing literature by engaging a strategy-as-practice perspective, e.g., [43,44,45], as a theoretical lens for making SISP a more manageable and predictable process rather than a mere conceptual research agenda for scholars. This study also removes information asymmetry and methodological ambiguity in IS literature [39,40] by providing an applicable micro-level, multi-method framework labeled CSF-MCDM for SISP.

This paper also presents several managerial applications. First, this study highlighted a real-world case for the SISP process that would further strengthen the replicability of organizations. As pointed out by diverse scholars, how organizations deal with SISP processes remains in question, leaving IS/IT managers confused about implementing an effective strategy. As described in this research, future managers will be able to conduct their SISP with more focus and effectiveness while reducing uncertainties. Second, the approach adopted in this research project may allow managers to diversify their methodological approaches further to formulate, plan, and implement IS/IT strategies. Mintzberg [112] pointed out that creative strategies are born from emergent and diverse thoughts. In this regard, managers can utilize qualitative methodologies, such as CLA used in this study, to incorporate each stakeholder’s diverse view and expertise to enhance the existing information system. Because the business phenomenon is multi-layered and multi-faceted [37], it would not be effective for a manager to adopt only one method for decision-making of importance. The different views drawn along the strategy formulation process can also be effectively evaluated and prioritized by MCDM, providing more concrete and practical management decisions for the future. As exemplified in this research, managers can involve stakeholders with different views and backgrounds and include those opinions in the strategy process to enhance organizational effectiveness. Third, the variables, drivers, and hidden thoughts made visible in this research represent the must-review agenda of the “New Normal era.” Business systems face new technological challenges regarding remote work, virtual meetings, contactless commerce, privacy protection, cybersecurity issues, data analytics, and data-driven capabilities [16]. It becomes more apparent that organizations must seek alignment with the upper-level strategy while satisfying the IS users to secure continued usage [18] for the desired organizational performance in the post-COVID-19 era. The CSFs derived in this research may match modern businesses’ needs to enhance the technology-supported business performance required in today’s competitive business environment.

7. Limitations and Future Research

This paper presents several limitations as follows. First, it may be difficult to remove the subjectivity of the stakeholders entirely. Generally, it is known that qualitative research is subject to validity problems. Moreover, a method such as CLA has its weaknesses of presenting “collective futures” or failing to examine an individual’s psychological aspects [113]. However, integrated qualitative research models may be of more value when supported by other quantitative methods [37], bringing more academic interest and business practicality. Moreover, future researchers may want to repeat the SISP processes based on the CSF-MCDM framework to secure more validity and the scientific replicability required for the subsequent studies. Second, a more diverse approach to implementing SISP could be considered. This study adopted a CSF-MCDM framework that assumes the merits of combining qualitative–quantitative methods. However, future researchers may diversify the SISP by adopting a new perspective, i.e., a quantitative–quantitative combination. Inducing CSFs using technology could be one suggestion that enables researchers to maximize the potential of AI/ML techniques, including text-mining. Third, the SISP literature pool needs to be enhanced by studies conducted in diverse cultures, countries, and contexts. This paper argues that to answer the academic call from Hughes and McDonagh [45], the SISP research stream should be diversified and enriched, expanding upstream and downstream of the entire IS/IT ecosystem regarding the significance of the technological advancements.