E-Learning Critical Success Factors during the COVID-19 Pandemic: A Comprehensive Analysis of E-Learning Managerial Perspectives

Abstract

During the COVID-19 pandemic, educational institutions were shut down all over the world, which impacted over 60% of students and caused a massive disruption of the education system. The goal of this paper was to identify the critical success factors for E-learning during COVID-19 using the multi-criteria Analytic Hierarchy Process (AHP) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) techniques to enhance the educational process. Data were generated by interviewing 69 E-learning managers in educational institutions during COVID-19 based on defined evaluation criteria and E-learning approaches through several channels. We found that technology management, support from management, increased student awareness to use E-learning systems, and demanding a high level of information technology from instructors, students, and universities were the most influential factors for E-learning during COVID-19. Among the five learning systems, blended learning was the most suitable learning system to practice. These results demonstrated that, regardless of how extraordinary the technology is in an educational institution, the readiness of E-learning execution played a large role in boosting the educational process during the COVID-19 pandemic.

1. Introduction

Prior to the COVID-19 pandemic, E-learning was growing approximately 15.4% yearly in educational institutions around the world without uncertainties or pressure on those institutions or on students [1]. However, as this research was conducted during COVID-19, the situation has changed dramatically. Educational institutions began providing most of their services online, including lecturers and different assessments via several platforms for over 60% of students around the world due to global restriction measures to minimize the spread of COVID-19 [2]. Referring to the data released by the World Health Organization (WHO), COVID-19 has been reported in over 216 countries, and there are areas with millions of confirmed cases [3]. Many countries have taken precautionary measures, including lockdowns of schools and universities, and switching to full E-learning mode during the spread of the Coronavirus, to avoid future expected waves [4]. This action was in response to social distance rules, which were strongly recommended by the WHO to prevent the spread of COVID-19 [5]. This lockdown began in the middle of the spring semester, which was unplanned for both instructors and students.

However, many studies have previously examined the critical success factors (CSFs) in the education sector from both the instructor’s and students’ perspectives for future improvement in the E-learning system. Organizations can determine the most valuable CSFs that should be achieved in order to boost a project mission. Thus far, these studies examined the CSFs of E-learning during typical times. Yet, the CSFs during the COVID-19 pandemic are expected to be different than the CSFs during typical times for many reasons. Firstly, during COVID-19, the switch to E-learning was for all educational institutions, which was unplanned. Not all institutions had the ability to switch smoothly as not all of them were previously implementing E-learning, unlike institutions that already offered E-leaning and were planning for and investing in the E-learning process. Secondly, during COVID-19, many factors other than educational ones, such as political and health factors, influence the process, which makes it an abnormal situation. For example, during typical times, students may visit the library, attend tutoring sessions, and even go to places with a good internet connection speed if they do not have a good internet connection at home, unlike during COVID-19 where students were in curfew situations. Thirdly, the course material of the classes that were taught through E-learning pre-COVID-19 were well-prepared, unlike during COVID-19 where courses were not planned to be taught through E-leaning.

The list could continue, not only for the CSFs but also for E-learning system approaches. These are just some of the main differences worth evaluating during the pandemic. In this paper, we discuss the essential success factors from the perspective of E-learning managers in various educational institutions using multi-criteria decision-making methods to ensure the continuity of educational objectives and students’ prosperity in their education while fulfilling the World Health Organization (WHO) social distancing recommendations. This provides the perspective of E-learning managers, which allows us to understand the best practices during uncertain crises that could force educational institutions to switch to E-learning. This can help policy makers in educational institutions to better execute the educational process during a crisis through improving the most critical factors to prepare for.

2. Literature Review

2.1. E-learning

E-education, distance-learning, and online learning are all different terminologies of E-learning. The authors in [6] defined E-learning as “the wide set of applications and processes which use available electronic media and tools to deliver vocational education and training”. Researchers [7] stated that E-learning is "the use of various technological tools that are web-based, web distributed, or web capable for education". E-learning has been growing year after year as there are many advantages, such as flexibility, internet accessibility, and cost-effectiveness [8]. These advantages could transform education into a lifelong learning process. According to [9], having access to lectures anytime, as many times as needed, allows students to better recall the information that is required for traditional education.

The flexibility of E-learning is a solution for people’s commitments to their family or work, which may increase the number of people who enroll in this type of education. In fact, this goes beyond the learners; it gives flexibility also for the instructors. In addition, educational institutions are implementing E-learning technologies to improve the communication among learners and instructors for better knowledge exchange as well as to strengthen the learning community to accomplish personal objectives [10].

In 2005, the Kingdom of Saudi Arabia established the National Center of E-learning and Distance Learning (NCEDL), in which at least nine universities were involved [11]. This central role was aimed at enhancing the experience of E-learning in educational institutions by adapting and implementing the most effective practices of the E-learning system globally [12]. According to the National Center of E-learning and Distance Learning, the NCEDL has been involved in several E-learning system projects, such as the Learning Portal, which helps students to access the online learning material remotely as well as train teachers in the use of E-learning tools. Furthermore, the center has created an Award for Excellence of E-learning to encourage educational institutions to utilize E-learning, which involves around 42 institutions.

In 2011, the government of Saudi Arabia established the Saudi Electronic University (SEU), with tens of thousands of students enrolled in its different programs, including undergraduate and graduate studies. Since the occurrence of these events, King Abdulaziz University implemented many technological tools in order to enhance their practice in the E-learning system, such as the Learning Management System (LMS), which supports fresh and junior students by providing access to over 16,000 e-books as well as other academic resources online [12]. All these efforts for E-learning impacted the education transformation from on-campus to distance learning during the COVID-19 pandemic, which happened suddenly without enough time to plan.

2.2. TOPSIS and E-learning

In 1981, Hwang and Yoon developed the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) method [13]. This method is commonly known for the Multiple Criteria Decision-Making practice to determine the best alternative among a set of alternatives [14]. This method essentially determines the distance of both the positive and the negative alternatives of the ideal solution [15]. The TOPSIS technique is widely used in many industries, including education; specifically, E-learning is becoming more popular and is growing over time.

For instance, researchers in [16] evaluated the criteria of five different approaches of E-learning, where each approach’s performance was rated and computed using the TOPSIS method. They concluded that the Flipped Classroom, a student-centered approach with online material provided to student’s prior to classes, was suggested as the most convenient E-learning approach, whereas ‘strategic readiness for E-learning implementation’ was ranked as essential criteria. Additionally, in [17], the authors evaluated and selected the learning objectives of e-content and educational material in web-based learning systems. They combined both the Analytic Hierarchy Process (AHP) and TOPSIS methods to ease the selection process. They suggested that the learning outcomes metadata is an effective procedure to produce e-content.

Eight criteria were evaluated using the Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE) and the TOPSIS method to examine the E-learning readiness as well as to weight the criteria based on Simple Additive Weighting (SAW) and AHP to determine the weaknesses and improve the implementation of E-learning [18]. Indeed, TOPSIS has been commonly used to evaluate different criteria and factors in E-learning as well as different educational system approaches. Therefore, conducting this technique will help to find the most CSFs for E-learning during COVID-19.

2.3. Analytic Hierarchy Process (AHP) and E-learning

In 1980, Saaty [19] developed the Analytic Hierarchy Process (AHP). This is an effective decision-making process where quantitative and qualitative features are considered [16]. When the decision relies on several criteria, the AHP method is one of the most effective techniques to use [16]. Although the method has been criticized over its process, almost all fields, including resource allocation, management, and education, have used this technique for making important and responsible strategic decisions [20]. For example, Dweiri [20] researchers investigated and prioritized the critical factors of violations of academic integrity in Saudi Arabia within educational institutions. They stated that twelve essential factors of E-learning were prioritized using an analytic hierarchy process. They concluded that the most critical factor was the inappropriate guidelines provided to students, whereas a shortage of feedback was the least critical factor amongst all factors.

In [21], the authors evaluated the CSFs of E-learning from both the instructor and student perspectives at Sebelas Maret University. They stated that they used AHP and fuzzy techniques to determine the ranking of the CSFs from both perspectives. They concluded that the five CSFs from the instructor’s point of view were fiscal policy, regulatory policy, course quality, relevant content, and technical support. Whereas course quality, relevant content, completeness of the content, attitudes toward students, and flexibility in taking courses were the five critical success factors from the students’ perspective. However, the focus in this paper was on E-learning during the COVID-19 pandemic, which is an abnormal situation. Thus, we implemented the AHP method in order to calculate the weight of the different criteria.

2.4. Critical Success Factors and Type of E-Learning System

The critical success factors are referred to as ”characteristics, conditions, or variables that, when properly sustained, maintained, or managed, can have a significant impact on the success of a firm competing in a particular industry” [22]. By finding the CSFs, stakeholders can boost these factors for better outcomes. The types of E-learning systems are identified in

Table 1. E-Learning system definition and prior research.

Type of E-Learning System	Prior Research	Definition
Blended Learning	[4,16,23]	Mix of traditional and online classes
Flipped Classroom	[16,23,24]	Student-centered approach with online material provided to students prior to classes
ICT Supported Face-to-Face Learning	[25,26,27]	Traditional learning supported by information and communication technology.
Synchronous Learning	[16,24,28]	A real-time interaction distance learning
Asynchronous Learning	[16,24,28]	Non-real time interaction distance learning

, to determine the most appropriate type of system associated with AHP and the TOPSIS technique during COVID-19. The factors that were evaluated in this paper are identified and defined in

Table 2. E-Learning criteria, definition, and prior research.

Factors	Prior Research	Definition
Student Characteristics	[25,29,30,31]	This factor focuses on the student’s environment while learning. It includes the student’s pace of learning, commitment, attitude, motivation, knowledge of computer systems, and demographics.
Instructor Characteristics	[8,25,29,30,31]	This factor focuses on the instructor’s environments while teaching. It includes the instructor’s attitude, flexibility, knowledge of learning technology, teaching style, and efficacy in student motivation.
Learning Environment	[8,25,29,31]	This factor focuses on the learning environment and facilities that are provided for both students and instructors. It includes a learning management system, technical infrastructure, interactive learning, and access and navigation.
Instructional Design	[8,25,29,30]	This factor focuses on the instructional system to meet the objectives of the institution. It includes the content quality, objective clarity, learning strategies, and learning psychology.
Support	[8,25,29,30,31]	This factor focuses on supporting both the instructors and students to enhance their experience. It includes communication tools, help disk availability, and training.
Information Technology	[21,25,26,29,30,32]	This factor focuses on the information technology system to deliver learning materials and objectives. It includes ease of use, reliability, efficiency, privacy, and information.
Technology Knowledge	[8,25,29,30,31]	This factor focuses on the knowledge of using technology for both instructors and students. It includes the use of computers, the use of software, and communication interaction.
Course	[8,20,21,25,29,31]	This factor focuses on the course material and objectives. It includes course evaluation, assessments, content development, and learning evaluations.
Level of Collaboration	[8,21,30,31]	This factor focuses on the collaboration level between faculty members. It includes the lack of social interaction, project team supervision, and managerial support.
Knowledge Management	[8,21,30,31]	This factor focuses on the management knowledge within the educational institution for faculty members and administration. It includes the management team, managing delivery and maintenance, time management, thinking strategies, and implementation expertise.

. Figure 1 summarizes the critical success factor hierarchy problem discussed in this paper based on the multiple-criteria decision analysis problem representation. Those criteria and alternatives were identified from previous studies on E-leaning and were categorized to represent the majority of the criteria that were evaluated in other studies under different conditions.

3. Materials and Methods

The methodology in this research consists of three parts; a survey, the AHP method, and the TOPSIS method. Further details will be provided in the next subsections.

3.1. Data Collection

Firstly, the research team visited 69 educational institutions and planned several meetings with the distance learning managers to create an appropriate picture of the problem.

Table 3. E-Learning management staff demographic data.

		Frequency	Percentage
Age	30–39 Years	17	25%
	40–49 Years	24	35%
	50–59 Years	28	41%
Gender	Male	57	83%
	Female	12	17%
Nationality	Saudi	49	71%
	Non-Saudi	20	29%
Academic Degree	Master’s Degree	7	10%
	PhD	62	90%
Job Title	Lecturer	7	10%
	Assistant Professor	19	28%
	Associate Professor	31	45%
	Professor	12	17%
Discipline	Education	7	10%
	Science	9	13%
	Arts	4	6%
	Business	22	32%
	Medicine	2	3%
	Engineering	19	28%
	Political Science	6	9%

summarizes the E-learning management staff demographic data in terms of age, gender, nationality, academic degree, and job title. This shows that most participants (90%) had a PhD, and that, with respect to job title, the biggest group were Associate Professors (45%). Furthermore, the majority were Saudi nationals (71%). The managers evaluated the criteria associated with each E-learning system. During the first meeting, the discussion of the decision criteria were as follows: Instructor Characteristics, Student Characteristics, Information Technology, Support, Technology Knowledge, Course, Instructional Design, E-Learning Environment, Level of Collaboration, and Knowledge Management. The decision alternatives were as follows: Blended Learning, Flipped Classroom, Information and Communication Technology (ICT) Supported, Face-to-Face Learning, Synchronous Learning, and Asynchronous Learning. The managers of E-learning rated each criterion associated with each e-learning system toward the CSFs of E-learning in the COVID-19 pandemic, as shown in

Table 4. E-Learning approaches and evaluation criteria.

No.	Alternatives	No.	Criteria
1	Blended Learning	1	Instructor Characteristics
2	Flipped Classroom	2	Student Characteristics
3	ICT Supported Face-to-Face Learning	3	Information Technology
4	Synchronous Learning	4	Support
5	Asynchronous Learning	5	Technology Knowledge
		6	Course
		7	Instructional Design
		8	E-Learning Environment
		9	Level of Collaboration
		10	Knowledge Management

.

3.2. The Analytic Hierarchy Process (AHP)

To apply the AHP method, the following steps were performed based on [19,33]

Step 1: Weight each criterion and decision alternatives. The ratings are given in

Table 5. Analytic Hierarchy Process (AHP) ratings.

Linguistic Rating	Numerical Rating
Extremely preferred	9
Very strongly preferred	7
Strongly preferred	5
Moderately preferred	3
Equally preferred	1

.

For instant, if the E-learning managers determined that, in the E-learning process and based on the criteria and the rankings shown in Table 5, the Instructor Characteristics were very strongly preferable to the Student Characteristics (in other words, the E-learning Instructor Characteristics had a greater influence on the learning process compared with the Students Characteristics), they will give rank 7 to the Instructor Characteristics in the pair-wise comparison matrix when comparing the Instructor Characteristics versus Students Characteristics. On the other hand, the intermediate ratings (2, 4, 6, and 8) were also used. The reciprocal ratings (1/9, 1/8, etc.) were used in cases where a second alternative was chosen over the first, assigning a rating in the case of comparing an alternate with itself, as shown in

Table 6. Pair-wise comparison matrix.

	Instructor Characteristics	Student Characteristics	Information Technology	Support	Technology Knowledge	Course	Instructional Design	E-Learning Environment	Level of Collabor-Ation	Knowledge Management
Instructor Characteristics	1	1/7	1/7	1/9	1/8	5	1/5	1/3	1/3	1/9
Student Characteristics	7	1	1	1/3	5	5	3	5	3	1/7
Information Technology	7	1	1	1/5	3	5	3	7	3	1/7
Support	9	1/3	5	1	7	7	5	5	3	1/5
Technology Knowledge	8	1/5	1/3	1/7	1	5	1	3	3	1/7
Course	1/5	1/5	1/5	1/7	1/5	1	1/5	3	1/3	1/9
Instructional Design	5	1/3	1/3	1/5	1	5	1	7	3	1/5
E-Learning Environment	3	1/5	1/7	1/5	1/3	1/3	1/7	1	1/5	1/7
Level of Collaboration	3	1/3	1/3	1/3	1/3	3	1/3	5	1	1/5
Knowledge Management	9	7	7	5	7	9	5	7	5	1
Sum	52	11	15	8	25	45	19	43	22	2

. For example, Student Characteristics pair-wise compared to Instructor Characteristics at 1/7, which indicates that the Student Characteristics were seven times more preferable to Instructor Characteristics from the managerial perspective.

Step 2: Develop the pair-wise comparison matrix and rate the relative importance among every pair of decision alternatives, as shown in Table 6. The alternatives are listed in the matrix horizontally (first alternative) and vertically (second alternative) as associated with the numerical ratings.

Step 3: Set up the normalized matrix by dividing all numbers in the pair-wise comparison matrix column by the sum of its column, as shown in

Table 7. Normalized matrix.

	Instructor Characteristics	Student Characteristics	Information Technology	Support	Technology Knowledge	Course	Instructional Design	E-Learning Environment	Level of Collaboration
Instructor Characteristics	0.02	0.01	0.01	0.01	0.01	0.11	0.01	0.01	0.02
Student Characteristics	0.13	0.09	0.06	0.04	0.2	0.11	0.16	0.12	0.14
Information Technology	0.13	0.09	0.06	0.03	0.12	0.11	0.16	0.16	0.14
Support	0.17	0.03	0.32	0.13	0.28	0.15	0.26	0.12	0.14
Technology Knowledge	0.15	0.02	0.02	0.02	0.04	0.11	0.05	0.07	0.14
Course	0	0.02	0.01	0.02	0.01	0.02	0.01	0.07	0.02
Instructional Design	0.1	0.03	0.02	0.03	0.04	0.11	0.05	0.16	0.14
E-Learning Environment	0.06	0.02	0.01	0.03	0.01	0.01	0.01	0.02	0.01
Level of Collaboration	0.06	0.03	0.02	0.04	0.01	0.07	0.02	0.12	0.05
Knowledge Management	0.17	0.65	0.45	0.65	0.28	0.2	0.26	0.16	0.23
Sum	1	1	1	1	1	1	1	1	1

.

Step 4: Determine the average priority vector for all the rows in the normalized matrix. Then use these averages to create the priority vector of all alternative preferences associated with the criterion where the sum of this vector is 1, as shown in

Table 8. Priority vectors.

Instructor Characteristics	Student Characteristics	Information Technology	Support	Technology Knowledge	Course	Instructional Design	E-Learning Environment	Level of Collaboration	Knowledge Management
0.025143	0.111681	0.106554	0.169233	0.068143	0.022556	0.076001	0.023157	0.049536	0.347996

.

Step 5: Calculate the consistency ratio and use it to measure the subjective input. A ratio of less than 0.1 was considered good. For the ratios greater than 0.1, the subjective input was considered for re-evaluation.

Step 6: Develop the priority matrix to obtain the priority vectors for each criterion using the results of Step 4.

Step 7: Develop the criterion pair-wise development matrix following the one used to create the alternative pair-wise comparison matrices utilizing the ratings of Step 2, normalize the matrix following Step 3, and develop the priority vector criterion in Step 4.

Step 8: Develop the overall priority vector through multiplying the criteria priority vector in Step 7 with the priority matrix in Step 6.

Microsoft Excel was used to perform all these steps.

3.3. The Technique for Order Preference by Similarity to Ideal Solution (TOPSIS)

To apply the TOPSIS method, the following steps were performed, based on [32,34]

Step 1: Form the matrix expressed as follows:

$$D=\left[\begin{array}{cccccc}{A}_1& X_1& X_2& \cdots & \cdots & X_n\\ A_2& X_{11}& X_{12}& \cdots & \cdots & X_{1n}\\ A_3& X_{21}& X_{22}& \cdots & \cdots & X_{2n}\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ A_m& X_{m1}& X_{m2}& \cdots & \cdots & X_{mn}\end{array}\right]$$

(1)

where:

Ai = ith alternative project and Xij = the numerical outcome of the ith alternative project with respect to the jth criterion.

Table 9. Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) matrix.

	Instructor Characteristics	Student Characteristics	Information Technology	Support	Technology Knowledge	Course	Instructional Design	E-Learning Environment	Level of Collaboration
Blended Learning	1.796	0.003	0.032	0.006	0.436	193.488	0.49	0.49	6.452
Flipped Classroom	2.103	0.017	0	0	0.194	112.148	0.04	4225	0.001
ICT Supported Face-to-Face Learning	0.823	0.004	0.125	1.328	0.16	2.465	0.686	0.656	23.329
Synchronous Learning	2.89	0.052	0.206	1.595	0.032	0.922	0.24	0.212	0.96
Asynchronous Learning	2.624	0	0.006	0.001	0.141	0.448	0.689	0.084	24.9
sum	10.235	0.076	0.37	2.931	0.962	309.47	2.145	4226.442	55.642
√(sum)	3.199	0.275	0.608	1.712	0.981	17.592	1.464	65.011	7.459

shows the structure of the TOPSIS matrix.

Step 2: Normalize the decision matrix (D) by applying the following formula:

$$r_{ij}=\frac{X_{ij}}{\sqrt{{{\displaystyle \sum }}_{i=1}^n{X}_{ij}{}^2}}$$

(2)

Table 10. Normalized matrix.

	Instructor Characteristics	Student Characteristics	Information Technology	Support	Technology Knowledge	Course	Instructional Design	E-Learning Environment	Level of Collaboration
Blended Learning	0.419	0.182	0.296	0.047	0.673	0.791	0.478	0.011	0.341
Flipped Classroom	0.453	0.473	0.033	0.006	0.449	0.602	0.137	1	0.004
ICT Supported Face-to-Face Learning	0.284	0.221	0.582	0.673	0.407	0.089	0.565	0.012	0.648
Synchronous Learning	0.531	−0.832	−0.747	−0.738	0.184	0.055	0.335	0.007	0.131
Asynchronous Learning	0.506	0.045	0.123	0.019	0.382	0.038	0.567	0.004	0.669
sum	2.193	0.088	0.288	0.007	2.095	1.575	2.081	1.035	1.792

shows the TOPSIS normalize matrix.

Step 3: Construct the weighted normalized decision matrix that is shown in

Table 11. Weighted normalized decision matrix.

	Instructor Characteristics	Student Characteristics	Information Technology	Support	Technology Knowledge	Course	Instructional Design	E-Learning Environment	Level of Collaboration
Blended Learning	0.0105	0.0203	0.0315	0.0079	0.0459	0.0178	0.0363	0.0002	0.0169
Flipped Classroom	0.0114	0.0528	0.0035	0.001	0.0306	0.0136	0.0104	0.0232	0.0002
ICT Supported Face-to-Face Learning	0.0071	0.0247	0.062	0.1139	0.0278	0.002	0.043	0.0003	0.0321
Synchronous Learning	0.0134	−0.093	−0.0795	−0.1249	0.0125	0.0012	0.0254	0.0002	0.0065
Asynchronous Learning	0.0127	0.005	0.0131	0.0033	0.0261	0.0009	0.0431	0.0001	0.0331
A*	0.0134	0.0528	0.062	0.1139	0.0459	0.0178	0.0104	0.0001	0.0002
A-	0.0071	−0.093	−0.0795	−0.1249	0.0125	0.0009	0.0431	0.0232	0.0331

by multiplying the normalized decision matrix by its relative weights. The following formula is used to calculate the weighted normalized value vij:

$$V_{ij}=W_{ij}{R}_{ij}$$

(3)

Step 4: Define both the positive and negative ideal solutions:

$$A^{*}=\left\{\left(max v_{ij}|jєJ\right), \left(min v_{ij}|jєJ^{\prime }\right)\right\} \forall J=1, 2, 3, \dots , n$$

(4)

$$A^{-}=\left\{\left(min v_{ij}|jєJ\right), \left(max v_{ij}|jєJ^{\prime }\right)\right\} \forall J^{\prime }=1, 2, 3, \dots , n$$

(5)

where:

J is associated with the benefit criteria and J′ is associated with the cost criteria.

Table 12. Priority vector.

	Si*	Si-	Ci*	Rank
Blended Learning	0.1191	0.5108	0.8109	1
Flipped Classroom	0.3063	0.2859	0.4827	4
ICT Supported Face-to-Face Learning	0.2788	0.3586	0.5625	3
Synchronous Learning	0.562	0.0398	0.0662	5

shows the TOPSIS positive and negative ideal solutions.

Step 5: Calculate the separation of all alternatives from the positive ideal as follows:

$$S_i{}^{*}=\sqrt{{\displaystyle \sum }_{j=1}^n{\left(V_{ij}-V_j{}^{*}\right)}^2} \forall \mathrm{i}=1,2,\dots ,\mathrm{m}$$

(6)

Step 6: Calculate the relative distance of Ai with respect to A*, the ideal solution, which is defined as:

$$C_i{}^{*}=\frac{S_i{}^{-}}{\left(S_i{}^{*}+S_i{}^{-}\right)}, 0\le C^{*}\le 1 \forall \mathrm{i}=1,2,\dots ,\mathrm{m}$$

(7)

where performance alternatives become better with larger values of Ci*.

Step 7: Rank the order of preference.

All steps were performed using Microsoft Excel.

4. Results and Discussion

The data were collected by interviewing 69 E-learning managers. The response rate for both the first and second rounds of interviews was 100%. Every factor’s weight was calculated using AHP software and Microsoft Excel 2013. The inconsistency ratio was calculated to achieve consistent weights among all participants. Table 8 shows that Knowledge Management (0.347996), Support (0.169233), Student Characteristics (0.111681), and Information Technology (0.106554) were the most critical success factors that influenced the E-learning process during the COVID-19 pandemic.

The primary purpose of the TOPSIS technique is to discover the best alternative that should have the shortest distance, which is the Euclidean distance, from the ideal solution. The data for the E-learning system alternatives based on their criteria are provided in Table 9, which shows the calculation of the square root of the squared summation for the given criteria for each E-learning system. The value in every cell was divided by the root summation of the square value, which gives the normalized decision matrix, as shown in Table 10. Table 11 shows the weighted normalized decision matrix, along with the ideal best and the ideal worst values. Table 12 shows the Euclidean distance from the value of both the ideal best and worst along with the performance score for every alternative using the TOPSIS method.

The research team used Microsoft Excel 2013 software to analyze and employ both AHP and TOPSIS techniques. The demonstration of the steps and analysis was provided in Section 3. As the result of the analytic hierarchy process for the E-learning critical success factors during the COVID-19 pandemic, Table 12 shows that Blended Learning appeared to be the best decision alternative for educational institutions to consider when selecting an E-learning system during the COVID-19 pandemic, with a total weight of 0.811. This was followed by Asynchronous Learning, which was considered to be the second-best alternative, with a total weight of 0.564, and then by the ICT Supported Face-to-Face Learning with a total weight of 0.563. This was then followed by the Flipped Classroom with a total weight of 0.483 and, finally, Synchronous Learning with a total weight of 0.066.

The findings revealed that the course type and contents did not have such a great impact on learning outcomes as previous studies showed, because it is a managerial perspective and all the courses are quoted with the same weight/importance [35]. A focus on providing the users (students and instructors) with more training in knowledge management would be wise [36]. Educational institutional support is very important in the success of E-learning. This is compatible with [37], which stated that providing computer and training support to students positively impacted the students’ use of the learning system. Our study found that student characteristics played a large role in educational systems during COVID-19.

The findings also indicated that students must understand their role during the social distancing measures, build their own attitude and commitment, and find ways to self-motivate in order to gain successful learning outcomes. The findings for student characteristics are consistent with a study from [26,38].

The primary objective of this paper was to identify and prioritize the critical success factors of E-learning system adoption during the COVID-19 pandemic. We employed AHP and TOPSIS, and this study may assist educational institutions in gaining a better understanding of the critical success factors for E-learning adoption during the COVID-19 pandemic. Focusing on crises, such as the COVID-19 pandemic, the current study is relevant as no previous research has discussed issues where the entire educational system worldwide was affected with many interruptions. Many educational institutions shifted from in-class education toward E-learning. E-learning system adoption is not an easy process, nor can one system fit all different types of disciplines and institutions around the globe. This research tackles the different possible systems and their critical attainment factors.

5. Limitations

This research tackled the different possible systems and their critical attainment factors. There are many studies regarding general education and E-learning; however, there was a lack of literature that included the effect of pandemics that would reflect the same situation that the world is facing during COVID-19. One of the drawbacks of the multi-criteria decision analysis tools is that they are very sensitive to the perspective the study is focused on. Therefore, the same applied tools would provide different results and findings based on whether the issue was tackled from the student perspective or the instructor perspective. Finally, this study took place in Saudi Arabia, and might not apply to other countries due to differences in perspective and regulations.

6. Conclusions

The educational process worldwide has been interrupted due to the COVID-19 pandemic. E-learning is becoming much more necessary, and is very important in education. Educational institutions during COVID-19 face the unique challenges of smoothly maintaining the process of learning while ensuring that it is still beneficial. Therefore, these institutions must understand what drives instructors and learners toward the E-learning system. The main focus of this study was to classify and prioritize E-learning systems during the COVID-19 pandemic as well as to recognize practical implications.

This study prioritized different systems of E-learning using multi-criteria approaches. We discovered that the most significant factors influencing E-learning success during the COVID-19 pandemic were related to technology knowledge management, support from management, increased student awareness of utilizing E-learning systems, and demanding a high level of information technology from the instructors, students, and universities. This finding should be seriously considered as no matter how great the technology is, readiness for E-learning implementation still plays the leading role in improving the educational process. Blended Learning was the most preferred E-learning system out of the five methods discussed in this study. The results of this study provide useful information to the E-learning managers of universities in their process of implementing modern technologies in education.