The Perception of SPADA-DIKTI Services in Underdeveloped Area Using Customer Satisfaction Index (CSI) and Importance Performance Analysis (IPA) ^†

Abstract

A good online learning system service performance can reduce the negative influence of the learning environment, which in turn can increase the effectiveness of the learning process. SPADA-DIKTI is one of the Indonesian government’s online learning system services. The purpose of this study is to measure the effectiveness of the service quality of the online learning system in the underdeveloped area that is referred to 3T areas (terdepan, terluar, tertinggal) in Indonesia. The Customer Satisfaction Index method is used to determine user perceptions of SPADA-DIKTI service performance, while the Important Performance Analysis is used to select attributes that need to be improved. The results showed that the CSI value was 75.39%, thus the users (student of underdeveloped area) of the online learning system services were satisfied. However, the number of Modules/Features according to the needs of the learning/teaching process) and Modules/Features according to the needs of the learning/teaching process need improvement to improve service quality.

1. Introduction

At the end of 2019, China announced the spread of an outbreak of the disease that became known as Corona Virus 19 [1] and on 11 March 2020, the World Health Organization (WHO) declared the novel coronavirus (COVID-19) outbreak is a global pandemic [2]. This condition has forced physical closures globally, something which has encourage all institutions to migrate to online platforms in carrying out their activities [3], including higher education institutions. Educational institutions began to close their campuses and the learning process shifted to online learning (E-learning). To suppress the spread of the COVID-19 virus, the Indonesian government officially enforced the study-from-home rules on 18 March 2020 [4]. This had an impact on the effectiveness of the learning process in higher education [5].

Underdeveloped areas are regions of the Indonesian state which have the characteristics of being left behind, isolated, and outermost. The areas are mostly gateways to Indonesia’s borders which are far from the provincial capitals. This condition causes the development of infrastructure and facilities to be hampered, which has an impact on economic growth and education [6]. The result is that the quality of education in underdeveloped areas is poor. There are many factors that influence the development of education, including the availability of teachers, the condition of students, the adequacy of infrastructure, the availability of funds and the geographical conditions of the area [7]. There are several problems related to the implementation of online learning in underdeveloped areas, namely (1) the tools used for online learning; (2) the scarcity of the communication network used to access education as, although there is a communication network to access the internet, it is not stable; (3) the availability of quality teaching materials; and (4) the availability of professional resources to implement an online learning system. To support universities to be more active in utilizing digital technology for learning to overcome obstacles in the learning process of higher education in underdeveloped areas, the central government provides grants online learning platforms consisting of Tablet-DIKTIEDU, Grounded/Mobile VSAT and SPADA-DIKTI (SPADI).

The purpose of this study was to determine how students from underdeveloped areas students feel towards SPADA-DIKTI. This is amongst the government grants issued to improve the learning quality in underdeveloped area during the COVID-19 pandemic. SPADA-DIKTI is an online learning portal developed by the Ministry of Education, Culture, Research, and Technology (https://spada.kemdikbud.go.id/) that covered the entire territory of Indonesia. It was built to solve the problem of inequality in the quality of education in Indonesia as (1) quality education is mostly located on the island of Java (2), there are inadequate universities and quality educational resources, and (3) there is lack of higher education services that are equitable and provide quality services [8].

2. Materials and Methods

The Learning Management System (LMS) is a web-based learning tool that can be used 24 h a day by everyone who has a cellphone, laptop, or tablet and a user ID to log in to the system [9]. This tool is used to manage online learning materials, learning systems, classroom management, content management systems, portals, and instructional management systems [10]. The existence of LMS allows students to be more flexible in accessing learning materials, as students can access learning materials anytime and anywhere. Meanwhile, teachers find it easier to make improvements to the material and monitor the learning process. In addition, communication between teachers and students can be done synchronously and asynchronously so that the interaction is more intense.

There are several features that are delivered by LMS for users in terms of learning online, including the following [11]:

• Class management features, enabling teachers to deliver relevant material in a timely manner to enrolled students.
• Assessment of learning outcomes is one of the features that functions to collect and store assignments that can be assessed, as well as to provide grades and feedback for each student.
• Tracking progress is a feature that used to track user engagement in the course, and accounts for the frequency of admission, time spent in various parts of the course, communication interactions, and the number of resources downloaded. Declines in student performance can be detected earlier and interventions can be carried out before students drop out.
• Final grade evaluation is an LMS feature that includes the ability to facilitate the dissemination of assessment information to students. Being able to provide complete reporting information such as class grades, item score analysis, and students who have the potential to fail is also an important requirement for teachers.
• Communication media is an LMS feature that allows teachers and students to communicate synchronously or asynchronously. This medium of communication is often considered important for replicating traditional classroom-based communication, thereby fostering a sense of community among online learners.
• One of the biggest criticisms of LMS technology is the lack of social interaction in online learning. Therefore, LMS has features that seek to build an online social environment through discussion forums, live chat, and video conferencing tools.
• Security features in an LMS include user authentication, access verification, password integrity control, and intruder detection. Privacy controls are also important to ensure that sensitive information is only available to authorized recipients.
• Couse of the increasing human dependency on cell phones being connected to the internet, ubiquitous access features are included given. Therefore, LMS providers must be able to design class content as responsive HTML pages that can be accessed everywhere using smartphones and portable computing devices.

SPADA-DIKTI is an LMS developed by the Ministry of Education, Culture, Research, and Technology to assist the online learning process in Indonesia, including in underdeveloped areas. Students and lecturers can conduct learning activities online using various interaction modalities including discussion forums, video conferencing links, chats, and notifications. In addition, student attendance can be automatically recorded by the LMS based on their learning activities and engagement. Finally, SPADA DIKTI is equipped with a dashboard monitoring system that can display every learning activity in detail.

As explained in the previous section, SPADA-DIKTI is one of the grants from the Ministry of Education, Culture, Research, and Technology to improve the quality of learning in the era of the COVID-19 pandemic, especially in underdeveloped areas. The quality of SPADA-DIKTI services can describe the effectiveness of SPADA-DIKTI grants. Various methods can be used to measure service quality, including the Customer Satisfaction Index (CSI) and the Importance Performance Analysis (IPA) [12]. Adapted from the American Customer Satisfaction Index (ACSI), CSI is a measure of how products and services meet or surpass customer expectations [13]. CSI can be defined as a technique for determining customer satisfaction by comparing the level of customer expectations and service quality performance [14]. Thus, the purpose of this study is to measure the quality of SPADA-DIKTI services in underdeveloped areas by using CSI and IPA methods.

The research team and DIKTI held a focus group discussion to determine the indicators used to evaluate the use of SPADA-DIKTI for underdeveloped areas. Based on the online learning environment for students in the underdeveloped area, we decided on six indicators to evaluate SPADA-DIKTI services as shown in

Table 1. The Attributes of SPADA-DIKTI Services Quality.

NO	Attributes
1	Number of modules/features according to the needs of the learning/teaching process
2	Modules/features according to the needs of the learning/teaching process
3	The material presented is easy to understand
4	The modules/features that have been made are quite interesting
5	Easily accessible modules/features
6	Availability of module updates

.

The CSI and IPA methods are carried out by comparing performance values (satisfaction) and expectations (interest) for each attribute as shown in Table 1: SPADA-DIKTI Service Quality Attributes. The level of satisfaction and importance of each attribute is measured using a Likert scale between 1–4. These measurements can be explained in detail as

Table 2. Scala Likert Interpretation of Performance and Importance.

Scala Likert (Value)	Performance	Importance
1	Very Unsatisfied	Unimportance
2	Unsatisfied	Less Unimportance
3	Satisfied	Importance
4	Very Satisfied	Very Importance

. The results of the CSI calculation in the form of an index value (%) will determine the level of satisfaction category as shown in

Table 3. Interpretation of CSI Index Number.

Index Value (%)	Customer Satisfaction Index
81–100	Very Satisfied
66–80.99	Satisfied
51–65.99	Quite Satisfied
35–50.99	Less Satisfied
0–34.99	Unsatisfied

[15].

3. Results and Discussion

Total respondents who were processed by the researchers were 529 respondents and the majority of respondents were female, who represented 51.2 percent of the sample [16,17]. The remaining 48.8% were male students, as shown

Table 4. Descriptive Statistics of Respondents.

No	Indicators	Variables	Percentage
1	Gender	Male Female	48.8% 51.2%
2	Region	Maluku NTT Papua	8.6% 46.9% 44.2%
3	Age	16–20 21–25 26–30 31–35 36–40	57.53% 37.84% 1.54% 1.37% 2.05%
4	Have email address	Have Do not have	96.7% 3.3%
5	Have social media account	Have Do not have	99.1% 0.9%
6	Know and operate Ms. Office	Know and operate Do not know and operate	96.6% 3.4%

. In terms of the distribution across three regions are showed by Figure 1, the highest number of students came from East Nusa Tenggara at 46.9%, followed by Papua and Maluku. Most of the students in the sample were aged 16 to 20 years, but there were also students who were above 30 years in age, who were around 3.42% of the sample. Based on Table 4, it can be found that there are 3% of students who do not have email, almost 1% do not have social media, and 3.4% can’t operate Ms. Office.

There are six (6) items used to evaluate the quality of SPADA-DIKTI services, as shown in

Table 5. Score Average of Importance and Satisfaction Level.

NO	Attributes	Average of Priority	Weight	Average of Satisfaction	Suitability Level (%)	Gap Level	Weight Score (C3 × C4)
C1	C2	C3		C4	C5	C6	C7
1	Number of modules/features according to the needs of the learning/teaching process	3.25	16.88%	3.02	92.79%	−0.23	9.82
2	Modules/features according to the needs of the learning/teaching process	3.23	16.74%	3.00	92.85%	−0.23	9.67
3	The material presented is easy to understand	3.25	16.85%	3.07	94.53%	−0.18	9.97
4	The modules/features that have been made are quite interesting	3.12	16.19%	3.00	96.00%	−0.12	9.35
5	Easily accessible modules/features	3.22	16.70%	3.03	94.19%	−0.19	9.76
6	Availability of module updates	3.20	16.63%	2.98	92.98%	−0.22	9.55
Total		19.27	100%	18.09	-	−1.18	58.12
Average		3.21	16.67%	3.02	93.88%	−0.20	9.69

. The results of the CSI calculation obtained a figure of 75.39%, thus the level of satisfaction of SPADA-DIKTI users was in the Satisfied category.

Importance Performance Analysis (IPA) is an analytical tool used to compare the level of performance/service perceived by service users compared to the level of satisfaction desired/expected. The quadrant that becomes the main priority for quality improvement is the high-priority quadrant. This quadrant represents a very high level of student expectations but has not yet felt a level of satisfaction for students. According to Figure 2: Importance Performance Analysis Diagram, we find that attribute-1 and attribute-2 can be high in priority. This supports the findings of the previous research, which states that students who take more modules on online learning platforms will be more likely to be able to improve their learning process and have the potential to further increase the use of these platforms in the student learning process of underdeveloped areas [18].

The keep up performance quadrant provides a description that services that are considered important by students have been fully served by SPADA-DIKTI. Thus, students of underdeveloped area have felt the level of satisfaction in accordance with their expectations on these attributes. The low-priority quadrant explains that according to student perception, the factors are considered to be unimportant to define SPADA-DIKTI service quality. There are two attributes in the low priority area, namely: (1) The modules/features that have been made are quite interesting, and (2) Availability of module updates. Even though they have a low priority, the service quality related to these two attributes has not met their expectations. Thus, improvements are still needed to meet student expectations.

4. Conclusions

Based on the results of the researcher’s study, students in the underdeveloped area were satisfied (the CSI value was 75.39%) with the SPADA-DIKTI grant assistance provided by the central government (Ministry of National Education). However, several attributes need to be improved related to SPADA-DIKTI services, namely attribute-1 and attribute 2.

Researchers suggest that the central government provide scholarships, grants and or other types of financial assistance to students and university administrators in underdeveloped areas. For example, the government could provide grants for the development of online learning materials and internet infrastructure that can be used by all students, especially students in underdeveloped areas. This would allow more students to enjoy higher education online, which can increase student interest in learning. In the future, education in Indonesia must take full advantage of geographically flexible online learning methods, especially in underdeveloped areas.