Sustainable Adoption of E-Learning in Romanian Universities after the COVID-19 Outbreak

Abstract

The COVID-19 pandemic disrupted economic processes and various facets of daily life, including education, necessitating adjustments to help society adapt to the temporary status quo, with Romanian educational institutions being profoundly affected, and a full transition to online learning was mandated by central authorities in March 2020. The paper’s scope is to assess the sustainability of e-learning in Romanian higher education in the aftermath of the COVID-19 outbreak. The study was conducted on bachelor students from three Romanian universities through an online questionnaire with a sample size of 505 valid responses. This study aims to investigate the relationships between information quality (IQ), system quality (SQ), service quality (SEQ), and quality of life (QL) within an integrated model, based on the variables of the technology acceptance model (TAM) and performance models of information systems (IS). Specifically, the research explores how these factors, along with the mediating roles of perceived usefulness (PU) and perceived ease of use (PEOU), influence students’ behavioral intention to adopt e-learning systems (BISE) and actual use of them (EUOES) as a sustainable solution for post-pandemic COVID-19 education. Partial least squares structural equation modeling (PLS–SEM) was the selected method for data analysis performed with SmartPLS 4.0 software. The research results demonstrated that PU and PEOU showed a positive correlation relationship and were significantly influenced by IQ, SQ, and QL in the educational setting. The study also revealed that PEOU and PU exerted a positive influence on students’ behavioral intention to adopt e-learning systems (BISE) sustainably and on their actual use (EUOES). This study benefits universities and higher education institutions by providing insights into enhancing e-learning platforms and integrating technology effectively, as well as by supporting the formulation of sustainable online learning strategies beyond the COVID-19 pandemic.

1. Introduction

The digital transformation in the past decade, after the COVID-19 pandemic, has changed the educational landscape of learning. Advances in Information and Communication Technology (ICT) have led to major changes in all connected areas [1]. The digital revolution, followed by the fourth industrial revolution, redesigned the foundational structure of the educational system. The benefits of the e-learning structure of education improve academic performance and prepare students for new digital areas, based on ITC technologies. E-learning methods of education represent a significant leap in the educational paradigm of flexibility that closely sustains teacher–student relations. The traditional learning process was directly influenced by the situation that caused the COVID-19 pandemic, and employees in the Romanian educational system were forced to accept the acceleration of the digital revolution. Improvements in the technological revolution, particularly in ICT have led to notable societal transformations and effects. ICT has introduced new paradigms, like cloud computing, the Internet of Things (IoT), big data, and blockchain, to enhance sustainability [1]. Education is the main element for the development of a sustainable community. A large number of educational institutions started to use the e-learning platform during the pandemic [2]. Sustainable education is essential for a positive future and to meet resistance to the accelerated changes from a technological, economic, and social point of view. Due to the disruptions created by COVID-19, the urgent need to adapt to e-learning technology has never been more obvious.

The global lockdown of all public institutions and private enterprises during the COVID-19 pandemic accelerated the replacement of traditional learning and communication processes. A study [3] conducted at the European level shows that the crisis generated by COVID-19 has affected and accelerated the process of digitalization and improved educational systems among European Union members. According to UNESCO, COVID-19 affected approximately 1.6 billion students globally [4].

As shown in multiple studies, the pandemic took the Romanian education system to another level; professors and students were forced to accept ITC technology and adapt to the new situation created, i.e., distance education, due to the global crisis. In Romania, e-learning platforms were used by 17% of teachers, a very low level, in the period before COVID-19 [5].

The focus of this research is to analyze the connections between information quality (IQ), system quality (SQ), service quality (SEQ), and quality of life (QL) using a comprehensive framework based on the Technology Acceptance Model (TAM) and information system (IS) performance models. The study delves into how these variables, along with the influencing effects of perceived usefulness (PU) and perceived ease of use (PEOU), impact students’ behavioral intentions to embrace e-learning systems (BISE) and their actual use (EUOES) as a sustainable educational solution in the post-COVID-19 era. Furthermore, this research also evaluates the moderating influence of digital literacy (DL) on the direct relationship between BISE and EUOES.

In line with the objectives of this research, the following key questions are formulated to guide the investigation:

• How do information quality (IQ), system quality (SQ), and service quality (SEQ) influence perceived usefulness (PU) and perceived ease of use (PEOU) in the context of e-learning systems?
• How do perceived usefulness (PU) and perceived ease of use (PEOU) influence students’ behavioral intentions to adopt e-learning systems (BISE)?
• What is the relationship between behavioral intentions to adopt e-learning (BISE) and the actual usage of e-learning systems (EUOES)?
• What role does digital literacy (DL) play as a moderator in the relationship between behavioral intentions to adopt e-learning (BISE) and actual usage (EUOES)?

This research significantly enhances the understanding of e-learning adoption by integrating the DeLone and McLean Information Systems Success Model with the Technology Acceptance Model (TAM), creating a comprehensive framework that addresses previously overlooked interactions among quality factors. By including four distinct dimensions—information quality, system quality, service quality, and quality of life—this study offers a nuanced perspective on their collective impact, effectively bridging gaps in the literature that have treated these elements in isolation. Furthermore, the identification of digital literacy as a moderating factor highlights its growing relevance in how students engage with e-learning systems.

Beyond these theoretical contributions, the study also provides practical recommendations for universities to enhance e-learning environments. By advocating for improvements in information quality and system design, along with strategies to address financial burdens, the research aims to better meet students’ needs in real-time. The emphasis on quality of life as a determinant of e-learning usage encourages ongoing exploration of external factors influencing technology adoption. Additionally, promoting digital literacy programs addresses a critical gap in preparing students for effective participation in e-learning, ensuring that educational institutions are well-equipped to adapt to the evolving landscape of digital education. Furthermore, the adoption of e-learning in Romanian higher education is under-studied, and the COVID-19 pandemic highlighted a lack of local understanding of factors that can affect the natural trend of digitalization in education.

The article is organized into several key sections: it starts with the Theoretical Background and Hypotheses Development, continues with Materials and Methods, followed by Data Analysis. The findings are then discussed in the Results and Discussion section, and the article concludes with Conclusion and Limitations.

2. Theoretical Background and Hypotheses Development

2.1. E-Learning as a Concept

The literature does not provide a generally acceptable definition for the concept of e-learning, the approaches ranging from e-learning being regarded as an “informational system” [6] to being described as a centralized system which allows the coordination of the classroom communication via media integration [7]. Additionally, e-learning as a concept could be described as learning with the help of IT equipment [8]. A similar approach could describe e-learning as a medium of information transfer from the trainer to the student through technology [9,10]. Online learning, commonly known as e-learning, as a concept, represents a term that is widely understood and referred to as the possibility to explore content through a digital device. However, the term has received the most attention owing to the COVID-19 pandemic, when most educational institutions had to adapt and ensure continuity of education [11] as a mandatory response to the pandemic outbreak, and the concept appeared and developed aggressively, together with the Internet itself. Therefore, e-learning has continuously developed and gained space ever since the Internet appeared and useful content was accessible through a digital device connected to the Internet [12].

As the use of the Internet has become highly relevant to the day-to-day life of each individual, providing reliable and secure access to information and data (digital content), a particular interest of educational institutions in e-learning has developed, to enhance educational capabilities and to gain, consequently, competitiveness across the educational sector. The adoption of e-learning as a solution to easily provide access to information represents a vital element of educational quality. Digital capabilities of educational institutions were at the beginning a necessity to stay relevant while, for its beneficiaries, such as students, this represented only an alternative. Because of the COVID-19 outbreak, e-learning has become crucial for all actors involved. A forced unprecedented adoption of e-learning has brought to light the qualities of e-learning capabilities and how they are perceived by students.

2.2. Key Factors in the Adoption of E-Learning

In the literature on the subject of e-learning, a plethora of key elements or factors were identified to be important, or even crucial, to the implementation of such a system, factors that were expected to have a big impact from the perspective of end users. The ease of use and the capacity of the system to meet the requirements of students, or the flexibility of the system, were all found to be important to the perceived usefulness of the e-learning platforms [6,13,14,15]. The settings and tools of such platforms enhance the positive effect reported by the users [7]. In the educational process, the need for constant collaboration between the students and their peers and instructors is high, and the fit between the need for a certain task and the capability of the system could affect perceived usefulness and system use [14].

Information quality was deemed as a fundamental part of the perceived usefulness of the system [6,7], even though the high quality of the informational content was not enough to influence the adoption of e-learning by students, with one possible reason for the phenomenon being the high dependency on e-learning tools encountered by them, which may be limited in particular cases, especially when the informational resources are available only via the e-learning platform [6], the same phenomenon being discovered for teachers as well [8]. On the other hand, the superior quality of information was seen as important in the perceived ease of use and usefulness, with findings showing an increase in participation in e-learning if the informational content was considered as improving one’s knowledge [14]; therefore, a lack of trust in the quality or origin of the information would negatively affect perceived usefulness [16]. The continuous use of e-learning was linked with the quality of information. Students who experienced a platform with relevant, precise, and updated information reported more satisfaction and were likely to continue using e-learning [10].

Regarding the technical quality of the e-learning system, a mixed response was discovered, some studies reporting the technical quality as not important for the beneficiary of the platform [6], contrasting with evidence suggesting a positive relationship between the ease of use and the user engagement [7], or a greater willingness of teachers to keep utilizing e-learning if the platforms are well developed [8]. The service quality of the e-learning platform was a key factor in the perceived satisfaction of the students; however, the same quality does not influence the perceived usefulness or adoption by the beneficiary [6,10,15]. The quality of a platform’s operational service can also determine the continued use of the system [8]. In addition, ease of use of the platform is correlated with perceived usefulness [13].

The instructor’s attitude toward the e-learning platform did not affect the usage rate of the platform [6], but the level of computer literacy did have an effect. Therefore, in the implementation of such platforms, the digital skills of professors must be accounted for by the university [17]. The universities must establish an environment geared toward the implementation of e-learning, providing resources for teachers to prepare them for the transition from traditional teaching to the internet-based approach [17], especially in the early phases of the switch to the e-learning methodology, where an open mentality which takes into account the fact that a more digital education is beneficial might help with the overall success of such a platform [16]. The willingness of students to utilize the platforms has a direct effect on the likeliness of the users learning how to operate the systems available [2,7,16], an effect that can also be seen in learning satisfaction [13]. This could suggest that a natural talent in the use of technology could play a part in the adoption of internet-based learning [8,18], especially given that a previous experience with e-learning in the high school situation was not registered to be impactful on the perceived ease of use or usefulness at the university level [15]. There were cases, as seen in Oman where, even though the infrastructure for e-learning had been developed, the adoption was low [8], suggesting the same natural inclination regarding the use of technology in education.

E-learning platforms have been recognized for their contribution to the growth of sustainable education [19]. E-learning systems have evolved from traditional classroom methods to sophisticated digital platforms, driven by technological advancements. Initially, the evolution of e-learning systems provided access to educational materials, but this was followed by progress toward interactive platforms, which significantly increased student engagement. The pandemic accelerated the adoption and implementation of these digital technologies, highlighting the critical role of e-learning platforms. Video conferencing during the COVID period played a crucial role in maintaining educational continuity, demonstrating that these platforms have great potential in supporting education during crises and beyond [20].

The evolution of e-learning systems, from traditional classroom teaching to the use of sophisticated digital platforms, has led to the development of immersive environments made possible by new information technologies. These advances have expanded the possibilities for immersive learning and interactive experiences, leading to the integration of virtual reality and augmented reality as components of the Metaverse, which offer additional opportunities to transform traditional educational methods into digital ones [21].

The accelerated implementation of e-learning highlighted the importance of technological readiness and adaptability, with educational institutions quickly adopting digital tools and platforms to ensure the continuity of the learning process. According to research, the pandemic accelerated the implementation of the e-learning system [22].

The attitude of the student towards the e-learning platforms affects the level of satisfaction reported, a positive attitude towards e-learning translating into a more satisfied user and an increase in the usage of such platforms [6,7,15,17,23], with student satisfaction with e-learning platforms also being related to the quality of the teacher [6]. This attitude could be altered by the experience of the user; perceived enjoyment was correlated with an increase in the use of e-learning [14], explaining that a platform, in order to be accessed and considered useful, must come with an element of entertainment, a perceived boring system negatively affecting the desire of the student to interact with it. In the aftermath of the COVID-19 e-learning experiment, students from Saudi Arabia started to access and use more digital platforms to expand their knowledge [7], highlighting a change in the mentality of the learners, a mentality which proved to be positively associated with the use of e-learning [16]. A relatively short period, of just one semester, was shown to improve the perceived digital ability of students, but with little changes registered at the highest level of competence [24]; thereby, in order to implement an e-learning element in the educational experience of the students, even in the absence of previous training, a small period of training is necessary to raise the level of self-perceived digital literacy.

The overall experience reported by students was good when e-learning tools were in use, but the lack of human interaction was felt [7], with the performance of the students in modules where e-learning solutions were utilized being similar to those in a conventional educational setting, as seen in the case of Malaysia [13]. If the platform meets the needs of the student, the benefits registered range from better academic results, time saved in the search for informational resources, less strain on the physical educational infrastructure, and even a reduced environmental impact, such as regarding the utilization of paper, the usage of the e-learning platform also raising the digital capability of the students [6,7,17,24].

2.3. E-Learning in Romanian Education

The perceived digital competency of Romanian high school students was high, with all the respondents declaring an average or above level of computer utilization skills and 93% reporting an intermediate or higher level of skill regarding the use of the Microsoft Office package [25]. It must be stated that the results were collected from a high school in the center of Bucharest, the largest and most developed city in Romania. Bucharest was found to be an exception to the typical Romanian county, often being regarded as one of the most developed county clusters in Romania, alongside three others [26], together accounting for less than 10% of all the counties in the country. A different picture was painted by the analysis of rural areas in Romania during the COVID-19 pandemic, the economically challenged community facing a series of obstacles in the implementation of the required e-learning system, such as the lack of internet accessibility, proper internet speed, the slow speed of response from the central authorities or the absence of proper hardware, such as laptops [27,28,29].

The teaching style at the start of the COVID-19 outbreak was not prepared for the shift to online classes, and students from Romanian universities reported a bigger homework load, due to more assigned tasks compared to classical teaching, and an imbalance between theoretical and practical elements of the learning process, leading to a loss of focus [27,28] The adaptation of courses is regarded as the biggest drawback of e-learning as seen by the teachers too [5]. From the perspective of the students, some of the professors were not up to the challenge imposed by e-learning, and teachers lacked the technical skills required for online teaching [27], a lack of competence which could be attributed to an absence of prior engagement with e-learning platforms, as only 17% of teachers had made previous attempts at online teaching [5]

Students were more inclined to consider the traditional way of teaching, face to face, as the most suitable option, e-learning platforms being regarded as an additional tool in the educational process, a tool which could help enhance the classical teaching method but was not viewed as a replacement [27,28,30], 50% of students preferring a text-based type of course [31]. The tendency could be highlighted when attendance in online versus classical format was compared, with students taking on average 3.7 h of courses in the traditional format relative to the 2.43 h reported for an e-learning situation [32], some papers suggesting that only 38% of beneficiaries were considering e-learning as a tool for facilitating the learning process [28].

As seen in the case of the University of Petrosani, the primary catalyst for the adoption and implementation of e-learning systems was the COVID-19 pandemic, with over 70% of the interviewed students reporting the first year of e-learning use being 2020 [32], but the learning curve was steep, reports from Gheorghe Asachi Technical University of Iasi showing that approximately 75% of the students questioned considered themselves having medium or high skills in the use of e-learning platforms in 2022 [28], suggesting that the period of pandemic restriction on the educational system improved the quality of the e-platforms and helped the widespread adoption of virtual learning tools among students [33]. Reports were suggesting that 75% of students were utilizing technology between one and three hours per day with the purpose of gaining new knowledge or to learn, while all of the students were spending at least one hour on electronic devices for purposes other than learning [25]. Additionally, e-learning increased the amount of time spent by children in front of the screen to up to six hours a day, and in the COVID-19 pandemic even subjects such as arts or physical education required access to technology [5,29], some students even reporting an incompatibility between these subjects and the virtual learning environment [28].

The transition to online learning has significantly increased the burden on parents, particularly in the case of younger children, who require additional support to navigate and utilize e-learning platforms effectively [5,29]. Given that, in 2021, approximately 56% of Romanians had a low or absent level of digital skills, and approximately 16% had not used the internet in the last three months, the situation worsened in 2023, when 61% of the population had low or absent digital skills [34], and it is questionable how prepared parents were for the task of helping their children in the transition to e-learning.

The principal element considered by students to be a necessity in the improvement of e-learning platforms was the interface with the IT solution, with improvement in the ways resources were made available on platforms and the communication with teachers being regarded as areas of much-needed development [31]. Furthermore, students were shown to crave a medium for discussion, two-thirds indicating as a solution for improving the efficiency of e-learning the development of blogs or forums designed for constant debate, or as ways for specialists to elaborate on topics [31]. Regarding the availability of resources, nearly all students were happy with the quantity of information provided through the virtual learning platforms [28].

A critical aspect of e-learning that warrants attention is the diminished interaction among students, with reports indicating a notable shift in children’s behavior following the implementation of online instruction [29], with Romanian students reporting a feeling of isolation, a shift observed also by parents [5]. This isolation might increase the stress level felt by students and thus skew the perception of the effectiveness of e-learning, reports suggest that Romanian students who were more stressed perceived smaller usefulness for e-learning during the COVID-19 pandemic [35]. The research is focused on the perception of e-learning as a digital tool for educational sustainability after the pandemic provoked by COVID-19 in the Romanian educational system.

2.4. Research Model and Hypotheses Development

The aim is to construct a multidimensional and integrative model for the adoption of e-learning systems as a sustainable educational solution for students within the Romanian higher education context in the post-COVID-19 period. This model will integrate variables from the DeLone and McLean Information Systems Performance Model [36] alongside those from the Technology Acceptance Model (TAM) [37].

Digital capabilities (DL) and the quality of e-learning can be assessed through the following primary elements: system (characteristic of System Quality—SQ), information (Information Quality—IQ), and service (Service Quality—SEQ). Additionally, it is considered important to assess the level of quality of life (QL), as this can influence how e-learning is perceived by its beneficiaries. This research evaluates how the characteristics of online learning potentially influence perceived usefulness (PU) and perceived ease of use (PEOU).

The integration of the Technology Acceptance Model (TAM), despite its extensive use, remains relevant due to its robustness in explaining technology adoption across various settings. By combining TAM with the DeLone and McLean Information Systems Success Model, this study addresses the evolving dynamics of e-learning, particularly in the post-COVID-19 educational landscape. This approach allows us to capture not only students’ perceptions of ease of use and usefulness but also how system quality, information quality, service quality, and quality of life collectively influence the adoption of e-learning systems.

According to Figure 1, it is proposed that both perceived ease of use and perceived usefulness have a significant impact on the behavioral intention towards sustainable e-learning, which is referred to as BISE. Similarly, it is anticipated that the behavioral intention to utilize the e-learning system will exert a positive influence on actual system usage. This implies that, as users’ intentions to engage with the system intensify, there will be a concomitant increase in the actual utilization of the system. Furthermore, in order to enhance the relevance of the study, the authors are appraising how Digital Literacy (DL) can possibly influence students’ intention towards e-learning usage and the effective use of e-learning, respectively. The question the authors are addressing is “Does a high level of digital literacy involve a sustainable adoption of e-learning and vice versa?”. The logic behind this correlation is to understand whether a higher level of literacy can serve as a cornerstone for the sustainable adoption of e-learning.

2.4.1. Information Quality (IQ)

Information quality is the main characteristic that can be used to describe a system or service output [38]. Quality of information can be defined based on information completeness, time and context relevance, and accuracy [8]. Information quality can be labeled as the most important element while describing a particular system or service, particularly e-learning systems. While system or service quality is highly relevant for any given e-learning environment, neither system nor service quality can compensate for information quality, while vice versa logic also applies. In previous research on e-learning, it has been found that information quality has a significant effect on perceived ease of use [39]. Additionally, a positive relationship was revealed between information quality and the perceived usefulness of an e-learning system [40]. Therefore, the following hypotheses were developed:

Hypothesis 1. 

Information Quality (IQ) positively affects the Perceived Usefulness (PU) of the e-learning system.

Hypothesis 2. 

Information Quality (IQ) positively affects the Perceived Ease of Use (PEOU) of the e-learning system.

2.4.2. System Quality (SQ)

The functionality of an information system, as it pertains to accessibility, dependability, safety measures for users, adaptability, aesthetics, user-friendly design, ease of navigation, download speed, and overall accessibility, is referred to as its system quality [41]. In the realm of electronic learning platforms, the assessment of system quality is a critical topic of interest. As a result, scholars persist in pursuing suitable methods for evaluating the quality of these systems [42].

In this study, five subscales were used to measure this construct: reliability, aesthetics, ease of use, reasonable response time, and support for interactive communication between teachers and students [8,14]. Previous studies have shown that service quality significantly influences the perceived usefulness and ease of use of e-learning systems [43,44]. As a result, the following hypotheses emerged:

Hypothesis 3. 

System Quality (SQ) positively affects the Perceived Usefulness (PU) of the e-learning system.

Hypothesis 4. 

System Quality (SQ) positively affects the Perceived Ease of Use (PEOU) of the e-learning system.

2.4.3. Service Quality (SEQ)

Service quality, along with system and information quality, was considered by Holsapple and Lee-Post (2006) to be a critical factor in the successful design of e-learning systems [45]. In the e-learning service sector, service quality has been conceptualized and operationalized as the extent to which an e-learning platform effectively and successfully facilitates the learning process accurately and reliably by providing systems and IT technical support structures. These include an e-learning site’s ability to provide clear instructions, detailed steps to follow, and online problem-solving manuals [46,47].

Five subscales were employed to measure this construct: clarity and completeness of instructions, availability and effectiveness of online support, IT staff’s ability to understand students’ specific needs and requirements, IT staff’s availability and cooperation, and the quality and promptness of the responses provided by the IT staff [6,10]. The relationship between service quality and perceived usefulness was empirically shown to be significant in the study by Al-Sabawy (2013) [48]. Other research concludes that e-learning service quality can play a crucial role in facilitating user acceptance of e-learning, as a result of the significant positive influence of e-learning platform service quality on both perceived usefulness and ease of use [49].

There is a limited body of research in the existing literature that explores the connections between service quality, perceived usefulness, and ease of use [6]. This situation may be caused by the inclusion of this dimension as a subordinate of system quality in the models. However, some researchers argue that it could be considered an independent variable because of the significant changes in the role of information systems in recent years [50]. Therefore, this leads to the following hypotheses:

Hypothesis 5. 

Service Quality (SEQ) positively affects the Perceived Usefulness (PU) of the e-learning system.

Hypothesis 6. 

Service Quality (SEQ) positively affects the Perceived Ease of Use (PEOU) of the e-learning system.

2.4.4. Quality of Life (QL)

Whilst system and service quality are highly tangible and easy to define, quality of life comes with subjective aspects that are difficult to quantify and measure [51]. Quality of Life key indicators can vary from one academic environment to another and it is highly impacted by demographic, environmental, economic, and social factors that are too complex to be generally streamlined. For the purpose of this study, quality of life was defined based on the following aspects: e-learning platforms allow saving of time and money, professional and personal opportunities, and better subjective interpretation of life quality.

The only research conducted in the field of e-learning in university education that has delved into not only the quality of information, the system, and the service related to e-learning, but has also conceptualized and operationalized the variable of quality of life, is the study carried out by Alqahtani et al. (2022). They demonstrated that the quality-of-life variable exerts a significant positive influence on students’ behavioral intention to use e-learning systems. However, the relationship between quality of life and behavioral intention is complex and mediated through perceived usefulness and ease of use of the system [2]. As a result, the following hypotheses emerged from this research.

Hypothesis 7. 

Quality of Life (QL) positively affects the Perceived Usefulness (PU) of the e-learning system.

Hypothesis 8. 

Quality of Life (QL) positively affects the Perceived Ease of Use (PEOU) of the e-learning system.

2.4.5. Perceived Usefulness

Davis’s definition of perceived usefulness is an integral component in understanding user acceptance of technology: “The degree to which a person believes that using a particular system would enhance his or her job performance” [37] (p. 320). According to Lin et al.’s (2010) findings, perceived usefulness (PU) in the e-learning environment refers to the extent to which users consider e-learning capable of assisting them in accomplishing teaching and learning objectives [52].

The adoption of more innovative and accessible technologies can be attributed to perceived usefulness, identified as a primary determinant of intention [53]. Therefore, the model of the study includes perceived usefulness, as it expects that this factor could positively influence behavioral intention towards sustainable e-learning.

Hypothesis 9. 

Perceived Usefulness (PU) has a positive effect on the Behavioral Intention toward Sustainable E-learning (BISE).

2.4.6. Perceived Ease of Use (PEOU)

Perceived ease of use is defined as a system’s ability to be simple to use [37], enabling, therefore, an individuals’ decision to employ it [54]. The higher the level of perceived ease of use, the more likely is an individual to sustainably adopt e-learning. As a counter-example, e-learning systems should not require high effort to navigate within and display information, otherwise intention to use is believed to decrease.

Numerous studies have substantiated that perceived ease of use is positively correlated with behavioral intention to use through both direct and indirect pathways [49]. Accordingly, the following hypothesis was formulated:

Hypothesis 10. 

Perceived Ease of Use (PEOU) has a positive effect on the Behavioral Intention toward Sustainable E-learning (BISE).

2.4.7. Behavioral Intention Toward Sustainable E-Learning

Behavioral intention represents an individual’s intention to adopt a specific technology for various tasks [55].

In addition, a more complete definition describes behavioral intention as: “a cognitive process of individuals’ readiness to perform specific behavior and is an immediate antecedent of usage behavior” [56]. In the field of technology acceptance, the actual use of technology is determined by users’ behavioral intentions. Individuals choose a particular technology if they intend to use it [57].

A substantial body of research has demonstrated that behavioral intention exerts a direct and significant influence on the actual utilization of e-learning systems. This relationship underscores the critical role that user intentions play in determining the effective adoption and sustained use of e-learning technologies within educational environments [58,59,60]. Salloum et al. also found that behavioral intentions to use have a positive impact on actual use, meaning that, as users’ intentions increase, the actual use of a system will simultaneously increase [61]. Consequently, it is anticipated that the intention to use will have a positive influence on the effective use of e-learning systems in this study. As a result, the following hypothesis was formulated:

Hypothesis 11. 

Behavioral Intention to Sustainable E-learning (BISE) has a positive effect on the Effective Use of the E-learning System (EUOES).

2.4.8. Digital Literacy

A prevailing definition describes digital literacy as: “a set of competencies possessed by an individual to apply digital tools well in the digital era, easily accessing, applying, evaluating, analyzing and synthesizing data, as well as creating new knowledge. With that, students will be able to communicate and present content through various digital technologies” [62].

Digital technologies have profoundly reconfigured culture in various spheres of human existence, including education. Consequently, the rapid development of digital technologies requires adaptation to the level of the educational structures [63]. In this context, the use of digital systems in educational institutions contributes to the formation of digitally literate students who are able to adapt effectively and innovatively to various technological contexts in their lives.

Therefore, it is particularly interesting to analyze the impact of these transformations on students, especially regarding the actual use of digitized systems. Many studies have determined that a high level of digital literacy increases positive attitudes toward a certain technology [64,65,66]. Nevertheless, research that integrates digital literacy, behavioral intention, and the effective use of e-learning systems into a model is limited. Moreover, the moderating effect of digital literacy on the direct relationship between behavioral intention and effective use has been particularly understudied [67].

Hypothesis 12. 

Digital Literacy (DL) has a positive effect on the Effective Use of the E-learning System (EUOES).

Hypothesis 13. 

Digital Literacy (DL) moderates the relationship between Behavioral Intention toward Sustainable E-learning (BISE) and Effective Use of the E-learning System (EUOES).

2.4.9. Effective Use of the E-Learning System

Over time, various academic efforts have been undertaken to define the concept of system usability. Goodhue and Thompson [68] (p. 218) defined it as “the behavior of employing technology in completing tasks”, and Islam [69] (p. 390) describes it as “User perception regarding their amount of e-learning system usage”. In contrast, DeLone and McLean (2003) believe that assessments of use should not only measure the frequency of use but also capture the complexity of use as a system phenomenon, including its nature, level, and appropriateness [36].

In the educational field, the use of e-learning systems gained particular importance with the emergence of the COVID-19 pandemic, as it ensured the continuity of education, provided accessibility and flexibility, reduced the risk of virus transmission, and developed the digital skills of students and teachers [7]. Recent studies have highlighted the significant impact of the COVID-19 pandemic on the use of e-learning systems, recommending that higher education institutions evaluate the e-learning system from the point of view of the quality of the information, service, and system. The purpose of this evaluation is to improve perceptions of usefulness and ease of use, variables that influence the current use of e-learning as a sustainable educational resource [2].

3. Materials and Methods

3.1. Target Population

The experiment from the current research includes bachelor’s degree students from two state/public universities, Bucharest University of Economic Studies and the University of Bucharest, and a private one, Artifex University of Bucharest. The area of study of the Bucharest University of Economic Studies and the Artifex University of Bucharest is economics. By contrast, the University of Bucharest offers a vast range of academic programs, including, in addition to economics, domains such as law, communication, natural, social and educational sciences. As of 2020, as a response to the COVID-19 pandemic, all three universities adopted systems of online teaching, thereby ensuring that all their students acquired the necessary skills to attend courses in digital format.

3.2. Data Collection

The Google Forms survey was used to collect the research data and the questionnaires were distributed by the university professors on the e-learning platform in the seminar or the course section or were displayed on the course slides during class in order to invite students to take part in the survey, while it being expressly specified that taking part in the survey would be carried out purely voluntarily.

In our research, we used a questionnaire, a method already justified by [70]. who confirmed that the questionnaire is the most suitable instrument for acquiring knowledge of the sustainable adoption of e-learning systems, both during and after the pandemic.

The distribution of the questionnaire in the online environment was based on the following reasons: the possibility of sharing it with a high number of students, thereby extending the area of obtained responses, approaching a target group with a heterogeneous geographical location, ensuring the anonymization of the participants, an element which contributes to the authenticity and the honesty of the responses, easy access to the collected data, the compatibility of the data format with the used statistical software, reducing the likelihood of missing data due to the design of the questionnaires, which may enforce filling in all mandatory fields before submission, and saving time for the researchers, as well as channeling the efforts towards other qualitative aspects of the study [71,72].

Data collection took place between 10 April 2024 and 20 May 2024 during the spring semester (2023–2024) in the three universities. The research team carried out the random distribution of 700 questionnaires, with 505 valid responses being obtained, which means that the response rate was 72.14%. The sample size of 505 valid responses falls within adequate parameters given the sample size, as a sample size of 380 responders from a population of 40,000 was anticipated [73]. There is a considerable difference between the sample size (505) and minor requests. Considering this, the sample size is suitable for evaluation by structural equation modeling.

3.3. Data Sample

The questionnaire subjects were bachelor’s degree students from the three Romanian universities given the assumption that they had attended an online flow course at least once (Zoom, Google Classroom, etc.), had accessed pre-recorded videos and course materials which were uploaded on an existing e-learning website [74], or had completed a homework or project-type working assignment which required its uploading to an existing e-learning website.

Table 1. Demographic characteristics.

Demographic Characteristics	Frequency	Percentage
Age-group
Under 18 years	0	0
18–24 years	425	84.2%
25–34 years	47	9.3%
35–44 years	2	0.4%
45–54 years	27	5.3%
Over 55 years	4	0.8%
Gender
Male	196	38.8%
Female	308	61%
Another category	1	0.2%
Type of university
Public	414	82%
Private	91	18%
Form of study
Full-time	497	98.4%
Part-time	5	1%
Distance learning	3	0.6%
Year of study
1st	185	36.6%
2nd	188	37.2%
3rd	120	23.8%
4th	8	1.6%
5th or more	4	0.8%
Frequency of e-learning use per week
Daily	125	24.8%
A couple of times a week	207	41%
Occasionally	118	23.4%
Rarely/never	55	10.9%

shows the demographic features of the responders.

3.4. Survey Development and Structure

The research model of this study was adapted from various theories and some variables that were included. The model embedded a total of nine variables, including four independent variables (IQ, SQ, SEQ, QL), a dependent variable (EUOES), three mediators (PU, PEOU, BISE), and a moderating variable (DL).

Table 2. Conceptual constructs and variables. Source: authors, based on the references mentioned.

Latent Variable	Measurement Items	Sources/ References
Information Quality (IQ)	IQ1: The e-learning system provides information that is relevant to my needs.	[8,14]
	IQ2: The e-learning system provides complete information.
	IQ3: The e-learning system provides updated information and content.
	IQ4: The e-learning system ensures organized content and organized information.
System Quality (SQ)	SQ1: The e-learning system is trustworthy.	[8,14]
	SQ1: The e-learning system is aesthetically satisfying.
	SQ3: The e-learning system is easy to use.
	SQ4: The e-learning system ensures a reasonable response time.
	SQ5: The e-learning system allows for interactive communication between teachers and students.
Service Quality (SEQ)	SEQ1: There are sufficient and clear instructions regarding the use of the e-learning system.	[6,10]
	SEQ2: The e-learning system provides adequate online help and assistance.
	SEQ3: The IT personnel understand the specific needs of the students.
	SEQ4: The IT personnel is available and cooperative when an issue is reported.
	SEQ5: The IT personnel offer a satisfying answer in a timely manner.
Quality of Life (QL)	QL1: Using the e-learning system helps with saving time and money.	[2]
	QL2: Using the e-learning system offers more opportunities to take part in activities.
	QL3: The communication channel with teachers and colleagues from the e-learning system saves me money and effort.
	QL4: Generally, using the e-learning system helps in improving the quality of my life.
Perceived Usefulness (PU)	PU1: Using the e-learning system is an efficient way to quickly access my courses.	[7,13,14,16]
	PU2: Using the e-learning system increases my efficiency when studying.
	PU3: Using the e-learning system improves my performance during class.
	PU4: Using the e-learning system improves my productivity during class.
	PU5: I consider the e-learning system is useful for my studies.
Perceived Ease of Use (PEOU)	PEOU1: I find it easy to learn how to use the e-learning system.	[2,13]
	PEOU2: I find it easy to find the necessary information by using the e-learning system.
	PEOU3: My interaction with the e-learning system is clear and easy to understand.
	PEOU4: I consider that the e-learning system is easy to use.
Behavior Intention toward Sustainable E-learning (BISE)	BISE1: I will regularly use the e-learning system in the future.	[16]
	BISE2: I intend to use the e-learning portal to frequently access the learning services.
	BISE3: I intend to use the e-learning system to interact with colleagues and teachers.
	BISE4: I consider that the way teaching materials are designed in the e-learning system influences the sustainability of my learning.
	BISE5: I generally like to use the e-learning system.
	BISE6: I will strongly recommend to other colleagues as well to use the e-learning system.
Effective Use of the E-learning System (EUOES)	I frequently use the e-learning system.	[2,75]
	I tend to frequently use the e-learning system.
	I spend a lot of time exploring the e-learning system.
	I get very involved in using the e-learning system.
	On average, I use the e-learning system less than three times per week.
	On average, during a day at the university, I spend more than 2 h when using the e-learning system.
Digital Literacy (DL)	DL1: I know how to solve my own technical problems.	[24]
	DL2: I can easily learn new technologies.
	DL3: I keep up with the new significant technologies.
	DL4: I know a lot of different technologies.
	DL5: I have the necessary technical skills in order to use the ITC (Information and communications technology) for learning and creating artifacts (for example, documents, reports, presentations).
	DL6: I have good skills in the area of ITC.
	DL7: I am confident in my ability to search and assess when obtaining information from the Web.
	DL8: ITC allows me to collaborate better with my colleagues when implementing projects and during other learning activities.

shows the sources of these constructs. Researchers changed the questions from previous studies for the research to be applicable. The elements of the questionnaire were examined by a group of four experts, including academic staff and PhD students, with significant experience in educational technology, the elements being analyzed in terms of clarity, brevity, relevance, and relatedness to the problem. The evaluation process showed that the level of agreement between the members of the panel was 90%.

The first section focuses on the personal data of the responders and the data regarding the experience of using the e-learning system. The second section is made of 21 items dealing with the PU of e-learning (PEOU), Behavioral intention regarding the sustainable embracement of e-learning (BISE), and Effective use of the e-learning system (EUOES). There are 18 elements in the third section that signify the quality of the e-learning system (IQ, SQ, SEQ, QL) and the last section includes 8 elements showing the degree of digital literacy (DL). A psychometric Likert scale was used to measure the 47 items. Responders expressed their opinions on a scale of 5 points, from 1 (strong disagreement) to 5 (total agreement), with questions derived from the elements that were mentioned in the proposed model.

4. Data Analysis

The collected data were analyzed by using partial least squares structural equation modeling (PLS–SEM) in order to examine the proposed hypotheses in the research model. PLS-SEM is a structural equation modeling technique that is used for assessing complex models of cause-and-effect relationships with latent variables [76]. Therefore, the PLS–SEM analysis was performed by using the SmartPLS 4.0 software [77]. The software was first used to evaluate the measurement model by testing the reliability of the internal consistency, the convergent validity, and the discriminant validity, and afterward to examine the structural model and the proposed hypotheses.

4.1. Assessment of the Measurement Model (The Outer Model)

The evaluation of the measurement model was performed with the help of convergent validity and discriminant validity [78], these two validities providing evidence that the measurement model is adequate. The role of convergent validity is to show the degree to which the theoretically similar constructs are correlated to each other and the role of the discriminant validity is to show the degree to which there are differences between two constructs.

To establish the convergent validity, the factor loading of the indicator, Cronbach’s alpha, composite reliability (CR), and the average variance extracted (AVE) has to be taken into consideration. As one may notice in

Table 3. Convergent validity and reliability.

		Cronbach’s Alpha	Composite Reliability (rho_a)	Composite Reliability (rho_c)	Average Variance Extracted (AVE)
II.1. PU		0.912	0.914	0.935	0.741
II.1.PU1	0.815
II.1.PU2	0.891
II.1.PU3	0.894
II.1.PU4	0.850
II.1.PU5	0.853
II.2. PEOU		0.927	0.928	0.948	0.821
II.2. PEOU1	0.912
II.2. PEOU2	0.892
II.2. PEOU3	0.909
II.2. PEOU4	0.911
II.4. BISE		0.938	0.941	0.952	0.766
II.4. BISE1	0.898
II.4. BISE2	0.881
II.4. BISE3	0.787
II.4. BISE4	0.868
II.4. BISE5	0.907
II.4. BISE6	0.904
II.5. EUOES		0.873	0.918	0.907	0.631
II.5. EUOES1	0.862
II.5. EUOES2	0.886
II.5. EUOES3	0.882
II.5. EUOES4	0.895
II.5. EUOES5	0.409
II.5. EUOES6	0.718
III.1. IQ		0.906	0.907	0.934	0.781
III.1. IQ1	0.872
III.1. IQ2	0.873
III.1. IQ3	0.899
III.1. IQ4	0.889
III.2. SQ		0.884	0.889	0.915	0.683
III.2. SQ1	0.825
III.2. SQ2	0.825
III.2. SQ3	0.833
III.2. SQ4	0.858
III.2. SQ5	0.789
III.3. SEQ		0.925	0.931	0.943	0.770
III.3. SEQ1	0.838
III.3. SEQ2	0.881
III.3. SEQ3	0.906
III.3. SEQ4	0.886
III.3. SEQ5	0.873
III.4. LQ		0.895	0.897	0.927	0.761
III.4. LQ1	0.874
III.4. LQ2	0.875
III.4. LQ3	0.876
III.4. LQ4	0.864
IV.1. DL		0.936	0.937	0.947	0.692
IV.1. DL1	0.809
IV.1. DL2	0.858
IV.1. DL3	0.840
IV.1. DL4	0.825
IV.1. DL5	0.856
IV.1. DL6	0.850
IV.1. DL7	0.830
IV.1. DL8	0.783

Source: Authors with SmartPls 4 [77].

, all indicator loadings were larger than 0.718, which exceeds 0.70, the acceptability threshold [79]. However, one may consider the weaker outer loadings between 0.40 and 0.70 if they can explain 50% of the AVE [79]. The Cronbach’s alpha values for all constructs also varied between 0.873 and 0.938, which is considered to be adequate [79]. In addition, the composite reliability of all constructs varied between 0.889 and 0.941, which exceeds 0.6, the acceptability threshold [79]. To check the data validity, the convergent measurement of the validity was checked by AVE and all values exceeded the minimal level of 0.5 [76].

Furthermore, the Fornell–Lacker criterion is used to assess the discriminant validity. This method compares the square root of AVE with the correlation of the constructs [76]. The values highlighted in

Table 4. Discriminant validity using Fornell–Larcker Criterion.

	II.1. PU	II.2. PEOU	II.4. BISE	II.5. EUOES	III.1. IQ	III.2. SQ	III.3. SEQ	III.4. LQ	IV.1. DL
II.1. PU	0.861
II.2. PEOU	0.676	0.906
II.4. BISE	0.763	0.639	0.875
II.5. EUOES	0.612	0.517	0.718	0.794
III.1. IQ	0.665	0.600	0.752	0.638	0.884
III.2. SQ	0.651	0.633	0.716	0.619	0.754	0.826
III.3. SEQ	0.558	0.504	0.640	0.582	0.686	0.746	0.877
III.4. LQ	0.721	0.596	0.773	0.682	0.724	0.719	0.651	0.872
IV.1. DL	0.504	0.537	0.492	0.458	0.551	0.542	0.452	0.550	0.832

show that the variance of the latent constructs for our own indicator is larger than that for other latent constructs [80].

The test of the hetero-trait–mono-trait ratio of correlations (HTMT) was performed as another evaluation of the discriminant validity [81]. An HTMT value larger than 0.85 is recommended to show a lack of discriminant validity. In

Table 5. Discriminant validity using hetero-trait–mono-trait ratio (HTMT).

	II.1. PU	II.2. PEOU	II.4. BISE	II.5. EUOES	III.1. IQ	III.2. SQ	III.3. SEQ	III.4. LQ	IV.1. DL
II.1. PU
II.2. PEOU	0.737
II.4. BISE	0.823	0.682
II.5. EUOES	0.656	0.559	0.773
III.1. IQ	0.732	0.652	0.815	0.705
III.2. SQ	0.722	0.692	0.787	0.703	0.840
III.3. SEQ	0.599	0.537	0.684	0.650	0.747	0.821
III.4. LQ	0.797	0.651	0.843	0.764	0.802	0.809	0.711
IV.1. DL	0.543	0.576	0.521	0.505	0.595	0.589	0.476	0.597	0.341

Source: Authors with SmartPls 4 [77].

, all HTMT values are lower than 0.85, which implies that a discriminant validity is present.

4.2. The Evaluation of the Structural Model

First of all, the collinearity symptoms were evaluated by generating the inflation factor of the VIF variance, as seen in

Table 6. Collinearity statistics (VIF).

	VIF
II.1. PU → II.4. BISE	1.843
II.2. PEOU → II.4. BISE	1.843
II.4. BISE → II.5. EUOES	1.320
III.1.IQ → II.1. PU	2.875
III.1.IQ → II.2. PEOU	2.875
III.2. SQ → II.1. PU	3.261
III.2. SQ → II.2. PEOU	3.261
III.3.SEQ → II.1. PU	2.510
III.3.SEQ → II.2. PEOU	2.510
III.4. LQ → II.1. PU	2.519
III.4. LQ → II.2. PEOU	2.519
IV.1. DL → II.5. EUOES	1.432
IV.1. DL × II.4. BISE	1.122

Source: Authors with SmartPls 4 [77].

. The obtained VIF values fall within the limits of the accepted thresholds (VIF < 5) [82]. Consequently, collinearity was not a problem in the data from this research process.

Furthermore, the fit indices of the model were evaluated during this stage. As depicted in

Table 7. Model fit indices.

	SRMR	d_ULS	d_G	Chi-Square	NFI
Saturated model	0.051	2.934	1.231	3731.509	0.835

Source: Authors with SmartPls 4 [77].

, based on [83], all indices are within the supported values, which suggests that the data set adequately matches the proposed model. Taking into consideration our recommendations, we calculated the standardized root mean square residual, which was found to be equal to 0.051, meaning less than 0.08, with a Chi-square of 3731.509 and a Normed Fit Index (NFI) of 0.835.

The bootstrapping procedure (5000 resamples) was used to examine the path coefficients. As mentioned in

Table 8. Results of hypotheses testing.

Hypothesis	Paths	β	T Statistics	p Values	Decision
H1	III.1.IQ → II.1. PU	0.221	2.949	0.003	Supported
H2	III.1.IQ → II.2. PEOU	0.200	2.416	0.016	Supported
H3	III.2. SQ → II.1. PU	0.175	2.940	0.003	Supported
H4	III.2. SQ → II.2. PEOU	0.352	4.473	0.000	Supported
H5	III.3.SEQ → II.1. PU	−0.014	0.274	0.784	Not supported
H6	III.3.SEQ → II.2. PEOU	−0.043	0.709	0.478	Not supported
H7	III.4. LQ → II.1. PU	0.444	7.459	0.000	Supported
H8	III.4. LQ → II.2. PEOU	0.226	3.597	0.000	Supported
H9	II.1. PU → II.4. BISE	0.610	12.197	0.000	Supported
H10	II.2. PEOU → II.4. BISE	0.226	4.100	0.000	Supported
H11	II.4. BISE → II.5. EUOES	0.653	16.628	0.000	Supported
H12	IV.1. DL → II.5. EUOES	0.176	4.505	0.000	Supported
H13	IV.1. DL × II.4. BISE → II.5. EUOES	0.082	3.524	0.000	Supported

Source: Authors with SmartPls 4 [77].

, IQ, SQ, and LQ are key facilitators of PU and PEOU. These results support the H1–H4 hypotheses, H7 and H8, respectively. SEQ is a major inhibitor of PU and PEOU. This result rejects the H5 or H6 hypotheses. The analysis shows that H9 and H10 are accepted because PU (β = 0.610, p value < 0.000) and PEOU (β = 0.226, p value < 0.000) were the main facilitators for BISE, since they had a positive influence on it. Besides, BISE (β = 0.610, p value < 0.000) and DL (β = 0.138, p value < 0.000) were shown to have a positive influence on EUOES, the H11 and H12 hypotheses being accepted.

In addition, IQ, SQ, SEQ, and QL explain 57.4% (R² = 0.574) of the PU variation and 45.6% (R² = 0.456) of the PEOU variation. These explanatory powers are considered to be moderate [78]. Both PU and PEOU explain 61% (R² = 0.610) of the BISE variation, showing a moderate explanatory power [78]. The BISE and DL variables also have the same explanatory power for EUOES (R² = 0.543).

Table 8 summarizes the findings when testing the proposed hypotheses. Results suggest that all the hypotheses were supported, except for H5 and H6. In particular, it was found that SEQ has a negative effect on PU (β = −0.014, p value < 0.784) and PEOU (β = −0.043, p value < 0.478). Even if it was shown that the path coefficients were negative, the lack of statistical significance, proven by the high p values, means that we cannot claim that SEQ negatively influences PU and PEOU.

Finally, the total indirect effects were examined. As shown in

Table 9. Total indirect effects.

Path	β	T Statistics (|O/STDEV|)	p Values
II.1. PU → II.5. EUOES	0.399	9.376	0.000
II.2. PEOU → II.5. EUOES	0.148	3.962	0.000
III.1.IQ → II.4. BISE	0.180	2.963	0.003
III.1.IQ → II.5. EUOES	0.118	2.977	0.003
III.2. SQ → II.4. BISE	0.187	4.118	0.000
III.2. SQ → II.5. EUOES	0.122	3.926	0.000
III.3.SEQ → II.4. BISE	−0.018	0.468	0.640
III.3.SEQ → II.5. EUOES	−0.012	0.469	0.639
III.4. LQ → II.4. BISE	0.322	7.219	0.000
III.4. LQ → II.5. EUOES	0.210	6.033	0.000

Source: Authors with SmartPls 4 [77].

, the total indirect effects of the independent variables on the dependent variables were significant. This suggests that the effects generated by the independent variables are transmitted through the mediator variables in order to influence the dependent variables. For example, PU (β = 0.399, p value < 0.000) has a significant total indirect effect on EUOES through BISE. Such a finding shows that an increase in PU may lead to an improvement in EUOES by strengthening BISE.

5. Results and Discussion

The research findings revealed that all the hypotheses were supported, except for H5 and H6. Specifically, the actual use of e-learning systems (EUOES) is significantly explained by both behavioral intention to use e-learning systems (BISE) and digital literacy (DL). Digital literacy (DL) was found to significantly amplify the effect of BISE on EUOES, playing a crucial role in reinforcing and adjusting this complex relationship. BISE is influenced by perceived usefulness (PU) and perceived ease of use (PEOU), which are in turn affected by information quality (IQ), system quality (SQ), and quality of life (QL). Notably, service quality (SEQ) did not emerge as a significant factor in shaping PU and PEOU, as variations in the perception of the e-learning system’s quality did not result in corresponding changes in the perceptions of its usefulness and ease of use.

Results show that IQ has a positive effect on PU and PEOU, thereby validating the H1 and H2 hypotheses. This implies that the more students perceive the course content and the resources as being complete, updated, organized, and relevant to their needs, the more they consider the platform as being more useful for achieving their educational goals, and experiencing an easy interaction at the same time. The results from previous research as regards the quality of information are varied, e.g., [61,84,85] show that the influence of IQ on PU and PEOU is positive, while [86] notes that the easier to understand and the more complete (IQ) the information offered by the e-learning system, the easier it is perceived to be to use the platform, with no significant direct effect of IQ on PU. This situation can reveal the existence of a high appreciation by the students when comparing the comfort and the simplicity of the immediate use of the e-learning platform to the long-term benefits of its usefulness. It is shown that students at Romanian universities deemed the quality of information from the e-learning system to influence almost equally the perceived usefulness (β = 0.221, p value < 0.003) and the perceived ease of use (β = 0.200, p value < 0.016). This situation can be determined on the one hand by the development of balanced expectations and abilities in the students, regarding the navigation of the online learning platform, as a result of the fast transition to e-learning induced by the COVID-19 pandemic, and on the other hand by the existence of some positive and consistent experiences with the e-learning system, which led to an equal appreciation of the high quality and easily accessible information.

The findings from this research confirm H3 and H4, according to which the quality of the e-learning system is a strong predictor of its ease of use and perceived usefulness. This suggests that the more students perceive the e-learning platform as being trustworthy, aesthetically satisfying, easy to use, prompt, and supporting of interactive communication between students and instructors, the easier to use and the more useful it is considered to be with respect to their needs. These results support the conclusions of previous studies [6,87]. Consequently, the quality of the educational system, including the existence of communication systems and interactivity features, the diversity of learning styles, and the provision of evaluation materials, such as tests and homework, strongly influence the perception of the e-learning system’s usefulness and ease of use, making students more likely to use it.

The research completes the previous results, showing that the students at Romanian universities deemed the quality of the e-learning system as having a higher impact on the perceived ease of use (β = 0.352, p value < 0.000) compared to the perceived usefulness (β = 0.175, p value < 0.003). This conjuncture may be explained by the fact that the ease of use can be more obvious and easier to evaluate during a short period, compared to usefulness. The ease of use also directly influences the performance of students, by their carrying out the academic tasks in an efficient manner, while the accessibility facilitated by several devices can play a role in this perception.

The H5 and H6 hypotheses were not supported (β = −0.014, p value < 0.784; β = −0.043, p value = 0.478); in other words, the quality of the services provided by the IT personnel do not contribute to the student’s perception regarding the e-learning system’s usefulness and ease of use. This result was consistent with several studies, in which researchers did not find any relationship between the quality of the services and their usefulness [6,88,89].

It is obvious that the quality of the service provided by the IT personnel is a catalyzer that can improve the general experience, but this does not directly influence the way in which students perceive the platform’s usefulness and ease of use. Therefore, students consider that a good IT service is beneficial, but not essential for assessing the e-learning platform’s efficiency and convenience, relying more on their direct experience with the platform’s interface and functionalities. The lack of perceiving an impact of the quality of the service offered by the e-learning platform on its usefulness and ease of use may be determined by reduced interaction of students with the IT personnel, due to the optimal functioning of the e-learning system.

Quality of life was included based on previous empirical studies in the area of information systems in order to extend and improve the Technology Acceptance Model (TAM), yet this was not previously considered within the educational context [90]. Recently, the inclusion of the quality-of-life variable in the TAM models has become an element more and more widely researched, taking into consideration the possible benefits that students may obtain as a result of using the e-learning system with the aim of ensuring educational sustainability. Therefore, the results from [2] show that the perceived usefulness and ease of use were positively correlated and influenced by quality of life in education. These discoveries are in line with those of this research, with the H7 and H8 hypotheses being supported. Romanian university students considered that quality of life influences perceived usefulness (β = 0.444, p value < 0.000) and perceived ease of use (β = 0.226, p value < 0.000).

The opinion of Romanian students that quality of life has a high influence on the usefulness of the e-learning platform compared to the ease of use is reflected in the following elements:

• Although the ease of use contributes to saving time by decreasing the learning curve, its impact on the total saved time is less compared to the time that was saved and used by students in other types of activities, such as academic, professional, or personal.
• Although an intuitive system is capable of reducing the costs of training and technical support, these costs are not noticeable by students, who rather notice the reduction of costs that directly affect them, such as transportation, accommodation costs, and the costs of physical educational materials.
• Although simple navigation of the e-learning platform may facilitate an opportunity for student’s participation in courses, the main importance lies in its usefulness, due to the non-stop access, the customized learning rhythm, the variety of available materials, the adaptability to a wide range of devices, and the fast feedback.
• Although the interface of the e-learning platform simplifies communication and collaboration for students and teachers, its role is secondary compared to the interaction facilities made available by the e-learning system (forums, discussion groups, chats) because these ensure an active interaction, which is essential for a complete educational process.

This analysis found a significant relationship between perceived usefulness, the platform’s perceived ease of use, and intentional behavior regarding the sustainable embracing of e-learning among the students prepared by higher education institutions in Romania, thus confirming the H9 and H10 hypotheses. These findings are similar to previous studies [13,61,91], and yet few studies have identified the lack of a relationship between perceived ease of use and intentional behavior regarding the embracement of e-learning [92,93].

In conclusion, perceived usefulness has a stronger influence on the intentional behavior of sustainably embracing e-learning (β = 0.610, p value < 0.000) because it offers tangible and concrete benefits, which are essential for the student’s academic and professional success, while ease of use (β = 0.226, p value < 0.000) is just a secondary factor, which facilitates the initial access to these benefits. The COVID-19 pandemic made a significant contribution to this situation, amplifying the influence of e-learning’s perceived usefulness on the Romanian students’ intention to sustainably embrace this type of system, because it was the only solution to continue their studies. Apart from this continuity, this “solution” provided flexibility, accessibility, and quality for the educational process, but in the long term it decreased the impact of the platform’s perceived ease of use, when considering it to be necessary only for initial use and access.

The study also supported H11, H12, H13. Behavioral intention, the direct and most significant construct, was identified as having a noteworthy association with the actual use by the students of the e-learning system (β = 0.653, p value < 0.000). The findings from this study showed that behavioral intention regarding the sustainable embracement of e-learning had a strong influence, showing an increased use of e-learning among students from the Romanian academic environment. The results were also consistent with previous research [59,94,95,96,97].

Another predictor of the actual use of the e-learning system was digital literacy (β = 0.176, p value < 0.000), thus the improvement in digital skills by students from Romania may lead to more frequent and efficient use of e-learning platforms, taking into consideration that a high degree of literacy allows users to overcome possible technological barriers more easily and to make full use of the available resources and instruments within the e-learning system.

Taking into consideration the positive and significant influence of digital literacy on the actual use of the e-learning system, the moderating effect of digital literacy was investigated in the relationship between behavioral intention regarding the sustainable embracement of e-learning and its actual use. It has been concluded that, although digital literacy intensifies the effect of the behavioral intention on actual use (β = 0.082, p value < 0.000), this effect is subtler. The moderating effect of digital literacy has a role in adjusting the behavioral intention–actual use relationship, but not fundamentally defining it.

This study stands out through its unique approach, including an acceptance model of the technology, the influence of digital literacy on actual use of the e-learning system, and the investigation of the moderating effect of digital literacy on the relationship between behavioral intention and actual use. Individually, there are studies that analyzed only the impact of digital literacy on attitude regarding the use of information and communications technology [98], and a study which places digital literacy between external factors and the actual use of e-learning, as a moderator in a simplistic model made of three constructs, was also identified, proving that this weakens the relationship between the external factors and actual use, although there is a direct and positive relationship between them [67].

5.1. Theoretical Implication

The first contribution of this study revolves around the development of a multidimensional and comprehensive model regarding the embracement of the e-learning system as a sustainable solution for education by students within the Romanian academic environment following the COVID-19 pandemic. The model was developed based on an intensive literature review and by analyzing ten approaches, which include variables of the DeLone and McLean information systems’ performance model and the Technology Acceptance Model (TAM).

Secondly, this study moves forward and offers an empirical investigation of the developed model, embedding the factors that influence the success of e-learning systems. Three types of quality factor (information quality, system quality, quality of life) are considered to be decisive for perceived usefulness, perceived ease of use, behavioral intention regarding the embracement of the e-learning systems, and their actual use.

The current research also investigated new relationships that had not previously been empirically tested, e.g., the relationship between information quality, system quality, service quality, life quality, and perceived usefulness and ease of use, since the previous studies focused only on their relationship with satisfaction. The model also captured the direct influence of digital literacy on the actual use of the e-learning system, but also the moderating effect of digital literacy on the relationship between behavioral intention regarding the embracement and actual use of e-learning.

The fourth contribution emphasizes the performance of the developed model, which explained 57.4%, 45.6%, 61%, and 54.3% of the variance in perceived usefulness, perceived ease of use, behavioral intention to adopt, and actual use of the e-learning system, respectively. In this way, the model significantly contributes to the DeLone and McLean Information Systems Success Model and the Technology Acceptance Model (TAM), while also validating, extending, and applying them within the Romanian academic context.

5.2. Practical Implications

Taking into consideration that the learning management system (LMS) has become an essential part of higher education and that during the last two decades universities have embraced digital platforms to offer online education, this omnipresence of the LMS has facilitated remote, face-to-face, and blended teaching and learning processes and, as a result, universities have invested heavily in the development of online education, positioning the LMS in the center of this investment [99]. The results of the study highlight significant problems and offer recommendations that should be taken into consideration in order to improve perceptions regarding the usefulness, ease of use, behavioral intention regarding embracement, but also the actual use of e-learning systems. The study provides practitioners with several important contributions:

• The results of this study revealed the importance of improving the quality of information from the e-learning platforms and its significant impact on usefulness and ease of use, as perceived by the students. Therefore, it is recommended that higher education institutions should invest in the development and constant updating of courses in order to ensure that students have educational materials that are complete, updated, well-organized, and relevant to their needs. A first step in attaining this stage may be made by the constant collection of feedback from students regarding the content’s quality but also by organizing regular training sessions for the academic staff on topics such as instructional design, multimedia content development, and the use of open educational resources.
• The findings of the research can assist universities and other institutions in recognizing the defining features of an efficient e-learning system, such as being trustworthy, aesthetically satisfying, easy to use, prompt, and supportive for interactive communication between students and instructors. Moreover, they can direct their efforts to maximizing the ease of use by simplifying the users’ interface and navigation experience. In order for students to perceive the usefulness of the e-learning system, the teaching group should be encouraged and trained to efficiently integrate technology during teaching with the aim of increasing the interactivity and attractiveness of online courses.
• Although the study did not identify a direct influence of the quality of the e-learning platform’s service on its usefulness and ease of use, it is recommended that universities maintain a functional and reliable platform for a satisfying educational experience by implementing the following: the regular collection of feedback from students regarding the needs and the challenges of using the system and the implementation of continuous updates to the platform in order to improve user experience; the prevention of technical problems, the provision of fast support for the platform’s functionality, and the reduction of students’ frustration; and a decrease in interaction with the IT personnel by developing detailed guides, video tutorials, and virtual assistants in order to offer automated and instant support to students.
• Quality of life, although rarely included in the information systems’ performance models and in the technology acceptance models, is an essential variable that deserves to be investigated by universities because it has a significant indirect effect on both behavioral intention and actual use of the e-learning platforms but also direct effects on the perception of their usefulness and ease of use. Therefore, it is recommended that universities propose measures to attempt to decrease the students’ financial burden. For example, measures can be implemented in order to facilitate access to online courses in order to reduce the need for travel, accommodation, and their associated costs. The use of digital educational materials can also be promoted, thereby significantly decreasing the costs related to physical materials. Apart from these factors, the continuous accessibility to educational resources allows students to access materials at any time and to study at their own pace.
• Taking into consideration that the perceived usefulness and the perceived ease of use were the most relevant factors in determining the behavioral intention regarding the sustainable embracement of e-learning and that this, in turn, is a significant predictor for the actual use of e-learning, it is recommended that universities, apart from providing flexibility, accessibility, monitoring, and continuous adjustment, promote the benefits of e-learning and make full use of this experience to develop sustainable strategies for embracing e-learning, efficiently integrating online learning in the traditional educational model, given the context in which the COVID-19 pandemics highlighted the dependence on e-learning.
• The results highlighted that digital literacy intensifies the effect of behavioral intention on actual use, playing an essential role in adjusting this relationship. As a result, universities should implement educational resources and programs in order to improve students’ digital literacy; these can include dedicated workshops and courses for developing the necessary technological skills for the efficient use of e-learning platforms.

6. Conclusions and Limitation

This research contributes significantly to the understanding of e-learning adoption as a sustainable solution in the post-COVID-19 Romanian academic environment. By developing and empirically testing a comprehensive model that integrates variables from the DeLone and McLean Information Systems Success Model and the Technology Acceptance Model (TAM), the investigation provides valuable insights into the factors influencing the success of e-learning systems. The inclusion of four quality factors—information quality, system quality, service quality, and quality of life—offers a nuanced understanding of their impact on the perceived usefulness, ease of use, behavioral intention, and actual use of e-learning. Additionally, novel empirical relationships are introduced, particularly the moderating role of digital literacy, and the robustness of the model is highlighted in explaining substantial variances in key e-learning adoption metrics. These findings not only extend existing theoretical frameworks but also offer practical implications for enhancing e-learning systems in higher education.

To summarize, this study offers valuable contributions to improving e-learning adoption in higher education. First, it emphasizes the critical need to enhance the quality of information on e-learning platforms, encouraging universities to consistently update and organize educational content to better meet students’ needs. Second, the findings highlight the importance of designing e-learning systems that are user-friendly, reliable, and aesthetically appealing to ensure interactive communication between students and instructors. Third, although the study did not find a direct link between service quality and perceived usefulness, it recommends that universities focus on maintaining functional platforms by gathering regular student feedback and offering automated support tools. Moreover, the study identifies the significant role that quality of life plays in shaping students’ perceptions and usage of e-learning systems. Universities should address the financial burden by promoting digital educational materials and ensuring continuous access to online resources. Since perceived usefulness and ease of use are key predictors of e-learning adoption, institutions should further promote e-learning as a sustainable educational strategy, especially after the pandemic. Finally, digital literacy was found to significantly enhance the use of e-learning systems. Therefore, universities are encouraged to develop programs to boost students’ technological competencies for effective platform usage.

Although the study results were quite interesting and played an essential role in describing the Romanian academic environment regarding the sustainable embracement of e-learning by students, the study is characterized by a few limitations.

First of all, the conclusions should be approached with caution given that the sample consisted of a limited number of universities. The attitude of students may significantly differ if universities with different study specializations or from various demographic areas were included. The size and type of university as well as differences in politics and strategies may have an impact on the results and may be analyzed in further research through a moderation analysis.

Another limitation lies in the fact that the study was exclusively geared towards students, who may differ in perceptions compared to teachers. Future studies may also focus on comparative analyses between students and instructors.

Thirdly, the research is of a transversal nature and determines the students’ perceptions and intentions for a defined time point. Therefore, the use of a longitudinal survey is recommended because the users’ perceptions and preferences may change as they gain more experience as time goes by.

Lastly, the proposed model explained 57.4% of the variance in perceived usefulness (PU), 45.6% in perceived ease of use (PEOU), 61% in behavioral intention to adopt e-learning (BISE), and 54.3% in the actual use of the e-learning system (EUOES). In other words, 42.6% of PU, 54.4% of PEOU, 39% of BISE, and 45.7% of EUOES were influenced by variables that were not examined in the model. Therefore, there remains room for further investigation into the quality factors influencing the sustainable adoption of e-learning.

Regarding future research directions emerging from the scope and findings of the current paper, several potential approaches can be explored. Firstly, a similar methodology could be applied to high school or even primary levels of education in Romania, which would offer insights into the future cohorts of university students. In addition, such an approach could assess perceived ease of use and perceived benefits at a more detailed level. Secondly, another promising avenue for research involves examining the perspectives of teachers, focusing on their perceptions of e-learning, thereby expanding the current understanding of the digitalization of education.