Impact of Learning in the COVID-19 Era on Academic Outcomes of Undergraduate Psychology Students

Abstract

The COVID-19 pandemic, and the containment measures adopted by the different governments, led to a boom in online education as a necessary response to the crisis posed against the education system worldwide. This study compares the academic performance of students between face-to-face and online modalities in relation to the exceptional situation between the months of March and June 2020. The academic performance in both modalities of a series of subjects taught in the Psychology Degree at the European University of the Atlantic (Santander, Spain) was taken into account. The results show that student performance during the final exam in the online modality is significantly lower than in the face-to-face modality. However, grades from the continuous evaluation activities are significantly higher online, which somehow compensates the overall grade of the course, with no significant difference in the online mode with respect to the face-to-face mode, even though overall performance is higher in the latter. The conditioning factors and explanatory arguments for these results are also discussed.

1. Introduction

The current COVID-19 pandemic has affected education in general and university student education in particular, and, although the digital transformation is not a novel phenomenon, accompanying higher education institutions for some years now [1,2,3], the university system has had to adapt quickly to distance learning as a result of the closure of higher education institutions [4,5].

The urgent need to create learning environments for future graduates has involved decisions and adaptations to meet not only student expectations but also the basic requirements for their necessary academic education [6]. The very terminology used for defining this situation points to categories such as emergency remote teaching [7] or emergency eLearning [8], as well as indicating the very problems associated with adapting online teaching, i.e., lack of faculty and student experience, and difficulties for teaching-learning in an extra-university setting [9].

This situation has contributed to the fact that many academic institutions that were previously reluctant in changing their traditional pedagogical approach have had no choice but to switch completely to the online teaching-learning modality [10]. These circumstances make it possible to situate the planning of different scenarios as an urgent need for academic institutions [11].

The experience of shifting from a face-to-face to a distance learning mode is certainly not new. But while this rapid and unexpected transition has brought several constraints, it also offers opportunities that need to be examined in detail [12,13]. An initial analysis of the literature confirms that a significant number of research conducted before the pandemic had already explored the impact of specific online learning programs, approaches, or resources as tools for improving the effectiveness of subjects related to the teaching-learning process [14,15,16,17], while other authors [18] have qualified that online learning is estimated as a complement toward traditional learning.

For the online teaching-learning process to be effective, it is necessary to learn more about its potential and use. Therefore, it is essential to go beyond emergency online practices and develop distance teaching and learning based on quality criteria resulting from careful planning and appropriate instructional design [2,19]. This period of change implies the need to provide a scientific, evidence-based perspective on what works and what does not, but most importantly, to understand the characteristics, processes, outcomes, and implications of online educational practices [20,21].

According to Arkoful and Abaidoo [22], there are several elements that highlight the possibilities of distance learning. These include efficiency (ease of access to a large amount of information), interaction among students and with the professor (use of discussion forums), inclusiveness (attention to the individual differences of students), and self-learning (the asynchronous form enables each student to study at their own pace). Other authors indicate that previous experience in online education for both teachers and students is a key element to ensure success [23,24].

In a similar line, Kaviyarasi and Balasubramanian [25] highlight the relevance of several factors involved in adequate learning. These are the personal skills and characteristics of both the student and the teacher, the level of learner-teacher interaction, the learning resources available, and the external environmental influences on the student. Similar results are shown by del Arco et al. [19], where quality online teaching is associated with adequate interactions between teachers and students (speed of teacher response and promotion of active student participation), as well as with the quality of the material resources offered (motivational resources, innovative and easily accessible).

For his part, Dawhan [10] also indicates that the online learning mode is easily accessible and can even reach rural and remote areas (accessibility). It considers it a mode of education with a lower cost of transportation, lodging, and overall institutional learning expenses (affordability). The student can schedule or plan their time to complete the available courses (flexibility). The combination of face-to-face lectures with technology results in blended learning and flipped classrooms (learning pedagogy). Lastly, students can learn anytime and anywhere, thus developing new skills in the process leading to lifelong learning (lifelong learning).

Despite the advantages that distance education seems to have, other authors [5,17,18,19,20,21,22,23,24,25,26] mention some drawbacks at the technical level ranging from download errors, installation problems, login, audio, and video problems, etc.

On the other hand, the loss of motivation toward studying during the pandemic is a well-documented phenomenon [27]. In this regard, some students find that online teaching can be monotonous and unengaging, the formative contents too theoretical or poorly elaborated, making it difficult for students to learn effectively [28,29] and others indicate that they do not find the right time to engage in education [30]. Personal attention is also a problem faced by this learning modality, as many students require two-way interaction which is sometimes difficult to implement. In addition, some students reported health problems and mentioned difficulties with vision and headaches after online study [31].

Teachers, on the other hand, were ambivalent about the change in the teaching modality, since they mentioned, among other characteristics, the depersonalization of teaching, becoming concerned over student academic success, and the need for technical support. The online approach must be properly planned and regulated and provide training to teachers on how to promote student motivation in this modality [32].

But it is not only the pandemic that has triggered the need in transforming face-to-face studies into online studies. Ayebi-Arthur [33] conducted a case study of a university in New Zealand that was severely affected by seismic activities. In his study, he found that the university became more resilient to online learning after this situation, and technology helped them overcome difficulties in that compromised situation. Therefore, it is important that institutions and organizations prepare contingency plans for facing challenges such as pandemics or natural disasters, among others [34] (Seville et al., 2012).

Along these lines, Popa et al. [35] assessed how educational agents relate to online teaching in crisis situations and explored opinions among teachers and students on the advantages, disadvantages, vulnerabilities, and aspects that can be improved. They observed that when this move to online education was initiated, many aspects were considered non-functional by the respondents (platforms, communication, and teacher-student interaction), with the participants mostly showing attitudes of rejection. This may be due to the extreme novelty nature of some of them, with the coping modalities closely related to the expressed adaptability. The results showed that only those who showed a high level of adaptability can seek help in developing digital competencies, by which it is necessary to review teacher training, incorporating a much more elaborate exercise of digital skills and online teaching.

For his part, Bao [36] proposes six educational strategies in improving student concentration and engagement in learning to achieve a smooth transition toward online learning. First, carry out emergency preparedness plans for unexpected problems (e.g., server overloads); break teaching content into smaller units to help students focus; teach faculty the use of their voice to enable students to grasp key knowledge points; work with teacher aides and receive online support from them; strengthen offline student active learning ability effectively.

Evaluative research becomes particularly important at times when program development conditions are affected by external conditions that force significant change. This involves analyzing quantitative and qualitative information from multiple and flexible research methodologies [37], that enable identifying elements for improvement. Academic performance or successful achievement is a fundamental indicator to determine permanence within the university system [38], and constitutes a quantitative measure of great application to empirically contrast results in different subjects and programs.

For all these reasons, the need for a holistic perspective on online education pedagogy seems obvious. This perspective should integrate technology as an inherent part of the online teaching and learning process and take into account the pedagogical possibilities associated with online tools, i.e., the appropriateness and possibilities of online resources (e.g., blogs promote discussions and facilitate reflective practice; videos help develop practical knowledge about the profession and connect educational theory to practice; forums enable sharing narratives that enhance comprehension skills; chat rooms foster belonging, high levels of support and cooperation that enhance comprehension skills; and chat rooms help develop practical knowledge about the profession and connect educational theory to practice; videos help develop practical knowledge about the profession and connect educational theory with practice; forums allow sharing narratives that enhance comprehension skills; chat rooms foster belonging, high levels of support and cooperation among group members) to ensure that the chosen tools or resources help learners achieve the desired outcomes [39].

Furthermore, this comprehensive perspective of online education pedagogy must recognize the complex interactions between the components involved in the teaching and learning process (teachers, students, task characteristics, and technology itself), including the prior experience of teachers and students with online teaching and learning tools, so as to develop effective educational experiences that address their different and changing roles [40,41].

Likewise, within the current world scenario in which the importance of designing and implementing sustainable education programs (both at the undergraduate and graduate levels) is becoming more and more evident, and which underlies the intention to expand educational options transcending the barriers of time and space, research that enables optimizing the procedures, methods, and technological supports of online/virtual education to overcome the difficulties or obstacles that it entails, acquires great importance.

From the aforementioned, the following research questions are raised:

Have the academic results of the students been affected by the implementation of the online study modality in the face of the crisis derived from COVID-19?

What guidelines could be used to improve online teaching based on the results of the comparison between both study modalities?

The present study is aimed at contrasting the effectiveness of the emergent design carried out at the European University of the Atlantic facing the situation derived from the COVID-19, and making a comparative analysis of the academic results of the program prior to the pandemic, and the program that faced this challenge.

2. Methodology

2.1. Sample

The sample took into account a total of 809 (n = 809) overall grades obtained by students at the European University of the Atlantic enrolled in the Psychology Degree, of which 444 are grades were from the face-to-face mode and 365 from the online mode. The grades divided by subjects are n = 113 for the face-to-face modality and n = 82 for the online modality in the Developmental Psychology subject (1st year); n = 122 for the face-to-face modality and n = 93 for the online modality in the Cognitive Processes II subject; Thought and Language (2nd year); n = 119 for the face-to-face modality and n = 83 for the online modality in the Social Psychology subject (2nd year); and n = 90 for the face-to-face modality and n = 107 for the online modality in the Organizational Psychology subject (3rd year).

2.2. Variables

The type of modality is considered as independent variables; face-to-face modality and online modality.

As dependent variables, academic performance is considered for the aforementioned subjects according to 4 grading parameters: the overall grade for each of the subjects, the grade of the final exam, and a series of evaluable activities throughout the subject classified in two blocks as continuous evaluation activities 1 and continuous evaluation activities 2.

2.3. Procedure

To carry out the study, the procedure was designed with the following characteristics: a series of subjects were chosen from the Psychology degree offered by the European University of the Atlantic, which were taken in-person during the 18/19 academic year and which, due to restrictions imposed by the Government of Spain during the COVID-19 pandemic, would be taken online during the 19/20 academic year.

The subjects selected were the 1st year subject of Developmental Psychology, the 2nd year subjects of Social Psychology, and Cognitive Processes II: Thought and Language, and the 3rd year subject of Organizational Psychology. The analysis was carried out considering the different moments of the evaluation and the final results.

The criteria for the selection of these subjects were for them to have been taught by the same professors in both years and that their contents, the exam, and continuous evaluation activities 1 and 2 had to be exactly the same for each subject for both the online and face-to-face modalities.

The only difference between the two modalities is that students were subject to a class attendance schedule in the face-to-face modality and the contents of the subject were carried out in a face-to-face manner, with direct interaction with the professors. In the online modality, students were not instructed to attend the subject at a specific time and the contents taught by the professor were carried out through previously recorded videos by the professors and uploaded to the Moodle platform. The professor–student interaction took place in specific forums for each subject within the Moodle platform and through virtual tutorials that were carried out synchronously using the Google Meet tool. The Questions and Answers Forum for each subject was also set up to answer operational questions related to the Virtual Campus, its use and management, or specific questions about content that could be solved in a more specific manner.

During the 2018–2019 academic year, the teaching and learning process for these subjects was developed in a prominently face-to-face setting. Very little use was made of educational resources within the virtual environment. The platform, which had been accessible since then although with fewer functionalities, was used as a mere repository of content to be studied, with links to external educational videos and storage of deliverables for some activities.

Likewise, these subjects began to be taught during the 2019–2020 academic year, which in its initial stages was just another classroom subject. After the confinement resulting from the pandemic, it became necessary to redirect the teaching process towards a virtual learning environment. The virtual campus platform, based on a Content Management System, was redesigned, expanding its potential to respond to the current emergency. The flexibility of the platform made it easier for each professor to find their own way of designing and prioritizing the use of the educational resources built for the acquisition of the learning outcomes for their subject. The final evaluation of the subjects was carried out by means of an exam in a classroom environment and continuous evaluation within a virtual environment.

The learning process during the mandatory virtual environment meant a transit through three stages (familiarization, deepening and evaluation) for each topic learned within the virtual learning environment. A specific combination of teaching and learning resources was developed, with some variations, for each of the stages as shown in

Table 1. Stages of application of teaching-learning resources.

Stage 1 Familiarization	Stage 2 Deepening	Stage 3 Evaluation
Reading	Development of virtual tutorials	Case study activities
Viewing prerecorded classes	Question and Answer Forum	Theoretical and practical work
Individual study	Self-assessment exercises	Forum Activities

Note: Own source.

:

Once these stages had been completed and the educational and evaluative activities corresponding to each subject had been concluded, the academic results in each case were subjected to statistical processing, comparing them with the results of the previous academic year.

2.4. Statistics

The SPSS 26.0 statistical package was used for data analysis of academic performance in the selected subjects. An intergroup factor was used as an independent variable, which was the type of modality in which the subject was taught (online or face-to-face).

The dependent variables used were the grades obtained in the subjects selected by the students in both modalities, according to four grading levels: the overall grade of each subject, the final grade in the examination of each subject, and the continuous development activities 1 and 2, respectively, in each of them.

Prior to the analysis of the variation of means, a nonparametric analysis was performed using the Kolmogorov–Smirnov (K–S) test, with the aim of analyzing the type of distribution, since the sample varied in number in terms of the online and face-to-face modalities. The bilateral significance of the K–S test indicates that our sample does not follow a normal distribution.

Therefore, the Kruskal–Wallis (K–W) test was used for comparisons of several means, which is considered the nonparametric equivalent of ANOVA.

Since there were only two groups according to the independent variables, no post hoc studies were carried out.

3. Results

The K–S test analysis for the total sample of grades provided the values (Z = 4.180) in the sample selected for the analysis of the overall final performance of the chosen subjects with a bilateral significance (bilateral sig. = 0.000), (Z = 1.700) in the sample for the analysis of the performance the exam grade of the selected subjects with a high bilateral significance (bilateral sig. 0.006), (Z = 2.244) in the sample for the performance analysis of the continuous evaluation activities 1 with a high bilateral significance (bilateral sig. = 0.000) and (Z = 4.199) in the sample for the performance analysis of the continuous evaluation activities 2 with a bilateral significance (bilateral sig. = 0.000) indicating that the sample selected for the analysis does not follow a normal distribution.

The overall performance for the selected subjects showed a significant main effect for the total performance of the subjects in terms of the grade obtained in the final exam in the test for the K–W analysis (H = 448.10 for the face-to-face modality and H = 352.58 for the online modality), Figure 1, where the interaction MODALITY × EXAM is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is highly significant (X² = 33.486; Mean difference = 0.654; p < 0.001) being favorable for the face-to-face mode. In the inter-subjects analysis, it was obtained R squared = 0.187 (corrected R squared = 0.092), with an observed power = 1.000.

There is no significant main effect for their total performance in terms of the grade obtained in the continuous evaluation activities 1 in the test for the K–W analysis (H = 392.25 for the face-to-face modality and H = 420.51 for the online modality), Figure 1, where the interaction is illustrated MODALITY × CONTINUOUS EVALUATION ACTIVITIES 1. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is not significant (X² = 2.993 mean difference = 0.336, p > 0.05), being favorable for the online mode. As the interaction is not significant, no studies of inter-subject effects are carried out.

A significant main effect was detected for their total performance in terms of the grade obtained in the continuous evaluation activities 1 and 2 in the test for the K–W analysis (H = 359.96 for the face-to-face modality and H = 459.76 for the online modality), Figure 1, where the interaction MODALITY × CONTINUOUS EVALUATION ACTIVITIES 2 is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is highly significant (X² = 36.687; mean difference = 0.772; p < 0.001), being favorable for the online mode. In the inter-subjects analysis, it was obtained R squared = 0.574 (corrected R squared = 0.530), with an observed power = 1.000.

However, there is no significant main effect for their overall performance in terms of the overall grade obtained in the final score of all selected subjects 1 in the test for the K–W analysis (H = 415.43 for the face-to-face mode and H = 392.32 for the online mode), Figure 1, where the interaction MODALITY × FINAL GRADE is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is not significant (X² = 1.968; mean difference = 0.029; p > 0.05), being favorable for the FACE-TO-FACE mode (Figure 1). As the interaction is not significant, no studies of inter-subject effects are carried out.

The following results were obtained in the individualized analysis for each subject in isolation:

The analysis of the K–S test for the analysis of the grades obtained in the Developmental Psychology subject indicated, as in the previous sample, that the sample selected for the analysis of the Developmental Psychology subject does not follow a normal distribution for the parameters final grade of the subject, continuous evaluation activities 1 and 2, but it does follow a normal distribution for the grade of the subject exam (Z = 1.135; bilateral sig. = 0, 152), Sig. > 0.05.

For the overall performance in the interaction MODALITY × EXAM for the subject of Developmental Psychology, it was decided to perform a single factor ANOVA since the distribution of grades of the same is normal, which showed a significant main effect for the total performance of the same in terms of the grade obtained in the final exam F (1, 193) = 21.680, p < 0.001. Figure 2, where the interaction MODALITY × EXAM of the subject is illustrated. The paired analysis of this interaction showed that the difference between the ONLINE modality and the FACE-TO-FACE modality is highly significant (mean difference = 1.619 p < 0.001), being favorable for the face-to-face modality. In the inter-subjects analysis, it was obtained with an R-squared = 0.453 (corrected R squared = 0.130), with an observed power = 0.998.

There is a significant main effect for the total performance of the same in terms of the grade obtained in the continuous evaluation activities 1 in the test for the K–W analysis (H = 70.05 for the face-to-face modality and H = 136.51 for the online modality), Figure 2, where the interaction MODALITY × CONTINUOUS EVALUATION ACTIVITIES 1 is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is highly significant (X² = 14.862; mean difference = 2.758; p < 0.001), being favorable for the online mode. In the inter-subject analysis, it was obtained as R squared = 0.745 (corrected R squared = 0.634), with an observed power = 1.000.

A significant main effect is also detected for the total performance in terms of the grade obtained in the continuous evaluation activities 2 for that subject in the test for K–W analysis (H = 84.77 for the face-to-face modality and H = 116.23 for the online modality), Figure 2, where the interaction MODALITY × CONTINUOUS EVALUATION ACTIVITIES 2 is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is highly significant (X² = 65.982; Mean difference = 0.872; p < 0.001), being favorable for the online mode. in the inter-subjects analysis, it was obtained R squared = 0.574 (corrected R squared = 0.530), with an observed power = 1.000.

However, there is no significant main effect for their overall performance in terms of the overall grade obtained in the final score of the Developmental Psychology subject in the test for the K–W analysis (H = 94.54 for the face-to-face modality and H = 102.76 for the online modality), Figure 2, where the interaction MODALITY × FINAL GRADE is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is not significant (X² = 1.009; mean difference = 0.205; p > 0.05), being favorable for the online mode (Figure 2). As the interaction is not significant, no studies of inter-subject effects are carried out.

The analysis of the K–S test for the analysis of the grades obtained in the Thinking and Language subject indicated, as in the previous sample, that the sample selected for the analysis of the Developmental Psychology subject does not follow a normal distribution for the parameters final grade of the subject, continuous evaluation activities 1 and 2, but it does follow a normal distribution for the grade of the subject exam (Z = 0.982; bilateral sig. = 0.290), sig. > 0.05.

For the overall performance in the MODALITY × EXAM interaction for the Thinking and Language subject, a one-factor ANOVA was performed since the distribution of grades of the same is normal and whose performance showed a significant main effect in terms of the grade obtained in the final test F (1, 213) = 23.822 p < 0.001. Figure 3, where the interaction MODALITY × EXAM is illustrated. The paired analysis of this interaction showed that the difference between the ONLINE modality and the FACE-TO-FACE modality is highly significant (Mean difference = 1.139 p < 0.001), being favorable for the face-to-face modality. In the inter-subjects analysis, it was obtained R squared = 0.536 (corrected R squared = 0.375), with an observed power = 1.000.

There is a significant main effect for the total performance of the same in terms of the grade obtained in the continuous evaluation activities 1 in the test for the K–W analysis (H = 143.50 for the face-to-face modality and H = 61.41 for the online modality), Figure 3, where the interaction MODALITY × CONTINUOUS EVALUATION ACTIVITIES 1 is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE modalities is highly significant (X² = 92.715; mean difference = 0.49; p < 0.001), being favorable for the face-to-face modality. In the inter-subjects analysis, it was obtained R squared = 0.845 (corrected R squared = 0.816), with an observed power = 1.000.

There is however no significant main effect for total performance in terms of the grade obtained in the continuous evaluation activities 2 for that subject in the test for K–W analysis (H = 109.19 for the face-to-face modality and H = 106.44 for the online modality), Figure 3, where the interaction MODALITY × CONTINUOUS EVALUATION ACTIVITIES 2 is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is highly significant (X² = 0.747; Mean difference = 0.285 p < 0.05), being favorable for the FACE-TO-FACE mode. In the inter-subjects analysis, it was obtained squared = 0.468 (corrected R squared = 0.374), with an observed power = 1.000.

However, there is no significant main effect for their overall performance in terms of the overall grade obtained in the final score of the Developmental Psychology subject in the test for the K–W analysis (H = 134.54 for the face-to-face modality and H = 73.33 for the online modality), Figure 3, where the interaction MODALITY × FINAL GRADE is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE modalities is highly significant (X² = 50.951; mean difference = 0.826; p < 0.001), being favorable for the face-to-face modality (Figure 3). In the inter-subjects analysis, it was obtained squared = 0.467 (corrected R squared = 0.321), with an observed power = 1.000.

The analysis of the K–S test for the analysis of the grades obtained in the Social Psychology subject indicated, as in the previous sample, that the sample selected for the analysis of the Developmental Psychology subject does not follow a normal distribution for the parameters final grade of the subject, continuous evaluation activities 1 and 2, but does follow a normal distribution for the grade of the subject exam (Z = 1.343; bilateral sig. = 0.054), Sig. > 0.05. Since the significance is minimal and is very close to the limit in this case, the variation of the means in this parameter was studied using a nonparametric test.

The overall performance for the Social Psychology subject showed a significant main effect for the total performance of the subjects in terms of the grade obtained in the final exam in the test for the K–W analysis (H = 102.74 for the face-to-face modality and H = 99.73 for the online modality), Figure 4, where the interaction MODALITY × EXAM is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is significant (X² = 4.890; Mean difference = 0.502; p < 0.05), being favorable for the FACE-TO-FACE mode. In the inter-subjects analysis, it was obtained squared = 0.332 (corrected R squared = 0.048), with an observed power = 0.983.

There is also a significant main effect for total performance in terms of the grade obtained in the continuous evaluation activities 1 in the test for the K–W analysis (H = 93.17 for the face-to-face modality and H = 113.44 for the online modality), Figure 4, where the interaction MODALITY × CONTINUOUS EVALUATION ACTIVITIES 1 is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is significant (X² = 5.882; mean difference = 0.49; p > 0.05), being favorable for the online mode. In the inter-subjects analysis, it was obtained squared = 0.355 (corrected R squared = 0.073), with an observed power = 0.991.

No significant main effect is detected for their total performance in terms of the grade obtained in the continuous evaluation activities 2 in the test for the K–W analysis (H = 100.44 for the face-to-face modality and H = 103.02 for the online modality), Figure 4, where the interaction MODALITY × CONTINUOUS EVALUATION ACTIVITIES 2 is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is not significant (X² = 0.096; Mean difference = 0.063; p > 0.05), being favorable for the online mode. As the interaction is not significant, no studies of inter-subject effects are carried out.

There is also no significant main effect for their overall performance in terms of the overall grade obtained in the final subject score selected in the test for the K–W analysis (H = 102.74 for the face-to-face modality and H = 99.73 for the online modality), Figure 4, where the interaction MODALITY × FINAL GRADE is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is not significant (X² = 0.130; Mean difference = 0.145; p > 0.05), being favorable for the FACE-TO-FACE mode (Figure 4). As the interaction is not significant, no studies of inter-subject effects are carried out.

The analysis of the K–S test for the analysis of the grades obtained in the Organizational Psychology subject indicated that the sample selected for the analysis of the subject does not follow a normal distribution for the parameters final grade of the subject, continuous evaluation activities 1 and 2, but it does follow a normal distribution for the grade of the subject exam (Z = 0.741; bilateral sig. = 0.643), sig. > 0.05.

The overall performance for the Organizational Psychology subject follows a normal distribution, we proceed to analyze a one-factor ANOVA which showed no significant main effect for performance in terms of the grade obtained in the final exam F (1, 195) = 3, 851 p > 0.05. Figure 5, where the interaction MODALITY × EXAM is illustrated. The paired analysis of this interaction showed that the difference between the ONLINE modality and the FACE-TO-FACE modality is not significant (Mean difference = 0.537 p > 0.05) being favorable for the online modality. As the interaction is not significant, no studies of inter-subject effects are carried out.

There is also a significant main effect for total performance in terms of the grade obtained in the continuous evaluation activities 1 in the test for the K–W analysis (H = 109.14 for the face-to-face modality and H = 90.47 for the online modality), Figure 5, where the interaction MODALITY × CONTINUOUS EVALUATION ACTIVITIES 1 is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is significant (X² = 5.949; Mean difference = 0.417; p < 0.05), being favorable for the FACE-TO-FACE mode. In the inter-subjects analysis, it was obtained squared = 0.346 (corrected R squared = 0.157), with an observed power = 0.999.

A significant main effect is also detected for their total performance in terms of the grade obtained in the continuous evaluation activities 2 in the test for the K–W analysis (H = 64.27 for the face-to-face modality and H = 128.61 for the online modality), Figure 5, where the interaction MODALITY × CONTINUOUS EVALUATION ACTIVITIES 2 is illustrated. The Chi-square analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is highly significant (X² = 61.829; mean difference = 1.853; p < 0.001), being favorable for the online mode. In the inter-subjects analysis, it was obtained squared = 0.843 (corrected R squared = 0.790), with an observed power = 1.000.

There is a significant main effect for total grade performance in terms of the overall grade obtained in the final subject score in the test for the K–W analysis (H = 87.23 for the face-to-face mode and H = 108.90 for the online mode), Figure 5, where the interaction MODALITY × FINAL GRADE is illustrated. The Chi-square contrast analysis of this interaction showed that the difference between the ONLINE and FACE-TO-FACE mode is quite significant (X² = 7.065; Mean difference = 0.625; p < 0.01), being favorable for the online mode in the case of this subject (Figure 5). In the inter-subjects analysis, it was obtained squared = 0.261 (corrected R squared = 0.021), with an observed power = 0.953.

4. Discussion

The virtual learning environment involves web-based educational instruction that does not require face-to-face classes [42]. Although synchronous interactions take place, it has the advantage that an important part of interactions can be asynchronous, overcoming the spatio-temporal restrictions of a face-to-face setting. These benefits also entail certain costs. The virtual learning environment, as it is structured with greater flexibility, requires more capacity for self-organization and time management on the part of the learner. The change could then bring about a challenge in terms of coping with the stress resulting from the new demands, which could in turn influence academic performance.

Regarding the results obtained, it should be noted that the differences in academic results contrasted with respect to one and the other form of continuous evaluation (face-to-face and virtual) are associated with specific differentiating elements. Compared to the face-to-face learning environment, continuous evaluation within the virtual environment as defined during confinement was characterized by:

• Shorter time lag between acquisition and evaluation for each subject.
• Greater flexibility and autonomy in their completion (access to reference texts and other resources while answering evaluative activities).
• Increased time frame for implementation and delivery.
• Increased access to personalized interactions with the professor during the time defined submitting the evaluative activities (subject to the previous point).
• Increased complexity or creative development of the tasks indicated (as a necessary compensation to the characteristics explained in points 2 and 4).

Another explanatory element would be the relationship between the educational activities or resources used and the forms of evaluation. Considering Bloom’s revised taxonomy [43], the following fundamental forms of knowledge can be considered: remembering, understanding, applying, analyzing, evaluating, and creating. In the virtual learning environment defined based on confinement, the teaching resources stimulated learning strategies that encourage application, analysis, and creation to a greater extent, which is congruent with the continuous evaluation instruments defined in these circumstances (case studies, discussion forum, and theoretical-practical work). This would explain the better results obtained in the continuous evaluation activities in most of the subjects. In the specific case of the Thought and Language subject, the results obtained in the CONTINUOUS EVALUATION 2 (highly significant difference with respect to the face-to-face modality X² = 0.747; Mean difference = 0.285 p < 0.05) would be explained from the following elements:

-

This is a subject with a high degree of abstraction from the theoretical point of view, so in this case, it is essential to pay attention to professor-student interactions and the professor’s intervention in the teaching-learning process.

-

The CONTINUOUS EVALUATION ACTIVITIES 2 were developed individually. Their complexity is compensated in the classroom environment thanks to the intervention of the professors in a systematic and individualized way. In the case of the online modality, this intervention (in the case of synchronous interactions) takes place based on the demands of the students, since participation in tutorials is voluntary, reducing the possibilities of the subject’s understanding, reproduction, and application of the contents.

-

In the case of CONTINUOUS EVALUATION 1, it is about teamwork, so in this case, the 5 points previously developed would be applied as argumentative elements.

In the case of the final evaluation, we note lower academic results for the 2019–2020 academic year. It is worth mentioning that there were no differences in the exams applied from one year to the other, since it was possible to return to the University to take the exams in person after the confinement. The final exams were carried out in the usual manner that they are in Spanish universities, that is, they included multiple-choice questions that demand greater memory and comprehension skills, focusing on remembering and understanding types of knowledge [44]. It could then be considered that the differences accentuated within the virtual learning environment with respect to better results in continuous evaluation and worse in final evaluation at the time of confinement can be explained by the relationship (congruent in the case of continuous evaluation and not congruent in the case of final evaluation) between the learning strategies promoted and the evaluation instruments used. In addition, this is strengthened by the fact that in the 2018–2019 year, which was entirely face-to-face, a part of the continuous evaluation used partial exams with similar characteristics to those of the final evaluation as an instrument. The students then had an assessment experience that could reorient their learning strategies for facing the final evaluation. This experience was not possible in the case of the 2019–2020 year, where for the continuous evaluation, the instruments were totally different from the one used for the final evaluation.

In the case we are dealing with interaction in the 2018–2019 academic year, interactions between students and professors took place in traditional contexts (face-to-face classroom spaces and individual/group tutorials also of a face-to-face nature). For the 2019–2020 academic year, synchronous and asynchronous modalities were integrated into these interactions. On the one hand, students had pre-recorded classes where the more complex topics were addressed and, on the other hand, virtual tutoring spaces were enabled in real-time in which questions could be answered, certain topics could be illustrated more extensively, etc. These tutorials were group-based, which greatly enriched interactions among students and with the professor.

For the learning resources available, there were also basically similar resources: reading recommendations, reading materials (basic original textbooks of the subjects), as well as links to external open-access resources, both bibliographic and in video format. The fundamental difference lay in the availability of recorded classes, which certainly offers more possibilities in terms of understanding the different contents, given the functionality of having videos that can be watched several times and paused to be analyzed in more detail, etc.

Regarding environmental influences external to the student, important variations can be defined, many derived from the exceptional situation of confinement. Compared to the previous year, students had more free time that they could use in developing their activities. But at the same time, during this period different reactions associated with the confinement due to the pandemic were described, which could hinder the different cognitive and affective capacities necessary for studying. In this sense, fears associated with contagion/disease/death, social isolation, and work/income problems were described [45]. Elevated levels of emotional impact from fears over the coronavirus, sleeping problems, and emotional symptoms (worrying, stress, hopelessness, depression, anxiety, nervousness, and restlessness) were also found [45]. Likewise, an increase in psychosocial risk factors such as violence, overcrowding, abuse of new technologies, and isolation were described in the case of the infant-youth population [45], all of which could influence teaching performance.

The present study highlighted the elements highlighted by Arkoful and Abaidoo [22] as facilitators of distance learning (efficacy, learner and teacher interaction, inclusiveness, and self-learning). Indeed, the implementation of the online work methodology made it possible to transcend the difficulties derived from confinement and provide the opportunity for all students to continue their studies regardless of their location. The interaction between students and teachers—the need for which is also highlighted by Kaviyarasi and Balasubramanian [25] and del Arco et al. [19]—was made feasible thanks to the virtual tools and spaces available, enabling more personalized attention according to the academic needs of each student. In fact, the absence of synchronous classes increased the time available for professors to clarify particular issues through private messaging or in the tutorials themselves. An interesting element to highlight regarding self-learning is that the University’s Psychopedagogical Office developed a Study Techniques Workshop that addressed issues aimed at supporting the organization and management of a student’s study time. Therefore, students had sufficient tools to carry out autonomous learning that could later translate into favorable academic results.

Another favorable aspect for the implementation of the online model in record time has to do with the previous experience of professors in distance work as advocated by some authors [13,23,40], since the professors of the four subjects analyzed had at least three years of experience in working with non-face-to-face teaching-learning modalities as part of their graduate teaching project in higher education. This was indeed a key element in the whole process.

The main limitation of the present study lies in the fact that the design was carried out a posteriori to the implementation of the strategies—although within the pandemic phase itself—which would have made it possible to take into account variables that could not be controlled and that would have provided significant data for the evaluation of the teaching-learning strategies implemented, and the corresponding comparison that is the object of the analysis that concerns us.

The practical implications of the analysis are derived based on the above, since the conclusions reached will enable enriching face-to-face teaching processes, as well as defining strategic lines of work for future contingency situations. On the other hand, the results enable us to identify feedback elements on the teaching practice of great value for professional work as teachers/educators.

5. Conclusions

From the present study, no significant differences were found between the academic results of the face-to-face modalities taught in the 2018–2019 year, and the 2019–2020 year in which teaching was carried out from emerging strategies within a virtual environment. Thus, the results of the strategy designed to meet the demands of confinement resulting from the pandemic were effective. On the other hand, regarding the differences found in the evaluated activities (better results in the continuous evaluation in the online modality and better results in the final exam in the face-to-face modality), it is pertinent to propose the following lines:

-

Promoting the design of exams where it is necessary to apply knowledge and/or create from what has been learned.

-

Promoting the design of final exams in which the student’s criticality with respect to the contents is also evaluated, rather than the mere comprehension of the contents.

-

Maintaining the operational and methodological characteristics of the teaching-learning process that can be applied to face-to-face teaching, such as the creative complexity of the tasks performed, the temporal distance between acquisition and evaluation for each topic, the flexibility and autonomy for their study (access to reference texts and other resources while answering evaluative activities) and the time margin for realization and submission.

As for future lines, it would be interesting to integrate other subjects in the post hoc analysis carried out. It would also be pertinent to implement the improvements defined as conclusions of this study and to carry out a comparative study in subsequent semesters to evaluate the effectiveness of these improvements.