A Systemic Framework to Evaluate Student Satisfaction in Latin American Universities under the COVID-19 Pandemic

Abstract

Latin American universities (LAUs) have been going through a serious lack of economic resources which has plunged them into a deep financial crisis. The COVID-19 pandemic has exacerbated this calamity. However, LAUs have implemented online teaching processes in order to mitigate the effects of scheduling and other classroom disruption. There is evidence that these modes of teaching have had a reasonable reception but the level of student satisfaction is yet unknown. This article takes a systemic view of the predicament facing LAUs. It represents the elements related to the disruption caused by COVID-19 in a rich picture, building a systemic framework to explore student satisfaction with remote teaching. Using a sample of 298 students from Brazil, Colombia, and Peru, the study analyzes their situation through: (a) Well-being, educational resources, and learning experience and; (b) General satisfaction with virtual classes. Applying exploratory factor analysis, this study identifies three dimensions: (a) satisfaction with support and adaptation in the virtual modality; (b) satisfaction with the interaction in the virtual classroom; and (c) satisfaction with the development of the study program. Medium/high scores for the dimensions indicate moderate/high levels of satisfaction. The findings suggest that there are still unsatisfied needs regarding access to digital resources and socio-emotional needs. This article could be of interest to Higher Education Institutions (HEI) planners dedicated to post-pandemic, virtual education.

1. Introduction

At the time of writing this paper [mid November 2020], the Coronavirus Disease 2019 (COVID-19) has infected more than 53 million people, causing 1.3 million deaths across the world [1]. The pandemic abruptly erupted in all areas of our society and education (at all levels) has been one of the sectors most affected. UNESCO reports the massive closure of educational institutions in more than 190 countries in order to prevent the spread of the virus and mitigate its impact and, in May 2020, more than 1200 million students of all educational levels have stopped having classes face-to-face. Among this group, more than 160 million were students from Latin America and the Caribbean [2]. Likewise, millions of university students are abandoning their university studies as a consequence of the changes that they were forced to carry out due to the implementation of strategies for continuing studies [3].

It is well known that even before the pandemic, education in the Latin America (LA), faced many unsolved problems. For some time now, this sector has been characterized by underfunding, inaccessibility, and extreme levels of inequality. However, over the last decades, governments in the region have made efforts to invest in basic education programs as well as to implement and improve the quality of universities. As a result of these efforts, since 2000, the enrolment figures for higher education in Latin America actually doubled from 21% to 40% to 2010 of the university population [4]. Even beyond such statistics, the pandemic has exacerbated the structural problems of education inequality and inaccessibility that still plague many Latin American countries.

The pandemic is putting the situation of access to universities at serious risk of setback. The Times Higher Education Supplement (THE) reports that Latin American universities were already lagging behind those in other regions of the world before the pandemic. According to THE World University Rankings 2021, there is not a single Latin American university among the top 200 in the world [2]. Representing this region in 240th place, the University of Sao Paulo, Brazil is followed by other Latin American universities, (yet, with the risk of falling lower in the ranking due to the closure of universities) [2].

The three countries referred to in this paper all share similar challenges in terms of accessibility to education and dropout from studies. In the last 10 years, Colombia had an improvement in its higher education coverage rate (Indicator that shows the relationship between students enrolled in the upper undergraduate level and the population between 17 and 21 years old) and a percentage increase of 21 points in 2015. This was evidenced when compared with the 2005 Census, which was 28.4% [5]. However, findings for 2018 reveal (for the first time) a change from a constant increase in students in universities to a decrease [6]. Meanwhile, the dropout rate by cohort shows 48.3% in 2005 and increased to 50.7% in 2015. Consequently, half of the students who start university studies drop out of the educational system [5]. In addition, due to the pandemic, enrolment is expected to decrease by up to 25% [6].

The situation in Brazil is similar. This country had already been facing accessibility problems. There is a very complex and unequal higher education system, an annual dropout rate of 18.3% and a phenomenon of expansion of the university system closely associated with the growth of the private sector [7]. In this country, a recent study concluded that 42% of students from private universities are at risk of abandoning their studies because they cannot pay their semester fees [8].

In Peru, according to the Instituto Nacional de Estadística e Informática (INEI) figures, between 1996 and 2010, the number of universities has grown from 57 to 100 but this was in an unplanned, disorganized, and haphazard manner. Additionally, the number of public universities increased by 25% (from 39 to 51 universities), while the number of private universities increased by 124% [8] (from 40 to 91 universities) to the year 2014) [9]. Likewise, the number of students enrolled in universities from 2007 to 2017 increased from 649,499 to 1,279,738 [10]. However, as a result of the pandemic, 174,544 students suspended their studies, mainly due to lack of connectivity to follow remote education and another cause was due to lack of financial resources [11].

The pandemic has meant that university students have been affected by the closure of their study centers, in Colombia (2,408,041), in Brazil (8,571,423), and in Peru (1,895,907) [12]. Likewise, universities are facing global challenges such as: opting for virtual teaching, having to guarantee continuity and commitment to study by their students, in addition to the traditional problems common to that level of studies.

One of the problems that universities face when implementing virtual teaching is the lack of a technological platform, despite efforts to integrate into the digital field since the 1980s. Since then, the main concern has been to strengthen students’ digital skills. This situation is common in Latin American and Caribbean countries that are unequally prepared to face this crisis by taking advantage of digitization [13].

For many students with problems of connectivity, provision of digital equipment and health due to socioeconomic problems, virtual teaching generated dissatisfaction mainly at the beginning of classes. Some students triumphed over these problems because of support from their institutions and their own families. Despite this, there are many cases that have ended in desertion of studies.

Evidence about student satisfaction under the new virtual modality is unknown. This represents a weakness for every institution. Such evidence would make it possible to provide solutions aimed at responding to the challenges of academic security, quality of education, and accessibility. This is in order to guarantee the training of professionals with improved competencies, irrespective of the current pandemic [14].

However, due to the urgency of the current situation, Higher Education Institutions (HEI) in LA have had to develop additional skills in their education processes. This includes the mechanisms for evaluating student satisfaction. A quick review of these efforts reveals that they are made from a mix of traditional methods of offering education and reference to different virtual education platforms.

According to Aristovnik et al. [15] (p. 2), several papers have already been published by researchers around the world presenting studies on various aspects of the COVID-19 pandemic crisis:

Particularly its consequences for physical and mental health, the economy, society and the environment. The biggest limitations of these studies about student life while their higher education institution is physically closed are generally: the data derive from early stages of the pandemic; a relatively small sample; and focusing chiefly on a restricted number of aspects of a student’s life. Indeed, most studies are limited to academic work or life issues, a student’s mental health or a combination of both.

Álvarez, et al. [16] consider that the student is not a client, nor the passive recipient of a service, researching student satisfaction clearly contributes to improving the quality of educational services, it is necessary to make improvements in educational services. Thus, inviting student commentary about their satisfaction with the quality and competitiveness of their educational institution is important.

The quality of the university’s service benefits, when students express their views about the characteristics of the teaching delivery, context, and interaction with their inclination to study. Such characteristics can be considered as external and internal aspects of the student life that influence the individual’s academic satisfaction. [17].

Finally, a practical approach to a satisfaction evaluation model is offered by Rosario-Rodríguez et al. [18], where they used a non-experimental cross-sectional exploratory-descriptive design, in a sample of 167 students from 17 different universities in Puerto Rico. The main objective of this research was to examine the technological, academic, and psychological demands that Puerto Rican HEI students may be experiencing during the COVID-19 pandemic. The main results entailed: problems related to stressors due to overload of assignments, poor implementation of online courses, lack of guidance from their universities, and little flexibility from their professors, among others. These results open the way to study sudden changes in educational modality (transfer from face-to-face to the online approach) and their impact on students, Rosario-Rodríguez et al. [18].

In this research, we wanted to investigate this further, and using systems concepts, we attempt to explore from the perspective of the students their experiences and satisfaction with the virtual modality of studies adopted by the three HEIs in our sample: Municipal University Centre of Franca (Uni-FACEF) in Brazil; Externado University of Colombia (UEC); and Jorge Basadre Grohmann National University (UNJBG) in Peru.

At almost ten months from the beginning of the pandemic, a period in which universities have implemented online teaching, there are very few studies assessing the adequacy of this methods and the satisfaction experienced by students. There were studies that have examined students’ satisfaction in conditions prior to the pandemic. The models to study satisfaction have used a mix of quantitative and qualitative approaches. This study acknowledges these studies but we believe that only by taking a systemic view, can the concerns and needs and therefore satisfaction of students can be assessed. To cope with the interruption, HEIs had to mobilize a set of software tools to replace normal methods, tools to re-write academic content of entire units, and then to evaluate their reception and efficacy. It is in this area of assessing the efficacy and effectiveness of the new modes of teaching and its reception by students that the present research concentrates. We aim to identify the key elements that underpin students’ satisfaction or dissatisfaction in three countries most badly hit by the pandemic in the Latin American region.

Until a few years ago, the nature of management sciences applications were characterized by the use of a particular technique, method, or methodology and by aligning itself with a particular unique paradigm, traditionally labeled as hard/quantitative or soft/qualitative. However, in the last two decades, management scientists and indeed systems practitioner, academics, and consultants have begun to combine methods, methodologies, and techniques not only from a single paradigm, but from various paradigms simultaneously [19,20]. This mode of practice has been coined as multi-methodological practice. The approach used in this paper follows that trend in that we combine systemic tools and concepts to explore the initial phases of the problem tackled and then we use quantitative techniques such as factorial analysis to investigate student’s expectations during the pandemic.

The paper is organized as follows: After this introduction, in Section 2, using some basic systems principles, we set out the systemic framework used to explore the complexity faced by universities at the outset of the pandemic. In Section 3, we outlined the research questions derived from the main relevant issues derived from the systemic analysis drawn from the previous section. In Section 4, we outline the methodological stages to explore the key satisfaction dimensions amongst a sample of students in the three countries. Results of survey and factor analysis: exploring students’ satisfaction are presented in Section 5. In Section 6, we discuss the results of the survey. In Section 7, we advance some final remarks as to how our findings might affect educational policy, we also acknowledged the limitations of our study together with some ideas for further research.

2. Conceptual Framework and Model

2.1. Systems Thinking: Exploring and Tackling Complexity of a Pandemic

The consequences of the pandemic have been felt in every sector of society and every aspect of life. At the moment of writing this article [mid-November 2020], there are fears of a second-wave of infections and there are no signs of a permanent solution to the problem or even of abating the epidemic consequences—at least in the short term. We might be approaching a scenario in which we expect only partial improvements of a messy situation. What is becoming more apparent is that the pandemic and its consequences cannot be explored using deterministic and reductionist-informed tools. These do not pay enough attention to ‘connections’—to the myriad of elements or parts that conform what can only be classified as an extremely complicated, problematic situation.

Our contention is that, to tackle the complexities of COVID-19, it is imperative to take a systemic (not systematic) view. Systems thinking (ST) as a discipline and as a way of thinking can be traced back to Aristotle: “The whole is more than the sum of its parts”. ST focuses and explores the interactions between the constituent elements of a system, offering a holistic inquiry of such systems in contrast to reductionist approaches. It has been applied extensively to many fields of knowledge and organizational settings and yields effective results when problematical situations or ‘messes’ are most recurrent. Ackoff [21] (p. 1) urges us to take a systemic perspective: “Almost every problem confronting our society is a result of the fact that our public-policy makers are doing the wrong things and are trying to do them righter.”

Furthermore, when we approach complexity, it is worth to remember what systems thinking is and why it is important to differentiate between a collection of parts and a systems. As Kim [22] (p. 2) states: ‘[…], a system is any group of interacting, interrelated, or interdependent parts that from a complex and unified whole that has a specific purpose. The key to remember is that all parts are interrelated and interdependent in some way. Without such interdependence we have a collection of parts, not a system’.

Debates about the handling of the effects COVID-19 have increased interest in systems thinking in the field of health, healthcare, and medicine. Two recent online publications suggest the necessity to use ST: (1) COVID-19 means systems thinking is no longer optional [23]; and (2) A systems approach to preventing and responding to COVID-19 [24].

Furthermore, the urgency for ST to cope with the COVID-19 aftermath has been echoed recently by the OECD [25]:

[…] policy interventions and priorities to address COVID-19 must incorporate principles of system resilience to systemic disruption now, for not doing so will limit future socioeconomic recovery for the next decade at least. Systems thinking is the most powerful tool we have at our disposal to accomplish this task, if it is part of a trilogy completed by anticipation and resilience.

In this paper, we aim to use some ST principles to first explore the situation the pandemic has presented; and tried to take various perspectives from which to assess its consequences. The context in which this is unfolded will be the processes that universities have put in place to mitigate the pandemic disruption effect. We plan to use the systemic tools to explore the success of these teaching and learning plans and, to assess how this fared, in terms of overall university student satisfaction, in the three Latin American countries.

2.2. Conceptual Framework of the Study

In order to carry out the study and try to answer the research questions (RQs), the conceptual framework informing the study adopted is based on systemic principles and follows the Four ‘As’ model or stages of a systems study proposed by Mingers and Brocklesby [26]. The four stages should be followed as an iterative (not sequential) process in that during the study, returning to the previous stage and refining the model are encouraged. The four As are: Appreciation, Analysis, Assessment, and Action. We start by appreciating and trying to make sense of the situation by surveying and structuring the problematic situation. The second analysis is aimed at identifying the possible variables and relationships between them, in the context of the problematic situation. We then assess or evaluate ways to control or eliminate those causes, while the fourth establishes and implements the action plans. The general framework is shown in Figure 1.

Having the relevant issues and research questions outlined in

Table 1. Relevant systems and research questions to study.

Relevant Issues and Relevant Systems	Research Questions
Continuity of classes; Safety of lecturers and students; Learning.	RQ1: What are the main issues of concern faced by LA HEI in dealing with the consequences of the pandemic?
Quickly trained lecturers for online teaching; Invested in video conference tools to make classes possible; IT support personnel for eventual contingencies.	RQ2: What are the main measures taken by LA HEI to deliver their teaching and learning objectives under the current crises?
Tool for accessing classes: cell phone, notebook, computer; Infrastructure problems: internet; Tiredness of students and lecturers in the face of the new situation; Possible diseases throughout COVID-19 process: Mental and Health problems.	RQ3: What are the main variables affecting students teaching and learning objectives under the current crises?
Adaptation of students/lecturers with the new protocol to “Teach and Learn”; Safety and tranquillity during learning in a Pandemic Context; Feedback from the students during this experience.	RQ4: How satisfied are students with the new teaching arrangements and how these can be improved?

Source: authors.

, we followed the four stages of this framework depicted in Figure 1.

• Appreciation: Firstly, the project team appreciated the situation by searching for the main variables that affected the process. Then, we built the rich picture with all the elements. This was in order to be able to interpret the whole problematic situation. We then structured the problematic situation to be able to see the whole—through the correlations between the variables.
• Analysis: Secondly, from the rich picture we selected four relevant research questions and designed the questionnaire by compiling relevant variables to student satisfaction under the pandemic context.
• Assessment: The third stage of assessment and interpretation of the data followed. We selected a sample to carry out a pilot study with 298 students in one of the countries involved. To complete the Analysis stage of our conceptual framework, we revised the questions and the relevant systems formed in the rich picture, and applied the survey with 298 Colombian, Brazilian, and Peruvian students. This was done using Google forms. As will be fully explained later on in the paper, the project team complied with ethical protocols set out by the three universities involved for this type of study. Specifically, we made students aware that they were being invited to academic research. There was voluntary participation and their anonymity and confidentiality were guaranteed.
• Action: Finally, we wrote the findings and conclusion together with and analysis of the systemic feasibility of changes that need to be implemented.

2.3. Rich Picture and Relevant Systems

To structure the situation and these questions, the project team worked via a series of Zoom meetings and similar platforms. We gathered perspectives and stories about how HEIs in Brazil, Colombia, and Peru are coping with the unfolding of the pandemic. We all agreed that this was a situation featuring hard and soft complexity which can be characterised as a mess [21], or a wicked problem in that ‘evades a definitive formulation’, [27,28].

We follow some of the concepts proposed by Checkland [29], when faced with a messy problem. We try to structure it by drawing a rich picture (RP) that contains as much of the information, variables, people, institutions, and relationships of problematic situations provoked by the pandemic. Rich picture techniques have been used extensively in systems studies, particularly as a first step to make sense of a messy situation. The UK Open University has developed a good repository of advice as to how to use this technique [30].

Using this basic and simple technique, we first drew a RP of the situation. This RP was constructed via an iterative exercise conducted via various Zoom sessions, with collaborative input from the project team members. Figure 2 shows the final version of the RP. It is worth stressing that this is by no means a picture that contains all the variables interacting in this situation but it contains the world views (arguably subjective) of the project team at the time; and, it is a snapshot of the situation at the moment in which it was constructed (end of October 2020).

Looking at Figure 2, we can see two distinctive parts: the left side, containing information when the problem started and the right side, with the consequences of the HEI lockdown.

On the left side, events of the beginning of March 2020 are depicted. The LA HEI authorities and staff started received phone calls and enquires from the students’ families, asking about: the particular university’s position regarding the appearance of COVID cases, worries about the students’ own safety, and concerns about the increase in the number of deaths. The LA university authorities decided to close their universities. This decision followed the world trend and the rules imposed by the government. Some universities, at this time, already had a pandemic committee created. Without face-to-face classes, the IT department had to provide technological options to make distance classes feasible.

On the right side of the picture, relevant results and consequences are presented together and the implementations are depicted: all the university staff started working online from home; a new protocol: “Teach and Learn” was created and the lecturers had to change and adjust their mental models from face-to-face classes to live online teaching. Difficulties in adapting to this new mode of teaching are depicted by the number of arrows relaying this item and other problems that emerged (such as difficulty with equipment, internet and wireless connections). As it well known, everybody had to develop new mental models and adapt quickly to the new unprecedented circumstances, creating anxiety (which in themselves caused more anxiety from the full lockdown). Mental and health problems definitely surfaced among the actors involved: university staff, lecturers, students, parents, etc.

In a systems study, the RP is to have a general view of the situation. As it can be seen in the RP, there are various areas of concern. The project team comprised mainly lecturers who, having experienced the stress and anxiety at first hand, decided to concentrate on issues affecting student learning and teaching and the impact on student satisfaction. In the next subsection, we develop these relevant systems in some detail, structured according to the shape of research questions guiding the rest of the study.

3. Relevant Issues and Research Questions

As mentioned above, from the rich picture, we extracted some relevant systems that the project team thought were actually connected to student satisfaction. Using the RP as an initial point (and, after various online sessions various issues were considered important), the team decided to address the following issues around four research questions (RQs). These are summarized in Table 1 immediately below.

4. Methodology: Using Factor Analysis to Explored the Students’ Satisfaction Factors

The study focuses on effects of the virtual teaching modality in the framework of the COVID-19 pandemic at a university in Brazil, Colombia, and Peru, respectively. Initial and preparatory stages have been completed by systematizing techniques and procedures that have allowed, through observation and measurements, to quantify the data of the study variables. Such systematization provided description objectives on aspects related to the experiences of students on well-being in the face of a pandemic study as well as their satisfaction with the university institutional performance. A quantitative approach was followed with an instrument that made it possible to configure three dimensions per data grouping.

4.1. Data & Method

An observational, analytical cross-sectional study was carried out in the months of August and September 2020 in the schools of Accounting and Business Sciences of the HEI of the Municipal University Centre of Franca (Uni-FACEF) in Brazil; Externado University of Colombia (UEC); and Jorge Basadre Grohmann National University (UNJBG) in Peru. Students were drawn from their 1st to 5th year; the study complied with the ethical protocols of each university and gained approval from the management of the mentioned schools.

4.2. Data Description

The population comprises students of the three universities programs; the sampling method was non-probabilistic. The inclusion criteria considered students enrolled in the first academic semester of 2020. The sample was made up of 298 students (Brazil 99; Colombia 80; Peru 119). All students in the sample were previously informed of the study objectives and their consent to answer a questionnaire in virtual mode was gained.

The data for the research come from an anonymous and self-administered survey in a virtual way through the Google Form computer application. The participants were located in some cases in their virtual study rooms, and in other cases, contacted through their groups organized by WhatsApp. Once contacted, the instrument location link was explained to them and the Google Form was sent in. The responsible researchers were the members of the research team, who coordinated and monitored the reception of the virtual surveys. Surveys obtained from students from other professional programs were excluded.

The questionnaire comprised 48 questions and the estimate to complete was between 12 to 15 min, previously checked in a pilot test. The first section comprised sociodemographic data (10 questions) including age, sex, careers, study funding, and housing situation; a next section for data related to their well-being within the framework of their educational environment in the COVID-19 context (10 questions); Another section requiring answers on whether they used or use didactic resources (2 questions). Finally, they were asked to answer 26 questions about satisfaction regarding the teaching-learning process with alternatives on the Likert scale of 5 alternatives: 1. Very dissatisfied; 2. Unsatisfied; 3 Neither Satisfied nor Dissatisfied; 4. Satisfied; and 5.Very Satisfied. The first column of

Table 2. Factor analysis results.

	Factor 1	Factor 2	Factor 3
SATISFACTION WITH SUPPORT AND ADAPTATION TO VIRTUAL MODALITY
HSAY(*) was supported by your study program before the pandemic? Q19(**)	0.770
HSAY with the way your university has handled the pandemic? Q18	0.695
HSAY with the general support of the university with online technology? Q15	0.679
HSAY regarding the commitment of the lecturer (s) in this virtual modality Q30	0.671
HSAY with the online library? Q17	0.627
HSAY with adaptation of the tutors-professors/professors in their academic work in the face of the pandemic? Q22	0.627
HSAY with the provision of university support services other than IT? Q16	0.620
HSAY Y with the use of technology used by the lecturer to facilitate learning? Q25	0.578
HSAY with the help of your career lecturers (program) during the pandemic? Q21	0.572
HSAY regarding the understanding on the part of their lecturers in the face of connectivity or equipment problems? Q38	0.569
HSAY regarding the change of treatment received from the lecturers in this virtual modality? Q24	0.543
SATISFACTION WITH INTERACTION IN THE VIRTUAL CLASSROOM
HSAY regarding communication with colleagues in this virtual mode? Q33		0.781
HSAY about openly expressing your doubts? in this virtual mode? Q31		0.714
HSAY regarding flexibility in the demands of lecturers? Q34		0.708
HSAY regarding motivation to study? Q35		0.692
HSAY regarding communication with the lecturer in this virtual modality? Q32		0.689
HSAY regarding time to reflect during class? Q39		0.671
HSAY with the interest, of your lecturers, regarding your health and well-being? Q40		0.575
HSAY regarding the duration of class sessions?Q29		0.468
SATISFACTION WITH THE DEVELOPMENT OF THE STUDY PROGRAM
HSAY regarding the type of evaluation in the virtual modality? Q36			0.796
HSAY regarding compliance with schedules in this virtual mode? Q27			0.757
HSAY regarding the ease of understanding of the courses in this virtual modality? Q26			0.687
HSAY regarding the variety of evaluation in the virtual modality? Q37			0.589
HSAY with your study program during the pandemic? Q20			0.548
HSAY regarding content compliance in this virtual modality? Q28			0.534
HSAY regarding the methodology of lecturers in this virtual modality? Q23			0.518
Total explained variance = 62.641	52.887	5.386	4.368
Alpha	0.93	0.908	0.897

* HSAY: How satisfied are you. ** Corresponding numbers in questionnaire. The first factor referring to institutional and teaching support and support, included 11 different items, shows a very high reliability with a Cronbach’s alpha of 0.93. These elements make up, as a whole, a dimension labeled as satisfaction with support and adaptation in the virtual mode. The scores for this dimension ranged from 11 to 55, with a mean 37.35, a median 38, and a deviation of 8.74. Such measures indicate that students assign it a medium/high satisfaction.

presents the list of questions asked.

In order to manage the data and identify reasonable dimensions of student satisfaction, an exploratory factor analysis was performed for the 26 questions on satisfaction.

Compliance with the prerequisites to enable factor analysis was verified. A Kaiser–Meyer–Olkin (KMO) index of 0.958 (greater than 0.6) was obtained; for Bartlett’s sphericity test less than 0.05. Once the requirements were exceeded, the analysis of components with varimax orthogonal rotation produced a 3-factor structure that explains 62.64% of the accumulated variance. All but one of the factor loadings exceeds the value 0.5. The last three columns of Table 2 shows the results of the factor analysis. The 3 factors represent, in the COVID-19 context, satisfaction dimensions regarding: (i) support and adaptation to the virtual modality; (ii) interaction in the virtual classroom; and (iii) the development of study programs.

The second factor captured the interaction in the virtual classroom that occurs between students and the lecturer. It expresses communication between students, lecturer–student communication, motivation, interest, and reflection in the virtual classroom environment. It was composed of 8 items with high internal consistency (Cronbach’s alpha = 0.91), the dimension was labeled as satisfaction with Interaction in the virtual classroom. Among students, scores ranged from 8 to 40, mean 25.82, standard deviation 6.95, and median 26. These measures indicate medium to high levels of satisfaction for this dimension.

The third factor includes aspects such as the type of evaluation, compliance with the schedule, understanding, methodology, and content. The incorporated responses generated a composition of 7 items that together also show a high index for internal consistency (Cronbach’s alpha = 0.90). This composition has been identified as satisfaction with the development of the study program. The students’ scores ranged from 7 to 35, with an average score of 22.32, standard deviation of 5.88, and median 23. This dimension also reaches a medium to high level of satisfaction

5. Results of Survey and Factor Analysis: Exploring Students’ Satisfaction

The outbreak of the COVID-19 pandemic caught LA HEIs between the 2020-I semester and the beginning of the 2020-II semester. The virtual modality was the response of the universities to the technological and connectivity service demands of the university community. In this section, we present the results of the survey starting with outlining some demographic characteristics of the groups in the three universities in Brazil, Colombia and Peru. We also present some experiences and well-being aspects brought to the surface as a consequence of the pandemic.

We then proceed to present the results of the level of satisfaction presented by the students according to the dimensions generated by the exploratory factor analysis. Such findings are presented below.

5.1. Results of Sociodemographic Characteristics

Regarding the sociodemographic characteristics of the university students: in Brazil, they were 57.68% women and 42.42% men, mostly under 23 years of age (82.90%); in Colombia, 58.85% were women and 41.25% men, 100% being under 23 years of age; lastly, in Peru, 62.18% correspond to women and 36.13% men, mostly under 23 years of age (89.90%). The undergraduate majors considered were Administrative Sciences and Accounting Sciences in Brazil (Uni-FACEF), Administrative Sciences and Government Finance and International Relations in Colombia (UEC), and Administrative Sciences and Commercial Engineering (UNJBG), evidenced in

Table 3. Demographic characteristics.

		Country
		Brazil	Colombia	Peru
Gender	Male	42.42%	41.25%	36.13%
	Female	57.58%	58.75%	62.18%
	Prefer not to say	0.00%	0.00%	1.68%
Year	Less than 20	35.35%	80.00%	50.42%
	21–23	47.47%	20.00%	39.50%
	24–26	11.11%	0.00%	6.72%
	27 or more	6.06%	0.00%	3.36%
Disability	Yes	2.02%	0.00%	94.96%
	No	97.98%	100.00%	3.36%
	Did not say	0.00%	0.00%	1.68%
Course	Accounting Sciences	25.25%	0.00%	0.00%
	Administrative Sciences	74.75%	13.75%	73.95%
	Govern. Finance and Int. Relations	0.00%	86.25%	0.00%
	Commercial Engineering	0.00%	0.00%	26.05%
Level of Study	1st.	36.36%	58.75%	0.00%
	2nd.	8.08%	40.00%	50.42%
	3rd.	10.10%	0.00%	1.68%
	4th.	45.45%	0.00%	45.38%
	5th.	0.00%	0.00%	0.00%

Source: authors.

.

In relation to the year of study that students take and percentage of participation, in Brazil, first and fourth year (80%); in Colombia, first and second year (90%) and in Peru, second and third year (95%). Regarding how they finance their studies, in Brazil, they work full time (38.4%) and their parents pay for their studies (37.4%). Students from Colombia and Peru state that their parents support their studies: 61.3% and 71.4% respectively.

Before the pandemic, students indicate that they lived with their families in Brazil and Peru with 94.9% and 93% respectively, and at the time of the survey, they continue to do so; the situation is somewhat different is in Colombia where 81.3% lived with their family before the pandemic and currently 93.8% do.

5.2. Well-Being Outcomes within the Framework of Their Educational Environment

Regarding aspects related to well-being within the framework of their educational environment in the current context, this study shows in the population of university students in Peru that 25.2% have suffered from the COVID-19 disease or a direct member of their family compared to 17.2% in Brazil and 15% in Colombia. Regarding other concerns in the midst of the COVID-19 pandemic, students from Brazil and Colombia indicate concern for mental health 37.4%, and 41.3%, for studies 35.4% and 33.80% and, for money, 34.3% and 26.3% respectively; in Peru 61.3% indicate concern about money, 40.3% about studies and 27.7% about mental health. Faced with the alternative of mentioning “other problems”, there were, altogether, 50 responses (11 Brazilian, 25 Peruvian, and 14 Colombian) with coincidences for problems with the technological infrastructure, anxiety, stress, depression, staying at home, family, and illnesses (COVID-19 or other). Regarding whether they have disabilities, less than 5% refer in Peru and Brazil (low vision, physical disability).

Despite the circumstances of the pandemic, most of the university students in the three countries indicate a positive attitude and on agreeing to incorporate more online aspects in the face-to-face learning activity when the pandemic ends, there is in Peru 71.4%, Colombia 42.5%, and Brazil 40.4%. Regarding disagreement, it is presented in Brazil 38.4%, Colombia 35%, and Peru 11%. In relation to feeling comfortable going back to university, the students of the three countries were mostly in agreement. When asked if they manage to balance studies with other commitments, including self-care responsibilities, more than 50% of Brazilians and Peruvians along with 35% of Colombians agreed.

In relation to didactic resources and learning experience, the most used teaching platforms are Zoom in Brazil and Colombia, and Google Meets in Peru. Before the closing in Brazil and Colombia, they used Moodle with 67.7% and 33.8%, respectively. Regarding use for academic debates with forum, Brazil 36.4%, Colombia 30%, and Peru, 32.8%. Use for Live conferences, Colombia 36.3% and Peru 27.7%.

5.3. Satisfaction Results in the Virtual Modality Study

In the measurement of the satisfaction variable in the virtual modality study in the COVID-19 context, 3 dimensions were included that showed high correlation with each other: support and adaptation in the virtual modality with interaction in the virtual classroom (0.79); support and adaptation in the virtual modality with development of the study program (0.804); finally, support and adaptation in the virtual modality with development of the study program (0.783).

6. Discussion of Results

One of the great challenges facing higher education institutions in the face of this pandemic situation is to prevent students from dropping out and to guarantee the quality of teaching. In Latin America, education opportunities in recent years with the efforts of governments have raised the figures for access to higher education in students of both sexes, although it is true not what was expected, close to 60% of students graduated in 6 years, with considerable variation by gender, ethnicity, and socio-economic level [31]. In Colombia, enrolment in university academic programs had already presented a downward curve that today in the face of the pandemic has accentuated [32,33], the same fate is emerging in Brazil, Peru, and Latin America in general and not only in enrolment but in terms of quality and equity [34].

The economic recession and the complex socio-health problem are aspects that, added to the effect of the pandemic, forced universities to implement strategies to meet the needs of the university community. Such efforts were aimed at maintaining spirits, motivation, well-being, allowing students to continue their studies and, above all, guaranteeing educational quality. This has led support to try to supply necessary physical resources, laptops, and Internet access, although these would not be the only concerns expressed by students in the study that reveal concern for their mental health and illness in the family [35,36,37].

Universities in their important social role in meeting the demands of supervisory bodies and concern for their own survival, have given importance to student satisfaction, recognizing that assessing it will contribute to making timely decisions in the face of management oriented to pursue the university quality [38]. In such a context, the power to keep the student body in the universities in the face of eminent desertion becomes more important, a problem that had been considered as the other pandemic that had already been afflicting the universities with its consequent social cost [39].

Various investigations have reported high and low levels of student satisfaction with the perceived quality of their studies [40,41] where quality indicators guided their measurement. In this study, students were taught some elements related to the virtual modality educational environment linked to the context of the COVID-19 pandemic from their study experience. The study located 03 satisfaction factors, the first referring to support and adaptation to the virtual modality, the second to the interaction in the virtual classroom, and the third development of the study program.

The surveyed students from the three countries showed average satisfaction with the first factor related to the support they have received from their institutions. The health crisis has made visible multiple shortcomings of the face-to-face educational system such as the lack of materials, ICT equipment, and incomplete preparation of lecturers and students for the virtual modality of studies. This context exacerbated the academic, technological, and psychological demands, forcing universities to develop urgent implementation policies and support programs for students with IT equipment and connectivity problems. As time progresses, they try to respond to the difficulties and barriers that arise independently of a positive attitude towards technologies in teaching–learning [37,41]. It is evident that the conditions of opportunity are not the same for everyone [42]. It should be noted that the information collection period falls on almost the end of the first semester 2020.

The second factor shows the need for soft skills such as communication and interest of the lecturer on the well-being of the student; students from the three countries show medium to high satisfaction. There is similarity in their comments on the teaching–learning perceived preoccupations of their lecturers, where they state: “That lecturers are more understanding with the schedule”, “I hope better communication from lecturers to students”, “That they are more flexible”, “That they be empathetic in the face of technology difficulties”, “More tolerance, solidarity, and understanding of lecturers”, and “Greater dynamism in the classroom”. What is expressed is unique since the bonding in the classroom is part of and articulated in a learning context. In the area of human development, researchers agree that “no one can replace the humanity of the lecturer in the classroom” [43,44]. This space of inter-subjectivity corresponds to a lecturer-student involvement [45] that will delimit the degree of cognitive and emotional satisfaction and consequently educational achievement [46].

The third factor related to the study program considered the elements of satisfaction based on content, methodology, understanding of virtual courses, evaluation, and compliance with schedules. This approach to the pedagogical-disciplinary field obtained average satisfaction. Since the pandemic began, the role of the lecturer had a great turn, being forced from one moment to another to assume the commitment to transform and adapt to the challenge of using new technologies in this teaching modality, which does not mean that the benefits that virtual education means (e-learning) can offer are not recognized [47]. In this context, some commentators have questioned the term ‘teaching-learning process under virtual modality’. The Vice-Chancellor (or Rector as it is called in LA HEIs) of the University of Quimes, Alejandro Villar [42] suggests the term ‘remote teaching’. This is because, in reality, there has been a rapid and heterogeneous response to the pandemic and “an explicit agreement where students and lecturers accept the advantages and limitations of this modality”.

The findings in this third factor cannot be ignored; this study allows us to have as a primary source, an approach to student satisfaction about their study program, about the response of their institution and their lecturers to this new scenario. All lecturers had to adapt to the lack of familiarity in this modality, as stated by Fernández-Regueira et al. [48] on whether the lecturer has taught online; shows that 68.07% have never done it, and they bet on a methodology and evaluation based on content transmission and present conceptual incoherence in the didactic dimensions of their proposals.

The interest in satisfaction measures is not recent, the institutions have been conducting evaluations in this regard during the pandemic. Some HEI decided to carry them out with the pertinent modifications, considering that it is unlikely to lift the state of emergency soon to go to complete normality. Satisfaction measurements are shown as a very useful tool to adapt to the conditions of each educational program due to the lack of knowledge about how the next academic year 2021 will develop [49].

7. Conclusions, Final Remarks, Limitations, and Further Research

In this paper, we reflect on the critical situation faced by Latin American universities as a consequence of the pandemic. Using simple systemic tools as rich pictures, we make sense of the complex situation and extract relevant systems of concern for the different stakeholders. We locate student satisfaction as an issue important to explore and measure to assess the usefulness of the online mode of teaching adopted due to the disruption caused by COVID-19.

Using a sample of 298 students from three of the countries more severely hit by the pandemic: Brazil, Colombia, and Peru, the article analyzes satisfaction towards online teaching using the following variables: (a) Well-being, educational resources, and learning experience; (b) General satisfaction with virtual classes. We applied exploratory factor analysis to these variables and identified three dimensions or clusters: (i) satisfaction with support and adaptation in the virtual mode; (ii) satisfaction with the interaction in the virtual classroom; and (iii) satisfaction with the development of the study program. Average scores for the dimensions indicate moderate/high levels of satisfaction. Our findings suggest that there are still unmet needs in relation to access to digital resources and socio-emotional needs. We are aware that due to the limitations of the sample size, it is not possible to generalize these results but nevertheless our findings, we believe will have the following implications which are relevant to lecturers and educational planners interested in teaching and learning in post pandemic times.

7.1. Policy Implications

Undoubtedly, one of the greatest consequences of the pandemic in the world is the economic recession that directly affected families with the smallest economy in Latin America. This aspect is manifested in the current situation of the students of the three universities sampled here, showing that more than a third in Brazil work to meet their expenses and more than 60% in Colombia and Peru are dependent on their parents who pay for their maintenance and studies. Thus, guaranteeing the continuity of studies is a latent problem, added to the health aspect. An important concern expressed is, even before money, mental health and the concern of being able to face this crisis that demands greater speed of adaptation in order to maintain the close “known normality”.

This situation shows clearly the weaknesses and, shortcomings of the universities regarding the use of teaching platforms, digital tools, and lecturer training. This has forced governments to implement policies that respond to demanding needs. However, despite the efforts of the universities to support the demands raised due to the pandemic, there is still a long way to go.

In this study, the aim was to answer research questions related to students’ satisfaction. (How satisfied are students with the new teaching arrangements and how these can be improved?) We did this to provide some evidence so the elements of the teaching–learning process can be enhanced, corrected, and improved in the next academic periods. Our findings indicate that there are still students who did not receive support from their institution on methodological management, bonding, and communication in the classroom. It is important to consider guaranteeing the safety, quality of teaching, and equal opportunities for educational institutions that face all these challenges in this context and strive to adapt for the benefit of their university community.

They also perceive the lack of understanding in the face of all the problems that continue their studies in the conditions of equipment failures and situations of health problems implies for them, which demands the understanding and flexibility of its lecturers. These aspects are important in the sense that the quality of the lecturer–student bond, the human climate, and lecturer support contribute to student learning.

7.2. Final Remarks, Systems Thinking and Multi-Methodological Practice, Limitations, and Further Work

Our study has shown that the systemic view taken has allowed us to explore one of the key stakeholders (students) immersed in this complex situation in which a myriad of variables interact. There are multi perspectives, many conflicting interests, and several implementation difficulties. In education, we highlight the problems involving need for technological platforms, problems with the incorporation of new digital technologies in classrooms, the connectivity difficulties of the online modality, the lack of equipment, and acceptance or rejection of the online modality by students. In addition, on top of all of this, there are mounting problems involving stress, anxiety for both students, staff, and university planners.

In this paper, we propose a systemic approach to explore the complexities of the new ways of teaching under the impact brought on by the pandemic. We linked some features of SSM with statistical analysis to give some indication as to how satisfied students are with the new ways of delivering the teaching material. We argued, following Mingers [19,20], that the multi-methodological practice in which methods from both hard and soft spectrum of management science are brought together under a systemic framework, can shed light into a complex situation. We believe that many multi-method applications draw only on different systems approaches, but it is perfectly legitimate to draw on a mix of systems, social science approaches and methods from the hard end of the spectrum. Using systems thinking principles does not equate with using systemic methods as such and that what makes a project systemic is how it is approached and not where the methods originated from.

Furthermore, systems thinking underpinning multi-methodological practice provides a backdrop and allows the participants to ‘immerse’ their minds and their actions when trying to have a systemic view of the problem(s) in front of them, and that is the contribution to the systems community that this paper proposed and we believe that we have followed these commitments in the paper. Systems thinking by its nature should encourage multi-perspectives to flourish. In our experience of practicing multi-methodology using systems thinking principles, see [50,51,52], one way of facilitating those is to assemble a multi-disciplinarily team. This project has been the product of a multi-disciplinary effort: One author is fully conversant on soft approaches and systems thinking practice [51,52,53]; one is expert in education [54]; other one is fairly familiar with applications of systems thinking in sustainability and public policy [55]; other in statistical analysis [56]; and international education formed the rest of the team. On reflection, at end of our exercise, the authors who were not familiar with systems thinking saw the benefits that this can bring and felt that they have acquired a useful tool to tackle complexity in future projects.

Finally, our contribution to the systems community can be summarized as follows: Based on commentators who advocate the use of systems thinking underpinning multi-methodological practice [19,20,26] and the evaluation of complex projects where multiple contexts were defined [57], we have designed a systemic framework to explore a complex problem (new ways of teaching under pandemic constrains). We conclude that the pandemic has brought a myriad of issues affecting the teaching and we have focus on one ‘relevant’ issue which is ‘students satisfaction’ and use some statistical analysis which provided some results that shed some light as an initial way to understand this problem. The results were brought back and discussed on the wider context of the pandemic impact in an attempt to ascertain which other issues might be worth pursuing using systemic approaches in future research. We do hope that readers will assess and value the use using systemic principles when using a tool of methods when tackling complexity.

This study has one important limitation. The study used a small sample of universities, which prevents making inference to the universe of universities in the countries involved. Likewise, it does not deepen in relation to the teaching–learning methodologies but it is oriented to make known the satisfaction of the students in front of the virtual modality in their studies.

The limitation mentioned is not impossible to address, so a suggestion as a future study is to consider first to explore further what systemic implications students’ satisfaction has brought to the fore. A future study should consider and discuss these issues against the initial findings presented here. This exploration as well the use of a larger sample can render a fuller view of this complex and problematic situation.

Finally, and although the questionnaire used (See Appendix A) contains a number of closed questions which have been used for the exploratory factorial analysis, it also contains a number of open questions. Interviewers have engaged fully with these open questions (see questions 45 to 47 in Appendix A) providing rich data about the feelings and perceptions of studying under pandemic conditions. This analytical approach concentrates on quantitative data, which can certainly be regarded as a limitation of the study. However, we plan to use these data in a future study using a qualitative approach. Analyzing these responses will indeed complement the systemic perspective taken in this study in that it will give scope to discuss the interrelations between quantitative variables useful to improve the performance of the processes used by the three universities and also will help to understand the perceptions of the students in a pot-pandemic environment.