Assessment in Higher Education during the COVID-19 Pandemic: A Systematic Review

Abstract

In the face of the COVID-19 pandemic, millions of students have been affected by the closure of educational institutions. This has forced a shift from face-to-face to distance education, facing numerous emergency educational measures, such as online assessment. This study aims to present a systematic review of the literature on the impact of assessment in higher education during the pandemic. The study has followed the methodology set out in the PRISMA statement, and includes 13 studies selected from a total of 51. The results indicate that faculty and students have faced numerous challenges in moving to virtual environments; on the faculty side the lack of training in online assessment techniques is the main problem, on the students’ side there is dishonesty and misconduct. However, it is concluded that continuous assessment, not focused on exams, but in a more qualitative way is the best way to assess at a distance.

1. Introduction

The emergence of the COVID-19 health crisis at university level has been an unprecedented challenge at political, administrative, and pedagogical levels. Universities, like other educational institutions, have had to continue with the forced and compulsory continuation of telematic teaching and to guarantee its quality as far as possible. This effort is limited to a pandemic that has had unprecedented consequences, and in which social gaps, connectivity, family reconciliation and other problems that already existed have been further exacerbated.

The difficulties in adopting the telematic modality in the university environment, beyond the additional problems arising from the circumstances, have led to a proliferation of debates ranging from the identification of the difficulties of tele-training in an educational context in which technological progress has been slow, to the comparison of face-to-face and distance teaching in terms of quality. In this sense, the health emergency generated by COVID-19 presents us with a great challenge related to the digital, despite authors such as Coeckelberg [1] maintaining that education outside technological development is not a problem of online education, but a problem of education.

University education must therefore be transformative and provide all students with the skills and attitudes needed to meet the challenges of today and tomorrow. Thus, one of the UN’s goals is to ensure inclusive, equitable, and quality education for all [2]. Online assessment therefore contributes to a sustainable education approach in which educational equity can be effectively achieved.

Current university education, in the European context, is circumscribed by the reform promoted through the European Higher Education Area (EHEA). A political proposal, with didactic implications, that pursues the objectives of mobility, employability, and quality. From the first two objectives, the system of competences and the European Credit Transfer System (ECTS) imply the training of students for access to the labour market, the situating of them at the centre of teaching and lifelong learning, and the embarkation of a gradual and continuous process that transcends the teaching timetable. The quality objective, for its part, favours the conception of assessment as an end, through a system of accreditation that acts as an indicator of curricular and institutional quality [3] and underpins the promotion system for teaching staff.

Most universities have made large investments to develop online teaching, hence the need to carry out evaluation mechanisms as the best way to study the impact that the teaching-learning processes have had on university students.

To achieve this, this systematic review seeks answers to the following questions:

• What is the general state of research on assessment in higher education during the COVID-19 pandemic?
• How has evaluation in higher education been conducted during the pandemic?
• What is the impact of online assessment on faculty and university students?
• Is online assessment recommended in higher education?

2. Assessment in Higher Education during the Pandemic

The assessment of learning has become a much sought-after topic in the last 20 years, where the concern lies in achieving quality assessment of learning. To this end, educational reforms have been generated that deviate from administrative matters that forget the central point, the academic, in which the concern should not be the introduction of the use of technologies, but how to carry out an evaluation that allows us to assess student learning and the role of all participants in the teaching-learning process to improve the achievement of the goals set.

Assessment is an integral part of the educational process, and it is a serious didactic risk to develop assessment mechanisms that are fragmented or disjointed from the teaching modality. When this happens, assessment strains the learning process and takes on a punitive character that differs from its true meaning [4].

Conceptualisation

Our work focuses on online assessment in higher education, hence the importance of delimiting the concepts. On the one hand, higher education is understood as that which is provided after the baccalaureate. The Final Report of the 1998 World Conference on Higher Education in Paris (UN 2018) establishes the following definition: “it is a component of a single system that begins with early childhood and primary education and continues throughout life”. The same document states that its mission is to: “educate, train, conduct research and, in particular, contribute to sustainable development and the betterment of society as a whole”.

On the other hand, student assessment is one of the central aspects of the transformation of higher education. According to Giuseppad’Aostino de Cersosimo [5], educational evaluation is conceived as a process aimed at verifying the degree of effectiveness and quality of all the elements that converge in the realisation of the educational event, in order to assess this degree against reference parameters and to decide what to do about it.

Casanova [6] conceives educational evaluation broadly as a “process of systematic data collection that provides continuous information about how teaching and learning takes place” (p. 84). The learning outcomes condition and determine the activities and methods of assessment. Defining an assessment approach for a subject, whether face-to-face or online, is an extremely important aspect that structures and integrates the entire teaching and learning process.

Having defined the concept of assessment, we must explain its purposes, which can be formative or summative [7]. The purpose of formative assessment is to obtain information about the student’s progress, and to be able to improve it through feedback. Summative assessment, on the other hand, is carried out in order to grade the student’s performance and compare it with peers or specific criteria. It is often done at the end of the course and leads to qualifications.

If online teaching was already a challenge, often a shock, for the university community, non-face-to-face assessment is a major obstacle to completing the academic year. In this sense, there are many factors that have an influence, such as peoples’ natural resistance to change, together with the technical limitations of systems prepared to provide specific computer support for activities that are mainly face-to-face.

Today, the use of online technologies in higher education institutions has become a tool of great importance due to the ease and speed with which all operations of any kind can be carried out. Numerous educational institutions make use of the tools available on the web to improve their teaching-learning processes, as well as the evaluation process. Proof of this is the study presented in 2013 at the International Conference in Beijing (Advanced Learning Technologies), where a comparative study of different web-based multiple-choice formative assessments was presented, the main purpose of which was to emphasise different strategies used by these systems.

Also, that same year, at the Global Engineering Education Conference, a paper was presented in relation to the Design and Implementation of an Intelligent Assessment Management System [8]. This study proposes the assessment of university courses, the integration of a virtual online assessment management system, and classroom examinations, ensuring that all students are treated fairly during the examination and that the proposed system improves the effectiveness of assessment.

On the other hand, Ruimin Shen [9], basing their work on the modern education model, investigates the importance of an assessment system in web-based distance education and establishes a model of intelligent assessment system characterised by good performance and adaptability.

In recent decades, the use of standardised tests in education has been very frequent and these tend to be of different types: mastery tests, objective tests, competence tests, certification tests, criterion tests, classroom tests, etc. [10]

Despite all the experiences, there are still many barriers to overcome. Along these lines, authors such as Cardenas & Luna [11], in a recent study following the COVID-19 health crisis, conclude that the main problems in the assessment process of a population that was studied during the transfer from a face-to-face mode to virtual environments were the following: in relation to the teaching staff, it is the lack of technological knowledge, both of purely technical elements and of pedagogical tools and formulas for assessment suitable for virtual environments; this technological obstacle has been overcome as they have gone along. On the students’ side, the main problem they identify is the lack of sufficient elements for the continuation of instruction (mainly internet connection), and its consequent evaluation, in this case the problem has not been overcome.

Similarly, the recent study by García-Peñalvo, F., Corell, A., Abella-García, V. & Grande [12] concludes that online assessment therefore requires redesigning the subject assessment system, i.e., the best approach is not to try to replicate traditional exams in the online ecosystem. This implies the need to have both methodological skills and technological skills in order to know the functionalities and limitations of IT tools; however it also implies the need for an understanding that technology is a mere facilitator of the assessment that by itself will not provide answers to the objectives of the assessment process.

However, as we know that every crisis leaves behind some positive elements in the form of lessons learned, COVID-19 should provoke a profound reflection on the design of education systems at all levels, with an impact on effective strategies to reduce the effect of the gaps that affect the use of technology in education in general and especially assessment as an integral part of the didactic process.

3. Method

After having conducted a literature search, we found that there is a significant lack of information on the research topic. The particularity of our study is not only to conduct a review on this topic, but to illustrate the object of study. Thus, the aim of this systematic review is to obtain an overview of the status and impact of online assessment in higher education during the pandemic.

3.1. Search Strategy

In order to answer the above research questions, this article undertakes a systematic review of the literature. The systematic review will allow us to map the evidence and synthesise the available information [13]. The review was developed following the Preferred Reporting Items for Systematic Reviews and Meta-analyses Statement (PRISMA) [14], searching two academic databases (Scopus and Web of Science). The search and selection processes were carried out following four phases: identification, screening, suitability, and inclusion of articles.

This review focused on the themes “online evaluation” and “COVID-19”. In this way, the search descriptors that the studies had to contain in the title, abstract or keywords were determined: “evaluation”, “COVID-19”, “assessment”, “exams”, “test” and “higher education”. The search was conducted in June 2021.

3.2. Elegibility Criteria

The following eligibility criteria were established to select the most relevant studies to answer the questions posed. The inclusion criteria were: (1) peer-reviewed research articles, (2) published between March 2020 and April 2020, (3) addressing online assessment during the COVID-19 pandemic. Articles were excluded if the studies: (1) did not undergo a peer-review process, (2) were published outside the established period, (3) addressed the topic of online assessment, but not during the COVID-19 pandemic, (4) belonged to the following document type: reviews, editorials, letters, books, opinions, conference proceedings, etc.

3.3. Study Selection

The initial search (identification phase) yielded a total of 45 studies in all databases, and 6 potentially valid studies were added after manual checking of the reference lists of the identified studies. The second stage (screening phase) involved a title review to remove duplicate studies, eliminating 20 studies. This resulted in the review of 31 studies against the eligibility criteria. Thus, 12 studies that clearly did not meet the eligibility criteria were excluded. The remaining 19 studies were reviewed in full text to ensure that the studies should be included in the review. Six studies were eliminated (eligibility phase). Finally, a total of 13 articles met the inclusion criteria and are included in this review. This review process is recorded using the PRISMA flow chart in Figure 1.

3.4. Data Extraction

A total of 13 published studies met the inclusion criteria and are included in the review (

Table 1. Details of the articles included in this review.

Study	Date	Country	Technologies	Key Ideas
Díez Gutiérrez, E.J. & Gajardo Espinoza, K. [15]	May 2020	Spain	Not specified	Continuous assessment, digital divide
García-Peñalvo, F.J. et al. [12]	May. 2020	Spain	Use of available technology (online assignments, videoconferencing, proctoring)	Online assessment is not replicating traditional exams
Linden, K. & González, P. [16]	May 2020	Australia	Proctoring, ZOOM	Supervision
Grande de Prado, M. et al. [17]	September 2020	Spain	Anti-plagiarism tools, Moodle, Proctoring, Videoconferencing, Google Forms	Importance of reducing or eliminating final testing
Afiqah, N. & Naing, L. [18]	October 2020	UK	Videoconferences, proctoring, self-assessment questionnaires and discussion forums, tests…	Taking into account negative student behaviour
Cardenas Cabello, F. & Luna Nemencio, J.M. [11]	November 2020	Mexico	Not specified	Continuous assessment not focused on exams
White, A. [19]	December 2020	Australia	Proctoring	Vigilance in online testing
Senel, S. & Can Senel, H. [20]	January 2021	Turkey	Moodle (online test)	There are constraints of time, instructions, feedback...
Elsalem, L. et al. [21]	February 2021	Jordan	Moodle, Proctoring	The student body suggested a reduced preference for remote examinations, Vigilance due to dishonesty
Khabart, F.F. & Abu Daabes, A.S. [22]	February 2021	U Arab Emirates	Email	Privacy concerns, environmental and psychological factors
Prigoff, J., Hunter, M. & Nowygrod, R. [23]	March 2021	USA	Proctoring	Virtual supervision to prevent student misbehaviour
Amzalag, M., Shapira, N. & Dolev, N. [24]	April 2021	Israel	Proctoring	Unethical student behaviours
Ortega, D., Rodríguez, J. & Mateos, A. [25]	April 2021	Spain	Handing in assignments, online tutorials, videoconferencing	Teaching methodologies need to be adapted, lack of training, lack of resources

). The authors then began data extraction using a form. The form for each article included details of: (1) author, (2) country, (3) date of publication, (4) technologies used, and (5) key research ideas. The information for each study is summarised in Table 1.

The data extracted through the form were synthesised quantitatively and qualitatively in order to answer the research questions.

4. Results

The analysis of the articles that passed all the eligibility criteria for selection was done from a quantitative perspective in order to obtain bibliometric information of interest. In this review, a total of thirteen articles were identified that address the use of online assessment during the COVID-19 pandemic at the university stage. In this regard, based on the information obtained from the reviewed studies, we present the following results.

In a short period of time, researchers from around the world have shared their work on the impact of online assessment in higher education during the pandemic. In relation to the country where the studies were carried out, the results are shown in Figure 2. This issue has been addressed mainly by European and Asian countries. Spain tops the list, participating in 4 studies.

The review aims to describe the scientific production on the evaluation of COVID-19 times recorded in the databases analysed. Figure 3 shows the productivity in this field in the period analysed (March 2020–April 2021). May, February, and April (in that order) were the months with the highest trend of publications in this field. As can be seen in Figure 3, it was not until May that the effects of online assessment during the pandemic began to be analysed. An exponential growth rate is also evident. This behaviour allows us to reflect on the interest that this topic has generated since the beginning of the pandemic due to the unknowns of both teachers and students about assessment and promotion during the months of the pandemic. It should therefore be noted that research in this area is likely to increase at a rapid pace.

The method of online assessment of student learning during online education due to the pandemic-related closure of university institutions has been a question mark for all. Through analysis of the selected studies, though not all articles specified the methods used, Moodle platforms (39.19%), which allow the application of multiple functions such as: test-type questionnaires, discussion forums; and videoconferencing (34.78%), which allows remote monitoring during online tests via ZOOM or Skype, stand out as two of the main instruments to carry out online assessment in higher education (Figure 4). Other more qualitative methods can also be found, such as the online submission of academic assignments, the use of email, online tutorials, or other tools such as Google Forms for conducting questionnaires.

In Figure 5, we can see the main perceptions of both teachers and students of the emerging situation in relation to assessment as a result of the pandemic. The data mainly reveal an increased effort on the part of teachers to supervise students during assessments, also called e-proctoring (32%). In order to prevent dishonest student behaviour, such as cheating, teachers may require the use of software, cameras, or plagiarism detection in order to prevent such behaviour. Other challenges include the lack of teacher training (23%) when designing different online assessment tests. The results also show the lack of support and technological means (digital divide) on the part of students and teachers (17%). These aspects (lack of confidence in the subject or technical problems) require more effort, mainly from teachers, which increases their working time and affects their mental health (10%). However, both teachers and students highlight online assessment as more flexible and enhancing autonomy (10%) and providing instant results (8%).

5. Discussion

As a result of the “state of alarm”, decreed by the COVID-19 pandemic, most university institutions closed their doors, reorganising the entire education system so that they could continue their distance education, as well as curb the pandemic [25]. This situation prompted educational measures to be taken to cope with this period, such as online assessment. In order to assess its impact on higher education, this review has analysed a total of 13 studies addressing this issue. The research questions are developed below:

Q1. What is the overall state of research on assessment in higher education during the COVID-19 pandemic?

The first question (Q1) aims to describe how the educational process has been evaluated during the academic year in this exceptional situation. University institutions and faculty had to improvise with the means available to them to conduct distance education [26], which led to student assessment also being carried out using online methods [24].

The results of this review support the idea that this situation has been a concern for university institutions around the world, with research being conducted mainly in European and Asian countries, throughout the pandemic. As we have seen, the thirteen studies reviewed suggest that research in this field will increase over time given the relevance of this topic in the field of online education.

Q2. How has assessment in higher education been carried out during the pandemic?

The entire education community has faced numerous challenges in advancing students’ education during the pandemic by moving to online education, including online assessment, using all available technology [18]. As neither teachers nor students expected this sudden change.

The studies reviewed provide us with an insight into the online assessment methods most commonly used during the pandemic. In this sense, the Moodle platform of the different university institutions stands out as a means of carrying out assessment. Here, teachers can use various free online tools and functions to carry out assessment, such as the configuration of different types of exam questions or discussion forums. Other main methods of assessment include the use of video-calling software (such as ZOOM or Skype). This method allows for tutorials, oral tests or as a proctoring tool during exam time [24].

Assessment other than examination has also been possible during the pandemic through a more qualitative assessment of learning, for example, through the submission of academic assignments, or by assessing their progress through the educational stage, through ‘continuous assessment’ [6]. This option, although considered the most appropriate, has mainly been carried out in the stages prior to higher education [15].

Q3. What is the impact of online assessment on university faculty and students?

Universities have been forced to make a radical change in their education. However, most were not prepared for this shift to online education. The teaching staff had to improvise with the technological means at their disposal to keep up with the students’ learning. This situation, which caught the entire educational community by surprise, led them to use their own computers and technological means from home to continue teaching [12], further exposing the digital divide suffered by a significant part of the population [15], related to the lack of access to technological devices or access to the internet.

Replacing a face-to-face assessment with a virtual one does not mean replicating these materials but requires redesigning the entire teaching and learning process [12]. Precisely, one of the great challenges facing online education is assessment [27]. This highlights the need to improve teacher training in techniques and methods for carrying out distance assessment, as teachers carried out this process by learning about new assessment methods on the go using technological platforms.

One of the main problems of online assessment through available technology is that they facilitate unethical behaviour of students, i.e., test security [28]. However, this requires a remote monitoring system called E-Proctoring. This system allows students to be monitored via video or audio, while they are taking the tests, to see if they are engaging in any inappropriate behaviour. This has led universities to use platforms, such as ZOOM, for exam proctoring [16].

Dishonest student behaviour occurs not only in online exams, but also through the submission of academic papers, which requires faculty to check papers through plagiarism analysis software or good test design [15]. From the analysis of the results, we can affirm that, unfortunately, some students make inappropriate decisions in order to pass the assessment tests. Among the many dishonest behaviours of students, we can find: cheating in exams, plagiarism of work or falsifying data. In order to increase the probity of students in online assessment, it would be necessary to change the focus of the assessment tests so that they lead students to reasoning and reflection rather than to the memorisation of content.

In order to improve online assessment, we must take into account the positive aspects of online assessment that are not necessarily taken into account in traditional assessment [19]. Based on experience and good practices, we will announce some of the positive aspects of online assessment in order to bring about an educational transformation:

-

Immediate feedback: results through a questionnaire provide immediate results, which promotes students’ motivation [28].

-

Flexibility: this strategy facilitates students to take the tests from any computer with internet access [15], allowing in this case the continuation of the students’ education.

-

Monitoring strategies: For teachers, virtual platforms are useful as a monitoring and follow-up strategy. They provide us with information about the files viewed by students or their accesses, but do not inform us about their understanding [29].

-

Autonomy: These tools improve students’ autonomy, but they require the guidance of teachers to serve as scaffolding.

However, although all these elements are essential in the teaching-learning process, students prefer face-to-face assessment. For many, this situation has affected their emotional and physical well-being due to the pressure of telematics education [22,30,31,32]. Lack of training, heavy workloads, and technical problems in online assessment affect teachers’ mental health [31].

Q4. Is online assessment recommended in higher education?

The methodological adaptations carried out in higher education have allowed institutions to continue the education of millions of students all over the world [25], as well as to face this adverse situation and to stop the spread of the virus in educational institutions.

Students and teachers accept emergency educational measures adapted in order to reduce interruptions in teaching, however, in order to carry out quality distance education, it is necessary for all students to have the necessary technological means and resources, as well as to improve the training and capacity building of teachers in relation to online assessment [4].

It has been found to be quite complicated to adapt to this new assessment modality, despite the efforts of the teaching staff. However, in order to improve online assessment, it will be necessary to improve teacher training to enable them to apply other strategies in the assessment of their students. On the other hand, the adaptation to online assessment affected countries with limited technologies. Thus, a facilitating factor is that most students have access to the internet and a computer at home.

However, the results of the study emphasise the importance of continuous assessment, which has both formative and summative components, to better address this situation whenever possible in distance education [7,11,19]. This approach allows students to be evaluated formatively through feedback, but at the same time they are graded according to a summative evaluation.

In order to avoid misbehaviour on the part of the students, the objectives of the assessment must be clear, and the appropriate method must be selected to obtain the desired results. Some alternative methods to the online exam are qualitative ones such as: reflective journals, essays focusing on the development of an argument, reports highlighting solutions to problems posed, project work, analysis of a topic, among others. Depending on our objective, we can select one or the other. It is recommended to increase the level of complexity of the questions or tests, so that the student reasons and reflects on the knowledge learned, avoiding memorisation. Other suggestions for innovative teaching methods that lead to the evaluation of students’ competences such as problem-based learning, future-based learning, flipped classroom, project work, etc. can be considered and used.

The need to ensure social distance during the COVID-19 pandemic has forced rapid decisions to be made about the online assessment of students. In this respect, the urgent adoption of new formats has created significant challenges for teachers. Among the most important are online supervision by the teacher (via webcam and microphone), plagiarism analysis of submitted work, academic dishonesty on the part of students, increased work demands on teachers (affecting mental health) or communication problems and the digital divide [21], so we recommend that, in similar situations, more qualitative assessment methods should be adopted, such as those mentioned above, as well as the use of virtual supervision whenever available [23].

6. Conclusions

The global pandemic of COVID-19 has led to the suspension of teaching at universities around the world. Universities have been quick to adapt, responding with innovative approaches and the rapid adaptation of new forms of teaching and assessment, however, both students and faculty have been affected. Among the responses implemented as a result of the coronavirus outbreak, universities have moved from face-to-face assessment to administering examinations online.

Thus, online assessment has been one of the most significant challenges faced. Several problems have been encountered in conducting online examinations: students and faculty must be trained in using the proctoring tools as well as the virtual platforms; student misconduct, as faculty and students perceived that it is easier to cheat in this type of examination; technical problems such as internet disconnection, power outage, or family emergency could affect test taking; the need facilitate access to the necessary technology (high speed internet and computer with camera and microphone); privacy issues, as not all students accept video recordings; as well as other problems [7,31]. Research also shows that online assessment in higher education promotes students’ motivation, satisfaction, outcomes, attitudes, and skills, which are highly effective for education for sustainable development [2].

In this sense, teachers have had to employ different software that allows monitoring, through webcam or by blocking other applications and internet browsers. However, the transition to online exams can help teachers to facilitate the marking process in a faster way, especially for multiple-choice questions, as well as minimising the cost of printing on paper.

In this sense, through experiences in different universities around the world, we can find other alternative solutions to online assessment through exams, selecting tests with a more qualitative approach that invite students’ reasoning and reflection.

Information and communication technologies (ICT) have provided us with new assessment methods, but their success will depend on theoretical and practical knowledge on the part of teachers. In this sense, as a result of the coronavirus pandemic, university institutions should become aware and accelerate change in education, since online education is an accelerating element of social transformation that is here to stay, improving the education and training of teachers to address the problems posed by online assessment through different strategies.

Apart from the importance of these findings, it is necessary to consider the methodological limitations of this review. The analysis of 13 studies is very limited, so future research could draw on a wider range of sources to reach new conclusions. Furthermore, the results of this review may be useful for reflecting on the development of educational policies aimed at improving online assessment in higher education institutions.