Practicality of Ad Hoc Online Assessments for Teaching ESP in Online Flipped Classrooms during COVID-19 ^†

Abstract

The COVID-19 pandemic has severely affected the livelihood of many ever since its first detection nearly a year ago. Up until now, almost every domain of societal norms has been drastically impacted—particularly education. Many people immediately opted for Internet-based apps and programs within their capabilities and financial means to cope with the learning progress for themselves and the people around them. Learning institutions and teaching staffs worldwide quickly adopted these technologies and later adapted everything within online contexts. From there, extensive changes had been made to cope with the teaching/learning issues caused by the global pandemic. By utilizing an adapted questionnaire, this study discussed and evaluated the practicality of online assessments that had been implemented posthaste to assess an Aviation English course for aircraft maintenance undergraduates in flipped classrooms. The discussion findings indicated that while there were certainly a number of issues that can be quickly improvised in the near future, the whole implementation was definitely a success—especially with the ongoing limitations experienced by both educators and students.

1. Introduction

Online assessments are not something new, and it can be considered as something common amongst many learners and educators. However, most learning institutions still utilized physical assessments up until the beginning of the COVID-19 pandemic in early 2020. No one would have expected all the ongoing physical assessments to be converted into online assessments. It has been reported that an estimation of 1.6 billion learners worldwide have been affected by the pandemic. The number closely reflects more than 90% of the world’s learner population from over 190 countries across all the continents. To make things worse, prior global issues concerning learners such as educational disparities and learning opportunities for all children regardless of socio-economic statuses have yet to be solved in many parts of the world [1].

Despite these hardships, educators and administrators worldwide have kept on experimenting and implementing numerous ways to overcome the problems caused by the lockdowns. One of the most common solutions is conducting online assessments [2] for all, which has always been an issue of resources and connectivity [3]. However, the emergency situation for many countries had hindered governments and policymakers from making drastic changes within a short period of time; hence, the ultimate lack of options has rendered online assessments to be the most convenient and practical method to continue the education flow [4]. In Malaysia, it all started in mid-March 2020; some of the first higher learning institutions (HLIs) that had gone online before others did the same were Universiti Malaya, Universiti Putra Malaysia, Universiti Sains Malaysia, Universiti Kebangsaan Malaysia, Xiamen University Malaysia, Universiti Malaysia Sabah [5] and Universiti Kuala Lumpur. In the midst of rush, confusion and panic among many people during these times, there were also rational, logical and calm decision-making processes involved. With careful planning and quick adaptation skills, educators began to initiate migrating contents into digital clouds and teaching them in online assessments—including the researcher who was lecturing Aviation English courses at the time.

As a subdomain under English for Specific Purposes (ESP), it was unbeknownst to the researcher regarding any previous documentations or publications produced to specifically highlight on matters concerning teaching Aviation English courses in flipped classrooms with a full online mode. The emergency situation entailed a number of possible risks and consequences; hence, the researcher has decided to evaluate if the online assessments carried out were practical or not—from the students’ perspectives.

2. Literature Review

To reiterate the previous statement, assessments carried out in online classes are definitely not something novel; even for teaching ESP, it is actually quite common and has been practiced by many for quite some time now. Quite recently, a group of researchers had even outlined extensive information on integrating ICT for teaching ESP [6]. One of the critical issues discussed was how can educators incorporate ICT elements with their highly-trained pedagogical methodology concurrently and effectively. It would be inaccurate to say that using PowerPoint slides to showcase ESP information is already sufficient to be considered as “integrating ICT elements with teaching ESP”. There are at least several conditions that need to be fulfilled to justify the notion.

First of all, both educators and students must have satisfactory background knowledge with technological components that may be used throughout the entire course. It would include basic system setup, possible applications, and accumulated experience of having utilized the components in previous occasions. Next, teaching ESP is ultimately still teaching how to use the English language appropriately within the intended domain [7]. Therefore, sound knowledge of assessing the acquired linguistic knowledge within the technological components’ parameters is vital. In addition, this could only be carried out if the contents of the assessments were taught using an effective collaboration of pedagogical skills and technological contexts.

For instance, a group of educators from several universities located in different countries had proven that a flipped classroom can be taught online effectively [8]. Although their participants were undergraduates taking the online assessments for engineering-based courses, it would still be relevant to be adapted and systematically applied in courses from other domains. The main focus still reflects both online delivery methods and online assessments performed. As stakeholders, the students themselves would occasionally know and realize the fact that this distinctive transition from physical assessments to online assessments could possibly have short-lasting or long-lasting effects (or both) on their academic performance and psychological conditions. With the freedom to evaluate what is given to them, it was hoped that they would make the most out of this rare opportunity.

3. Methods

The objective of this study was to evaluate the practicality of online assessments implemented in an Aviation English course from the perspectives of undergraduates learning in flipped classrooms.

Prior to answering the questionnaire, the students had managed to undergo flipped classrooms (with their respective class groups) once a week; (i) in physical classes from Week 2 to Week 5, and (ii) in online mode from Week 6 to Week 12. Supposedly, there would be two written tests scheduled (Test A in Week 6 and Test B in Week 9. However, due to the major conversion (from physical class to online mode) that happened in between Week 5 and Week 6, the researcher had promptly initiated alternative media for both written tests before Week 6 started. When the time came, the lecturer asked for a clear image of their answer script after Test A questions had been displayed in students’ Learning Management System (LMS) university portal. The test was conducted within 60 min, just like how it would normally be if the test were to be given in a normal physical class.

However, for Test B in Week 9, the researcher not only utilized a different medium (Google Form, for both multiple-choice and open-ended questions), but it was also carried out with extended test time. The supposed 60 min was extended to 180 min as per requested by many students right after Test A was finished. The request was deliberated with administrators, and permission to extend the test time was granted because of the extenuating circumstances faced by students. Due to the societal and financial impacts of COVID-19, a number of students had limitations when it came to technological and Internet networking capabilities.

3.1. Participants

This study utilized a specifically tailored questionnaire adapted from Chaisuriya and Shin (2019) [9] via Google Form to collect data from undergraduates enrolled in Aircraft Maintenance Technology programs at Universiti Kuala Lumpur—Malaysian Institute of Aviation Technology (UniKL MIAT), Malaysia. During the point when this study was initiated, all of the nearly 400 students who took the Aviation English course had already begun continuing their flipped classrooms in online mode by utilizing Microsoft Teams. They were divided into three different program majors; each major had at least three different groups with maximum of 28 students per group. Hence, cluster random sampling was used to ensure representatives from all three program majors were selected. For that, the Google Form link of the questionnaire was posted online—on their respective groups’ Microsoft Teams announcement walls.

The data was collected from Week 15 to Week 18 of Semester January–June 2020. Every step of the data collection process had been performed virtually following the strict COVID-19 protocols imposed by the university. The virtual questionnaire was optional and not related to any of their course assessments. Hence, the number of participants was expected to be on the lower side. With the cluster random sampling technique, it was recorded that only 184 undergraduates answered the questionnaire.

3.2. Instruments

The questionnaire had four sections; two close-ended and two open-ended sections. The demographic section, which was the first section, was brief to prevent bias elements and ease students to do it online without any reservations: it had age, current location, name of program, and name of major.

The second section had 10 statements reflecting on their perceptions toward the online assessments implemented for the Aviation English course. The statements considered several criteria that can be associated with practicality such as format, convenience, awareness, availability, and frequency [9]. The third section had five statements pertaining to the assessments’ contents and mode of delivery. Both the second and third sections utilized a 4-point Likert scale (1-Strongly disagree; 2-Disagree; 3-Agree; 4-Strongly Agree) to questionnaire the students’ agreement/disagreement with the statements listed. The researcher thought that it was imperative to omit any ‘neutral’ standpoint in this matter (such as neutral, not sure, don’t know, etc.) as this would concern prompt real-life implementations and therefore could possibly have implications in the near future. The values for interpreting the findings are as in

Table 1. Percentages for each proficiency level.

Mean (M)	Interpretation
1.00 < M < 1.50	Strong disagreement
1.51 < M < 2.50	Disagreement
2.51 < M < 3.50	Agreement
3.51 < M < 4.00	Strong Agreement

below:

The data collected was quantified with SPSS software to calculate the mean and standard deviation; the numbers were then be interpreted as final conclusions of the questionnaire. The fourth section was an open-ended question that gave the opportunity to all participants to write their suggestions and/or personal feedback regarding their abnormal semester, which saw a drastic change from physical assessments to online assessments in the middle of the semester.

4. Results

The results in

Table 2. Mean and standard deviation for second section.

Online Assessments’ Practicality	M	SD	Meaning
1. The format of the test seems familiar to me.	3.50	0.789	Agreed
2. I can only attempt this test once.	3.46	0.816	Agreed
3. It is convenient for me to attempt this online test.	3.55	0.759	Strongly agreed
4. It is convenient for me to submit this online test.	3.42	0.852	Agreed
5. I am fully aware that this test exists virtually.	3.50	0.843	Strongly agreed
6. I do not need additional guidance on how to view this test online.	3.48	0.843	Agreed
7. Good Internet connection is readily available in my current location.	3.33	0.814	Agreed
8. Nowadays, I can communicate with lecturers and classmates anytime.	3.48	0.843	Agreed
9. I have no problem taking online tests for more than once this semester.	3.38	0.752	Agreed
10. Having more online assessments is better than physical assessments.	3.52	0.789	Strongly agreed

and

Table 3. Normality of distribution for both groups’ pre-test scores.

Assessments Contents and Mode of Delivery	M	SD	Meaning
1. The number of questions in the online assessments are sufficient.	3.55	0.730	Strongly agreed
2. The time limit for the online assessments is adequate.	3.50	0.803	Agreed
3. The speaking assessment was done without individual presentation because of time constraint and extensive mobile data usage.	3.55	0.715	Strongly agreed
4. The audio files for listening assessment were clear enough.	3.46	0.788	Agreed
5. It is better to answer the open-ended question(s) in Google Form rather than writing the answers with pen and paper.	3.45	0.815	Agreed

show that majority of the 184 participants agreed on the practicality criteria asked in the questionnaire:

Overall, the numbers indicated that almost all students agreed with the statements above. The highest mean (3.55) was recorded for second section’s Question 3 and third section’s Questions 1 and 3. The lowest mean (3.33) was recorded for second section’s Question 7.

The small difference between the highest and lowest means proved that most students found the online assessments to be practical enough for their predicament of having to almost instantaneously adapt to online mode. One possibly major factor is students’ demographic background. Having born in the years post-2000, which had been dubbed as the starting point of modern and digital millennium, they were already exposed to extensive usage of Internet, mobile phones, computers and other technological hardware. They could cope with the sudden change by participating in the online assessments and attempting the online assessments. Regardless of their respective assessments’ marks and achievements, their perspectives on the online assessments were clear—they were practical; at the very least, they were practical enough to be carried out for typical university students.

However, even though the numbers for both sections look positive in general, the minority did express their thoughts with the Likert scale and their respective feedback. Figure 1 below shows some of the students’ feedback as to why there were (at most) six students who marked “1-Strongly disagree” and 14 students who marked “2-Disagree” for certain statements.

In this case, it can be acknowledged that a few students may have felt that the online assessments held did not meet their expectations in terms of quantity, as they requested for more assessments and also additional classes.

In addition to that, the suggestion, “Always give a chance tu student who fail do assigment because have some Internet problem”, may have involved a handful of them who had been affected by the subpar Internet connectivity in their respective housing areas. Whilst Malaysian government and several telco operators did launch a nationwide package of free Internet data for every local citizen (both provided 1 Gb of data for every registered phone number), the coverage limitation was still an issue.

In Malaysia alone, there have been many news reports that showcased pictures, videos and social media posts that described citizens’ hardship in accessing good Internet connectivity [10]. Most of them were located in rural and/or hilly areas without any telecommunications tower nearby. In addition, the free data was only made available starting from 1 April 2020, while telco operators made their free data package available starting from 10 June 2020—which was quite late for UniKL MIAT students’ semester that had begun in January 2020. Nevertheless, it is also important to note that the Malaysian government did extend this free 1 Gb free data assistance until 31 December 2020; from initially within a 10 h period daily (8am–6pm) to an updated availability of 24 h period every day [11]. It was not much, per se, but it did help a lot of students to download scaled-down videos of lectures and soft copies of notes and assessments (such as .pdf, .doc, and .ppt files). It was also crucial for students to be able to log into their student portal and email for various purposes.

Figure 2 shows a screenshot from another class, with the additions of more suggestions. The suggestion, “The lecturer should talk more loud and clear”, may indicate another technological problem—hardware and possibly software as well. This reflects the third section, which may be related to the audio files or the researcher’s voice whenever he was lecturing. For the record, the researcher had been using Fantech HG11 headset and Realtek HD Audio Manager software preinstalled with desktop computer’s Gigabyte motherboard. There have been instances when some default audio–video settings needed adjustments; for example, (i) when everyone logs into Microsoft Teams, users have the options to choose preferred audio and camera settings, and (ii) when multiple users had their microphone setting to be “unmute” and noises can be heard.

For some educators, they may perceive suggestions such as, “Do physical class”, (during COVID-19) and, “Give a lot of example in our daily life”, (while teaching Aviation English, which included real-life aviation contents and contexts) as less useful in a serious reflective teaching discussion. However, the researcher strongly believes that there were suggestions such as these because some students may need, or at the very least prefer, for their assessments to be more of an experiential learning experience rather than a plain, virtual class. Moreover, someone actually suggested, “Give more individual assignment than group activities”, which would be an indicator of intrapersonal learner qualities.

Another interesting aspect that can be seen from the suggestions is nonverbal communication, which is clearly reflected from, “Need to smile a lot” (while teaching). In this case, the researcher agrees with the notion of body languages such as facial expressions and postures do play a significant role in effective teaching. For online mode, the visual limitations are to be expected as the students can only see what their educators allow them to see. The external cameras’ screen resolution can never replace normal human vision. Both educators and students are bound to a limited frame, and it significantly limits the capabilities and effects of body languages.

5. Recommendations

Based on the discussions, the researcher has compiled specific recommendations for future researchers and practitioners to solve and prevent the problems faced by UniKL MIAT students not only in terms of online assessments but also some other things that had been suggested.

5.1. Number of Assessments

As per the respective syllabus requirements and teaching/learning standards set by learning institutions worldwide, educators do not have the flexibility or freedom to change the number of formal assessments for students; it can only be done under special circumstances with administrator’s permissions and systematic moderation.

However, that is not the case with informal assessments that do not affect students’ coursework marks. Hence, proper planning and effective communication strategies are essential, be it with colleagues who are teaching the same courses or with students. Usual pre-semester preparation will have to include additional online setups such as separate online groups and channels, various online templates for exercises and assessments, group email lists and online calendar items such as reminders and future tasks. Setting these up will indirectly ease assessment distributions later. Hence, educators can timely brief students on the possible number of assessments (for both formal and informal) from the get-go; the recommended frequency for this particular briefing is weekly or bi-weekly. Depending on the nature of courses taught, it can also be done monthly. This way, students can also prepare physically, mentally and technologically as they undergo the online learning process. It will also be advantageous should there be any sudden changes or events that can disrupt the teaching/learning flow; as everything has been planned out within specific periods, it will be easier to cross-check possible dates or sessions for replacement/retest purposes.

5.2. Internet Connectivity

In the event of weak Internet connection problem, several recommendations have been outlined based on personal experiences, directives from administrators, guidelines from the government and feedback from colleagues.

For UniKL, students had been advised to return to campus and stay indoors at all times as early as 5 April 2020. With 12 campuses across the Malaysian peninsula, students were also given options to mitigate their home-to-campus distance problem by staying at the nearest UniKL campus to their hometown. This was to avoid students from spending additional costs to pay for their travel fares. With strict enforcement from hostel fellows and security personnel, students could ensure good internal Internet connection whilst staying at a proper learning environment. This has also been adapted as standard practices amongst many public and private higher learning institutions in Malaysia as per the SOP guidelines announced by Ministry of Higher Education Malaysia [12].

Other viable options mainly include SIM cards; either specific perks and benefits at discounted prices offered by telco operators [13] with the extended awareness campaigns by university administrators; or free unlimited broadband data packages offered by universities and/or other organizations. UniKL offered this package to every eligible student who had legally registered as a person from a low-household-income family (also known as “B40”) starting from 12 June 2020—which was roughly two weeks before the new semester started in July 2020. Other higher learning institutions had already begun these similar efforts; some even had collaborations with external organizations to boost their funding, and they were able to extend their number of recipients significantly [14].

5.3. Learner Types

As experienced educators, it would normally be easier to identify learner types of students in a physical class rather than in an online class. For example, close observations on how students speak, interact, move, and react face-to-face would normally be adequate to have some basic ideas in determining whether they are interpersonal or intrapersonal learners. However, this is difficult for online assessments especially in situations when students had to mute their audio and close their camera for various reasons.

The silver lining would be the responsibility of the educators to have both audio and camera switched on from beginning until the end of their assessments. From this perspective, it would seem as if the class is already teacher-centered. This notion is not entirely wrong because the students would naturally have minimal participation rate. Thus, the previous recommendations for online assessments must be customized to include in-class activities as well. Students must be made aware of the fact that they would always have the chance to not only participate but to do so actively and interactively with both their classmates and educators alike. Occasionally, educators would be able to pinpoint relevant learner types in the class and carry out appropriate learning activities and practices for the students.

6. Future Research

More experimental action research [15] is needed to further evaluate the practicality issues of online assessments for other courses and domains, which include (but are not limited to) aviation [16,17,18], engineering [19,20,21], health [22,23,24] and management [25]. The multitude of educational programs offered by various levels of learning institutions have been mostly affected by the pandemic. As highlighted, the practicality of online assessments is paramount as the education world is still adapting to the new social norms, and educators need to ensure their assessments are still applicable and effective in measuring the levels of acquired knowledge gained.