Flipped Classroom and Gamification Approach: Its Impact on Performance and Academic Commitment on Sustainable Learning in Education

Abstract

The onset of the COVID-19 global pandemic has negatively impacted sustainable learning in education (SLE). During city lockdowns, higher education institutes (HEIs) have transitioned from adopting solely traditional didactic classroom teaching to including innovative, flexible learning approaches such as flipped classrooms. Gamification is a new techno-pedagogy that has been integrated into flipped classrooms to promote learner achievement and engagement. Grounded in self-determination theory, the objectives of this exploratory study were to analyse the influence of the flipped classroom and gamification on SLE concerning learner achievement and engagement. Participants were recruited from postgraduate business education programmes in China, and three instructional interventions were applied for a semester of 10 weeks. The three instructional interventions applied were: gamified flipped classroom (n = 25), non-gamified flipped classroom (n = 24) and gamified traditional classroom (n = 19). A mixed-methods approach was used, and both quantitative and qualitative data were analysed. The results indicated gamified traditional classrooms promote learner achievement, and the gamified flipped classrooms promote learner engagement. Furthermore, learning culture, such as teacher-dependency, also influence learner achievement and engagement. The class observation reports and learner interviews suggested that both gamified flipped classrooms and gamified traditional classrooms support SLE in the time of academic uncertainty during the COVID-19 pandemic.

1. Introduction

The onset of the COVID-19 global pandemic has imposed a negative impact on the Sustainable Development Goals (SDGs) of Education 2030 (SDG 4) declared by the United Nations in 2015 [1,2]. The goal of SDG 4 is to equip adult learners with the knowledge and skills necessary for the benefit of global sustainable development by providing flexible education pathways in higher education [3]. Higher education institutes (HEIs) have to equip adult learners with the knowledge and skills of sustainable learning in education (SLE) to cope with challenging and complicated circumstances [4,5]. Thus, HEIs are responsible for assisting adult learners in overcoming barriers and interruptions by providing sustainable quality education, which is essential for the country’s SDGs [6,7]. HEIs were compelled to rapidly redesign teaching approaches and classroom arrangements to offer flexible and sustainable learning pathways for SLE in the times of COVID-19 lockdown [8,9].

Instructors in China innovatively planned their teaching by providing learners with pre-class self-study materials during in-campus classroom lessons due to the uncertainty of intermittent lockdowns [10]. Advancements in information communication technology have made such learning arrangements that combine pre-class and in-classroom learning activities feasible. These arrangements are critical for the sustainability of education programmes under enforced home confinements and campus lockdowns during the pandemic [11,12]. Most importantly, these new pedagogical approaches can alleviate the threats of the pandemic on the sustainability of the country’s education system [13] and promote SLE amid COVID-19 [4].

Learners with flipped classroom approaches were provided with instructional materials for self-study at home before participating in practical learning activities in the classroom, reducing the reliance on face-to-face teaching instead of a traditional classroom arrangement that is totally reliant on it [14]. Learner achievement and engagement are two important indicators of the success of flipped classrooms in HEIs [15,16]. Research also indicates that the success of flipped classrooms depends on sustained learner engagement in learning activities [17,18]. According to self-determination theory (SDT), gamification has the potential to promote learner achievement and motivate learner engagement in the learning activities of flipped classrooms [19]. SDT is an empirically derived theory in social contexts that differentiates human motivation in terms of autonomous and controlled, in which autonomously motivated learners thrive in educational settings [20,21]. Gamification is the use of game-design elements in non-gaming contexts and has the potential to motivate learning [22,23]. Based on SDT, gamification is hypothesised to foster intrinsic motivation and satisfy the psychological needs for autonomy, competence, and social relatedness in flipped classrooms [19].

Flipped classroom approaches have also been adopted in HEI programmes during the COVID-19 pandemic for sustainable learning in higher education [24]. Instructors provide prerecorded instructional videos to enable learners to study on their own during home confinement periods. In addition to the flexibility, the flipped classroom approach also allows more in-class time for promoting SLE. The key components of SLE are (1) active learning; (2) independent learning; (3) collaborative learning; (4) renewing and relearning; (5) knowledge and skills transferability [4]. The key learning activities in flipped classrooms are pre-class self-study (i.e., independent learning), knowledge application (i.e., transferability), problem-solving (i.e., renewing and relearning), and peer-assisted learning (i.e., collaborative and active learning) [8,19,25,26]; these are especially advantageous in promoting SLE. HEIs in China flexibly adopted flipped classroom approaches in response to the government’s policy of “suspension of classes without interrupting learning” during the COVID-19 pandemic [27,28]. However, additional research is required to investigate the impact of the abrupt pedagogical change caused by flipped classrooms in the long history of Chinese education, which is dominated by traditional didactic teaching [29].

This study investigated the impact of the flipped classroom and gamification approaches on learner achievement and engagement in postgraduate business programmes for SLE in China during the COVID-19 pandemic. Three classes with different instructional interventions were applied: gamified flipped classroom (GFC), non-gamified flipped classroom (NFC) and gamified traditional classroom (GTC). Furthermore, the study was guided by the following two research questions:

• RQ1. How does the adoption of the flipped classroom approach influence (a) learner achievement and (b) learner engagement compared to non-flipped approaches?
• RQ2. How does the adoption of gamification influence (a) learner achievement and (b) learner engagement compared to non-gamified approaches?

2. Literature Review

We have the following three sections of the literature review. First, we discuss the SLE and flipped classroom approach during the COVID-19 pandemic. Second, we review how the SDT framework supports and motivates SLE and flipped classrooms. Third, we explore the integration of gamification with flipped and traditional classrooms for SLE.

2.1. SLE and Flipped Classroom during COVID-19 Pandemic

SLE is a philosophy of learning and teaching, which is not limited to education for or about sustainability [30]. SLE supports SDGs in education for adult learners, whose past knowledge and skills are viewed as flexible and receptive to modification [31]. SLE is an emerging and timely concept designed to enable learners to keep pace with technological and social changes [8]. HEIs with SLE initiatives provide learners with individual and group learning [32]. SLE promotes learners’ willingness to participate and contribute to the learning process, reflected in learners’ behavioural, emotional, and cognitive engagement in learning activities [33,34]. Ben-Eliyahu [4] summarised the key components of SLE as:

• Active learning: refers to seeking information actively and intentionally;
• Independent learning: refers to self-sustained learning, which involves learning by oneself, being autodidactic, identifying and finding sources for what needs to be learned;
• Collaborative learning is conducted in groups with peers of different levels of knowledge and is a process of acquiring and creating knowledge;
• Renewing and relearning is the self-reflection and awareness of our knowledge that might have been forgotten or need to be relearned because of advancements in a field;
• Knowledge and skills transferability: refers to using knowledge and skills learned in one setting (e.g., in the classroom) to another setting (e.g., at work).

SLE could be integrated into education settings with information communication technology (ICT), online courses (e.g., MOOC), and pre-class and in-class learning [8]. Such flexible pedagogical approaches are especially beneficial to ensure learning sustainability when classroom teaching is suspended during uncertain and complex situations [35].

The COVID-19 pandemic impedes learning sustainability by imposing academic uncertainty on HEIs. Learners reported negative emotions and decreased satisfaction due to interruptions to their sustainable learning [36,37]. HEIs in China needed to rapidly adjust their instructional practices to anticipate the changes in the country’s confinement policies [38]. In sync with this development, educators in China quickly adopted the flipped classroom approach with the aid of information communication technologies [28,29]. The flipped classroom approach is a technology-enhanced pedagogy that free ups class time by providing pre-class instructional videos [39]. Flipped classrooms emphasise learner-centric and problem-solving activities inside the classroom [40], which enable independent and collaborative learning [41]. Peer-to-peer and peer-to-teacher interactions promote active learning, and problem-solving activities encourage knowledge renewing and relearning of SLE [4,29]. In addition, problem-solving activities for knowledge transferability also enhance perceived learning and learner achievement [42,43].

Chinese learners are very concerned about the delays in their academic progress [44]. Therefore, they favoured flipped classroom approaches as they could benefit from the self-paced independent learning enabled by pre-class instructional materials, such as pre-recorded videos, even during confinement and closures of campus. They could then look forward to increased engagement in collaborative learning once classroom teaching was allowed [17]. However, the flipped classroom approach was not common in HEIs worldwide or in China before the onset of the COVID-19 pandemic [45]. HEIs in China have long adopted teacher-centric didactic pedagogy in which programmes are primarily conducted face-to-face with scheduled timetables [46]. The COVID-19 pandemic precipitated an abrupt transition in traditional classroom teaching approaches. Thus, additional research with a solid theoretical foundation is required to investigate the impact of this abrupt pedagogical change on learning, mainly learner achievement and engagement [24,47]. This is especially important during the interruptions and frequent transitions between pre-class and in-class modes of learning caused by the COVID-19 pandemic [48,49,50].

2.2. Self-Determination Theory

Motivation is the main catalyst for sustainable learning behaviour [51]. SDT posits that humans inherently possess the propensity to be curious and interested in learning and developing [52,53]. The desire for sustainable development of business executives and entrepreneurs helps to attain the SDGs of the country [54]. SDT states that when the three basic psychological needs of learners (i.e., autonomy, competence, and relatedness) are fulfilled, they are motivated and are more likely to engage in education contexts [55,56]. Learner engagement is characterised by learners’ investment in learning, desire to exceed standard requirements, and preference for challenges of varying difficulty levels [57]. Learner engagement includes behavioural, emotional, and cognitive dimensions [58]. Studies by Abeysekera and Dawson [40] and Sergis et al. [59] indicate that SDT could be drawn upon to support learning in flipped classroom environments. As less class time is consumed by lectures, learners have more opportunities for self-regulated, independent, peer-to-peer collaborative learning and hands-on problem-solving activities for knowledge transfer [59]. Hence, flipped classrooms facilitate autonomy, and supportive feedback from teachers and peers promotes competence and relatedness [60].

A recent study of HEIs during the COVID-19 pandemic found that learning persistence and sustainability were also directly impacted by learners’ needs for autonomy, relatedness, and competence [61]. Therefore, higher engagement resulting from motivation, as explained by SDT, is a prerequisite for the sustainability of education programmes during the COVID-19 pandemic [62]. Supportive feedback from teachers is especially crucial for engaging learners and can be conveyed via game-design elements such as points, badges, and leaderboards. Learner engagement is positively related to perceived learning and sustainable learning [5,63,64]. Thus, flipped and gamified classrooms based on the SDT framework may promote learning through knowledge transferability in problem-solving activities and collaborative and active learning [19,65]. Furthermore, the adoption of flipped classrooms and game-design elements based on SDT has the potential to promote SLE in HEIs during the COVID-19 pandemic [4,66].

2.3. Gamification

Gamification is the use of game-design elements in a non-game environment [67], which can be a sustainable method to achieve the United Nations’ Sustainable Development Goals for quality education (SDG 4) [68]. In the context of education, both gamification and SDT aim to promote learner achievement and engagement [69]. Gamification is typically used in flipped classrooms to provide feedback to encourage learners to take on task challenges with progressively increasing difficulty levels and to motivate independent and collaborative learning [70]. Feedback and challenges are important to promote learning achievement and engagement [71]. A progressive increase in the difficulty level intrinsically motivates learners to complete more challenging tasks, especially those tasks that are relevant to their personal goals [54,72]. One major personal goal for learners in higher education business programmes is to translate knowledge to practice (i.e., knowledge transferability), as it enhances their employability [73]. Gamification promotes active learning and engagement in problem-solving activities that bridge the knowledge-to-practice gap [74,75]. Game-design elements engage learners by helping them to build new knowledge (i.e., renewing and relearning) and support learning achievement [70]. Therefore, pedagogies with gamification promote SLE (i.e., renewing and relearning, collaborative and active learning, and knowledge transferability) [4,76].

The most common game-design elements used to motivate learning in higher education are points, badges, and leaderboards [22]. These game-design elements serve the following specific purposes [77] (

Table 1. Game-design elements and feedback.

Game-Design Element	Picture	Feedback	Description
Points		Granular feedback	- Rewards for specific actions and participation in learning exercises. - Encouragement for pursuing optional and increasing difficulty levels of tasks.
Badges		Sustained feedback	- Rewards and recognition for progress and contributions to problem-solving exercises in the activity groups (e.g., case studies).
Leaderboards		Cumulative feedback	- Ranking of learners in activity groups according to total badges earned.

):

• Points capture granular feedback directly related to learners’ specific actions, e.g., participation in in-class learning exercises and pursuing optional and challenging tasks.
• Badges capture sustained feedback to recognise learners’ progress and contributions to the activity group tasks.
• Leaderboards capture cumulative feedback on a series of actions performed by the learner and their contributions to completing tasks by displaying the number of badges earned in activity groups during the entire course.

Gamification can be adopted in flipped classrooms to motivate learners to increase their efforts and participation during the COVID-19 pandemic [78]. Recent studies found that gamification [79] and flipped classrooms [70] support positive learning achievement and engagement. Further, SDT provides the theoretical foundation on which gamification and flipped classroom pedagogies can be grounded [59,80]. Nevertheless, there is a paucity of research on gamification applied in flipped classrooms for business education among adults [81].

3. Research Methods

We adopted an explanatory sequential design with a mixed-methods approach in which both quantitative and qualitative research methods were used. When the quantitative phase is followed by the qualitative phase, it helps to explain the mechanism behind the quantitative results [82]. This approach can also be adopted to provide an insightful comparison between flipped classrooms with gamification to traditional classrooms in the existing study [60]. In this section, we first introduce participants, followed by the research design, data collection, and analysis.

3.1. Participants

Participants were adult business executives (aged 25–42 years) studying postgraduate business programmes of HEI in eastern China. Their participation was voluntary, and participants could withdraw from the study without negative consequences at any time. They were assigned to one of the three different instructional approaches (GFC, n = 25; NFC, n = 24; GTC, n = 19) for the second module in weeks 6–10. For flipped classrooms (i.e., GFC and NFC), the learners were provided with pre-class instructional videos recorded by the teacher via the institution’s learning management system (Moodle) four weeks before the scheduled in-class lessons. The videos were intended to impart basic knowledge before the in-class lessons. After watching the videos, the learners could complete self-evaluation multiple-choice quizzes during their free time. Owing to the high degree of trust and coordination between the government and Chinese higher education institutes [14], universities reopened soon after local COVID-19 cases were contained. Rigorous hygiene measures enabled the resumption of regular face-to-face lessons in classrooms. Teachers focused more on advanced topics during the in-class lessons once the institute was allowed to reopen. In this study, the learners in the flipped classrooms spent more in-class time on learning activities such as knowledge application for solving real and simulated business problems and assignment discussions.

The learners in the gamified classrooms (i.e., GFC and GTC) were given an account and access code to Qitoupiao. Guidelines and descriptions of expectations throughout the pre-class (GFC only) and in-class learning stages (GFC and GTC) were provided. These learners were not graded or provided marks for the points or badges they gained in the learning activities to prevent them from aiming at high scores instead of being motivated by the game-design elements. A summary of the guidelines and necessary information was made available on Moodle, as shown in Figure 1.

Participants in the three classrooms had to submit a post-class assignment in an essay of around 2000 words after completing classroom lessons at the end of each module. Assignment questions mainly focused on knowledge application for solving real business problems the learners are facing at work. Furthermore, all assignment questions were evaluated and approved by the programme academic team and external examiners.

3.2. Research Design

The study was conducted in the context of adult postgraduate business education programmes in China during the COVID-19 pandemic from November 2021 to January 2022. The programmes consisted of two modules. Each module lasted for five weeks and consisted of 16 h of in-class lessons (eight hours each day for Day-1 and Day-2). Three kinds of intervention, namely the gamified flipped classroom (GFC, experimental group-1), the non-gamified flipped classroom (NFC, experimental group-2), and the gamified traditional classroom (GTC, experimental group-3), were introduced in the second module, which started at the sixth week. To evaluate the influence of flipped classrooms on learner achievement and engagement (i.e., RQ 1), we compared the GFC and GTC experimental groups, and to evaluate the influence of gamification on learner achievement and learner engagement (i.e., RQ 2), we compared the GFC and NFC experimental groups. The experimental design is summarised in

Table 2. Experimental design.

Experimental Groups and Approaches	Experimental Group 1: Gamified Flipped Classroom (GFC)	Experimental Group 2: Non-Gamified Flipped Classroom (NFC)	Experimental Group 3: Gamified Traditional Classroom (GTC)	Research Question and Group Comparisons
Flipped classroom	Yes	Yes	No	RQ1: GFC and GTC
Gamified classroom	Yes	No	Yes	RQ2: GFC and NFC

.

3.2.1. Class Rundown

All in-class lessons were conducted face-to-face after city lockdowns were lifted. In the first module (weeks 1–5), the students in all three experimental groups learned under the traditional classroom approach (i.e., teacher-centric didactic approach with no pre-class videos and gamification). The assignment marks from the first module were recorded as pre-intervention references. The three instructional interventions were adopted for the respective experimental groups in the second module during weeks 6–10 (Figure 2). For the flipped classrooms (GFC and NFC), eight sessions of pre-recorded instructional videos with a duration of 30 min each were recorded by the module teacher and provided via the institution’s learning management system (Moodle). Self-evaluation multiple-choice quizzes with only ten questions were offered to avoid heavy cognitive load in the pre-class stage of learning [40,83].

3.2.2. Lesson Schedule

The face-to-face lesson schedules were the same across the three experimental groups for the first four morning hours of lecture lessons. This was followed by two hours of case study lessons in the early afternoon for all three classrooms (GFC, NFC, and GTC). The case study lessons served to expand the scope of learning and knowledge through peer interactive and collaborative learning. For the flipped classrooms (GFC and NFC), there were two hours of knowledge application and assignment discussion lessons in the late afternoon of Day-1 and Day-2, respectively. In contrast, for the non-flipped classroom (GTC), there were lecture lessons in the late afternoon of both Day-1 and Day-2. Figure 3 provides details of the lesson schedule.

The case study lessons format was the same for all three classrooms, which started with a business case relevant to the morning lectures. Learners were grouped into activity groups of five to seven peer members to discuss the case according to the guiding questions. For example, the learners discussed the case of emotional pricing and its implications for marketing strategy in Figure 4. Each activity group could then choose one of three levels of difficulty to present their results: a summary report of the discussion (Easy level), real application with examples (Medium level), or real application with a business plan (Hard level). The learners’ choices were recorded in class observation reports by teaching assistants.

For the two flipped classrooms (GFC and NFC), there were two hours of knowledge application (Figure 5) and two hours of assignment discussion (Figure 6) during the late afternoon of Day-1 and Day-2, respectively. In the knowledge application lessons, the teachers introduced a topic or questions that focused on applying the knowledge learned. For example, as shown in Figure 3, the learners learned different pricing strategies in the morning lessons and then attempted to apply them in real or simulated business scenarios. These lessons were intended to strengthen their ability to apply their knowledge for solving real or simulated business problems, which was knowledge transferability and most relevant to their jobs and employment. Lastly, the assignment discussion lessons on Day-2 allowed the learners to discuss the individual assignment questions and learn with their peers collaboratively. Peer-to-peer interactive and collaborative learning could inspire further thoughts, ideas, and solutions to the business problems listed in the assignment questions. Learners could renew and relearn in the process. Because the learners in the flipped classrooms (GFC and NFC) were aided by the knowledge application and assignment discussion lessons, they subsequently required less time to complete the same assignment writing required for all three experimental groups after the classroom lessons. By contrast, those in GTC had to work on their assignment writing from the beginning and required more time to complete it by themselves after classroom lessons.

Learner surveys, interviews, class observation reports, and written feedback were collected from all three classes at the end of Day-2 after the completion of all the in-class lessons of the second module (i.e., in week 10).

3.2.3. Application of Game-Design Elements

During the case study lessons, the learners were divided into activity groups of 5–7 peers per group. Game-design elements were applied for the gamified classrooms (GFC and GTC). To ensure that gamification was effective, points, badges, and leaderboards were displayed through the Chinese classroom application Qitoupiao (‘Voting Together’). These game-design elements and their use in the two gamified classrooms are described below and in

Table 3. Application of game-design elements for gamified classrooms (GFC and GTC).

Game-Design Element	Purpose	How They Were Awarded
Points	Feedback to encourage task completion and progress towards more advanced and challenging learning activities.	Point(s) for in-class learning and presentation activities based on level of difficulty: ● Easy—one point for a summary report ● Medium—two points for real application with examples ● Hard—three points for real application with a business plan
Badges	Recognition of learning efforts, participation, and contributions; promoting social recognition.	Badges were given to learners who presented good ideas, new knowledge or innovative solutions to a problem.
Leaderboards	Inspire intragroup peer learning and intergroup competition for learning advancement.	The number of accumulated badges obtained in each activity group.

.

• Points: Learners received points on completing tasks in the in-class learning activities. Points serve as feedback to encourage learners to complete subsequent learning activities and achieve more advanced challenges together in a group [84,85]. One point was allotted to activity groups that complete their discussion and presented the results as a summary report, which was the lowest level of difficulty (Easy) to complete their task. Two and three points were allotted to activity groups that chose the intermediate (Medium) or the most challenging (Hard) levels of difficulty to present their discussion results in a real application with examples and a business plan, respectively.
• Badges: Learners received badges when they provided innovative ideas and solutions to problems and questions during in-class learning activities. Badges serve as recognitions of a learner’s contributions and encourage participation during learning activities [60]. Badges also promote social validation as they provide opportunities for learners to show their conformity and progress towards the expected learning behaviour with their peers [86]. Learners in an activity group obtained one badge when any member of the group presented a good idea, new knowledge, or an innovative solution to a problem or question. A ‘good idea’ refers to a new way to apply the knowledge learned in the class, ‘new knowledge’ refers to a point that was not mentioned or taught in the class, and an ‘innovative solution’ is a solution to a problem-solving task that was not mentioned or taught in the class.
• Leaderboards: Teamwork was encouraged within each activity group and the learners were also encouraged to compare their performance with those of other activity groups in the same classroom [87]. Thus, the numbers of accumulated badges of each activity group were displayed on the class leaderboard. The leaderboard was intended to inspire intragroup peer collaborative learning and a healthy intergroup competition amongst the groups to achieve a prominent position on the leaderboard by contributing to learning activities for more badges [88,89].

3.3. Data Collection

Both quantitative and qualitative data were collected during the in-class and post-class stages (

Table 4. Data sources and collection.

Stage	Data	Purpose	RQs Addressed
In-class	Class observation report for levels of difficulty and participation	To evaluate learner engagement under the three instructional approaches (GFC, NFC, and GTC).	Learner engagement (RQ1 and RQ2)
Post-class	Learner survey	To evaluate learners’ perceived achievement and engagement under the three instructional approaches.	Learner perceived achievement and engagement (RQ1 and RQ2)
	Learner interview
	Assignment marks	To evaluate the learner’s overall achievement under the three instructional approaches	Learner achievement (RQ1 and RQ2)

). Class observation reports were collected by teaching assistants during the in-class stages. Data were from the post-class stage and were collected based on three sources: learner survey, learner interview, and assignment marks. The survey used a 5-point Likert scale that ranged from ‘Strongly Agree’ (5) through ‘Neutral’ (3) to ‘Strongly Disagree’ (1). The learner survey and interview focused on the learners’ perceptions and their suggestions for pedagogical improvement.

The effects of classroom approaches (i.e., GFC, NFC, and GTC) on learner achievement were evaluated based on the assignment marks. The teachers graded the assignment strictly according to the marking scheme and rubrics provided by the institute. All in-class learning activities, topics and assignment questions were approved by the school appointed external examiner and the academic programme committee. Thirty per cent of the assignment marks were countermarked and sent to the external examiner for final review to ensure accurate assessment. Discrepancies in marks were discussed and were resolved in the board of examiners’ meetings.

The learners were invited to complete a survey at the end of classroom lessons in the second module. The survey consisted of questions on four themes: (i) perceived learning (Items 1–3), (ii) behavioural engagement (Items 4–8), (iii) emotional engagement (Items 9–13), and (iv) cognitive engagement (Items 14–17). It also contained one open-ended question (Item 18). The learners were free to respond, and their responses were tagged using anonymised labels, e.g., GFC-Learner 1, NFC-Learner 2, and GTC-Learner 3. Sample items for each theme from the survey are presented in

Table 5. Sample items of the learner survey.

Survey Items	Supporting Citation
Perceived learning (Items 1–3) I learned more because of the classroom format. (Item 2)	Lo and Hew [60]
Behavioural engagement (Items 4–8) I participated in in-class activities and discussions. (Item 6)	Skinner et al. [90]
Emotional engagement (Items 9–13) The class was fun. (Item 11)	Skinner et al. [90]
Cognitive Engagement (Items 14–17) I was so involved that I forgot everything around me. (Item 17)	Rotgans et al. [91]

.

Qualitative data sources included Item 18 of the learner survey, class observation reports, and learner interviews. Class observation was adopted for evaluating learner engagement in in-class activities in accordance with the recommendation of a prior study [92]. Studies have found learners’ behavioural and cognitive engagement to be reflected in their participation in learning activities [60,93]. Another study found that the willingness and effort to exceed minimum requirements and adopt more challenging presentation methods indicated a higher learning engagement [94]. Thus, teaching assistants recorded class observation reports which focused on two aspects of the learners’ engagement: (1) the level of difficulty that they chose for their case study results presentation and (2) their learning participation. Learning participation ranges from passive receiving, active manipulating, and constructive generating to the most engaging interactive dialoguing [92]. Learner engagement and their participation also reflected the different components of SLE [4]. Teaching assistants recorded the learners’ participation in in-class activities by checking one box each for the level of difficulty and level of participation, respectively, that most closely matched their observations of classes during the in-class case study lessons (Figure 7).

Interviews help to understand learners’ behaviour, feelings, and interpretation of the learning environment [95]. Therefore, we conducted learner interviews after classroom lessons in the second module (Week 10). The interview adopted a semi-structured approach with a protocol (

Table 6. Sample questions of the semi-structured learner interview protocol.

Dependent Variable	Question	Citation Reference
Behavioural engagement	How did the classroom approach change the way you prepared for studying this module differently from other or previous classes you have attended?	Fredricks et al. [96]
Emotional engagement	What did you find most/least interesting in your studying of this module?	MaCullum et al. [97]
Cognitive engagement	Did you do anything extra that helped your learning when studying this module

). The topics covered in the interview were based on the engagement framework of Fredricks et al. [96] and McCallum et al. [97]. Topics for exploring learner engagement included (1) behavioural engagement (e.g., task participation and completion), (2) emotional engagement (e.g., enjoyment or boredom), and (3) cognitive engagement (e.g., investing effort in learning). All interviews were conducted and recorded in Chinese by the researcher. Some transcripts were translated into English for reporting purposes. The interviewees were invited to check all original transcriptions, and discrepancies were corrected to ensure accuracy. Learners’ participation in the survey and interview was voluntary, and no incentive was offered. After the completion of the second module lessons, we received responses for 49 surveys (GFC n = 20, NFC n = 17, GTC n = 12) and 26 learner interviews (GFC n = 7, NFC n = 11, GTC n = 8).

3.4. Data Analysis

3.4.1. Quantitative Data Analysis

To determine the most appropriate statistical test for quantitative data analysis, quantitative data were first tested for normality [98]. A Kolmogorov–Smirnov test showed a significant deviation from normality for learner achievement (according to assignment marks) across the three classes (first module p < 0.001; second module p = 0.017). Therefore, non-parametric tests were used to analyse the quantitative data on learner achievement [98].

The Kruskal–Wallis non-parametric test for multiple group comparisons was performed on the assignment marks from the first module, which was taught in traditional classrooms, to check the initial equivalence of the three experimental groups (i.e., GFC, NFC, and GTC). The results showed that the three experimental groups were statistically equivalent (H = 0.256, p = 0.880) for the first module. The assignment marks from the second module of the three experimental groups were then tested with the Kruskal–Wallis test at a significance level of 0.05. Multiple Mann–Whitney tests were conducted for post hoc pairwise comparisons when significant differences were found [98]. Bonferroni correction was applied to avoid Type I errors [99]. Therefore, the post hoc analyses reported effects at a significance level of 0.05/3 = 0.0167. The effect size (r) was then calculated using the following formula [100]:

$$r=\frac{z}{\sqrt{N^{\prime }}}$$

where z is the z-score, and N′ is the number of participants in the two experimental groups in each pairwise comparison.

3.4.2. Qualitative Data Analysis

The qualitative data were transcribed in Chinese and analysed thematically by organising the data into categories. Direct quotations from participant interviews were used to ensure data validity [101]. The qualitative analysis procedures followed the suggestion of Creswell et al. [82]. Coding started with the shortest interview transcripts using some exact wordings of the participants and concepts from the literature. All assigned codes were reviewed and grouped with redundant codes to produce a preliminary list of codes, which were then used to analyse the rest of the qualitative data. Exemplary quotes that clearly illustrated new emerging themes were identified and added to the list of codes, and similar codes were organised into subthemes.

Double-coding was adopted when data were descriptively and inferentially meaningful but not in neat or isolated units [102,103]. Thirty percent of the qualitative data were double-coded by an independent coder, and the intercoder agreement rate was checked. Disagreements between the coders were resolved through discussion. The data were validated by member checking to avoid any misunderstanding or misinterpretation [104]. Qualitative data were translated into English for reporting in this study.

Qualitative data from the written comments, feedback from learner surveys (Item 18), teaching assistant interviews, and class observation reports and teacher interviews were analysed with triangulation for comprehensive understanding [105] and to better explain the quantitative results [82].

4. Results

4.1. Learner Achievement

The Kruskal–Wallis test indicated a significant difference between the three classrooms in terms of the assignment marks obtained in the second module (H = 7.550, p = 0.023). Figure 8 shows a boxplot of the results.

Mann–Whitney tests for pairwise comparisons showed no statistically significant differences between GFC and GTC (p = 0.199) or between GFC and NFC (p = 0.117). However, the learners in GTC scored significantly higher assignment marks than those in NFC (U = 112.00, z = −2.875, p = 0.004) with an effect size r = 0.410.

Table 7. Pairwise comparison of assignment marks between NFC and GTC for the second module.

Assignment Marks	n	Mean	SD	Mean Rank	Pairwise Comparison
NFC	24	75.44	3.73	17.17	GTC > NFC *
GTC	19	78.79	3.05	28.11

* p < 0.0167 (Bonferroni correction).

shows the results for the pairwise comparison of assignment marks between NFC and GTC.

Although the qualitative data from the interviews revealed several benefits of gamification for learning achievement, the learners trusted knowledge delivered by teachers more than knowledge received from their peers, regardless of whether their peers had more firsthand experience in specific business scenarios. Interviews with both learners and teaching assistants indicated that the knowledge learned from teachers was reflected more frequently in the learners’ submissions of their assignments, which required them to apply the knowledge learned in the module to provide a solution to a real business problem in around 2000 words.

Table 8. Benefits of GTC.

Theme	Example Quote
Excitement/curiosity	‘Very excited, and engaged’ (GTC-Learner 3)
Peer learning	‘Classmates encouraged each other to choose the difficult learning option’ (GTC-Learner 8)
Exchanges/interactions	‘More feedback from the teacher, enabled by the rhythm of theteachingprocess, made me more attentive’ (GTC-Learner 5)

shows the major benefits of GTC identified from the learner interviews. These benefits included the generation of excitement and curiosity and the promotion of interactions and discussions in the classroom. In contrast, the learners in NFC did not experience a sense of ‘belonging’ (NFC-Learner 8) to the class. Keywords that occurred multiple times in the interviews of learners from NFC were ‘boring/dry’ (NFC-Learners 1 and 7) and ‘no interaction makes me sleepy/passive’ (NFC-Learners 8 and 10).

4.2. Learner Engagement

The Kolmogorov–Smirnov test showed that the data from the learner surveys significantly deviated from normality (p < 0.001); therefore, nonparametric Kruskal–Wallis tests were performed. The difference between GFC and NFC was significant (p = 0.014) for perceived learning. Post hoc pairwise Mann–Whitney tests showed that the learners in GFC scored significantly higher than those in NFC (U = 102, z = −2.448, p = 0.0140), with effect size r = −0.402. These results are shown in

Table 9. Comparison between GFC and NFC for learner survey Item 2.

Item	Survey Question	Class	n	Mean	SD	Mean Rank	Pairwise Comparison
2	I learned more because of the classroom format.	GFC	20	4.80	0.41	22.40	GFC > NFC *
		NFC	17	4.35	0.61	15.00

* p < 0.0167 (Bonferroni correction).

.

Class observation reports reflected that the learners in GFC engaged at the interactive dialoguing level during learning activities [93]. This shows that GFC was the most engaging of the three classrooms and performed all five SLE components. Those in NFC engaged half their time at the active manipulation level and the other half at the constructive generating level. This reveals that NFC was the least engaging among the three classrooms. Gamification elevated the participation level of learners in GTC to constructive generating, which showed three out of the five SLE components, indicating that GTC was more engaging than NFC (

Table 10. Level of difficulty and participation in in-class learning activities from class observation reports with SLE components. The order of the classrooms in terms of learner engagement and the number of SLE components that were performed was GFC > GTC > NFC.

Classroom	Level of Difficulty	Level of Participation	SLE Components
GFC	Medium	Interactive dialoguing (100% *)	Active learning Independent learning Collaborative learning Renewing and relearning Transferability
NFC	Medium	Active manipulating (50% *)	Active learning Independent learning
		Constructive generating (50% *)	Active learning Independent learning Renewing and relearning
GTC	Medium	Constructive generating (100% *)	Active learning Independent learning Renewing and relearning

* % of the time learners engaged at the level of participation.

).

Keywords that occurred multiple times in learner interviews from the gamified classrooms were ‘active participation’ (GFC-Learners 2 and 8), ‘focused and engaged’ (GFC-Learner 7, GTC-Learner 3), and ‘the class was interesting, inspired my thoughts and stimulated deep learning’ (GFC-Learner 5). One point of contrast in the feedback between GFC and GTC related to peer-dependent vs. teacher-dependent learning:

GFC-Learner 2: ‘The presence of such exceptional classmates made me more actively engaged (in learning)’.

GTC-Learner 5: ‘In the classroom, I followed the rhythm and pace of the teacher, who enabled me to focus during the interactive learning lesson’.

The feedback from the interviews with those in NFC, in contrast, reflected that the learners faced certain obstacles to active participation:

NFC-Learner 1: ‘I wish the teacher could control classmates that engaged too much in irrelevant discussions.’

NFC-Learner 5: ‘I am not very familiar (close) with the classmates, and the (learning) interactions and bonding between us were not strong’.

In summary, there was a significant difference in learner achievement between GTC and NFC (p = 0.004). The learners in GTC scored significantly higher than those in NFC for learner achievement (U = 112.00, z = −2.875, p = 0.004), with effect size r = 0.410. Regarding learner engagement, the results of class observation reports and learner surveys indicated that the learners in GFC had the highest level of participation and perceived learning, and performed all five components of SLE.

5. Discussion

In this study, we compared the influence of flipped classrooms and gamification on learner achievement and engagement for SLE in the context of postgraduate business education during the COVID-19 pandemic. The findings are discussed in the following three subsections. First is the role of flipped classrooms and gamification during the COVID-19 pandemic. Second, teacher-dependency and learning interactions. Lastly, lessons learned for the further improvement of flipped classrooms and gamification pedagogical design for SLE in HEIs.

5.1. Role of Flipped Classroom and Gamification during COVID-19 Pandemic

Flipped classrooms were welcomed by learners [106] and helped to maintain the sustainability of learning programmes in higher education during COVID-19 lockdowns [24]. However, learner motivation declines in the absence of learning interactions [48,107]. Gamification promotes learning interactions and social networking [108]. The qualitative results of class observation reports from the teaching assistants on learner engagement concur with those of Lo and Hew [60] that learners in gamified classrooms (GFC and GTC) exhibit a stronger sense of engagement, as evidenced by their level of participation during in-class learning activities. In the context of adult postgraduate business education, the quantitative results of the post-class assignment in our study showed that GTC offers advantages for learner achievement and GFC for learner engagement, respectively. GFC improved learners’ perceived learning and engagement, but there was no significant impact on learner achievement. Several other studies have found similar results, suggesting the need for further investigation [109,110,111]. Our results confirm the observation of Bredow et al. [81] that the benefit of a simple flipped classroom (i.e., NFC) may not be apparent. According to McLean et al. [110], one reason for this might be that flipped classrooms challenge the learners’ perception of the teacher’s role as a knowledge provider to playing the combined role of knowledge application and synthesis facilitators.

5.2. Teacher-Dependency and Learning Interactions

A prior study found that, despite the autonomy that the flipped classroom pedagogy affords to learners, in-class interactive learning was still preferred over pre-class self-regulated study [39]. Peer-to-teacher interaction and learner preference for teachers to play the role of authority for knowledge delivery are dominant themes in the Asian context [112]. In flipped classrooms, teacher roles change from being like those of sages to those of facilitators, thereby conflicting with existing expectations [113]. Like previously published findings [114], our results found better learning achievement in a traditional classroom approach supplemented by gamification (i.e., GTC), as teacher recognition and feedback in the form of game-design elements were valued more than responses from peers [115]. Such teacher dependency was also observed in previous studies [116,117,118]. Teachers should introduce the importance and value of lesson content at the beginning of each lesson [119], offer a brief review of pre-class materials [120], and facilitate peer-to-teacher and peer-to-peer interactions [25].

In-class learning interactions played an important role in flipped classrooms [121], and there is a need to explore this in further detail [60,122]. Learning interactions can be either peer-to-peer or peer-to-teacher interactions. Our interviews with the teaching assistants indicated that the learners were used to a teacher-dependent style of learning, which may impede peer-to-peer learning. Learners needed the teacher to motivate the discussion when the learners’ participation waned. The following is a transcript from an interview with our experienced teaching assistant (JG):

‘Although the flipped classroom and gamified classroom may be the trends in future education, our learners are relatively older adults, and the education model that they experienced during their school ages was traditional. Many people are still inclined towards the traditional methods of education. We have to use different teaching modes according to the learners and their preferences and rely on teachers to facilitate changes’.

Therefore, to facilitate SLE in flipped classrooms, the cultural background and learning styles of the learners, especially the type of schooling that the learners experienced, must be considered. This is imperative if more instructional content is going to be imparted using the flipped classroom and gamification approaches during the COVID-19 pandemic [24].

5.3. Lessons Learned

The flipped classroom and gamification designs adopted in this study were theoretically grounded in SDT [56]. The three pedagogical interventions (GFC, NFC, and GTC) provided valuable insight into the practical application of the theories in the context of adult business education during the COVID-19 pandemic. Based on our findings, we provide three observations and recommendations for the design of flipped classrooms and gamification for SLE.

First, gamification plays an important role in motivating adult learners for learning participation and engagement, which in turn promotes SLE [123]. From the perspective of SDT, timely and evaluative feedback from teachers is vital to promoting learners’ sense of engagement [55]. Second, teachers should facilitate learning interactions at the earliest stage of flipped classrooms, such as at the beginning of the pre-class learning stage. Similar inferences have also been made in other studies [112,124]. Knowledge learned from pre-class materials must be recollected to set as the foundation of more advanced in-class learning [125]. This is critical, especially in a highly teacher-dependent learning culture. Teachers’ early participation and facilitation in both pre-class and in-class lessons would trigger learners’ curiosity and interest and stimulate initial discussions, leading to more learning participation [83]. Last, although we used the same game-design elements in both GFC and GTC, learner achievement in the flipped GFC was lower than that of the traditional GTC. Our results resonate with the findings of Jensen et al. [124] that the adoption of flipped classrooms to create more time for in-class learning may not be effective even with gamification. Teacher-dependency and learning culture must be considered. Adult learners in our study benefitted more from teacher-led lectures (i.e., from the traditional GTC) than the two flipped classroom approaches (GFC and NFC). This mirrors the study of Magana et al. [126].

Nonetheless, the flipped classroom is an effective pedagogical approach to ensuring the sustainability of education programmes in HEIs during COVID-19 lockdowns [24]. Pre-class self-study videos and materials ameliorate the anxiety of learners caused by academic uncertainty and interruptions in learning [127]. For example, one learner from GTC remarked, ‘if the school can provide pre-class videos, I can prepare in advance, integrate my previous knowledge and develop clarity on the areas in which I need to learn more (in the coming in-class lessons)’ (GTC-Learner 3). Therefore, a flipped classroom with gamification is a valuable pedagogical approach for the sustainability of higher business education, especially under the dynamic COVID-Zero strategy in China [128]. However, teachers cannot rely solely on flipped classrooms or gamification pedagogies. Indeed, the early presence of teachers along the entire learning journey, including in the pre-class stage, to facilitate peer-to-teacher and peer-to-peer facilitations is deemed necessary.

6. Conclusions and Recommendations for Future Study

This study compared learner achievement and engagement across three pedagogical approaches (i.e., GFC, NFC, and GTC). The learners in GTC performed the best in terms of learner achievement, whereas those in GFC reported the highest perceived learning and engagement for SLE. Qualitative findings from surveys and class observation reports reflected that gamification made the lessons more interesting and exciting, and promoted participation in in-class learning activities. Learners in GTC and GFC showed higher levels of participation and performed more key components of SLE, which were the reasons for higher learner achievement and engagement in comparison to NFC.

We provided three recommendations based on the study. First, flipped classrooms can be leveraged to maintain the continuity and sustainability of education programmes, especially under the uncertainty of lockdowns during the COVID-19 pandemic. Second, gamification plays a key role in improving learner achievement, and should therefore be integrated into flipped classrooms with the consideration of the learning culture and styles of the learners. Third, teachers must facilitate and be involved in all stages of flipped classrooms, especially in highly teacher-dependent learning cultures.

Notwithstanding the aforementioned contributions, there are certain limitations in the study. First, the sample sizes of the three classrooms were small, which reduces the generalisability of the results. Second, the study was conducted in a postgraduate adult business education setting and may therefore not be equally applicable to other educational contexts. Third, the duration of the study was limited to 10 weeks owing to restrictions caused by the COVID-19 pandemic. Therefore, the findings must not be interpreted as establishing that any one of the pedagogical approaches is better than the others. Further studies are necessary to explore the differences between flipped classrooms with and without gamification in larger samples and across a longer duration.