Augmented Reality and the Flipped Classroom—A Comparative Analysis of University Student Motivation in Semi-Presence-Based Education Due to COVID-19: A Pilot Study

Abstract

Background: This study aims to analyze university student motivation during the period of semi-presence-based education due to the COVID-19 pandemic, based on the type of educational material used: Augmented Reality (AR) videos and traditional videos associated with the Flipped Classroom (FC) methodology. Methods: A sample of 129 university students were asked to take the Instructional Materials Motivation Survey (IMMS). Results indicate that participants in the study were generally more motivated by AR educational material than by FC material; although mean motivation levels with both instruments are extremely high, with scores of 4.57 and 5.64 for FC and AR, respectively. Conclusions: Therefore, a conclusion was met which determines that these results are a starting point to continue the research of the motivational impact of using different types of videos in active methodologies.

1. Introduction

Worldwide education have been greatly affected by the Covid-19 pandemic. Perhaps the most significant alteration, caused by the pandemic, has been the nearly immediate switch, from the traditional learning setting to an online/remote learning environment. This situation is having a direct affect at college, universities, and other institutions of higher learning at the worldwide level.

This development is uniquely historical from an educational philosophy perspective. At no time in history could human beings be virtually connected in almost all corners of the world. This technology extraordinary, however this sudden advancement in communication technology has forced the higher education system to make drastic changes to both content and pedagogic delivery. Modification to the higher to the education system would most certainly give greater influence to remote learning. This is no easy task, replacing face-to-face with virtual learning consequently requires teaching staff to change and provide educational methodologies and resources that optimize virtual classes.

As suggested by Castaño-Garrido [1], teaching staff must encourage their students to be active during virtual higher education classes. Activity and productivity can be increased by providing, both technological and pedagogical, advice and facilitating the overall management of the remote learning environment. According to Cebrián [2], a scenario of technological innovation in remote learning involved the teacher taking on a role of adapter and producer of educational materials in different media, and motivator and facilitator of resources.

The virtual classroom teaching-learning process has the ability develop metacognitive skills in students. This increases self-reliance and responsibility in their own learning process. Nevertheless, Sanglier-Contreras, Martínez-Cepa, Serrano-Fernández & Zuíl-Escobar [3], warn of gaps in the quality offered and some students sometimes show little to no motivation for this new modality. While not proven successful for all students, this pedagogical transformation leads teaching staff, almost prescriptively, to use motivational techniques to make virtual classes more appealing by using creative and active methodologies [4] as well as innovative educational resources that encourage, motivate and awaken the interest of students [5].

One of the leading emerging digital technologies in education is augmented reality (AR) [6]. AR enables students to interact with the physical world around them. Typically, this is accomplished by combining the three dimensions of objects generated on real images and videos [7,8,9]. This techno-pedagogical tool combines digital and physical information in real time [10,11].

Research also reveals that the use of AR in educational processes favorability in student attitudes and satisfaction levels. It is also shown to increase motivation to learn [12,13,14,15]. It is considered a highly motivating resource due to its relevant appeal that encourages students to explore owing to its great element of surprise [16].

Another academic motivational approach is titled, the Flipped Classroom (FC). FC is another pedagogical model used to promote effort and activate student cognitive resources. This methodological format is based on reorganizing time in and outside the classroom. FC fosters flexibility and active participation of learners in educational processes [17]. FC puts students at the forefront and teachers take on a role of coordinator and moderator and facilitator of educational materials.

FC must tak into account that this model revolves around three axes: (1) tasks in the classroom, (2) tasks outside the classroom, and (3) the assessment process [18]. Studies highlight that FC optimizes student performance, but above all, it increases interest and motivation towards the subjects studied [18]. This coincides with other authors who affirm that applying this model developed a positive attitude to learning [19,20]. Motivation is a psychological skill, and teachers must strive to develop this skill in their students. Motivation consists of activating, guiding and maintaining the desired conduct. Keller [21], designed a motivational model to provide teachers with necessary information on their students’ lack of interest in educational processes, based on developing four psychological variables: (1) attention, (2) relevance, (3) confidence and (4) satisfaction (ARCS). Sanglier-Contreras, Martínez-Cepa, Serrano-Fernández & Zuíl-Escobar [3], concluded their study by asking researchers for more analysis of psychological variables to verify how immersion of active methodologies influence students. Considering the aspects detailed above, the current research is proposes to describe and verify the level of motivation of university students in virtual classes during the COVID-19 pandemic, using active methodologies such as AR and FC.

2. Materials and Methods

2.1. Design and Participants

The research sample consisted of a total of 129 students from University of Seville: 48% were male (62) and 52% were female (67). The students sampled had a mean age of 20.44 years (SD: 2.31). Students were enrolled in Primary Education Degree (third year) and Physical Activities and Sport Science Degree (first year), and coursed “Fundamentals and Curriculum of Physical Education in Primary Education” subject and “Motor Skills and Systematics of Exercise” subject, respectively. The first group of students (n = 65; 20.3% male and 79.7% female, mean age 21.03 ± 1,96) from the Primary Education Degree, followed Flipped Classroom (FC) classes, while the second group of students (75.4% male and 24.6% female, mean age 19.86 ± 2.50) from the Physical Activities and Sport Science Degree used Augmented Reality (AR).

2.2. Instrument

The Instructional Materials Motivation Survey (IMMS) was used, Keller [21], based on Keller’s ARCS motivation model (1987), comprising 36 items divided into four factors: Attention (12 items), Relevance (9 items), Confidence (9 items), and Satisfaction (6 items). The IMMS measures learners’ motivation levels by applying a 7-point symmetrical. The Likert Scale, ranges from 1 “Totally disagree” to 7 “Totally agree”. There are also 10 reverse items (e.g. item 3 of the confidence subscale) within the IMMS instrument. In the reverse item, the lower score the learners give to the reverse items, the higher learners’ motivational score is. The scores of the reverse items were manually reversed. Higher total score values indicate a greater level of motivation.

To accommodate the Flipped Classroom and Augmented Reality courses setting, minor modifications were made to the IMMS survey. The proposal by Di Serio, Ibáñez & Delgado (2013) was used in this case, obtaining suitable internal consistency in both subscales (attention α: 0,94; confidence α: 0,82; satisfaction α: 0,92; and relevance α: 0,70) and the global scale (α: 0,96) according to the acceptable value of 0.70 established by McMillan (2008) and Lowenthal (2001).

2.3. Procedure

Degrees offered by the University of Seville are taught face-to-face, but during the 2020–2021 academic year, due to Covid-19, the university was forced to a blended learning approach. The students who participated in the study received blended classes consisting of a face-to-face (only 33% of students present on a rotating basis) and a virtual component (Online Learning Platform with videoconferencing rooms). Active methodologies were used in the subjects, some with AR and others with FC. The virtual learning platform used was Blackboard Learn, provided by the University of Seville.

In this research, AR digital technology was implemented among first year “Motor Skills and Systematics of Exercise” students of the Bachelor’s Degree in Physical Activity and Sport Science. The teacher designed a video with AR on mechanical and kinesiological aspects of movement in order to develop the specific content of the systematics of exercise block (Figure 1). The video was uploaded to the virtual platform for students to view and then analyze the content, both virtually and within the traditional face-to-face classroom setting.

As for the FC procedure followed, it was applied to third year students taking the “Fundamentals of Physical Education in Primary Education” subject, which makes up part of the Bachelor’s Degree in Primary Education. The implementation of this methodology made it possible to reorganize time, focusing on tasks in and outside the classroom. Content was taught using traditional explanatory videos, which were recorded in advance by teachers. Students were provided with videos through the virtual teaching platform while using the Kaltura Media tool, which they watched during virtual classes. During face-to-face classes, students focused on resolving disagreements, carried out tasks and checked what they had assimilated with teachers to follow a formative and shared assessment, answering a test that provided weekly feedback.

The survey was distributed to all students online, at the end of term and during class hours. The objective of the research was explained to students, ensuring them that their data would be anonymous and they were given instructions on how to fill out the survey. All students took part in the study voluntarily. The researcher was always present to resolve any doubts.

3. Results

Data were analyzed using SPSS version 26.0. The Kolmogorov-Smirnov test was used to assess data normality, recording a p = 0.00 and therefore non-normal distribution of data. Descriptive statistics were applied to calculate mean scores and standard deviations of the survey items and factors, and the Mann-Whitney U test was used to assess whether the difference in means between the groups was significant, establishing a significance level of p ≤ 0.05.

Specific differences between groups—Flipped Classroom (FC) versus Augmented Reality (AR)—in the 36 survey items distributed into four factors were analyzed first.

Table 1. Mean, Standard Deviation and difference in means by Attention factor item.

	FC		AR
Attention	M	SD	M	SD	Sig.
2 There was something interesting in the materials that caught my attention.	4.39	1.28	6.02	1.03	0.00 *
8 The technology applied caught my attention.	3.81	1.57	5.97	1.10	0.00 *
11 The quality of the material helped me to pay attention during the task.	4.30	1.54	5.89	1.27	0.00 *
12 (inverse) The material was so abstract that it was difficult to pay attention to it.	5.18	1.53	5.87	1.16	0.00 *
15 (inverse) The images, videos and texts I discovered in the lesson were not very attractive.	4.67	1.54	5.86	1.18	0.00 *
17 The manner in which information was organised by using this technology helped me to pay attention.	4.33	1.54	5.82	1.13	0.00 *
20 The information discovered in the activity awoke my curiosity to keep on learning.	4.16	1.41	5.51	1.33	0.00 *
22 (inverse) The amount of repetition in this subject sometimes bored me.	4.48	1.45	4.86	1.75	0.13
24 I learned some things in the subject that were surprising and unexpected.	3.78	1.37	6.28	0.96	0.00 *
28 The variety of subject material helped me to pay attention during the task.	4.06	1.54	5.94	0.96	0.00 *
29 (inverse) The subject material is boring.	3.42	1.40	1.72	0.99	0.00 *
31 (inverse) The subject has so much content that it is irritating.	4.90	1.59	5.70	1.45	0.00 *

shows data on Attention factor items. There are significant differences in 11 of the 12 items. Specifically, in all items except the item referring to repetition in the subject.

In all cases where differences are significant, these are oriented towards AR, with the mean being higher than that of the FC. Items 2, 8, 11, 17, 24, 28 and 29 stand out for showing a greater difference. Therefore, it can be concluded that AR attracts the attention of students more to the material and helps them to maintain their attention; it allows them to organize the information better in order to engage their interest and facilitates more variety in the use of materials, making it more appealing, which in turn allows students to learn in a novel and creative way.

Table 2. Mean, Standard Deviation and difference in means Relevance factor item.

	FC		AR
Relevance	M	SD	M	SD	Sig.
6 The content of the material is related to things I already know.	4.05	1.44	4.92	1.31	0.00 *
9 There were images, videos and texts that showed me how the material could be important to some people.	2.81	1.48	2.52	1.57	0.16
10 Completing tasks successfully was important to me.	5.95	1.41	5.66	1.13	0.04 *
16 The content of this activity is relevant to my interests.	4.75	1.43	5.38	1.55	0.00 *
18 The activity has content that can be used for your knowledge.	4.97	1.42	6.00	1.14	0.00 *
23 The content and audiovisual material of this subject make understanding the topic seem worthwhile.	4.03	1.35	5.98	1.15	0.00 *
26 (inverse) This subject was not relevant to my needs as I already knew the content presented.	5.96	1.23	5.78	1.49	0.67
30 I could relate the content of this activity with exercises in my daily life.	4.72	1.09	5.23	1.49	0.00 *
33 The content of this activity will be useful to me.	4.86	1.51	5.88	1.11	0.00 *

FC: Flipped Classroom; AR: Augmented Reality; M: Mean; SD: Standard Deviation.

once again shows significant differences in practically all items (7 of 9). In this case, mean scores are once again almost always higher for AR than FC. Only in the case of the importance the student gave to completing tasks successfully is the mean score of FC (5.95) slightly higher than AR (5.66). In all other cases, how content relates to prior knowledge, adaptation of content to interests, relationship with daily life or perceived usefulness, the mean AR score is higher than FC.

Table 3. Mean, Standard Deviation and difference in Confidence factor item.

	FC		AR
Confidence	M	SD	M	SD	Sig.
1 Upon first impressions, the task looked easy.	4.78	1.27	5.17	1.19	0.06
3 (inverse) This material was more difficult to understand than I would have liked.	5.18	1.35	5.78	1.08	0.01 *
4 After seeing the tasks and receiving the information about what to do, I knew what I was supposed to learn from this part of the subject.	4.63	1.67	5.85	0.97	0.00 *
7 (inverse) There was so much information that it was difficult to remember the important points.	4.34	1.47	5.26	1.42	0.00 *
13 While working on this task, I was learning the content of this part of the subject.	4.98	1.21	5.85	1.16	0.00 *
19 (inverse) The tasks of this subject were too difficult.	4.85	1.54	5.38	1.46	0.03 *
25 After working on the tasks, I was confident that I could pass the subject exam questions.	4.66	1.33	5.46	1.13	0.00 *
34 (inverse) I did not really understand the material of this subject.	6.06	1.08	6.03	1.36	0.70
35 The material was well presented and helped me to be confident in my learning of it.	4.17	1.53	5.92	1.00	0.00 *

FC: Flipped Classroom; AR: Augmented Reality; M: Mean; SD: Standard Deviation.

shows data on Confidence factor items. There are significant differences in 7 of the 9 items. Specifically, in all items except the item referring to how easy the subject is (3) or to not being able to understand the material (34).

Regarding the differences indicated, once again all cases show a higher mean AR score than FC. Students taking this subject using AR perceived the material to be simpler, better understood what they had to do and learn, were able to remember important points, learned the subject content, perceived less difficulty in tasks, and was confident in passing the exam subject and learning its content, more so than students taking the subject with FC.

All Satisfaction factor items (

Table 4. Mean, Standard Deviation and difference in Satisfaction factor item.

	FC		AR
Satisfaction	M	SD	M	SD	Sig.
5 Completing these tasks made me feel satisfied with my achievements.	4.70	1.76	5.78	1.11	0.00 *
14 I enjoyed this task so much that I would like to learn more about its contents.	3.86	1.51	5.60	1.11	0.00 *
21 I really enjoyed studying this lesson with this activity.	4.03	1.55	5.95	1.12	0.00 *
27 The learning achieved helped me to feel rewarded for my effort.	4.89	1.48	5.66	1.05	0.00 *
32 I felt good completing the tasks successfully.	4.80	1.58	5.69	1.03	0.00 *
36 It was a pleasure to work with such well-designed content.	4.12	1.58	6.38	0.91	0.00 *

FC: Flipped Classroom; AR: Augmented Reality; M: Mean; SD: Standard Deviation.

) show significant differences between FC and AR; the mean score was once again higher for AR.

Students who worked with AR felt more satisfied when completing the tasks, were more eager to learn, felt rewarded for their effort, were good at completing the tasks and enjoyed working on the content.

The

Table 5. Mean, Standard Deviation and difference in factors and global scale.

	FC		AR
	M	SD	M	SD	Sig.
Attention	4.38	1.06	5.83	0.90	0.00
Relevance	4.36	0.81	4.98	0.75	0.00
Confidence	4.85	0.82	5.63	0.77	0.00
Satisfaction	4.40	1.31	5.84	0.84	0.00
TOTAL	4.57	0.88	5.64	0.73	0.00

below ilustrates the trend described above when analyzing the different items. There are significant differences in mean scores within the scale factors: Attention, Relevance, Confidence and Satisfaction, all in favor of AR. These differences are especially high in the cases of Attention (5.83 compared to 4.38) and Satisfaction (5.84 compared to 4.40) factors, with a difference of over 1 point.

The global scale score (Table 5) is also higher in AR (5.64) compared to FC (4.57); this difference is significant. As can be seen in the Figure 2, the greatest frequency of student scores in the case of AR is at the top of the scale, between 6 and 7 points. While in the case of FC, the greatest frequency of student scores ranges between 4 and 5.

4. Discussion

The present research has illustrated some reactions with regards to material presented in both AR and FC environments. The findings revealed that the majority of participants had high levels of motivation and were satisfied with the materials supplied during the pandemic. It is important to note that these data do coincide with Huang & Hew [22]. It can be observed that the attention of students was captured during classes, which, in turn persuaded them to continue to develop their interests. This is considered an extremely positive result.

According to data the provided by participants, mean motivation levels with both instruments are very high, with a score of 4.57 and 5.64 for FC and AR, respectively. These scores are like other similar research studies [22] the authors used the educational robot-based learning systems which were proposed as a teaching tool to motivate and engage students. These scores were significantly higher than in other studies such as Rosales & González [23], which obtained scores of 3.67, or Huang & Foom [22] with scores of 3.69, but without reaching 4, scores exceeded in our study with both instruments.

One aspect to emphasize and consider is that, of all mean factor scores, Satisfaction was valued the highest. Barroso, Cabero & Moreno [24] explained that being able to generate satisfaction indicated that the students involved in the study would continue to retain and further develop this motivation to learn. In our case, AR and FC obtained mean values of 5.64 for AR and 4.57 for FC; both scores are high if we compare them with other studies.

It is important to stress that regarding relevance with FC, with a mean of 5.95 subjects indicated that completing tasks successfully was important to them. This was compared to a mean score of 5.66 for AR, that is also high but slightly lower than the use of the other study instrument. These data also coincide with the study by Rodgers & Withrow-Thorton [25]. We believe that the interaction and instant feedback provided in the FC has created opportunities for students to engage cognitively, as explained by Cairncross & Mannion [26], ‘it causes them to think about the material presented, what it means, its relevance, how it can be applied and in what contexts’.

On the other hand, the results of this study were in line with the results of Yildirim, Ozden & Aksu [27], where the confidence factor illustrated higher levels. We agree with these authors that — in both AR and FC — as materials are available to students from the first day of class, that control over the material and well described tasks to complete from day one has motivated student learning. In addition, this has lead to a greater feeling of confidence that, in turn, lead to even higher levels of motivation for the sampled population.

We also agree with Rodgers & Withrow-Thorton [25] and Bailey [28] that there are other theoretical explanations for the success of AR and FC. From a constructivist’s viewpoint, they lead to a more student-centered, self-directed approach that allows each learner to manage their own learning process.

In terms of limitations, ensuring that the quality of the material designed for AR and FC are similar. There may some differences that impacted the motivation of students, so diligent consistency is required. In the future, the authors suggest the sample should be extended to a greater N, as this would allow the authors to increase the capacity to generalize results based on the data and the analysis. Likewise, other variables should be controlled (e.g., students level, Degree and subject) in order to improve the research design and comparisons settings in forthcoming research. For this reason, our findings should be interpreted with caution.

Despite these limitations, the authors can conclude that the tentative results presented here represent a starting point for investigating the motivational impact of using different materials such as those used in this research for AR and FC in active methodologies.

Future research is needed to, investigate whether age and gender could affect student motivation regarding the use of these materials in the active methodology. Further research also must, consider interviewing students to collect data based on their internal motivation needs and propose. Our everevolving world will continue to require innovative educational solutions and new proposals to improve the attention factor in future learners.