An Interactive Information System That Supports an Augmented Reality Game in the Context of Game-Based Learning

Abstract

Mobile augmented reality applications are gaining prominence in education, but there is a need to design appropriate and enjoyable games to be used in educational contexts such as classrooms. This paper presents an interactive information system designed to support the implementation of an augmented reality application in the context of game-based learning. PlanetarySystemGO includes a location-based mobile augmented reality game designed to promote learning about the celestial bodies and planetary systems of the Universe, and a web application that interacts with the mobile device application. Besides face-to-face classes, this resource can also be used in online classes, which is very useful in social isolation situations as the ones caused by the COVID-19 pandemic. Furthermore, it is the inclusion of the web application, with a back-office, in the information system that makes it possible to include curricula contents according to the grade level of students. Moreover, it is intended that teachers use the information system to include the contents they find appropriate to the grade level they teach. Therefore, it is crucial to provide their professional development to be able to use this resource. In this regard, a pilot study was conducted with teachers who participated in a STEM professional development programme in order to assess if the system is appropriate to be used by them. It is concluded that teachers found this resource relevant to motivate students to learn, and also acknowledged that the web application facilitated the introduction of appropriate curricula contents and also was useful to assess student performance during the game. Teachers need support, however, to implement these types of technologies which are not familiar to them. The necessary support can be provided through collaboration among the researchers and teachers in their schools. Besides engaging students to learn about celestial bodies, it is concluded that the information system can be used by teachers to introduce appropriate curricula contents and to be implemented in class.

1. Introduction

Besides promoting forms of entertainment, nowadays digital games have a growing social and cultural impact in our society. In addition, digital games have been playing an increasingly important role in education. Moreover, interactivity in games is recommended for enhancing students’ creativity and engagement, and also to provide students’ knowledge acquisition [1]. In particular, emergent technologies, such as augmented and virtual reality, are being introduced in education because it is argued that they contribute to motivating students to learn about curricula contents and improve their school performance [2,3,4].

The striking evolution of mobile devices, such as smartphones or tablets, in terms of hardware, software and affordability, enables the incorporation of these emergent technologies, which allows greater accessibility to them [5,6,7]. For this reason, Mobile Augmented Reality (MAR) is an important dimension that is being used in game development, in particular with game-based learning purposes [8,9]. In fact, several games are designed for purposes other than only entertainment as is the case of serious games [10,11,12]. For example, they can be used in industry, scientific exploration, health, marketing and education, amongst others [13,14]. In addition, games that provide physical movement and social interaction in the real world are included in the pervasive games’ paradigm [15]. Regarding education, serious games can have a crucial role in promoting the interest and motivation of students to learn school curricula [11,12]. In this regard, gamification in Augmented Reality (AR) applications is suggested to increase player engagement and immersion during the game [9].

However, the literature refers to few studies that highlight the use of game-based AR applications in education and, also, about the need to design enjoyable games targeted to educational contexts [9,16]. Furthermore, most applications are implemented by their designers and there is a lack of studies about teachers implementing AR applications in the classroom, in particular related to STEM (Science, Technology, Engineering and Mathematics) education [17]. Therefore, this scenario justifies the need to develop research in this matter, namely by providing solutions that may be implemented in schools, in particular by the teachers.

This paper presents an interactive AR information system designed to promote STEM education in formal learning environments such as schools. The information system, entitled PlanetarySystemGO, includes a web application and a location-based MAR application with game-based learning purposes, related to the planetary systems of the Universe. In the MAR game, the real world, captured by the mobile device camera, is the environment where the player is inserted, and the virtual objects are celestial bodies that appear on the screen of the mobile device [18]. During the game, players gain points as they succeed in capturing the objects and correctly answering the questions about the celestial bodies they found.

Furthermore, the information system includes a web application with a back-office that allows adapting the contents to the school curricula and to the grade level of the students who play the MAR game, namely the information about the celestial bodies and the questions about them. An important upgrade of the information system is the possibility of using the web application to model planetary systems in a georeferenced map, which can be done in the classroom or online classes. This last feature is important in situations of social isolation such as the ones caused by the COVID-19 pandemic, as will be exemplified in Section 5 and Section 6.

This paper presents the design and new developments of the architecture of the PlanetarySystemGO information system with the aim of providing an effective implementation in schools, according to the grade level of students. In particular, it describes the MAR and role of the back-office on the information system and how it may be used by the teachers in order to introduce content related to planetary systems and how to assess students’ performance during the game.

However, until now, the contents were introduced by the developers of the PlanetarySystemGO who are not experts in school curricula. Therefore, it is crucial to involve teachers in the process to be able to use the system and to introduce the contents by themselves. In this regard, another contribution of this paper is the presentation of the PlanetarySystemGO information system in a teachers’ Professional Development Programme (PDP) in order to teach them to use the information system and to assess their feedback about this resource. In fact, besides a few studies highlighting the use of game-based AR applications in education [16], most of them are implemented by the researchers and rarely refer to teachers’ knowledge and skills about reproducing them in schools. The involvement of teachers in the process is crucial regarding the sustainability of the proposed approach. In fact, they are the main ones responsible for education in schools and are the ones who better know the curricula to be introduced to their students.

The paper is structured as follows: Section 3 presents the background and context of the study, Section 2 the literature review, Section 4 the methodology, Section 5 the description of the information system architecture, Section 6 the results and discussion. Finally, in Section 7, conclusions are presented.

2. Literature Review

Digital games have been gaining more and more impact in modern societies and, also, have an increasing role in education. In addition, game interactivity is recommended for enhancing students’ creativity, and engagement and also to provide students’ knowledge acquisition [1]. In this respect, the same authors advocate that gamification has the advantage of offering immediate feedback and reflection, competition, collaboration or self-study, which is beneficial in learning contexts. Moreover, challenging interactive tools help to improve students’ motivation and focused attention, which promotes their learning in the classroom [19].

In particular, there is an increasing call for the introduction of emergent technologies in education to promote students’ interest to learn curricula contents and to better prepare them for real-world challenges [20]. In fact, the rapid development of technology has been changing the paradigm of education by creating new opportunities and resources to improve the quality of teaching and learning [21]. In particular, AR is an emerging technology that allows combining the real world with virtual objects, that runs interactively in real time [22,23,24]. In addition, it is recognized to engage students and improve their learning performance [2,3,4,25]. Furthermore, AR contributes to improved benefits in the teaching and learning environment [21,26]. In a systematic review of literature about the use of game-based AR applications in education [16], the authors suggest that AR-enabled learning environments can improve student learning because it provides high interactivity and immersion.

Recently, the literature also refers to the importance of AR in STEM education [17,26]. In fact, the integration of the subjects included in the STEM acronym is defended in the literature to better prepare students to the increasing challenges of the real world and to promote critical thinking and creativity amongst other competencies that are considered crucial to contribute to innovation, scientific and economic growth in the countries [27,28]. For this reason and, also, to meet the growing needs of careers and professionals in these areas, STEM education is part of the curriculum of several countries and the importance of increasing students’ interest in STEM during early school years is widely recognized [29]. However, the topic of AR in STEM education is quite new and there is a lack of study and research thereon [20].

With the increasing improvements in mobile devices from hardware to software, until its affordability, a growing number of AR applications are running on these devices, which enables more users to benefit from this technology anytime and anywhere, through the mobile device interfaces [5,6,24]. For this reason, MAR includes applications in several areas such as marketing [30], tourism [31], industry [32], navigation [33], or medical training [34], among others [35]. Regarding education, MAR is suggested to increase the quality of experience in the teaching and learning process, but it is important to design enjoyable games to be used in educational contexts such as classrooms [9].

The great improvement in mobile devices functionalities such as battery autonomy and the inclusion of sensors such as accelerometers, compasses, and integrated GPS, amongst others, permits the introduction of location-based MAR applications, which provides more interactions with the real environment [7,18]. However, most proposals with game-based learning purposes are related to marker-based AR [26], which may be related to the fact that the tracking process of markers is more effective and stable compared to other tracking techniques [3]. For this reason, the PlanetarySystemGO is an important contribution in this respect, because it includes a MAR location-based application that offers an interactive experience in the real world, which is the real environment where the player is moving while capturing planets that circulate around a star, amongst other celestial bodies [18].

However, despite the increasing calls and references defending AR in education, several authors continue to argue that there are few studies providing students with assistance in carrying out learning activities [36] and about the use of MAR applications in formal learning environments such as schools [37,38]. In fact, there is a need for educational AR applications with sufficient learning content and, also, only some students benefit from AR-enriched offers in the classroom because few teachers are deploying these tools in their classes [17]. Although some authors refer to studies carried out at schools most of them use marker-based AR applications [26]. Moreover, there is a lack of references highlighting the use of game-based AR applications in education [16]. Furthermore, based on a literature review on the use of AR technology to support STEM learning, in [36] the authors sustain that researchers need to design features that allow students to acquire basic competencies related to STEM disciplines.

Serious games are more than mere entertainment, but definitions may vary depending on their applications and according to the point of view of the authors who provide them [39]. The same authors define serious games “as digital games that have an intended impact on cognition, behaviour or motor skills” (p. 32). According to the authors of [40], serious games are related to successful AR applications that provide an immersive and interactive experience, which can be used for learning purposes. In this regard, the authors propose a prototype that includes a head-mounted device and a vibrotactile feedback jacket to explore and interact with the AR serious game.

Pervasive games provide the users with physical movement and social interaction in the real world while playing the game [15]. Through mobile devices that include new technologies such as sensors, software applications and network communication, pervasive games can provide a new way of social interaction [5]. Regarding Serious Pervasive Games, in [41] the authors advocate that the environment and social context lead to a more meaningful impact on the gameplay. In this regard, they present research related to location-based games in the context of a natural park and its species.

3. Background and Context of the Study

In this section, we provide the description of the broader project where the PlanetarySystemGO information system is inserted. The Academy of Science, Art and Heritage (AcademiaCAP) is a pedagogical project that is being conducted in a polytechnic higher education institution in Portugal (http://www.academiacap.ipt.pt, accessed on 29 October 2021). Besides several activities during students’ holidays, and in the community, targeted to primary and secondary schools, it also fosters teachers’ professional development. In addition, the AcademiaCAP promotes the development of hands-on experiments and prototypes with the collaboration of higher education students under the supervision of teachers and researchers [18,42] and in the framework of problem-based learning [43]. Problem-based learning is an active engagement learner-centred approach that is very relevant for engineering students and empowers them to develop a viable solution to a defined problem [44,45].

The experiments and prototypes are developed with a design research methodology that includes various stages of design research until approved to be launched to the public. In the first stage, they are tested in informal learning environments, such as during students’ holidays on the college campus. This way, the developers have the chance to correct any mistakes and improve performance issues to be tested in the next opportunity. The following stage is to implement in schools, in order to assess if they are adequate to curricula contents and well accepted by students. This is one of the most important stages, where the feedback and collaboration of teachers is crucial to prepare the final step, which is to include the solutions in teachers’ PDP.

It is in this context that the PlanetarySystemGO has been designed, since 2015, by higher education students. Every school year teams of three or four Computer Engineering BSc final project students keep upgrading the project based on the result of the design research cycles. The first version was called SolarSystemGO and consisted of a MAR game hard-coded on the application that only permitted exploring our Solar System. Although this was a preliminary version, it was tested in informal learning environments and the authors concluded that children were engaged during the game and that this approach promoted learning of interdisciplinary subjects related to the Sun and the planets of the Solar System [46]. Because of this positive student feedback, the game designers decided to keep upgrading the application to include more functionalities in order to be implemented either in informal or formal learning environments. The last version is an information system that includes a location-based MAR game designed to promote learning about the Universe. When playing the game, in addition to exploring planetary systems, the players have access to information about the learning objects such as celestial bodies and, also, need to answer multiple-choice questions related to them. Moreover, at each stage of the game such as finding orbits, planets and answering correctly to the questions, the players collect points, thus providing a gamification experience (Figure 1).

Since the first implementation test in informal learning environments [46], the MAR application is being implemented in primary schools with very good results, as has been described in previous studies by the authors of this paper [18,47]. These good results include the enthusiasm of students and their teachers about the game, in particular the recognition, by teachers, that this approach improved students interest and motivation to learn school curricula content.

Figure 2 shows students playing the game in their schools, before the COVID-19 pandemic, where joy on their faces is visible. Having fun is fundamental to the effectiveness of the game in the students’ performance, because if they find the game boring, they will give up playing it, and consequently will not learn from the application. This is in line with [9], where it is stated that it is important to design enjoyable games to be used in educational contexts. Furthermore, because the MAR application has learning purposes and provides physical movement and social interaction of the students in the real world, it is a serious pervasive game [15,41].

However, until now all implementations on the field were performed by the developers and researchers team. For this reason, the authors decided to include this resource in a PDP, which is an important contribution of this paper. Moreover, there was the need to include more webservices as described in Section 5 and Section 6.

4. Methodology

The information system is being developed with a design research methodology, where each academic year a new team of computer engineering students continues to upgrade the system (Section 3). In fact, the literature supports the need for new approaches regarding educational technology research [48]. In this regard, the close collaboration between designers and practitioners is crucial to developing interactive learning environments in the contexts in which they will be implemented. The present study is in line with this recommendation because it provides collaboration with teachers to develop the information system, in order to be implemented in schools.

The PlanetarySystemGO information system was introduced in a three-hour workshop, included in a PDP, in October 2020. The PDP includes several workshops related to STEM with a duration of two to three hours. Because of the COVID-19 pandemic, all workshops were online through the Zoom platform and some hands-on experiments were exemplified through videos or with live demonstrations. Regarding the PlanetarySystemGO workshop, a live hands-on demonstration of how to use it was presented to the teachers. This was a pilot study because it was the first time that the information system was implemented in a PDP and, for this reason, it was important to assess its usability and suitability to be implemented in class by the teachers with students.

The main goal of the workshop was to introduce teachers to the information system, explaining its potential and characteristics, and how to use it to provide hands-on activities to engage students to learn about celestial bodies and planetary systems. In this regard, the workshop was organized as follows: (i) introduction to the workshop and raising awareness about the importance of introducing new tools and methodologies to engage students to learn curricula contents; (ii) presentation of the PlanetarySystemGO information system; (iii) live hands-on demonstration of how to use the information system, in particular how to use the web application; (iv) focus group to discuss and assess teachers’ perceptions about this resource; (v) filling in a questionnaire.

To develop the present research, documentary analysis was carried out to frame this study in the literature. Moreover, this paper uses a mixed methodology with a qualitative and interpretative approach that includes participant observation and questionnaires applied to the participant teachers in the PDP [49]. Participant observation occurred during the online workshop, where two authors of this paper were the facilitators. In order to assess teachers’ perceptions about this resource, namely if it is adequate to their needs and if they consider that it is suitable for the primary school curriculum, teachers were invited to fill in the questionnaire at the end of the workshop.

A total of 16 primary school teachers (15 female teachers and 1 male teacher) voluntarily made their inscription on the PDP. The teachers, from twelve schools of Central Portugal, participated in the workshop and answered the questionnaire.

Table 1. Age of the teachers who participated in the workshop.

Age Range	≤30	30–40	40–50	50–55	55–60
Frequency	1 (6.3%)	3 (18.8%)	5 (31.3%)	5 (31.3%)	2 (12.5%)

and

Table 2. Distribution of the teachers by the grade level they teach (ES—Educational Support).

Grade Level	1st	2nd	3rd	4th	5th	6th	ES 1st to 4th
Frequency	3	4	3	3	0	2	1
	(18.8%)	(25%)	(18.8%)	(18.8%)	(0%)	(12.5%)	(6.3%)

give information about the first two questions related to the age of the teachers and about the grade level they teach, respectively. In Portugal, the first level of elementary school includes four grade levels (1st to 4th grade, 6 to 9 years old), and the second level of elementary school includes two grade levels (5th to 6th grade, 10 to 11 years old).

As shown in Table 1, about $75.1\%$ of the teachers are more than 40 years old, and $43.8\%$ are more than 50 years old. Table 2 gives information about the grade level they teach.

Most of the teachers are from the first level of elementary school and a quarter of them teach the second grade. Only one teacher did not have her own class because her role was to support other teachers in their classes. Discussion about the answers to the other questions of the questionnaire is provided in Section 6.

5. Description of the PlanetarySystemGO Architecture

This section is devoted to the presentation of the PlanetarySystemGO architecture. AR-based architectures have gained increasing momentum in many pedagogical contexts, and in most recent years a considerable effort has been made in applications that range from Mathematics teaching [50] to Pervasive [15,51] and Serious Games [52,53].

The PlanetarySystemGO information system is mainly composed of two components, an Android application, and a web application, that share data on a common server. The MAR game is played on the Android application developed in Unity. The web application runs on a WAMP server (Windows, Apache, MySQL and PHP) and includes web-accessible back-office and front-office systems (Figure 3).

The server provides an API (Application Programming Interface) through REST (Representational State Transfer) webservices to be consumed by the Android application. All communication in this API uses JSON (JavaScript Object Notation) standards for data representation.

5.1. Description of the Back-End Architecture

The back-end layer of the PlanetarySystemGO architecture serves data and provides coherence to the remaining components. This layer is composed by a server, database and software resources, in order to comply with the front-end requests. Therefore, the back-end carries most of the computational burden of the system, as we are relying on a server-side rendering architecture [54].

We now describe in some detail the components of the back-end, and start by the database created for the system. For our implementation, a relational database model was chosen [55] and the implementation was conducted using MySQL. Figure 4 shows the model of our database.

Next, we summarize some examples of functionalities that are provided by the back-office:

• clone celestial bodies (includes physical characteristics such as textures, orbital radius, velocities, and also information about the celestial bodies and questions to assess students’ knowledge);
• clone planetary systems already parametrized in the repository;
• introduce new planetary systems and celestial bodies;
• introduce information about the celestial bodies;
• introduce multiple-choice questions;
• provide information about the results of the game.

The construction of planetary systems involves the definition of many parameters of the celestial bodies such as textures, and different physical quantities to give realism to the didactic game that will be played in the mobile application. The multimedia elements and the physical parameters of the celestial bodies, in addition to providing visual realism, also allow students to get a sense of the distances between planets and elliptical orbits, amongst others, thus getting an idea of the constitution and behaviour of planetary systems, as well as its relative dimensions. Those are challenging and complex concepts that are presented to students in a playful and interactive way through a MAR game. However, the definition of celestial bodies is a lengthy and complex job, which requires investigation to define the physical quantities and computational ability to build the multimedia elements. For this reason, to avoid making it difficult for teachers to use this resource, a system of repositories was created (Figure 5) to simplify their work:

• Public repository: set of objects accessible to all users of the system;
• Institutional repository: set of objects accessible to users of an organization;
• Private repository: set of objects only accessible to the user.

We continue this section by discussing the implementation of the programming layer of the back-end. The coding was performed in PHP [56], and in its model design, we considered three parts: Views, Web Services and Assets.

Views allow page building, encapsulating the required HTML, CSS and JavaScript code that enable the PHP code to interact with the server, namely to pick up valid planetary systems and provide them to the user. Furthermore, there are also views related to specific roles and tasks:

• Administration, for managing events, users, among all other management items, accessible to administrators only;
• Events, to list, create and delete events,
• Organizations, that deal with all aspects related to organization management and user interaction;
• Questions, to deal with their interaction, namely the creation, edition and deletion;
• Planets, to obtain information about celestial bodies, and deal with related tasks, such as listing, creation, edition and deletion;
• Systems, similar to the Planets view, with the same functionalities, but adding system cloning and repository change;
• Users, allowing the edition of each user profile and management actions.

Regarding the implemented webservices, our system features two types: services that communicate with the MAR application and services that communicate with the front-end. All these services were implemented in PHP, and [57,58] provide the details to the interested reader, namely in what concerns the webservices that were designed to implement communication with the MAR application.

The front-end component of this system provides tools for the game master to create new celestial bodies and new planetary systems, as well as sets of questions and answers, and also allows the extraction of gameplay statistics. With this goal in mind, the front-end system encloses two main functionalities: to receive data from events in the MAR application and transform them into statistical graphs, and another one that sends data from the back-office to the MAR app to configure events.

In the remainder of this section, we will discuss the features of our front-end. We start by presenting the latest changes in terms of user interface. The system allows for user registration through email authentication and password creation and changes, an almost universal feature in today’s web systems and applications (Figure 6). The front-end system reflects the organization created at the back-end level, and specific views for Administrators and Organizations are available, presented in Figure 7 and Figure 8, respectively.

An event is created and modelled in the front-end. For this purpose, the instructor is provided with a set of graphical tools to input the celestial bodies of the planetary system (images and textures) as well as to determine their correct placement so that the game experience may begin. This process can be conducted in two phases, as illustrated in Figure 9. In the first phase, the system has a set of functionalities for the user to input the above-mentioned data, leading to a first iteration of the planetary system location, still unmodelled. A sample is shown in Figure 9a. The front-end system then provides tools for the user to adjust the location and relative dimensions of each body in the game arena. In Figure 9b we present an example of a modelled planetary system.

Once the centre of the celestial body has been defined by the user, it is also possible to change its size, speed and radius, as well as the centre, as is shown in Figure 10.

Apart from the gamification dimension, PlanetarySystemGO also has an important pedagogical role and with this in mind, it is possible to create a detail page for each celestial body carrying relevant information, through the front-office. Those are static pages that contain animated models of the celestial body in stake, as well as some astronomical information. Figure 11 shows an illustrative example.

5.2. Description of the MAR Application

This subsection is devoted to the presentation of the architecture of the MAR application, which was developed in Unity3D [59], which is one of the most popular game engines currently in use [60]. On the design of this component, three important aspects had to be addressed: player interaction, the connection between the gaming experience, and finally the gamification dimension. In what concerns the design of the player interactive experience, it was developed with two game modes in mind: offline and online playing. In the offline mode, the game experience is based on a previously downloaded planetary system. Furthermore, at the beginning of the game, the star is located at the player initial position. On the other hand, the online mode allows the player either to download a set featuring both a planetary system and questions, created using the back-office component of the system. In this mode, it is also possible to start the game at a given location given by its GPS coordinates.

The quality of the 3D representations and animation of the celestial objects, enhance the quality of the user experience when playing with the MAR system. The surface of the celestial bodies is, as described in [61], represented by textures, that are made available to the MAR application either stored internally in the mobile device, in the offline mode or from the game master using the previously mentioned back-office component of the PlanetarySystemGO system. This requires that the Unity3D simulations be converted to Javascript and HTML5, to be featured in the web application.

The process of dynamically creating planets relies heavily on the scripting capabilities and on the rich set of C# primitives and types available in Unity3D and goes beyond the scope of this paper. The interested reader will find this information in [62].

Another important issue on this type of interactive system is the possibility of zooming objects, namely the representation of the celestial bodies, and the orbits, to provide a more engaging experience to the player. Our MAR application used the mouse-based zooming classes developed for [46]. Another aspect that was taken into consideration was the movement representation, and for that purpose, some C# scripts were designed, that address the elliptic movement of the celestial objects [62].

We now discuss the second aspect of the implementation of the MAR application, which has to do with the connection of the gameplay with the real game arena. For the game experience, it is essential that the mobile application is location and player movement aware, and this calls for an AR layer on the system. For this purpose, our system relies on ARCore, a platform for creating AR apps for Android devices that uses three key capabilities [62]: motion tracking (that allows the device to track its position based on the information received from the real world), plane detection (that allows the device to discover any surface size and location, such as horizontal, vertical and angled surfaces) and light estimation (that allows the system to evaluate the current lighting status of the surrounding physical space).

The gamification component of the mobile application is now addressed. This goal is achieved both by setting up a challenge and reward system, that corresponds to reaching the orbits and celestial bodies, as well as to correctly answering questions created by the game master. Another dimension of the game experience has to do with the player navigation through the mobile application. The game experience starts with an initial menu, presented in Figure 12. This menu also features a quick game mode, to be played only on offline mode, and therefore no statistics are sent to the server when the quick game mode is engaged. In Figure 12 we also present the menus for offline and online playing modes. The system also features menus for transferring an event from the server, so that the player can start the game experience. During the game play, it is also possible to trigger a menu in which the user can get information about the closest celestial body. Finally, questions that were generated by the game master on the back-office component of PlanetarySystemGO are made available to the player by means of menu entries, such as the one presented in Figure 12.

A set of celestial bodies, such as stars, planets and moons, is available in the public repository, with information about their physical characteristics and multimedia elements that allow their visualization in AR in the mobile device, and also a set of questions to assess students’ knowledge about them. Celestial bodies are grouped in planetary systems, with the star in the centre, the planets orbiting it and the moons circulating around the planets. Teachers can use this model, or they can build their own planetary systems that will be stored in the private repository. In this way, teachers can build a new planetary system by making clones of existing systems and parameterizing it for the curricula content they intend to teach their students. The parameterization of the systems allows deleting, inserting and modifying existing objects. For example, it allows that teachers change both the physical quantities, their multimedia characteristics and the questions associated with them. The parameterization of the system has a double purpose: to adapt the contents of the simulation to the syllabus contents of schools and to make the game “playable” in the real environment where the students will “hunt” the planets, for example in the school playground.

The system also allows the introduction of new content created from scratch. This process is more complex since it involves the definition of a wide set of numerical parameters, which are related to each other, and related to the other objects of the planetary systems, as well as multimedia elements that require some expertise to have a realistic appearance when used in the MAR application. All new elements can be shared so that they can be used by other teachers. This sharing can be done in the public repository, where any user has access, or in the institution’s repository where only teachers from the same organization can clone it. After parameterizing the planetary systems, an event is defined to be played in the mobile application. Once the game is played, players’ performance data is collected and sent to the back-office via webservices to be analysed and viewed by teachers. In Figure 13, data include the scores of the game (Figure 13a), results from answering questions about the celestial bodies, and the time playing the whole game as well as the time that each player took to answer the questions about each celestial body (Figure 13b), amongst others.

Figure 13a shows the results of the scores obtained by seven students who played the game. As can be seen in the figure, Vasco had the best score with 40 points and Maria the worst score with 32 points. By placing the mouse cursor over the graphic, the exact score is revealed, as can be seen in Maria’s column. Figure 13b gives an example of the results of the game that was played by the same students, displaying the time, in minutes, that each player took to answer the questions about each of the celestial bodies, which in this example are the Sun and the planets of our Solar System.

Based on the functionalities described in this section, the PlanetarySystemGO information system is a resource that can be used by teachers to introduce content according to school curricula, and also to assess the performance of the students during the game. The next section gives the results of the implementation of the information system in a teachers’ PDP.

6. Results and Discussion

The first subsection describes the implementation of PlanetarySystemGO in primary schools through online workshops taking advantage of the web application. Next, the results of the implementation of the web application in a teachers’ PDP is presented.

6.1. Implementation of the MAR Game in Primary Schools through Online Workshops

Because of the COVID-19 pandemic, since March 2020, no face-to-face sessions with the MAR game have been held in schools. For this reason, in 2021, an online workshop was prepared taking advantage of the new functionalities of the information system provided by the web application [57,58]. The workshop, entitled “Journey through the Solar System”, is presented using the Zoom platform. The first ones have been held with the students at home (until 12 March 2021) and the following with the students at school. Figure 14 shows the facilitators of the workshop and the teacher in the classroom with her students.

As was usual in the face-to-face workshops, the online session begins by explaining to the students what is AR and how the game is played on a smartphone. Additionally, because they need to answer questions about the Solar System during the game, a PowerPoint with information about the Solar System is presented. Then, the web application is used to model the Solar System on a georeferenced map, where the school is located, to provide students with knowledge about movements of the planets around the Sun and about its relative dimension, amongst other contents (Figure 15).

During this stage, students are very excited because they recognize the location of their school and the city garden where the Sun is placed on the map.

In addition, by using the menu on the front-office, different interactions can be provided such as changing the scales of the Solar System by using “Increase” and “Decrease” buttons or visualizing more details about the planets, amongst other functionalities (Figure 10). For example, the zoom application may be used to better visualize satellites or more details of certain celestial bodies.

This example of modelling the Solar System draws on subject matters included in the STEM acronym [17,26]. Therefore, this system contributes to STEM education in schools, which better prepares students for real-world challenges and consequently to innovation, scientific and economic growth in the countries [27,28].

The following figures illustrate other contents that may be explored such as seasons, days and nights, eclipses and moon phases, just to name a few. In Figure 16, on the left, it is winter in the northern hemisphere and summer in the southern hemisphere. On the right, it is summer in the northern hemisphere and winter in the southern hemisphere.

Figure 17, shows a full moon on the left and a solar eclipse on the right.

In Figure 18 and Figure 19 more simulations and details are shown that can be provided through the web application.

If this was a face-to-face workshop, after all the presentations in the classroom the students would go to the school playground to play the game with smartphones (Section 3). However, this was not possible in an online workshop. Therefore, to provide the game experience, a video was presented showing how the game is played. During the video presentation, the facilitator describes how the game is played and asks questions about the celestial bodies that appear on the screen of the smartphone. In addition, the facilitator pauses the video every time an object is captured to give the students an opportunity to answer the questions. Students are very participative and always raise their hands to answer the questions provided by the application. This is a way of providing interactivity with students to promote their engagement to learn about the proposed contents [1].

The following figures give some sequences of the game according to the video presentation (Figure 20 and Figure 21). The questions are in Portuguese because the school was located in Portugal, but the application can be used in English or another language because the PlanetarySystemGO is a multilanguage system [57].

The Sun is the first object captured and consequently, the first question is about it: “the Sun always rises in …”. Afterwards, the orbit of Mercury is caught and when the planet is captured the following question appears: “How long does it take me to make a complete turn around the Sun?” As can be seen in Figure 20, the player is winning points: one when he captures the object, and four when he answers the question correctly at the first attempt. This is important because gamification can motivate students and promote their attention, which is beneficial in learning environments including classrooms [9,11,19].

When the orbit is captured, it changes its colour from white to yellow and when the planet is captured it changes to green. When the question is correctly answered, the celestial body gains a flag with the logo of the AcademiaCAP project. At the end, when all planets are captured, a congratulations message, the total score, and the time it took to play the game is displayed on the screen of the smartphone. As is shown in Figure 21, the player took 8 minutes and 19 seconds to play the game and had a score of 43 points.

Finally, a focus group is performed with the teachers and the students to assess if they liked the experience. Usually, students appreciate and say that they learned things that they did not know about the Solar System. For example, some students thought the Earth was bigger than the Sun. Additionally, teachers recognize this approach is very important and thank the facilitators for providing this experience to their students:

Teacher 1—

I already knew that students liked this game because you were last year in our school in other classes. We like your experiments very much, its always very fruitful for our students. You helped us a lot with this presentation because we were teaching these contents to the students at the moment.

Teacher 2—

Thank you very much, it was very helpful because this way students learn better about the Solar System. There are always things to learn from your presentations, even if it is from a distance.

This result is rewarding for the team that intends to continue developing the application including more functionalities according to school curricula.

6.2. Implementation of the PlanetarySystemGO in a Teachers’ Professional Development Programme

This subsection describes the implementation of the information system in an online teachers’ PDP. In a three-hour workshop, the facilitators used the Zoom platform to introduce the PlanetarySystemGO to the participant teachers through a live hands-on demonstration of how to use it. At the same time, teachers were experimenting with their own computers on how to use the system and asked questions whenever they needed help or had doubts of how to use it.

As usual in these platforms, the first step is to make the registration for the new users on the back-office (Figure 22). In this task, difficulties were diagnosed in some teachers in making the registration, which shows that they are not familiar with this type of tool. These difficulties were solved with the facilitators exemplifying how to do it and also asking teachers to share their screens.

After registration, the facilitators explain how to use the back-office and its resources. For example, Figure 23 shows details of the repository with the Solar System that can be edited by the teachers to introduce information and questions about the celestial bodies [57].

Figure 24 shows how to introduce a new question and the four possible answers.

The previous examples are in Portuguese but, because the PlanetarySystemGO is a multilanguage system, it is possible to have all the information in English, as shown in Section 5. Moreover, other planetary systems can be introduced such as the Trappist System (Figure 5). After explaining how to use the back-office, it was explained how to perform modelling of the Solar System on a map (Figure 15), and also how to create an Event to be played on the mobile device as described in Section 5.

At the end of the workshop, besides focus group with the participants, a questionnaire prepared with Google Forms was applied to teachers, who voluntarily answered it ($N=16$), with the aim of assessing their perceptions about this resource, consisting of the following questions:

Q1:

Do you usually carry out hands-on activities related to science, namely astronomy? If the answer is yes, give some examples.

Q2:

Is PlanetarySystemGO information system important for primary school curricula?

Q3:

Is the back-office important to contribute to the effectiveness of PlanetarySystemGO by introducing appropriate curricula contents in class and assessing students’ performance?

Q4:

Quantify the importance that these activities can have in favouring learning over the traditional approach.

Q5:

Observations/suggestions for improving the system.

The characterization of these questions is presented in

Table 3. Questions of the questionnaire.

Question	Type	Main Result
Q1	Yes/No	15 (93.8%) for “No”
Q2	Likert scale	16 (100%) for ≥ “Important”
Q3	Likert scale	16 (100%) for ≥ “Important”
Q4	Likert scale	16 (100%) for ≥ “Important”
Q5	Open question	—

.

More than 90% of the teachers answered that they usually do not “carry out hands-on activities related to science, namely astronomy” (Q1). Only one teacher, who answered “Yes”, gave the following example: “Plastic expression activities, model building …”. According to this information, it is concluded that almost all teachers do not perform hands-on activities, and consequently do not use interactive technology in their classes such as AR applications. Therefore, it is necessary to support teachers in helping them to implement this type of activity in class, which is in line with Jesionkowska [17], where it is stated that few teachers are deploying these tools in their classes.

Table 4. Frequency of the answers to the questions with a five-point Likert scale.

	Not Important	Little Important	Important	Very Important	Very Much Important
Q2			4 (25%)	8 (50%)	4 (26.7%)
Q3			2 (12.5%)	10 (62.5%)	4 (26.7%)
Q4				8 (50%)	8 (50%)

shows the results of the answers of the teachers to the questions about the suitability of the PlanetarySystemGO to the primary school curricula (Q2) and about the importance of the back-office to introduce adequate content in class (Q3). As can be observed, all teachers answered at least “Important” with 76.7% (Q2) and 89.2% (Q3) answering “Very important” and “Very much important”. Therefore, all teachers recognized that the PlanetarySystemGO is suitable to the primary school curricula and also the importance of the back-office to contribute to the effectiveness of PlanetarySystemGO by allowing the introduction of appropriate curricula contents in class and assessing students’ performance. All teachers chose “Very important” and “Very much important” for question Q4, which means that they agree that it is very important to promote this approach instead of traditional approaches. This result reveals that it is crucial to invest in these resources.

A positive result to Q2 and Q4 questions was expected because of the good results obtained during the implementation of the MAR game in schools [18]. However, in the present paper, this question refers to the whole information system, in particular with an emphasis on the back-office, which is confirmed by question Q3. In fact, the back-office permits that teachers introduce curricula contents according to the grade level they teach. Therefore, it is crucial that they are able to use it in order to prepare the classes by themselves. Until this PDP, previous information about the celestial bodies and the multiple-choice questions on them had to be introduced by the designers of the system.

However, some technology difficulties were observed in the teachers during the live hands-on demonstration. In fact, they needed the help of the facilitators to register and also to use the tools provided by the system. In addition, during the focus group discussion, some teachers revealed that they were not used to employ this type of technology in their classes, and it was not easy to manipulate some of them. Moreover, some teachers also referred that they did not have the equipment to implement this approach. This is another dimension that justifies the need to create more support for the teachers in this matter. Furthermore, collaboration among teachers and designers should be provided to improve the system making it more user-friendly from the point of view of the teachers. This strategy can help more teachers to develop this type of approach in class. This is important to face the need for teachers to implement new methodologies in class as referred to in the literature [17].

Finally, in Q5 (open question), about suggestions for improving the system, only some teachers gave answers such as:

• It would be important to highlight the natural satellite Moon (phases, seasons, …)
• Adapt more to the 1st cycle—simpler questions
• Moon, Moon phases, asteroids
• Difference among comets, asteroids, meteoroids, meteors, meteorites

As referred before, the information and questions about the celestial bodies were introduced by the team who designed the PlanetarySystemGO, who are not experts in primary school curricula. Teachers’ suggestions are about the need to introduce the contents they teach, which again justifies and reinforces the importance of collaboration between the team of designers and the teachers, in order for teachers to be able to introduce the contents by themselves.

Unfortunately, because it was an online workshop, it was not possible to go outside and play the Events created by the teachers during the live demonstration. Furthermore, because schools were closed due to the COVID-19 pandemic, it was not possible to observe teachers in their classes using the PlanetarySystemGO and preparing their own Events to be played by the students in their schools. Because our final goal is that teachers use the PlanetarySystemGO to introduce curricula content in their schools, it is important to know if they are able to do it and to assess their perceptions about this approach. This will be accomplished as soon as possible.

7. Conclusions

Digital games play an increasingly important role in modern societies, which is also reflected in education. In particular, interactive challenge tools in gamification can motivate students and promote their attention, which is beneficial in learning environments including classrooms [1,19]. Emergent technologies like AR provide high interactivity and immersion, which can improve learning achievement, particularly in STEM education [17,26,36]. In addition, because of the increasing accessibility to more equipped mobile devices, MAR has been gaining prominence in applications with game-based learning purposes, namely in the context of Serious and Pervasive Games [41].

However, studies on AR are lacking in formal learning environments and there is a need to design enjoyable games to be used in educational contexts [9,16]. In addition, AR studies related to STEM education, in particular with solutions implemented by teachers in the classroom, are missing [17,20].

This paper presents an interactive information system designed to promote STEM education in schools. The PlanetarySystemGO includes a web application and a location-based MAR application with game-based learning purposes related to the planetary systems of the Universe. Because it requires GPS, the game is played in an outdoor environment, which is the real world, captured by the mobile device camera, and the virtual objects are celestial bodies that appear on the screen of the mobile device [18]. During the game, the players need to move in the real world to capture virtual objects, which are stars, planets, and other celestial bodies. Because it provides physical movement and interaction with the real world, this game is also included in the pervasive games’ paradigm [5,15]. In fact, joining the surrounding environment (physical world) with the fictive game’s world improves the player’s engagement and promotes situated learning approaches [41]. When the player captures a celestial body, a question about its characteristics appears on the screen of the mobile device with four possible answers (Figure 20 and Figure 21). When finding the virtual objects and correctly answering questions, the players gain points, which is related to gamification advocated in the literature because it increases the players’ motivation and immersion in the game [9].

Furthermore, the web application in the information system is an important tool to be used by teachers to adequate the content to the grade level they teach in school. Besides introducing the information about the celestial bodies and the questions to appear in the MAR application, teachers can create an Event with the experience of the game they intend their students to have, for example, according to the available outdoor space of their schools (Figure 9). After playing the game, the statistics of each player will be available to the teachers, such as final scores, the questions that were correctly answered, time to play, etc., which allows assessing students’ performance during the game (Figure 13b).

In addition, a new important contribution is the web application that can be used to model different planetary systems in georeferenced maps. This can be done in the classroom or online, for example using the Zoom platform, which is very useful in social isolation situations caused by the COVID-19 pandemic (Figure 15). With this tool, it is possible to place the Sun in the school position on a map, and to discuss with the students where the planets will circulate in their village or town, according to a chosen scale, amongst other possibilities. This new tool was implemented in the online workshops and students showed great enthusiasm and were excited when they recognized their school and city garden (Figure 15). Therefore, this tool promotes STEM education [17,26], which contributes for schools to better prepare students to respond to real-world challenges, and consequently to innovation, scientific knowledge and economic growth in the countries [27,28].

The PlanetarySystemGO MAR application is being implemented in schools since 2018, where it was concluded that the game increased students interest to learn about the Solar System [18]. However, the information system was only manipulated by its designers as well as the contents introduced in the back-office. Therefore, there was a need to assess if teachers were able to use the web application, namely the back-office to introduce school curricula content, and also their perception about the importance of this resource to be implemented in their classes. In this regard, the PlanetarySystemGO information system was presented in a PDP. Based on participant observation, focus groups and questionnaires applied to the teachers, it was verified that all of them recognized that the PlanetarySystemGO is suitable for the primary school curricula and also acknowledged the importance of the back-office to introduce appropriate curricula content in class and assessing students’ performance. Moreover, all teachers recognized the importance of introducing this kind of approach with the students by answering “Very important” and “Very much important” to question Q4 (Table 3 and Table 4). Therefore, we conclude that the implementation of the information system in the PDP is an important step for the implementation of the PlanetarySystemGO by teachers, which is important regarding its sustainability. However, although all answers were positive, some technical difficulties were observed in the teachers during the live hands-on demonstration and some of them revealed that they were not familiar with these tools, which confirms the need to support teachers to help them implement this type of activities.

In summary, based on this study, it is argued that the PlanetarySystemGO MAR application is a location-based serious pervasive game that engages students to play it and to learn school content. In addition, the web application, that can be used by teachers to model the Solar System and to introduce content they intend their students to learn, is an important step in this direction. However, although teachers recognized the importance of this resource, some technical difficulties were observed. Finally, it is concluded that the PlanetarySystemGO information system is suitable to be implemented in schools and allows teachers to introduce the contents according to the grade level they teach, but it is necessary to support teachers in this process for them to be able to introduce this resource in school by themselves.