Design of a Mobile Augmented Reality Platform with Game-Based Learning Purposes

Abstract

Augmented reality (AR) is an emergent technology that overlays virtual objects into the real environment. Lately, AR is gaining prominence in education because of its increasing affordability through computers or mobile technologies. In addition, research sustaining the benefits of AR to promote student’s engagement to learn is increasing every day. However, the literature identifies lack of studies about the use of AR in education, namely, studies focused on the development of AR games designed over specific learning objectives (game-based learning). This paper presents a mobile augmented reality platform with learning purposes. The platform includes a mobile application that consists of a location-based game targeted to promote learning about the universe. Furthermore, it includes a back-office that allows teachers to introduce information about celestial bodies and also develop a set of multiple-choice questions to assess student’s learning about the subject matters they teach. The mobile application provides the users with physical movement and social interaction in the real world, while playing the game and for this reason it is included in the pervasive games’ paradigm. Besides engaging the students to play the game, we argue that this platform may be used as a resource to be implemented in informal and formal learning environments.

1. Introduction

There is an increasing call for the introduction of technology into education to engage students to learn and enhance their learning performance. In particular, Augmented Reality (AR) is an emerging topic that has been gaining prominence due to its potential to combine the real world with virtual objects [1,2] and to engage students in practice-based activities [3]. Other emerging technologies that have been gaining more prominence in the last years are mobile technologies [4,5]. According to Koutromanos and Avraamidou, mobile technologies, such as mobile games, can be a learning tool in a variety of formal and informal learning environments [6]. A survey related to Mobile Augmented Reality (MAR) technology refers that the increasing improvement in mobile devices permits the use of AR applications, which gives an easier access to this tool [7].

However, the literature identifies a lack of studies about the use of AR in education, namely, studies focused on the development of AR games that are designed over specific learning objectives (game-based learning). In particular, there is a gap in the literature about the use of AR within mobile games and applications [6]. Furthermore, the design of MAR applications targeted to formal learning environments is missing. For example, some authors argue the need to conduct more studies that include AR applications with a pedagogical approach or instructional strategy [8]. Regarding these issues, it is relevant to keep developing research in this matter.

This paper presents a MAR platform for learning purposes called PlanetarySystemGO. The platform includes a mobile application that consists of a location-based game targeted to promote learning about the Universe. The real environment captured by the mobile phone camera is the place where the player is inserted. The virtual objects are celestial bodies that appear on the screen of the mobile phone such as stars or planets. The platform also comprises a back-office that allows teachers to introduce information about celestial bodies and develop a set of multiple-choice questions to assess student’s learning about the subject matters they teach.

Because it is a location-based game that requires Global Positioning System (GPS), PlanetarySystemGO is an outdoor version. The players need to use a smartphone with the AR application (app) that will guide them during the game. When they start the app, their position in the real world has a coordinate that represents the star of the planetary system. Next, they need to walk in the real environment to find the orbits of the planetary systems and the planets, in the least possible time, answering questions concerning each of the celestial bodies they encounter. In this regard, PlanetarySystemGO may be included in the pervasive games’ paradigm because it provides interaction with the real world. Pervasive games are becoming more popular because they may provide the users with physical movement and social interaction in the real world while playing the game [9].

During the design of the game, several implementation tests of the mobile app took place with primary school students in informal and formal learning environments, such as school holidays and primary school classes. In this context, questionnaires were applied to students and their teachers. Also, a qualitative analysis and an interpretative approach was used [10]. In this regard, data collection also includes participant observation and informal interviews to the participants (children aged 7 to 14 years old who played the MAR game).

The focus of this paper is to discuss the PlanetarySystemGO game in the framework of game-based learning. In this regard, our research contributes to the literature because this platform may be used as a resource to be used in informal and formal learning environments. Considering that the majority of the studies targeted for teaching and learning purposes use marker-based technology [8], our system features an innovative contribution, in the sense that it provides an AR location-based application for educational purposes. In the following sections, we present the background of this study and literature review. Next, we describe the architecture of the mobile augmented reality platform and the impact of several implementations test in the field, with primary school students, followed by discussion. Finally, conclusions and implications for the future are presented.

2. Background and Context of the Study

PlanetarySystemGO is a mobile augmented reality platform that is being developed by undergraduate higher education students under the supervision of higher education teachers from a Portuguese polytechnic institute as part of their final graduation project. The game may be played with a smartphone that includes camera, GPS, accelerometer, and gyroscope. The mobile app consists of a kind of planet’s hunt, where the players, starting from a given coordinate (the Star), and guided by an AR application (app), try to find the orbits and the planets in the least possible time gaining points at each stage of the game. When the planet is found, information about its characteristic may be provided by touching the planet on the smartphone screen. Also, when it is captured, a multiple-choice question needs to be answered. At each stage (finding the orbit, “hunting” the planet, and answering the question) the players gain points (Figure 1 and Figure 2). The player with the best score and the best time wins the game [11]. These characteristics are related to gamification that promote the player’s engagement and motivation provided by the competition among the peers [12].

Since 2016, every school year, a group of higher education students takes the challenge of developing the mobile augmented reality platform to promote STEAM (Science, Technology, Engineering, Arts and Mathematics) education. In the school year 2016/2017, the challenge was to explore the concept of having virtual objects (planets) moving in the real environment visualized in a smartphone screen. To achieve this goal, it was necessary to design a mobile app with the aim of exploring our Solar System. This first version only included the eight planets of our Solar System and a set of a multiple-choice questions hard-coded on the app. In the school year 2017/2018, an information system was designed to include any planetary system of the universe and to include information about the celestial bodies and more sets of multiple-choice questions. In 2018/2019, the back-office was improved in order to be more user-friendly for primary school teachers. The next challenge is to implement the PlanetarySystemGO platform in any level of education from primary school to University.

While developing the application, in a first stage, several implementation tests were performed with primary school students in informal learning environments in order to diagnose any problems related to its performance and to assess their interest to play the game. The first version of the game started to be designed in 2016 by two students from Digital Content Production (Master’s degree) and Computer Engineering (Bachelors’ degree). This version consisted of a mobile application targeted to promote learning about our Solar System and for this reason it was called SolarSystemGO [11]. The SolarSystemGO game was implemented in informal learning environments, during students’ school holidays in the Polytechnic’s campus. After the last experience that occurred in the 2017 summer school holidays, the authors concluded that the game engaged students to play it and promoted learning about the Solar System [11]. Because of these good preliminary results, we decided to keep upgrading the game in order to provide the users with more diversified experiences.

With the aim of improving the game, a new challenge was proposed to computer engineering students. In the school year 2017/2018, the supervisors of the project asked the students to include new features on the mobile application (app). This challenge requested the development of an information system that communicates with the mobile app in order to provide the player with different experiences every time he plays the game. Also, because of the existence of exoplanets and their planetary systems in the universe, it was decided to extend the experience to any planetary system of the universe. Three computer engineering students chose this challenge as their final project before graduation. These projects used problem-based learning methodology, which is recommended in the engineering curriculum to facilitate learning and enable students to achieve 21st century skills [13]. The project provided for the inclusion of a back-office that allows teachers to introduce contents according to the grade level they teach.

Currently, the platform continues to be improved by another group of computer engineering students with the objective of providing the user with more experiences and improve its performance.

3. Literature Review: MAR in Education

Because of the increasing performance of mobile technologies augmented reality is becoming mobile (Huang et al., 2013). In fact, built-in camera, sensors, computational resources, and power of cloud-sourced information are resources that allows the use of AR in mobile devices [7]. In addition, no matter some constraints related to hardware, its portability makes the use of MAR more popular than on traditional desktop computers. This leads to the increasing use of MAR in several areas. For example, several authors developed research related to marketing [14], tourism [15], industry [16], navigation [17], or medical training [18], among others.

This section discusses the importance and impact of MAR in education. In fact, the literature presents several studies that advocate the introduction of AR and MAR in education. In this regard, in [19], the authors developed a thematic review of the literature regarding Mobile Technologies and Augmented Reality in K-12 education. Based on their study, they conclude that there are several benefits in using these tools in education. For example, they state that “AR is a tool that can be used to support students in becoming 21st century thinkers and problem solvers” (p. 288). In addition, they argue that AR “can extend and enhance teaching and learning” by highlighting four affordances such as “authentic learning, student-centered learning, contextualized learning, and visualize subject content” (p. 288). In this line, some reviews have been undertaken by several authors [3,8]. In the context of formal education, Saltan and Arslan conclude that AR may improve students’ academic performance and increase their engagement and motivation to learn [8]. Concerning primary education (6–13 years old), a systematic review related to augmented reality game-based learning is provided by some authors, who conclude about several advantages brought by this approach: knowledge gain, increased motivation, augmented interaction, and enhanced collaboration [3]. Moreover, they argue that AR technology provokes positive attitudes on students towards the learning process, and thus improved learning performance. Another example is a gaming approach to support AR-based learning activities related to ecology conducted in real-world contexts [2]. Based on an experience conducted on an elementary school, they conclude that this approach can improve students learning attitudes and their learning performance in the field. In fact, “Empirical evidence shows that AR can positively make a difference to how students learn and there is promise and potential for the future of AR” [19] (p. 288). Ozdemir, Sahin, Arcagok, and Demir carried out a study based on meta-analysis intended to determine the effect of AR in the learning process [20]. In this study, the authors conclude that “AR applications increase students’ academic achievement in the learning process compared with the use of traditional learning methods” (p. 177).

Because of the increasing performance of mobile technologies, augmented reality is becoming mobile [7]. According to the authors, built-in camera, sensors, computational resources, and power of cloud-sourced information are resources that permit the use of AR in mobile devices. In addition, no matter some constrains related to hardware, its portability makes the use of MAR more popular than on traditional desktop computers. Although there are several references advocating the introduction of MAR in education, the truth is that they also mention problems in its implementation, especially in formal learning environments. For example, based on a systematic review of literature, Akçayır and Akçayır [21] argue that there are several challenges regarding AR, namely usability issues and technical problems. Furthermore, Fotaris et al. [3] refer lack of research in primary education regarding the educational potential and affordances of AR technology. Also, they verified that more than 50% of the studies used marker-based AR, which seems to indicate that most educational applications use this technology. For example, the gaming approach proposed by Hwang et al. [2] to support AR-based learning activities was based on QR code technology. An explanation for the preference on marker-based AR may be related to the fact that the tracking process of markers is more effective and stable comparing with other tracking techniques [3]. However, some authors recognize that “this is an old and mature method in current AR technology” (p. 1903), and, for this reason, they recommend the use of other tracking technology in order to offer more interactions with the real environment [2].

Concerning other tracking technologies, location-based AR applications are increasing due to the availability of sensors in mobile devices such as accelerometer, compass, and integrated GPS [3]. Technically, a player’s position in a location-aware game is either determined by GPS satellite signals, WiFi, or GSM signal strength and/or cell ID, or by using short range proximity-sensing technologies such as RFID, infrared beacons, or ultrasonic emitters [9]. The same authors refer that location-based games may be an opportunity to bring more physical movement and social interaction into games. However, there are several issues that may compromise the game’s performance related to the unpredictable nature of location-based games [22].

The PlanetarySystemGO platform includes a location-based MAR app that provides the users with physical movement and social interaction in the real world while playing the game and for this reason it is included in the pervasive games’ paradigm.

4. Architectures Related to MAR Platforms

MAR application can use different architectures [23]. It may run on a handheld system such as a smartphone or in a device connected to a remote server, it may run as a Web Augmented Reality application (WebAR) [24], it may run on a cloud with a thin client, or it may even profit of combinations of local and remote system. Except for the first possibility, all the other mobile architectures require some type of network connectivity. This does not mean that a permanent Internet connection is necessary—some MAR applications only need to be connected to the Internet for the purpose of content management and/or additional computational power and memory.

Another key issue on MAR applications is the presence (or absence) of markers. This allows the division of MAR applications in two broad categories: marker-based and location-based applications. The first type of system relies on predefined markers to trigger the display of AR overlays on the top of the image. Location-based apps use several position sensors (e.g., GPS) to display AR objects on top of physical ones. Marker-based apps are based on image recognition—they use black and white markers as triggers to display AR content. To see the augmented content, the user points the device camera on a marker position anywhere around him. Once the device recognizes the marker, the MAR app overlays the digital data on this marker.

On the other hand, location-based applications work without markers. They detect the user position with the help of sensors such as GPS or accelerometer, among others, and overlay the augmented reality objects on top of real physical places. Some known MAR location-based applications are Pokemon GO [25] and Geocaching [26]. These apps can send notifications to the user based on their location to provide new AR content related to a given place. For example, “Le Bar Guide” (https://itunes.apple.com/us/app/stella-artois-le-bar-guide/id335624129) provides recommendations about the best bars nearby, and “Find Your Car with AR”(https://www.augmentedworks.com/find-you-car-with-ar/) allows the user to retrieve his vehicle inside a huge parking lot using GPS.

In what target platforms are concerned, the two current major players are iOS and Android, and the choice of the best one between both will always be the subject of some controversy [27]. The first one—iOS—is a proprietary mobile operating system created and developed by Apple Inc. exclusively for its hardware. Apple iOS is based on the Mac OS X operating system for desktop and laptop computers. The iOS developer kit provides tools that allow for iOS app development.

The Android platform for mobile devices uses a modified version of the Linux kernel, was introduced in 2007 by the Open Handset Alliance, and runs in a very wide array of devices from different manufacturers. This allowed the creation of several development frameworks for this platform, such as Cordoba SDK, Xamarin, JQuery Mobile, ReactJS, and Unity3D, among others. The latter also acts as a game engine very well known in the game development community, and therefore includes game-related features useful for the development of visual and interactive systems such as MAR applications.

The development of a MAR system may not simply rely on the user/gamer interaction component. Key aspects such as gamification or student and teacher engagement—requiring access to further information in a persistent way, such as player scores or gaming objects—may lead to the integration of MAR applications in a cloud or server ecosystem comprising elements such as Web and Database servers and a set of Web applications and services, that manage the flow of information among the other elements, as a part of the Web of Things [28]. As far as MAR systems are concerned, the architectures have been following the trends of general mobile and web development.

Most mobile architectures rely on web services as a standard medium to propagate communication between the client and server applications on the Web [29]. A web service is a software module which is designed to perform a certain set of tasks, is searchable over the network and can be invoked. Its role is to provide functionality to the client that invokes the web service. The underlying architecture of web services consists of three distinct roles: the provider, which creates the web service and makes it available to the client application that will use it, the client application (the requestor) that looks for some sort of functionality via a web service, and the broker, which is the middle agent that enables the client application to locate the web service. There are two major APIs to deploy web services: SOAP and REST. Simple Object Access Protocol (SOAP) was the first to emerge and provided a standard to ensure that programs built on different platforms and programming languages could exchange data in an easy and transparent manner. Representational State Transfer (REST) was designed specifically for working with components such as media components, files, or even objects on a hardware device, making it particularly advisable for MAR applications. Any web service that is defined on the principles of REST can be called a RestFul web service. As REST is simply an architectural pattern, it can make use of SOAP as the underlying protocol for web services.

5. Results and Discussion

In this section, we give an account of the architecture of the PlanetarySystemGO platform and present the implementation tests of the game that were performed with the targeted public and their results.

5.1. PlanetarySystemGO Platform

The architecture of the PlanetarySystemGO platform is represented in Figure 3 and is divided into three components: Platform server, Web application, and Mobile app. Platform server includes database that is responsible for managing all data in the system. There are three types of data: data management, which includes the administration of users (teachers and students) and events; learning objects such as planetary systems; and the results about the implementation of the game on the students.

Learning objects and data management is administered by a back-office through a web application running in a web browser. Learning data is managed by the mobile app through the web services. The mobile app was developed in Unity3D that provides a rich environment to develop AR contents.

Table 1. Model of planetary systems.

PlanetarySystem
+ Name: string
+ StarPosition: point
+ Scale: float
+ Objects: list <CelestialObjects>

represents the model of planetary systems. It is identified by a name, which in the case of our planetary system is the Solar System. In addition, the position of the star, which is located at a focus of an ellipse, corresponds to the GPS coordinates in the real world. The scale represents the orbital ray of the last planet of the planetary system. Finally, there is a list of celestial bodies.

In our system, celestial bodies include star and planets (

Table 2. Model of celestial objects.

CelestialObjects
+ Name: string
+ Information: string
+ Texture: image
+ DistanceToStar: float
+ Dimension: float
+ Foco1: point
+ Foco2: point
+ Velocity: float
+ Questions: list <Question>

). Each celestial body has a name; information about its characteristics; its texture, which consists of the skin of the object; and distance to the star (DistanceToStar), which is a proportion of the planetary system scale and the dimension of the body. In the case of planets, they move in an elliptical orbit with two foci. Focus 1 represents the star position of the planetary system and the second, focus 2, together with the distance to the star and velocity, provides the location of the planets in the real world. The inclusion of the second focus allows the inclusion of moons of planets. In this case, the two foci are in the position of the planet. Finally, it includes a list of multiple-choice questions related to each celestial body.

Each question has content, a set of four answers and information about the correct answer (

Table 3. Model of questions.

Question
+ Conten: string
+ RightAnswer: string
+ Answer: string [4]

).

The learning objects are managed by teachers through the Web application. In this environment, they may create planets and group them in a planetary system. This process may be from scratch or by cloning existing models of celestial bodies. Also, they may use information and questions already existing in the system or include more information and questions about the celestial bodies. An Event is defined by a planetary system that is parametrized to be played in the real world. In this regard, it is necessary to provide the GPS coordinates of the star and orbital radius of the last planet (scale). Each Event is associated to a code that may be used by the mobile app to download data from the game. The game may be played in online or offline mode. Internet is only necessary to download the Event and to send data from the game to database through Web services. When the game is played in offline mode, the information is stored in the device and sent to the server when it gets online.

5.2. Implementation Tests of the Game

As mentioned before, the MAR platform has been in development since 2016 by higher education students supervised by higher education teachers in the context of their final project before graduation. Throughout the platform development, several implementation tests were performed with the targeted public to evaluate its impact and identify any performance problems (

Table 4. Implementation tests of the game.

Event	Date	Location	Children	Teachers
Christmas holidays 2016	21 December 2016	Polytechnic campus	16	-
Summer holidays 2017	26 June 2017	Polytechnic campus	17	-
International Children’s Day	1 June 2018	Mata dos Sete Montes	76	11
Summer holidays 2018	26 June 2018	Polytechnic campus	20	-
Primary school class (Pilot)	October 2018	Santarem district	20	1
Primary school classes	2018–2019	Santarem district	102	10

).

This research is inserted on a broader project of intervention targeted to promote STEAM hands-on experiments in primary schools. During students’ holidays in the polytechnic campus, children’s parents signed a document about agreement or not related to data collection. In primary school classes, the school leaders authorized the study and informed the parents about it. When we go to primary school classes, the teachers always have the documents signed by the children’s parents, and data is only collected from the children whose parents authorized.

In a first stage, the game’s target audience was primary school students and the first version aimed at exploring our Solar System. In the school year 2016/2017, two implementation tests took place on the campus of the Instituto Politécnico de Tomar during Christmas 2016 and Summer 2017 holidays. In this version, it was possible to visualize the planets on the smartphone but, because this was an incomplete version, QR code-based markers were used to give information about the planets of the Solar System [11]. Also, we used different models of smartphones according to their availability, and concluded that low level models may seriously compromise the game performance. This preliminary study allowed the authors to conclude that this approach was efficient to catch children’s attention and promote learning about the planets of the Solar System. For this reason, it was decided to upgrade the game in the following school years.

In the school year 2017/2018, an information system that communicates with the app (Figure 3) was developed and may provide the player with different experiences every time he plays the game. For example, it is possible to create pedagogical events with other planetary systems of the Universe besides our Solar System. In addition, the back-office allows teachers and instructors to introduce contents according to the grade level they teach and, for this reason, it is possible to extend the experience to any grade level [30]. Furthermore, the mobile app informs the Platform server about the results of the pedagogical event, namely the scores of the players.

To implement the game on the field, we decided to buy 6 medium-level smartphones that guarantee a good experience with the app. After some tests, the system chosen was the smartphone SAMSUNG Galaxy J5 2017 16 GB model. This device made possible to begin deploying the game outside the polytechnic campus. On the International Children’s Day (2018 1st June), an experience took place at Mata dos Sete Montes with 76 children (Figure 4 and Figure 5) with the support of Tomar City Council.

In this experience, we had limited space and many trees around it, which made it difficult to find the planets (Figure 6). Also, every 30 min a new class of students arrived to join the activities, which gave limited time to play the game with each class: ~20 min. However, again, children were very excited when playing the game.

In the end, a questionnaire was applied to them and to the eleven teachers who accompanied the classes. First two questions applied to students were: “Did you have fun playing the game?” and “Would you like to play it again?”. As can be observed in Figure 7, most students answered that they enjoyed playing the game and would like to play it again. The blue color (on the left) is for “Yes” and the orange color (on the right) for “No”.

Another question applied to students was: How much did you like the game? Following a Likert scale, the options to answer this question were: “I did not like it at all”, “I liked it a little bit”, “I somewhat liked it”, “I liked it”, and “I really liked it” (Figure 8). From the 76 students, three did not answer the questionnaire because they wanted to keep looking for planets. According to Figure 8, no one answered “I did not like it at all” and “I liked it a little bit”, six students answered “I somewhat liked it” (gray color), fourteen students “liked it” (yellow color), and fifty-three students “I really liked it” (light blue color). Explanations for not liking the game were related to difficulties to find some planets and also because in some groups some children did not have the opportunity to hold the smartphone.

The questionnaire applied to the teachers included the following questions. “Do you think your students had fun playing the game?” and “Do you find this game adequate to primary school curriculum?”. Results are in Figure 9. All eleven teachers answered “yes” (blue color) to both questions.

After this experience the designers of the game improved some aspects that they found in the last test. For example, planets were above the orbits (Figure 5). Also, there was a need to adequate the game to the available game arena. In this regard, four scale options, representing the distance of the last planet (Neptune) to the Sun, were introduced: 50 m, 100 m, 500 m, or 1000 m (Figure 10). With this option, the player may choose the game arena according to the outdoor space available to play the game.

A few weeks later, another deployment test took place at the Polytechnic campus with a group of twenty students during their Summer holidays. The chosen scale was 100 meters which allowed to have a much wider game arena than in the city garden with no big trees around. However, there were some buildings and cars spread in the game arena. After the game that occurred for approximately 40 min, some interviews were applied to the participants. One question was: what did you like more in the game? Below are some answers.

• Being an outdoor game
• Playing with the smartphone
• The game is easy to play
• The duration of the game

In what concerned the students’ major dislikes, we highlight the following answers.

• Not be able to find all the planets
• Planets crossing walls and cars
• Last planets were far away and it took too much time waiting for its journey around the Sun
• Having to wait for the planets that were lost in the walls

After reflection and discussion, several adjustments were performed to improve some of the identified issues, namely, last planets velocities around the Sun. Also, to better identify the orbits and planets that were captured, several improvements were performed. For example, when the player gets closer to the planet its size is increased. In addition, when the orbit is found, it changes its color from yellow to green (Figure 11), and when the planet is captured, it receives a flag with a logo from the broader project (http://www.academiacap.ipt.pt).

Finally, based on the good results in informal learning environments we decided to implement the game in a formal learning environment at a primary school in a 4th grade class with 20 students (Pilot experience). In class, after a small presentation about the Solar System, the students were organized into five groups according to the number of available mobile phones with the application. Figure 11 shows some sequences of the game.

After the students played the game, they completed a questionnaire in order to assess the impact of the game on the class. First two questions were: “Did you have fun playing the game?” and “Would you like to play it again?” (Figure 12). As can be observed in Figure 12, most students answered that they enjoyed playing the game and would like to play it again.

Another question was: How much did you like the game? Following a Likert scale, the options to answer this question are: “I did not like it at all”, “I liked it a little bit”, “I somewhat liked it”, “I liked it”, and “I really liked it” (Figure 13). According to Figure 13, no one answered “I did not like it at all” and “I somewhat liked it”, only one student answered “little” (orange color), three students “liked it” (yellow color), and fifteen students “I really liked it” (light blue color). The student who answered “I liked it a little bit”, was an autistic child. He explained that he did not like to run, and his team was running all the time.

Also, the teacher answered a questionnaire, containing the following questions: “Do you think your students had fun playing the game?”, “Do you find this game adequate to primary school curriculum?”, “Does this approach favour learning comparing to traditional methods?”, and “What are the advantages of this approach to the Solar System?”. The answer to the first three questions was “Yes”. In the last question, she referred that the game provided the students with a real experience by visualizing the planets that improved their learning about the Solar System contents in a ludic approach.

Another feature that was tested in the last experience was the information transmitted by the mobile app to the Platform server. Besides providing the development of the application in the mobile phone, the back-office is in charge of processing and presenting data collected during the game. Therefore, it gives access to the scores of each student and to the answers that each student gave to each question. Figure 14 shows the results of the final score of each of the five teams who played the game with the information provided by the back-office.

The winner team, named “Mistério” (Mistery), scored 39 points and the worst score was 19 points from “Grupo Especial de Planetas” (Special Group of Planets). Unfortunately, one of the mobile phones had a lower performance than the other four and this was one the main reasons for this worst result. In fact, variables related to sensors such as gyroscope and accelerometer may compromise the efficacy of finding orbits and planets and the problem increases with lower performing mobile phones.

Because of the positive feedback of this pilot experience, since 2018 the game is being implemented with success with students aged 8 to 10 years old, in the context of a broader intervention project targeted to implement STEAM hands-on experiments in primary schools. Until June 2019, the game was implemented in several 3rd and 4th grade classes. Questionnaires were applied to students and to teachers in charge of classes after previous participants consent. A total of 102 students and nine teachers agreed to answer the questions and an exploratory data analysis was performed. Concerning the question “Would you like to play it again?”, 97 students answered “yes” and the other 5 answered “no”. Results of the answers to the question “How much did you like the game?” are presented in

Table 5. Results of the answers of students to the question: How much did you like the game?

Answers	Frequency	Percent	Valid Percent	Cumulative Percent
I did not like it at all	1	1.0	1.0	1.0
I liked it a little bit	4	3.9	4	5.0
I somewhat liked it	0	0.0	0.0	5.0
I liked it	10	9.8	9.9	14.9
I really liked it	86	84.3	85.1	100
Total	101	99.0	100	100
Missing System	1	1.0
Total	102	100.0

.

As can be observed in Table 5, 86 students answered “I really liked it”, which means that most of the students really enjoyed playing the game. Again, some students complained they did not had the opportunity to use the smartphone and for this reason it was not a great experience. To improve the results we intend to buy more smartphones to provide a better experience in the future.

The teachers’ questionnaires contained the following questions. “Do you find this game adequate to primary school curriculum?”, “Try to quantify on a scale of 1 (very little) to 5 (very high) the importance that this approach may have in favoring learning comparing to traditional methods of teaching?”

The responses were organized on a Likert-type scale with five categories (organized from one to five). For the first question “1” means “Not adequate” and “5” means “Very adequate”. For the second question “1” means “Not important” and “5” means “Very important”. The ten teachers only chose the “4” and “5” options for both questions as can be observed in

Table 6. Results of the answers of teachers to the question: Do you find this game adequate to primary school curriculum?

Answers	Frequency	Percent	Valid Percent	Cumulative Percent
Adequate	1	10.0	10.0	10.0
Very adequate	9	90.0	90.0	100
Total	10	100.0	100.0

and

Table 7. Results of the answers of teachers to the question: Try to quantify on a scale of 1 (very little) to 5 (very high) the importance that this approach may have in favoring learning comparing to traditional methods of teaching?

Answers	Frequency	Percent	Valid Percent	Cumulative Percent
Important	1	10.0	10.0	10.0
Very important	9	90.0	90.0	100
Total	10	100.0	100.0

, which confirms the opinion of the teacher from the pilot intervention in a primary school.

The sample size available is small, but the teachers involved recognized the game is adequate to primary school curriculum and it favors students learning comparing to traditional methods of teaching. Currently, more sound instruments of measuring are being considered in order to provide a more complete statistical analysis.

During the implementation of the game, it was observed the great enthusiasm of the children. From the beginning, they were very interested in the topic and showed great commitment to play the game. In addition, the spirit of mutual help and communication between peers was observed, for example to choose the correct answer to the question displayed in the smartphone screen. After playing the game in primary schools, focus group was performed between the students, the teachers and two of the researchers (first two authors of this paper). As well as various manifestations of enthusiasm about the game, suggestions for improvement are also made, such as the introduction of asteroids, among other celestial bodies. Almost all students end up expressing interest in continuing to play by asking how they can install the game on their smartphone or tablet. Teachers also participate by reinforcing the importance of introducing these approaches to motivate children to learn.

After the good experience with students and teachers, the priority was to provide a back-office that would be user-friendly for primary school teachers, because our final goal is that they may implement the game with their students adapting the information about the planets and questions to the grade level they teach.

The celestial objects and the planetary systems that group them together are stored in repositories that may allow the sharing of pedagogical contents among teachers. Figure 15 is an example of our Solar System that is included in the public repository.

Figure 16 gives an example of a webpage where questions about the planets may be introduced. In this particular case, the multiple-choice question is related to Mars. Other webpages with questions about other planets have a similar image.

Menus are still in Portuguese but our next priority is to introduce multi-language support in the next version of PlanetarySystemGO, allowing instructors to set information and questionnaires in their own language, thus reaching audiences at a more global level (http://PlanetarySystemGO.ipt.pt).

In summary, the game consists of discovering a planetary system that includes a set of celestial objects and is parametrized to be played in a real environment in the context of augmented reality provided by the application in a smartphone. The back-office is still under development but it already allows teachers to introduce information about the planets and questions to assess students’ knowledge about the contents they introduce.

6. Conclusions

AR is an emerging topic that has been gaining prominence into education due to its potential to engage students in practice-based activities [3]. In addition, the increasing improvement in mobile devices allows the use of AR applications, which gives an easier access to this tool [7]. However, despite these recommendations, the literature continues to identify a lack of studies related to game-based learning, namely, about the design of MAR applications targeted to formal learning environments [8] and in particular location-based games [3].

This paper presents a mobile augmented reality platform called PlanetarySystemGO, with learning purposes intended to promoting learning about planetary systems. The PlanetarySystemGO platform includes three components: Web platform, database server, and mobile app. The mobile app, which may be used in smartphones, consists of a location-based game that requires GPS and consequently is an outdoor version. When the player starts the app, his position in the real world has a coordinate that represents the star of the planetary system. Next, guided by the app, they need to walk in the real environment to find the orbits and the planets of the planetary system. At each stage of the game (finding the orbit, “hunting” the planet and answering the question) the players gain points (Figure 1). The player with the best score and the best time wins the game, which promotes the player’s engagement and motivation provided by the competition among the peers [12]. Also, because it provides interaction with the real world with physical movement and social interaction, it may be included in the pervasive games’ paradigm [9].

The MAR app has been the subject of several implementation tests in the field with the targeted public that demonstrated the game can engage children to play it and to learn about the Solar System. Also, questionnaires were applied to primary teachers who considered that the game is adequate to the school curriculum and promotes students’ interest to learn.

The back-office allows teachers to introduce information about celestial bodies and, also, develop a set of multiple-choice questions to assess student’s learning about the subject matters they teach. In addition, it gives access to the scores of the players and the answers each player gave to questions. For this reason, we argue the PlanetarySystemGO platform may be implemented in formal learning environments and can be adapted by the teacher to any level of school curriculum. Furthermore, the introduction of multi-language support in future versions allows the implementation of the platform in any country of the world.

Our next step is to develop implementation tests about the use of the back-office by teachers to assess its usability and provide improvements whenever necessary according to teachers needs and suggestions. However, due to the positive opinions of teachers about the implementation of the game in informal and formal learning environments, we believe this stage will be successful.

In summary, the MAR game has been the subject of several implementation tests with primary school students in informal and formal learning environments. Also, primary school teachers answered questionnaires related to the impact of the game on their students. Based on the results, we conclude that the game promotes students’ interest and engagement to play it and to learn about the Solar System. Finally, we argue that this platform may be used as a resource to be implemented in informal and formal learning environments.

7. Limitations of the Study and Work for the Future

As mentioned before, the PlanetarySystemGO platform is still under development with the aim of improving it and introduce new features. For example, the implementation tests occurred with the Solar System, and we intend to introduce other planetary systems of the Universe. In addition, we intent to include moons, asteroids, and other celestial bodies in the next iterations of the game which is already being upgraded.

Moreover, there is the need to improve the game performance in what concerns its stability related to smartphone sensors such as GPS, accelerometer and gyroscope. In fact, there are some technological difficulties related to the game performance such as instability provoked by GPS and incorrect gyroscope reading. Furthermore, it is necessary to develop implementation tests with different models of smartphones to assess its performance on different models. We already observed that low level models may seriously compromise game performance but more research is necessary in this matter.

In addition, to promote the internationalization of the platform, the introduction of a multi-language support in future versions of PlanetarySystemGO is a priority. This tool allows instructors of any country to provide information about the celestial bodies and to set questionnaires in their own language. Also, our goal is to publish the software system (e.g., through the Android store). The globalization of this product will have, as a consequence the collection of several performance indicators, such as the number of downloads, active users, user time, among other relevant statistics.

Finally, we intent to introduce a better evaluation of the game using more sound instruments of measurement to provide more information about its impact on the students. Our final goal is to include the PlanetarySystemGO platform in a continuing teachers’ professional development program in the context of our STEAM broader project.