An Exploratory Analysis of the Implementation and Use of an Intelligent Platform for Learning in Primary Education

Featured Application

This study introduces the analysis of how different areas of knowledge use digital materials for learning in schools and outlines future directions to favor development.

Abstract

Smile and Learn is an intelligent platform with more than 4500 educational activities for children aged 3–12. The digital material developed covers all courses of primary education and most of the subjects with the different topic-related worlds with activities in the field of logics and mathematics, science, linguistics and tales, visual-spatial and cognitive skills, emotional intelligence, arts, and multiplayer games. This kind of material supports active learning and new pedagogical models for teachers to use in their lessons. The purpose of this paper is to explore the usage of the platform in three pilot groups schools from different regions of Spain, outlining future directions in the design of such digital materials. Usage is assessed via descriptive analysis and variance analysis, with data collected from users interacting with the intelligent platform. The results show a high use of STEM (Science, Technology, Engineering and Maths) activities among all the activities that could be chosen. Cross-curricular activities are also used. Continuation in the development of such materials is concluded necessary, focusing integration of different fields, accentuating games over quizzes, and the value of teacher training for improving their use in schools.

1. Introduction and Background

The use of digital materials and electronic devices has boomed since their emergence, and has influenced how humans interact with the world [1]. Implementation of digital material in schools has increased due to the large number of devices available: digital whiteboards, tablets, mobile phones, computers [2,3,4]. This led some schools to allocate funds to replace textbooks with electronic devices implementing a “one to one” model. In this model, each student is provided with an electronic device to work with in class [5,6]. In many centers there is some sort of technology to support student learning, nonetheless there are still limitations in many of them that do not have the required resources for a full digitalization. Another model followed by certain schools is called “Bring your own device” (BYOD), in which families are responsible for providing the device to the student and taking it to the school to be able to use it in the classroom [7,8]. These models are similar in that each student has their individual device. Conversely, there are models involving shared devices. Many schools or families cannot afford the models discussed above and have a computer or tablet room for one or two classes that rotate through the school. Another example is the case of digital whiteboards. These shows that, being wide interactive elements, they allow teamwork and generalized use in the classroom when individual devices are not available [2]. This may introduce a limitation for measuring time of use or digital resources implementation in the classroom, even when teacher training and motivation are favorable. Other factors that constrain their implementation and use in the classroom include overloaded curricula, lack of training, tight class schedules and lack of time, need for qualified technical support, and limited budgets [9,10].

In a similar fashion, particularly a few years ago, a limitation in this type of educational models was the lack of material to use in the classroom [9]. Improvement at this point has led to an increase of educational companies developing digital material for learning [1]. Thus, various types of materials and platforms can be found nowadays, such as Learning Management Systems (LMS) platforms, apps, online activities, digital serious games, educational videos, etc. [1,11,12,13]. In this way, it is intended to generate active learning environments for students by making such environments interactive [14].

Thus, different digital materials supporting teachers’ work can be found. In a broad sense, these materials can be divided into two categories. First, we could find platforms without educational content structured for uploading said material, selected by the teacher, and managing the class. This category includes management platforms or systems, such as Learning Management Systems (LMS) [1,13]. In the second category, digital activities developed for educational purposes can be gathered. In some cases, this content can form a learning platform itself. Here we could find games, videos, and online activities [11,12,15]. These new materials seek to increase long-term retention skills to achieve meaningful learning and the development of the skills that can be transferred to further tasks [16,17].

The present study introduces an exploratory analysis on the usage of Smile and Learn’s educational platform after the first year of implementation within primary schools in three pilot groups. Smile and Learn is an intelligent platform in the field of educational technology that develops material for children aged 3–12. Under certain agreements among Smile and Learn and some regions in Spain, this material is being implemented at several public schools that collaborate with the research project. The general motivation of this study is, on the one hand, to analyze and describe the usage of the platform’s activities after the implementation of the material in the classrooms. On the other hand, other motivation is to contrast whether there are differences in use among the schools of the different regions studied and at their different moments of implementation.

This analysis pursues to answer the following research questions: Which ‘worlds’ (topic-related ambiances in the app) and ‘activities’ (main element of the platform) are the most used items by teachers? Are ‘games’ more used than ‘quizzes’ (two main types of activity) in the classroom? Is there any difference in use between ‘courses’ and ‘pilots’ (two manners of launching activities)? Is there a relation between the amount of material developed for each cycle and the usage of students in such cycle? Finally, has the implementation of digital material been adequate in the classroom?

In this way, the study intends to analyze altogether which kind of activities can be more useful for teachers as all their subjects are incorporated in a single platform which offers digital materials in different formats. So far, there are studies evaluating materials in the classroom for specific areas or topics [12,15,18,19]. With this educational platform, the possibility of using different types of materials that cover the entire primary school curriculum to test which ones are most useful is incorporated. In this sense, those most widely used activities can guide future design lines for these materials. Likewise, this allows to focus attention on which areas of knowledge rely more on technology, when there is homogeneity between the different locations and temporality of implementation.

The remainder of this document is structured as follows: A contextualization of previous research in the area of digital materials usage at schools is located in Section 1. Section 2 offers the description of the platform and the digital material used in the study. Materials and methods are described in Section 3 and are immediately followed by the exposition of results in Section 4, where the two primary school educational cycles and special needs groups of children are analyzed in order to better answer the previous research questions. Statistical analyses are then discussed in Section 5 with limitations found during the analysis. Concluding remarks are provided in Section 6, along with potential future lines of work.

Related Work

The digital contents developed tend to afford higher interaction with the user as they are based on games or online activities that allow their constant repetition [20,21]. Thus, activities can be repeated until the learning objectives are achieved [22]. On the other hand, the use of technological resources in classrooms also favors student engagement towards the tasks to be performed [16,18,23]. This combination supports an important factor in learning and motivation. This factor is important for the achievement of academic goals by promoting the predisposition to learning and studying [13,24]. It also influences the performance of certain types of tasks, persistence and acquisition of knowledge that can be transferred to other activities [25]. Some studies suggest that using digital materials at class can help students in developing their motivation and attention while they connected with learning goals and achievements [16,26,27]. It can also help in building and practicing new knowledge, supporting cooperation in the classroom, further on developing more solid cognitive skills [2,28,29]. These contents help to enhance active learning by working with interactive practices [2,30], along with additional features, such as customization possibilities, that adapt the platform’s usability to the needs of the students [10,30]. Digital materials aim to help learning self-regulation as well via delivering continuous feedback and offering the possibility of repeating activities as many times as required by the student to improve performance [21,28].

Thus, by using these resources students can improve their visuospatial skills, problem solving, organization, reasoning, planning, processing speed, etc., while acquiring knowledge of the subjects [22,23,27,31]. These skills need to be worked in the classroom to achieve competence development, necessary for academic and work life [32]. Similarly, digital materials can help teachers to incorporate active learning in the classroom, provoking interest in their students and motivating them towards a better learning. This type of digital materials would support various teaching methodologies such as Flipped Classroom [33], CLIL (Content and Language Integrated Learning) [34] and project learning [16,32]. In this last point, they can provide support as complementary material for classroom work or information consultation. Other forms of classroom implementation would come with activities to practice acquired knowledge, reinforcement activities, review or homework and teaching assistance for explanations in the classroom [13,16].

On another note, the actual use of digital material by teachers mostly depends on their perception of personal comfortability and skillfulness with technology. Other factors include their knowledge about the devices and their different contents, as well as their professional abilities when integrating these into teaching programmes [35]. In addition to training, another factor constraining implementation is the attitude that teachers express towards these materials [2,35]. Bigger skills in handling them relate to more positive attitudes towards the use of digital material. This conclusion underscores the importance of teacher training in the curricular adaptation of these resources [35,36]. Teachers also play an essential role in making the usage effective through their commitment to integrate technology into the teaching process [37]. Therefore, the use of digital material increases in teachers that feel motivated to learn about it and use it [29]. Although students are digital natives, it is critical for achieving educational goals that teachers emphasize the point of using such devices is to serve as tools for work, not recreation [30,38]. In that sense, knowing how to properly use digital resources can lead to support meaningful learning in the classroom [1], not leaving apart the central fact that learning consists of many more variables, such as social interaction and cooperation, needed to be trained along too [4].

Another factor to highlight is the design of the material. A better design, accessible to diversity and consistent, will increase the amount of teachers and students using it [2]. Designing digital educational material is a multidisciplinary field in which pedagogical, computer engineering and psychological profiles get involved with game designers in a common task [10,36]. Although certain frameworks have been developed, attempting to cover both pedagogical and software bases for the design of digital educational material, there are still discrepancies and variables to incorporate in order to achieve the goals of empowering students’ learning and skills development [20,36,39].

Pedagogical models or frameworks for content development are diverse. They range from those that seek to relate game mechanics, incorporated in many of these activities, with learning [20,39]. Others show materials that focus design from pedagogical theories for learning, as the cognitive theory of multimedia learning [40], other cognitive or constructivist theories [27,36,39], and models for materials based on multiple intelligences [41,42]. Interactions made via electronic devices can improve and support better learning, should an effective and consistent design be developed among the various disciplines involved [36]. There is still no guide that defines unitary designs or how to evaluate these materials. This is perhaps due to the diversity of variables involved in these processes: educational areas, age ranges, amount of teachers, types of interaction.

This entire scenario has led to a booming growth of scientific studies in the area of educational technology [3]. Researchers detach different studies on the use of technology in scientific areas [43,44], mathematics [19,45], literature [30,38], and language learning [46] among others. In this sense, contemporary investigation has served to clarify that technology can actually help learning in elementary students [14]. In relation to the studies according to the types of materials implemented in the classroom we can find examples of various investigations on the use of videos [11], online activities thus requiring Internet connection [47], and other computer activities [12,27,48]. Current studies reach Learning Management Systems (LMS) [1,32] or digital materials and learning platforms with more complex development mechanics [18,49].

As a final point, it is worth noting the improvement of these materials in recent years with the help of Artificial Intelligence (AI), and by the incorporation of learning analytics systems [49,50,51]. This helps to specify the most relevant variables for designing digital educational material, as well as for its curricular evaluation, and its functional assessment. Their presence in platforms can help in the progress and orientation of the contents developed by monitoring activities and performance [52]. However, the limitations and controversies of generalizing and standardizing the variables required for the evaluation of learning are high and must be considered carefully, as well as the interpretations extracted from the results, rooted within the data collected by such intelligent platforms [53].

Data gathered by digital materials when incorporating these systems are useful for both the personalization of student learning, and the guidance and evaluation [5]. Such evaluation of the students’ performance serves as advice and support for the teacher, while facilitating the work of monitoring the students in their accomplishment of tasks [10]. In such a manner, educators can regulate students’ learning individually and according to specific needs. Likewise, a last benefit of using intelligent platforms comes with these undertaking many automatic tasks that rest teachers time of actual dedication to the student [29,49].

2. Description of Smile and Learn Platform

Smile and Learn is a multidevice platform: the products created can be used in digital blackboards, computers, tablets or mobile phones, available for the most common operating systems (Windows, Android, iOS, Linux). As a result, new educational material can be used by educators under different teaching methods in their lessons, helping implementation of technology at schools (information for accessing the platform is provided in Supplementary materials).

Inside the platform, as of December 2019, there are more than 4500 activities to be selected, classified among ‘games’, ‘quizzes’, ‘tales’, ‘videos’ and ‘theory’. Activities are grouped according to Gardner’s theory of multiple intelligences [54]. Following this structure on the main menu, eight topic-related worlds can be found, named as (Figure 1):

• Science: includes activities in the field of science, to work scientific methods and concepts.
• Spatial: includes activities working visual–spatial skills and cognitive–spatial skills.
• Logic: activities work logico-mathematical skills and others related with said field of knowledge.
• Literacy: contains activities as interactive tales, vocabulary, word games, etc., that work verbal-linguistic skills.
• Emotions: involves activities to develop emotional and intrapersonal skills.
• Arts: artistic activities such as painting, music, playing instruments.
• Multiplayer: includes games against the machine or classmates.
• An additional world named after the user, where the student can build two virtual villages and manage the resources earned by the activities done.

The platform includes some game elements that teachers can use in class if they want to introduce gamification patterns for their lessons. Each time users complete an activity, they earn the completion-token ‘smileys’, which are going to help them to progress and increase their level leading to getting a resource-token called ‘gems’, which can be used to buy clothes, buildings, personalize the user’s house at the virtual village, etc.

In addition, activity contents have been translated into five languages (Spanish, English, French, Italian and Portuguese). Translations are done by bilingual profiles of each language to adapt the content for learning in each language. Moreover, most of the activities draw from international curricular contents based on global elementary education standards or cognitive skills acquisition, that can help students perform better in their grades.

One of the aims of Smile and Learn is to support teachers in their work [49]. Following that purpose, the team includes new activities each month through the platform increasing the provision of materials. Moreover, the educational department elaborates didactic guides for each activity to show teachers significative pedological goals and how those can be fulfilled in the class. Catalogues of educational contents are also made by the educational department in order to facilitate the selection of contents (catalogues are provided as supplementary materials), where we can find 117 apps with activities that include in-class topics, 92 video activities offered for children aged 3–6; 132 apps and 105 videos for 7–8; 100 apps and 81 videos for 9–10; and 76 apps and 61 videos for 11–12.

Every time an activity is accessed by a student, duration and specific additional feedback about their performance is recorded at the learning analytics system. Those data allow the company to analyze the interaction of a particular child over time, study general behavior patterns and design a more personalized experience (legal terms are included in the Ethical statements). The learning analytics system shows these data to teachers to be used as evaluation factors, should it be required. In addition, the content, personal profile, passwords, group customization… can be managed by teachers on the website (Figure 2).

Likewise, the platform has an activity recommendation system based on artificial intelligence. By the activities recommender, on the initial screen the child is shown exercises that can improve their progress, and activities that other children, similar to their profile, have played. Information about the model of Smile and Learn’s platform recommender can be found in Baldominos and Quintana [55].

3. Materials and Methods

3.1. Participants and Cohorts

To analyze the platform’s implementation at schools, data were collected from three pilot groups in regard to their first year of use. Groups belong to different regions from the Spanish territory. Smile and Learn has signed agreements with various regions of Spain to implement digital material in schools with electronic devices. These schools are selected by said regions, or through registration processes regulated by the educational delegations in charge. A common characteristic of these schools is that they are public schools. With the bilingual model (Spanish and English) this material helps to have bilingual content to use. With the same idea, the implementation of the digital material agreement established that this material provided to schools is free and complemented with teachers training by the education team. In so doing, these schools have free access to all the content and functionalities of the educational intelligent platform.

In the case of this study, the pilots analyzed in their first year of implementation in public schools have Spanish as their official language, without a second official language in these regions. Therefore, the use of the implemented digital material has been carried out mainly in Spanish, with the exceptions of classes in the bilingual plans (which also used English), and the subject English.

The two following criteria were established in order to build the working database for admissions into groups, in the aim of avoiding noise for further analyses. (1) The first criterion attunes inclusion parameters, assigning the platform to be used in schools for primary education and excluding those with children aged 3–6—this criterion was set to facilitate the comparison between curricular contents and the catalogue of educational contents elaborated for the platform. (2) The second criterion helps database systematization, disposing all the groups to include the name of the class (e.g., ‘1C’) and the age of its students, excluding random names or nicknames for the classes. The selection of these groups is related as follows: first grade (aged 6–7), second grade (7–8), third grade (8–9), fourth grade (9–10), fifth grade (10–11), and sixth grade (11–12). Those groups can be organized as 1st Cycle (groups 1st, 2nd, and 3rd) or 2nd Cycle (groups 4th, 5th, and 6th) of primary education. Groups working with children with special needs were classified as Special Needs group properly.

Regarding cohorts, this study analyzes the use of the platform in three pilot groups composed by public schools from different locations where the implementation was done in different moments of the academic course 2018–2019. (1) The first group covers 49 schools, with 238 classes (18 with special needs; 134 of 1st Cycle; 86 of 2nd Cycle). The training period for this group was from January to February of 2018, and the timeframe selected for assessing implementation and usage was between March of 2018 and June of 2019. (2) The second group is composed by 29 schools with 251 classes (16 classified with special needs; 117 of 1st Cycle; 118 of 2nd Cycle). The training period in this region was from May to June 2018, and the timeframe of usage spans from September of 2018 to June of 2019. (3) The third pilot group has 26 schools with 122 classes (9 with special needs; 60 of 1st Cycle; 53 of 2nd Cycle). The training was scheduled in October of 2018, with a timeframe of usage from November of 2018 to June of 2019.

Smile and Learn’s platform was used to collect this information from users via a learning analytics system. In order to examine user data, the following variables were set: ‘time of usage’ (in seconds) addressing each type of intelligence, ‘number of activities’ for each type of intelligence, and the ‘score’ obtained (average shown as a percentage) from the activities listed as game and quiz during the platform’s usage period.

3.2. Pilots Training and in-Classroom Experience

Before implementation, a two-step preliminary plan was developed for training teachers and the executive board of each school. This training consists in introducing the methodology and usage know-how of the Smile and Learn’s platform.

The first step addressed a general training period, coordinated by the government of the involved Spanish regions, together with the educational team. This training consists in explaining the implementation and technical characteristics, how to download the platform, or how to work with the school’s free user. Additionally, they were exposed the platform’s management and the methodologies they were able to apply should they wanted to continue with the project further on. The use format in these schools came through a ‘superuser’ for the school, that managed the different classes and users, and that was able to create unlimited profiles. Each teacher also had a free access account to all the content in order to manage their students and see their progress.

The second step addressed a more specific training for each collaborating school, allocated in the different provinces within the participating regions of the country. These partnered with the team in the agreement for explaining all the resources needed and detailing how to manage the platform’s activities. For that training, members from Smile and Learn’s educational team were prepared for explaining and resolving any questions that may appear, creating the lists of users for each school, and for sharing the material content, the catalogues of contents and tutorial videos supplementing the training process. During the entire academic year technical support was provided to any teachers that required so. Likewise, didactic guides offered examples of methodologies to apply in class, explaining several ways for accommodating content and complementary activities through the platform.

As a result, these games can be used by teachers in their lessons under several teaching methods. For example, as the platform is developed in five languages by natives, it can be used for CLIL methodology [34]. Activities like ‘Experiments’ can be worked at classroom to learn expressions and verbal tenses used for researching goals within thought experiments. Meanwhile, students will also be learning scientific concepts. As an example of a Flipped Classroom [33], teachers can schedule home readings of the tales incorporated in the ‘Literacy’ world of activities, to be worked at class the next day by a discussion, a topic presentation, some group activities, etc. As a different example, if teachers want to develop an environmental project by groups, they can use activities like ‘Green City’ (a game in which children need to save a polluted city) and ‘Recycle’ (to learn how to separate waste). The possibility of having more than 4500 activities, with combined methodologies, gives a wide range of opportunities to adapt and support teachers’ lessons with active learning.

3.3. Data Acquisition

Smile and Learn records data resulting from the interaction of the children with the platform. The regulation of the data and the Smile and Learn’s agreement allows the usage of information anonymously for research projects. Third-party distribution for commercialization or other purposes is not allowed. This regulation is included in the legal terms accepted when entering the platform according to Article 13 of the EU 2016/679 Law (this information is expanded in the Ethical statements section). These data comprise two different dimensions: app usage and feedback from the activities. In the case of app usage, every time that a child opens an app, a session is started. This session finishes when the child closes the app, then the total usage time and closing timestamp are recorded in the database. In the case of feedback recording, ad-hoc data are gathered for each different app, describing the child’s performance per activity. A common framework is shared among most of them: common information includes the type of activity (theory, quiz, game, etc.) and performance scores (which consists on hits/mistakes in the case of quizzes or some games, but may appear different in other apps).

Such data are recorded in real time while children are using the platform. The mobile app calls API endpoints, where the information to be recorded is bundled inside the request. In the case that the device appears disconnected from the Internet (which can happen as a security mechanism to prevent children from accessing undesired contents), the request is cached and sent when the device gets connected again. All such data are stored in a relational database, and all records are timestamped.

3.4. Data Analysis

Following the study goals, pilot groups were analyzed individually, due to the difference in implementation time variable among them. Pilot groups from the different regions were compared too, in order to see whether use is homogeneous or varies according to the location of the school at the time of implementation. In the first place, for examining usage in the three pilot groups, descriptive statistics were applied for each group analyzing individual use of the platform. Moreover, specific intergroup comparisons were made via a one-way ANOVA analysis, with each pilot group divided into Special Needs (aged 6–12), 1st Cycle (1st, 2nd and 3rd grades, aged 7–9), and 2nd Cycle (4th, 5th and 6th grades, aged 10–12). The factor in that case is the division of these groups (coded: 0, 1, 2). This analysis was performed to answer the questions on the use of different worlds and the kind of activities played by each pilot group, as well as a description of such usage.

Dependent variables were ‘time of usage’, ‘number of activities’, and ‘score’. Data were analyzed by a Factorial ANOVA in order to explore whether there were different uses among the three groups of pilots as first factor, and groups of classes, as second factor. To deepen in the analysis of the platform’s usage, the study targeted each topic-related world available in the platform under the variables of ‘time of usage’ and ‘number of activities’. For the variable ‘score’, an overall percentage of activities (specifically designed to measure hits/mistakes) were collected when showing correct in the games and quizzes played by the user. This analysis attempts to clarify if usage is homogeneous among the different regions and moments of implementation. In addition, those data provided information to describe their motivation using the platform.

4. Results

4.1. Pilot Group 1

Results obtained from Pilot Group 1 via descriptive statistic analysis show the most used word was Science (mean: 19,869.85), followed by the world Logic (mean: 15,596.81). Should all activities done in all the worlds at the platform Smile and Learn be contrasted, the featured worlds were: Science (mean: 77.51), Logic (mean: 69.32) and Arts activities (mean: 64.26). It could be inferred that the students’ use of Arts activities was large, nonetheless the time spent with this activities was smaller than that of other worlds, which suggests that such activities can be done more quickly in comparison with Science or Logic activities (Figure 3).

4.1.1. Pilot Group 1 Usage of the Different Worlds

To analyze use (‘time of usage’ and ‘number of activities’) among the classes’ groups in Pilot Group 1, all descriptive statistics were calculated by aggrupation into Special Needs classes, 1st Cycle and 2nd Cycle classes (

Table A1. Descriptive statistic of the Pilot Group 1 at platform worlds. The use is measured in seconds and the activities done are collected as numbers.

	Artistic Use					Artistic Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	170	9826.56	17,121.06	3.82	127,423.20	174	64.26	94.17	1	595
1st Cycle	98	10,782.82	16,163.01	5.60	111,609.87	101	66.98	83.81	1	413
2nd Cycle	61	9766.35	19,700.78	15.94	127,423.20	62	68.74	114.55	1	595
Special Needs	11	1640.96	2904.61	3.82	9992.91	11	14.09	13.07	1	40
	Emotion Use					Emotion Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	119	1774.16	3060.49	25.64	24,036.19	124	10.58	14.19	1	108
1st Cycle	72	2063.91	3624.60	41.90	24,036.19	75	11.39	15.81	1	108
2nd Cycle	39	1421.99	2013.38	25.64	9874.73	41	9.85	12.22	1	59
Special Needs	8	883.18	415.44	234.95	1350.78	8	6.75	3.69	1	12
	Logic Use					Logic Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	186	15,596.81	27,272.71	16.33	151,940.04	195	69.32	110.59	1	802
1st Cycle	112	19,327.72	32,165.74	16.33	151,940.04	114	83.75	127.55	1	802
2nd Cycle	60	11,419.98	17,442.54	26.63	88,205.33	66	55.21	83.48	1	350
Special Needs	14	3650.21	3966.94	575.47	13,970.74	15	21.73	27.43	1	105
	Science Use					Science Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	183	19,869.85	27,187.81	10.69	172,020.99	187	77.51	104.766	1	539
1st Cycle	108	23,814.70	30,216.55	16.16	172,020.99	109	87.52	104.24	1	532
2nd Cycle	66	15,220.57	22,099.90	10.69	88,906.35	66	71.24	111.39	1	539
Special Needs	9	6626.48	7095.50	374.02	18,304.90	12	21.00	29.56	1	84
	Literacy Use					Literacy Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	176	12,405.35	26,158.33	5.57	222,286.25	173	38.73	69.57	1	468
1st Cycle	107	15,190.27	30,987.36	6.66	222,286.25	106	42.13	73.72	1	468
2nd Cycle	54	9165.22	16,861.79	5.57	70,050.54	52	38.88	69.88	1	310
Special Needs	15	4204.15	590.31	60.86	24,115.12	15	14.13	13.611	1	40
	Spatial Use					Spatial Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	170	8741.88	12,544.72	11.19	89,509.55	182	50.81	66.02	1	381
1st Cycle	95	9641.73	11,621.43	22.62	51,541.68	104	56.19	66.29	1	381
2nd Cycle	62	8220.88	14,736.52	11.19	89,509.55	65	47.38	70.29	1	326
Special Needs	13	4650.80	5336.12	490.91	19,238.01	13	24.92	25.79	2	80
	Multiplayer Use					Multiplayer Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	142	7948.80	12,223.45	1.83	69,380.18	137	25.71	36.25	1	209
1st Cycle	83	7381.98	11,149.32	7.02	69,380.18	81	22.26	25.49	1	112
2nd Cycle	51	9826.25	14,349.57	1.83	64,100.06	48	34.96	50.14	1	209
Special Needs	8	1860.88	3285.24	13.56	9784.16	8	5.13	4.27	1	10

). It was detected how primary education 1st Cycle learners were more engaged users, with more time spent working with the different materials of the platform. It is important to mention that, in the Pilot Group 1, classes grouped in 1st Cycle were more numerous than groups of 2nd Cycle or Special Needs. No significant differences were found when comparing between averages of activities done in worlds by 1st and 2nd Cycles. A possible explanation is that 1st Cycle students would need more time to complete the suggested activities. One exception to this is that 1st Cycle learners showed to be the most dedicated group at the Multiplayer world, while the highest use in both factors, ‘time of usage’ and ‘number of activities’, was achieved by 2nd Cycle classes. If we focus on ‘number of activities’, Science world activities were in first position (mean: 77.51), however the world with the maximum of activities played was Logic (number: 802). The less used world at Pilot 1 was Emotions, with its highest average reaching 11.39 ‘number of activities’ and 2063.91 seconds as ‘time of usage’—results from 1st Cycle population.

In order to compare whether there were any major differences among groups in Pilot 1, a one-way ANOVA analysis was run. At first place, results obtained from ‘time of usage’ of worlds in the platform were significant in those of Logic, F(2) = 3.167, p = 0.044 (< 0.05) and Science F(2) = 3.248, p = 0.041 (< 0.05), as represented in Figure 4. This can be explained by the huge usage of such worlds in the 1st Cycle of elementary education. Nonetheless, no significance was found among all other worlds.

In relation to the Multiplayer world, comparing the ‘number of activities’ done by class groups via one-way ANOVA, a significant effect is observed in this world world: F(2) = 3.329, p = 0.039 (< 0.05) (Figure 5). Results associate with the explanation followed through descriptive statistics, outlining 2nd Cycle in Pilot 1 as the one most involved with the Multiplayer world. No significance between other groups and worlds in ‘number of activities’ was found.

4.1.2. Pilot Group 1 Games and Quizzes Scores

In order to clarify whether activities were used with learning purposes, motivating students to rank higher scores, instead of just for obtaining rewards, scores obtained were analyzed. The average score on games by students among the different groups in Pilot 1 was 79.86 out of 100 right answers, which represents a high score of right answers during the games played. For the quiz, the average score obtained was 71.76, decreasing some points with the percentage of right answers obtained at games, although means were equally homogeneous among all groups. It should be mentioned that students with special needs got higher average scores than the 1st and 2nd Grades of primary school, however the average number of activities played was much lower. On the other hand, there was a higher use of interactive activities, listed as games, than that of quizzes (

Table A2. Descriptive statistic scores at all games and quizzes played by Pilot Group 1.

	Games					Quiz
Groups	N	AVG Score (%)	SD	AVG N. Activities	SD	N	AVG Score (%)	SD	AVG N. Activities	SD
Pilot group	152	79.86	17.04	257.83	1157.920	163	71.76	15.41	42.89	121.74
1st Cycle	93	78.82	17.67	270.17	1192.90	98	70.82	15.94	54.13	153.74
2nd Cycle	44	79.91	16.50	312.55	1280.83	53	72.14	13.71	28.55	37.02
Special Needs	15	86.21	13.89	20.80	26.75	12	77.61	17.89	14.42	19.06

).

No significance was observed in the scores obtained between games and quizzes after performing the one-way ANOVA analysis, due to the similarity of the means obtained in the three class groups. That shows a constant progress in both kinds of activity, which can mean that such activities are useful for teachers and for students to learn.

4.2. Pilot Group 2

Analyzing usage with descriptive statistics in Pilot Group 2, a higher ‘time of usage’ stood out in the world of Science (mean: 65,525.20), followed by the world of Logic (mean: 48,472.53). When analyzing the ‘number of activities’ done for each world, results also showed that there was a bigger number of activities in the world of Science (mean: 232.3), followed by the world of Logic (mean: 196.5), and Arts (mean: 164.26) (Figure 6).

4.2.1. Pilot Group 2 Usage of the Different Worlds

When performing descriptive statistics among Pilot 2 class groups, higher ‘time of usage’ and ‘number of activities’ was detected increased in comparison to the 1st Cycle in primary. The maximum number of activities carried out was reached in the world of Science (number: 4649). After this, the maximum number of activities played shows in the world of Arts (number of activities: 2673), being their average below Logic activities. The fact that the world of Arts has a smaller ‘time of usage’ result when compared with a similar average of activities per class can be interpreted regarding the time dedicated to fulfil artistic activities, which was designed to be shorter and therefore much less than in activities carried out within Science and Logic worlds (

Table A3. Descriptive statistics of the Pilot Group 2 at platform worlds. The use is measured in seconds and the activities done are collected as numbers.

	Artistic Use					Artistic Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	193	27,719.85	68,252.91	9.86	531,629.10	199	164.26	359.32	1	2673
1st Cycle	90	33,505.70	79,639.93	9.86	531,629.10	93	206.16	431.50	1	2673
2nd Cycle	93	24,898.82	58,927.72	16.65	410,792.62	96	138.57	290.05	1	1910
Special Needs	10	1882.84	2204.86	158.60	7370.25	10	21.10	33.17	3	113
	Emotion Use					Emotion Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	157	4446.52	8812.83	9.98	55,829.48	161	26.55	51.61	1	397
1st Cycle	77	5332.82	9412.81	15.97	50,824.00	81	30.62	50.09	1	284
2nd Cycle	69	3926.84	8723.41	9.98	55,829.48	69	24.90	56.79	1	397
Special Needs	11	1502.27	1819.15	34.63	5552.54	11	6.91	6.139	1	19
	Logic Use					Logic Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	218	48,472.53	83,509.39	22.76	419,728.11	223	196.58	329.86	1	2289
1st Cycle	101	64,579.17	93,543.72	24.69	419,728.11	104	249.01	352.87	1	1536
2nd Cycle	104	36,945.43	74,620.57	22.76	395,131.96	106	162.01	318.12	1	2289
Special Needs	13	15,553.14	29,776.27	42.60	111,901.73	13	59.00	60.78	1	201
	Science Use					Science Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	199	65,525.20	131,669.22	5.05	981,210.48	205	232.30	523.24	1	4649
1st Cycle	101	75,372.20	128,321.35	5.05	919,615.53	102	280.31	587.11	1	4649
2nd Cycle	89	60,165.53	140,741.02	56.15	981,210.48	94	199.13	467.54	1	3528
Special Needs	9	8021.09	10,176.30	1008.89	33,795.67	9	34.67	44.89	2	148
	Literacy Use					Literacy Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	195	31,659.10	56,349.86	7.71	277,126.51	192	89.47	153.32	1	1083
1st Cycle	96	39,578.60	64,180.87	7.71	277,126.51	92	110.88	170.92	1	1083
2nd Cycle	86	24,798.65	47,093.59	33.67	210,477.69	87	75.10	140.88	1	793
Special Needs	13	18,561.21	44,604.50	29.67	165,803.56	13	34.08	45.30	1	165
	Spatial Use					Spatial Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	194	25,151.84	48,206.98	5.11	312,566.48	203	151.51	288.29	1	2010
1st Cycle	96	31,147.13	53,913.52	5.11	312,566.48	98	178.39	284.69	1	1680
2nd Cycle	87	19,815.37	43,617.55	20.30	272,838.11	94	134.39	306.13	1	2010
Special Needs	11	15,036.10	13,091.39	218.67	39,115.06	11	58.36	41.75	4	111
	Multiplayer Use (seconds)					Multiplayer Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	171	29,609.54	65,553.53	7.19	412,884.92	169	86.13	192.65	1	1449
1st Cycle	82	26,188.26	45,509.16	12.10	263,742.04	82	75.01	131.38	1	822
2nd Cycle	76	37,801.07	85,437.46	23.82	412,884.92	76	109.21	250.89	1	1449
Special Needs	13	3301.03	4592.43	7.19	14,096.72	11	9.55	10.65	2	31

).

The one-way ANOVA analysis showed significance between groups with the ‘time of usage’ on the world Logic, F(2) = 3.986, p = 0.020 (<0.05). The average of ‘time of usage’ in 1st Cycle of primary education doubled the ‘time of usage’ in the 2nd Cycle, much higher than ‘time of usage’ present in classes with special needs. No significance was found with other worlds at the platform Smile and Learn, thus, general usage must be interpreted similar among such groups. Significance was also found along groups in their use of Logic activities, F(2) = 3.083, p = 0.048 (<0.05). This is also related to ‘time of usage’: in this case, the 1st Cycle of primary education in the Pilot 2 group has been much more active in this world than in the others. Further similarities can be observed in the rest of the worlds regarding ‘number of activities’ and ‘time of usage’ (Figure 7).

4.2.2. Pilot Group 2 Games and Quizzes Scores

In order to clarify whether activities were used with learning purposes, motivating students to rank higher scores, instead of just for obtaining rewards, scores obtained were analyzed. The average scores obtained by these students came with games activities, reaching around an 80% of hits (mean: 81.37%). For the quizzes, the average scores of right answers obtained corresponded with a 71.11% of the hits. On the other hand, the highest results in the games (mean: 83.33) were obtained by the 2nd Cycle students of primary education, while the highest scores in the quizzes (mean: 71.63) are obtained by the students of the 1st Cycle, from elementary school. The number of games achieved by the 1st and 2nd Cycle groups was less than the number of quizzes. However, in the Special Needs group, the usage of games stands out over the rest of the groups (mean: 127.92), opposed to the number of activities, listed much lower than the rest of the groups (mean: 11.18) (

Table A4. Descriptive statistics scores at all games and quizzes played by Pilot Group 2.

	Games					Quiz
Groups	N	AVG Score (%)	SD	AVG N. Activities	SD	N	AVG Score (%)	SD	AVG N. Activities	SD
Pilot group	188	81.37	15.52	30.,66	563.85	172	71.11	14.67	83.36	152.42
1st Cycle	94	79.67	14.26	35.,50	509.81	83	71.63	13.51	95.80	156.79
2nd Cycle	82	83.33	16.763	26.,82	647.25	78	70.80	15.60	80.31	156.48
Special Needs	12	81.34	15.86	127.92	243.97	11	69.40	17.45	11.18	8.38

).

No significance was found via a one-way ANOVA for the factor ‘score’ in the platform Smile and Learn between games and quizzes. That could be relevant to show that this kind of activities are considered useful by schools, and both support students’ learning. Furthermore, it shows there was no major difference in their performance.

4.3. Pilot Group 3

It was observed through the general results obtained via descriptive statistics that the Pilot Group 3 devoted more usage time to the world of Logic (mean: 94,201.08), followed by the worlds Multiplayer (mean: 80,237.00) and Spatial (mean: 56,694.08). Similarly, when analyzing the number of applications played, a higher activity was observed in the worlds of Logic (mean: 117.42), followed by Spatial (mean: 88.15) and Science (mean: 85.77). This indicates that the time dedicated to Multiplayer activities was much higher when compared with the number of activities done (mean: 23.55). It is worth to mention that this pilot group used two worlds transverse to their curricular content (Multiplayer and Spatial), both of which develop social and cognitive skills for learning (Figure 8).

4.3.1. Pilot Group 3 Usage of the Different Worlds

The results of the descriptive statistics among Pilot 3 groups show a greater ‘time of usage’ and ‘number of activities’ in the primary 1st Cycle group. The fact that the number of classes in the 1st Cycle group was bigger than in the 2nd Cycle classes explains this event. In relation to the ‘number of activities’, the maximum was obtained by the primary cycle classes in the world of Logic (N = 1000) (

Table A5. Descriptive statistics of the Pilot Group 3 at platform worlds. The use is measured in seconds and the activities done are collected as numbers.

	Artistic Use					Artistic Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	84	31,250.71	163,063.60	12.73	1,491,804.00	91	71.07	112.75	1	497
1st Cycle	47	51,860.93	216,275.02	12.73	1,491,804.00	49	107.57	125.03	1	477
2nd Cycle	31	5293.09	18,044.86	35.04	101,180.99	35	28.40	83.46	1	497
Special Needs	6	3918.33	6329.017	87.99	16,733.36	7	28.86	50.94	1	143
	Emotion Use					Emotion Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	60	29,524.73	178,283.01	7.21	1,369,570.00	64	16.69	26.77	1	176
1st Cycle	33	10,108.32	38,330.58	7.21	222,313.00	35	18.69	19.66	1	85
2nd Cycle	20	71,348.58	305,680.25	28.38	1,369,570.00	22	15.73	38.35	1	176
Special Needs	7	1562.57	1959.80	63.33	5234.92	7	9.71	10.31	1	27
	Logic Use					Logic Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	99	94,201.08	345,837.19	50.83	2,584,213.00	101	117.42	183.32	1	1000
1st Cycle	52	57,222.47	79,994.35	50.83	427,713.25	53	169.47	216.90	1	1000
2nd Cycle	38	115,441.40	465,793.07	196.25	2584,213.00	39	60.72	123.61	2	583
Special Needs	9	218,173.95	626,554.53	518.53	1888,822.00	9	56.56	62.42	10	199
	Science Use					Science Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	106	36,608.86	130,440.98	12.76	1,315,118.00	107	85.77	130.70	1	564
1st Cycle	54	36,704.26	39,020.71	19.95	144,816.80	54	130.43	146.98	1	564
2nd Cycle	44	42,023.57	198,589.15	12.76	1,315,118.00	45	43.51	99.31	1	562
Special Needs	8	6184.08	13,864.81	261.42	40,421.66	8	22.00	39.76	2	120
	Literacy Use					Literacy Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	95	53,621.70	200,797.90	31.29	1,640,275.00	173	67.44	107.48	1	545
1st Cycle	51	56,068.24	153,437.73	54.05	1,082,520.00	51	91.63	124.51	1	545
2nd Cycle	36	13,724.16	27,516.70	31.29	151,525.55	36	39.06	82.60	1	483
Special Needs	8	217,563.95	574,925.59	2037.86	1,640,275.00	8	41.00	32.82	6	95
	Spatial Use					Spatial Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	92	56,694.08	250,706.69	5.79	2,004,300.00	101	88.15	138.04	1	724
1st Cycle	49	93,531.61	337,794.80	5.79	2,004,300.00	52	133.92	167.15	1	724
2nd Cycle	34	14,313.01	54,684.76	57.53	320,106.00	40	31.12	70.82	1	420
Special Needs	9	16,240.44	14,445.52	269.56	43,843.75	9	77.11	74.99	2	231
	Multiplayer Use					Multiplayer Activities
Groups	N	AVG	SD	Min.	Max.	N	AVG	SD	Min.	Max.
Pilot group	77	80,237.00	370,084.40	5.25	2,865,876.00	76	23.55	37.78	1	187
1st Cycle	41	116,061.52	472,703.70	68.72	2,865,876.00	41	30.10	39.69	1	168
2nd Cycle	29	47,629.73	219,409.06	5.25	1,185,030.00	28	15.71	35.48	1	187
Special Needs	7	5494.89	10,867.64	296.96	29,861.98	7	16.57	32.25	1	89

).

In order to verify differences between the groups in Pilot 3, a one-way ANOVA was performed comparing the usage time per groups and activities carried out. Significance is obtained in ‘time of usage’ with the Literacy world along the groups, F(2) = 3.565, p = 0.032 (<0.05) (Figure 9). Pilot 3 shows a high use of the Literacy world by the 1st Cycle of primary school, above average (mean: 56,068.24), and very high in the Special Needs group (mean: 217,563.95). This usage may have been increased in some class groups by their preference for such material, however, no significant differences in use were found in the other worlds of the platform, with a fairly homogeneous use.

When performing the one-way ANOVA analysis to compare the ‘number of activities’ carried out by the groups, significant differences were observed in the Arts world, F(2) = 6.208, p = 0.003 (<0.05); Logic F(2) = 4.844, p = 0.010 (<0.05); Science F(2) = 7.214, p = 0.001 (<0.05); Spatial F(2) = 7.065, p = 0.001 (<0.05) (Figure 10). This points out the value of fulfilling activities in the 1st Cycle of primary school, in all of these worlds, contrasted to the use and implementation with the rest of classes, which include Special Needs and 2nd Cycle of primary education groups.

4.3.2. Pilot Group 3 Games and Quizzes Scores

In Pilot 3, games scored an average of 80% (mean: 82.15%). That average of right answers shows students were involved in their performance instead of getting the rewards given for the activities. Quizzes, however, while remaining at a remarkable level, decreased their overall score in comparison with the percentage of right answers obtained in games (mean: 74.33). The highest scores obtained, both in games (mean: 84.26) and quizzes (mean: 76.86), were achieved by groups with educational needs. Likewise, the use of games prevailed over quizzes in all class groups (

Table A6. Descriptive statistics scores at all games and quizzes played by Pilot Group 3.

	Games					Quiz
Groups	N	AVG Score (%)	SD	AVG N. Activities	SD	N	AVG Score (%)	SD	AVG N. Activities	SD
Pilot group	86	82.15	14.89	167.58	345.47	82	74.33	13.12	55.17	73.52
1st Cycle	47	84.25	10.33	249.57	434.95	45	73.08	13.59	81.22	81.45
2nd Cycle	30	78.24	19.78	67.57	155.53	29	75.59	11.77	23.62	49.99
Special Needs	9	84.26	15.36	72.78	73.95	8	76.86	15.94	23.00	34.28

).

No significance was found via one-way ANOVA analysis in the ‘scores’ at the platform Smile and Learn in activities classified as games and quizzes. That shows a constant progress in both kinds of activity, which can mean that both activities are useful for teachers and for students to learn. Performance results are similar among the groups (1st cycle, 2nd cycle, Special Needs).

4.4. Analysis and Description of Usage among the Pilot Groups

Once the individual usage by pilot group was analyzed, specific analyses were applied in order to compare if there were differences in usage among regions in Spain. This comparison takes place among Pilots Groups (1, 2, and 3) and Classes Groups (1st cycle and 2nd cycle) depending on their use of the activities by area (Science, Logic, Spatial, Literacy, Art, Multiplayer, Emotions).

In order to verify data consistency, a reliability analysis was carried out. For the variable ‘time of use’ the in Pilot Group 1 was 0.695, and 0.817 in Pilot Group 2. For ‘number of activities’ the α-Cronbach obtained was 0.825 in Pilot Group 1, and 0.901 in Pilot Group 2. It should be noted that in Pilot Group 1 the variances were larger than in Pilot Group 2. Equally, there were greater differences in ‘time of use’ among the different worlds in Pilot Group 1. This variability in the variance may reflect the teachers’ predisposition towards certain activities. Usage can also be influenced by the very school. Groups with special educational needs did not have enough sample for this analysis, so they could not be considered for comparison in the factorial analysis.

In a first place, a Factorial ANOVA was introduced along with the following factors: the first factor was the pilot group (1, 2 and 3), the second factor was class classification: by Special Needs, 1st Cycle or 2nd Cycle classes. The interaction effect that these two factors establish shows significance with the ‘time of use’ for the world Emotion, F(2) = 6.104, p = 0.003 (<0.05), ηp2 = 0.050, p = 0.885. This indicates the time of use dedicated to Emotion’s world was different in each of the groups and the pilots—in other words, these activities are presenting with a higher variety in the ways users implemented their resources for the world of Emotion. Another explanation is that some schools dedicate more time to emotional intelligence for skills development. It was the case of schools in Pilot Group 3, as results point out in Table A1, Table A3 and Table A5. Moreover, the three pilot groups of the different regions showed significance in the worlds of Logic F(2) =10.699, p = <0.005, ηp2 = 0.084, p = 0.989; Science, F(2) = 10.121, p = <0.005, ηp2 = 0.080, p = 0.985 and Literacy F(2) = 5.203, p = 0.006 (<0.05), ηp2 = .043, p = 0.826. These differences in ‘time of use’ among the pilots are due to the higher use of these worlds by schools in Pilot Group 2 and Pilot Group 3. This fact could indicate that implementation time was not decisive for an increased use of activities. Pilot Group 1 was the first to carry out the training and implementation, however showing a significantly lower usage.

On the other hand, between the 1st and 2nd cycle groups there was significance in the world of Literacy F (1) = 6.168, p = 0.014 (<0.05), ηp2 = 0.026, p = 0.696. Nonetheless, the strength of the effect found in this difference in use was lower than the use when comparing pilot groups. A higher use of the world of Literacy stood out by 1st cycle students of primary education (Table A1, Table A3 and Table A5).

Tendency of usage was high in all the pilot groups analyzed. No significance was found in the use of the Artistic, Spatial, or Multiplayer worlds, nor among the pilot groups schools, nor between 1st and 2nd cycle. It can be read that usage was homogeneous. This can mean a consensus of use by teachers to develop the skills of these areas via using technology. The differences in use detected may have also been due to the use of devices in the centers, or the teachers’ motivation for a certain type of activities.

In a second place, by measuring the usage through the factor ‘number of activities’, results showed significance at pilot groups in all the worlds: Artistic, F(2) = 52.550, p = 0.001 (<0.05), ηp2 = 0.053, p = 0.921; Emotion, F(2) = 5.304, p = 0.006 (< 0.05), ηp2 = 0.042, p = 0.834; Logic, F(2) = 10.525, p = < 0.005, ηp2 = 0.079, p = 0.988; Science, F(2) = 7.452, p = 0.001 (< 0.05), ηp2 = 0.058, p = 0.940; Literacy, F(2) = 4.055, p = 0.019 (< 0.05), ηp2 = 0.032, p = .718; Spatial, F(2) = 8.828, p = < 0.005, ηp2 = 0.067, p = 0.970; Multiplayer, F(2) = 9.557, p = < 0.005, ηp2 = 0.073, p = 0.980 . In this sense, significance can be observed in the groups of classes with the Logic world, F(2) = 3.103, p = 0.047 (< 0.05), ηp2 = 0.023, p = 0.594. These results show differences in the activities performed along the worlds between different regions. This could be explained by the differences in usage time according to the activities done. Similarly, another explanation for this result could be the preference (or form of use) of the different activities by teachers in the classroom.

To conclude this point, there appeared to be neither significance for the factor ‘scores’ among games and the number of games played by pilot groups, nor by 1st and 2nd cycles. That is to say: there were no differences in student performance based on the percentages of successes achieved in the activities. In general, a high percentage of successes was observed, which affected the motivation for the correct performance, both in games and in quizzes. This homogeneity showed a constant use between different schools and regions. However, there was a significant difference for the factor ‘scores’, F(2) = 4,429, p = 0.013 (<0.05), ηp2 = 0.027, p = 0.760; and number of quizzes performed, F(2) = 3,359, p = 0.036 (<0.05), ηp2 = 0.020, p = 0.632; among the three pilot groups. This may indicate that those teachers with more traditional teaching methods preferred to analyze the progress of their students by quizzes of specific questions.

5. Discussion

From the analysis of the three pilot groups, in relation to the factor ‘time of usage’ the most used worlds extracted were Science and Logic, being this higher usage independent from the moment of implementation. Some teacher may prefer traditional methods for these subjects, if we consider the differences among pilot groups of different regions. This high use of Science and Logic worlds coincides with the results obtained in a different analysis for mobile applications [3], which highlighted the use of scientific activities. Apart from this study, there are no studies of platforms that contain all kinds of curricular activities for the different elementary subjects in the literature. There are, however, studies of specific areas in which science projects [15,18,26,43,44] and logic [10,12,19,45] stand out. These studies coincide with the worlds and types of activities most used in the analyzed platform, in which all the subjects could be worked. Following these worlds, also for the variable ‘time of usage’ come the worlds of Arts, Multiplayer, and Spatial for the schools composing each pilot group. This points out the interest in these areas and activities supplementing school curricula, working cognitive skills necessary for the development and learning process of the students. The high usage of cross-curricular activities can also indicate that teachers are aware of their usefulness and have found ways to implement it in their classes properly. Opposite to this, the world of Emotions is the least used, however being of much import how emotional management and development of affective skills are addressed in the educational environment. As for the Literacy world, usage time was expected to show higher rates than obtained, for they are longer activities, very much parallel to those in the worlds of Science and Logic. Projects for language learning through digital resources [46] and literacy [12,30,38] are also notable in the literature reviewed. A possible explanation for this event could be the reluctance to use digital materials for encouraging reading. However, there are in the literature cases of success in digital materials implementation compared to more traditional methods for this area of learning [30]. The differences observed among the groups, Special Needs, 1st Cycle and 2nd Cycle groups, correspond to a greater dedication in 1st Cycle at Science, Logic and Literacy worlds. This is favored by the variety of material within the platform focusing these ages if compared to the other groups.

Regarding the factor ‘number of activities’, usage of Science, Logic, Arts and Spatial activities predominates. However, it is worth mentioning that, although the number of activities in the world Multiplayer is small, it implies a great dedication in time per activity. As in the previous case, the world with the least use of activities in general is Emotions. This fact could be explained by this world showing a smaller number of activities as opposed to the material developed for other worlds. On the other hand, the differences observed among 1st Cycle or 2nd Cycle groups show great variability in all the worlds. This will depend on the pilot group analyzed, and the activities performed in each world. As a general feature, a higher use in 1st Cycle was found compared with the rest of groups, excepting activities in the Multiplayer world for Pilot Group 1, which underlines the use of such activities by 2nd Cycle students. According to Baptista [31], this kind of activities (games genre puzzle and quizzes) may help students to enhance their operating skills performance, decision making, problem solving, technical skills and planification, among others. A more specific analysis in schools would be necessary to isolate the variables that may be influencing the diversity of use of the activities, as well as the teachers’ approach towards these activities in the classroom.

The third factor analyzed was the ‘score’ per type of activity. Higher scores in games versus in quizzes have been observed among all class and pilot groups. Such better results for games may be due to the fact that they have also been the most preferred activities, doubling and triplicating the number of games versus quizzes played. This reinforces the value of good and clever design, along the selected methodology of use, favoring the design of activities that motivate students to learn [28,43]. Likewise, these results show a high percentage of correct responses, which can be associated with a predisposition towards learning and students’ well-performing [24,25]. Should percentage of successes be lower, it would seem necessary to question whether students carry out activities only because of the rewards they can obtain. That would mean that only extrinsic motivation would have been able to be promoted [13]. This point is related to the motivation the use of technology in the classroom can arouse, however more studies are needed to clarify disposition in the area of motivation. In the same way, in all the groups, performance obtained by the students among different schools and locations shows similar. It would be positive to investigate further evaluation processes of the platform, being able to contrast progress in the activities with students’ progress in the classroom.

The present study allowed an investigation on the use of digital materials in different regions of Spain through an intelligent platform with digital educational content. This covers virtually the entire body of curricular areas presented, and allows a general analysis of those areas in which technology is used the most as support to the teacher. Similarly, based on the results obtained, some lines of improvement can be established for further use and implementation of materials:

• A preferential use of applications within the STEM area was observed—the use of cross-curricular activities such as those found in the world of Spatial being notable. To maintain the use and interest in this area, it would be necessary to continue developing new content to be able to provide teachers with a variety of materials from which to choose, being able to adapt them to the classroom.
• In the case of the world of Emotions, materials shall continue with a deeper development in order to analyze whether there is an increase in their use by incorporating bigger diversity of contents, involving a motivating offer.
• The monitorization of pilot groups, as the incorporation of new pilot groups, raise the possibility of a longitudinal study in the use of technology and its implementation. To do this, it will be necessary to continue expanding the digital materials available through the platform. In that sense, a study of design changes and User Experience Research in conjunction with such implementation would be able to clarify how and where to improve the organization and management within the platform in order to increase its use.
• Emphasis is also to be placed on how to train teachers for an enhanced use of the activities within the Literacy world, such as debates and colloquies, which would serve to evaluate the reading comprehension of the present tales and narratives.
• In regard to the activities within the Multiplayer world, it would also be necessary to continue with the development of contents that can grow the time of dedication. Activities from this world can help to work interpersonal skills, cooperation and the underpinning social relations required in learning [28].
• As a new line of content development, the design of activities addressed to work in cooperation, and applied in different worlds, is suggested to analyze whether this type of material can help to improve performance by incorporating social variables, thus increasing students’ motivation, following many contemporary strategies in Collective User Experience Research and Social Gaming Interaction [21].

These new directions of design and development of content can be supplemented with studies on the variables that would be appropriate to collect in each activity, analyzing different types of assessment and finding new applicable metrics observing the students’ learning process [51,53]. Teachers would be desired, thus, to have more technological support for their tasks. The final goal of implementing this material is to help the teacher in changing the pedagogical model, mainly by facilitating mechanical tasks so that they can guide students in a more personalized fashion [4,49]. This would allow a better work, with quality attention, regarding the relationships and interactions established in the classroom.

Study Limitations

Despite the limitations that have been found in the study, the teams involved have reacted to minimize their impact on the material implementation. Many of the platform’s usage limitations have been foreseen through previous research. Example of this was the continued support to teachers, who contacted the education team for the use of the material in the classrooms. Another damping effect was the fact that digital materials could be previously downloaded, prior to use, without connection in the classroom [47]. Should it be considered that in many occasions both implementation and usage processes are limited by the resources of the center: in some schools, there were devices available only for one classroom that were nevertheless managed to be used for all of the classes and courses. Another limitation, present from the beginning of the study in the selection of schools by the regions of Spain, has been the lack of required material, or abundance of obsolete devices, for the installation of the platform. In those centers that were able to introduce the materials despite the mentioned limitations, teacher training was always considered a key point. This training helped educators to integrate the content material in the classrooms, along with the additional managerial and support material provided [29], an assistance for teachers that can be reinforced to continue improving training on the use of digital resources.

Similarly, in some classes, groups were identified by nicknames. In many of these cases, the average age of the students has been used to resolve the educational stage of the group; therefore, this has not constituted a significant problem except in a very limited amount of groups, which have been excluded from the study. Additional limitation during the study was that data were retrieved at a group level, instead of individually. While such input could prevent a more detailed study about the dynamics of each student, this did not cause any problem within the scope of the research and goals of this work. For future studies, we are working to design an assessment plan that allows the analysis of students at an individual basis.

Because of the agreement between Smile and Learn and the Spanish regions where the educational institutions are based, we do not have full access to the academic records of the students nor to other internal academic information at schools, due to GDPR limitations. Although this information could be useful for further studies, it has not been required in this work, which is constrained to an analysis of the very platform’s usage. For further studies, we will observe the possibility of extending this agreement. We are also elaborating a research proposal for recording more data regarding platform use with a Learning Analytics System in order to improve students’ performance evaluation.

Many platforms just evaluate via internal data, as this article proposes. In order to differentiate analysis from other types of evaluation, a control group should be launched in a school with which to compare the teacher’s evaluation of students and grades. This part is however subjective for the teacher. Results obtained in games and quizzes show correct success rates and whether a correct usage is achieved. However, it would be necessary to develop more research about evaluative aspects, so that the correct successes in the activities of games or quizzes may correspond to evaluation standards. Likewise, the marks upon memory-driven results would be not enough. This makes the adaptation to an international evaluation framework a positive contribution to be able to approximate performance assessment to the required students’ competences [32].

6. Conclusions

This study proposed an exploratory analysis of the usage of an educational intelligent platform, after the first year of implementation in public schools of three pilot groups from different regions of Spain. The main goal of this study was to analyze and describe for each pilot group the ‘time of usage’ and ‘number of activities’ at the platform according with their classification in topic worlds (Science, Logic, Spatial, Literacy, Emotions, Arts and Multiplayer) and the ‘score’ of activity (games or quiz). Comparison between pilot groups was carried out to clarify whether there were differences of use among the different regions and the moment of implementation.

In summary, the evaluation of the platform implementation and the assessment of the use of digital materials are both satisfactory. High usage by all pilot groups schools was observed. Useful results informing of improvements for future lines are to be taken into account, especially applicable to the design of digital materials and activities. These mainly point out the significant role of personal preferences, teachers’ tendency towards specific topics, and the effect of planning motivation through the design of these activities. STEM areas stand out as those in which technology can have better reception by students, and application by teachers at class. This may be inferred from a higher usage of Science and Logics apps at the platform. Further research is advised in order to clarify why certain activities maintain homogeneous usage while others vary.

This study additionally establishes the basis for future research to be carried out by educational cycles or classes of students with special educational needs. This is possible as the study presents a general analysis of the platform’s use of material in all subjects of the school curriculum. As a major future line of work, there is the need to continuing emphasizing the improvement of educational material design expanding personal skills development. Likewise, it would be necessary to keep on deeper research of the variables to be collected by the learning analytics system, that will serve to improve personalized students’ performance evaluation. Teachers’ work can be facilitated through the platform by implementing skills assessment, and using contrasting methods via international standards. In the same way, evaluating teachers’ perspectives would be positive, contrasting the obtained results with their information, and thus keeping progress in developing support with teachers’ training and adapting material. As a final remark, expanding the proposal to collect new metrics via the platform’s Learning Analytics System would allow future statistical predictive analyses of use per schools.