Development of Teacher Digital Competence in the Area of E-Safety through Educational Video Games

Abstract

There is a clear need to promote motivating and effective training actions for the development of teachers’ digital competence, especially in the area of e-safety. Although educational video game-based learning has proven effective to improve motivation and learning outcomes, the existing evidence about its effectiveness for the development of teachers’ digital competence is very limited. This study examines the use of educational video games in an online course in MOOC format with the aim of developing teachers’ digital competence in the e-safety area. A total of 179 teachers from nonuniversity schools in the region of Castilla y León (Spain) participated in this study. A pre-test and a post-test were used to measure the knowledge acquired by the participants, and a questionnaire was used to measure their perceptions. The obtained results suggest that game-based learning using educational video games is an effective and viable approach to train teachers in the e-safety area of digital competence.

1. Introduction

Digital competence is “that which involves the creative, critical and safe use of information and communication technologies in order to meet the goals related to work, employability, learning, use of free time, inclusion, and social participation” [1]. The development of this competence is considered essential for all citizens to enable them to integrate properly into society, develop as individuals, meet the challenges and seize the opportunities brought about by the digital transformation [2,3]. The European Commission, with the aim of helping Member States enhance their citizens’ digital competence through education, developed and published the “Digital Competence Framework for Citizens (DigComp)” [4]. Afterwards, the European Commission developed and published the “European Framework for the Digital Competence of Educators (DigCompEdu)” [5], which defines the digital competence that teachers should have in order to make students digitally competent. This competence that teachers should have is known as “teacher digital competence” [6]. In addition to the DigComp and DigCompEdu frameworks, the European Commission has promoted different initiatives to promote the development of digital competence in the education sector [3]. At the Spanish level, it is worth highlighting the “Common Framework for Teacher Digital Competence” [7], which was developed as an adaptation of the DigComp and DigCompEdu frameworks with the aim of providing a reference framework for the diagnosis and enhancement of teachers’ digital competence. At the international level, one of the most relevant initiatives is the media and information literacy curriculum for teachers developed by UNESCO [8], which defines a competency framework that complements the ICT competency framework for teachers developed by the same organization [9].

Despite the various efforts made to date in digital competence training, there is strong evidence, not only that students currently have insufficient digital competence [10,11,12], but also that there is a worrying gap between the digital competence that teachers should have in order to successfully develop the digital competence of their students and the one they currently have [13,14,15,16,17,18,19,20,21]. Therefore, there is a clear and urgent need to promote training actions for the development of teachers’ digital competence. One of the main dangers of not addressing this need effectively in the short term is the emergence of a digital divide, which would not be due to a lack of access to technology but to a lack of digital competence [11,22].

Among the areas of teacher digital competence where the lack of training is of particular concern is the area related to e-safety, which encompasses the safe and responsible use of technology. Specifically, according to the DigCompEdu framework developed by the European Commission [5], this area covers competences related to the protection of devices and digital content, protection of personal data and privacy in digital environments, understanding risks and threats in digital environments, appropriate use and sharing of personal data, prevention against health risks and threats to physical and psychological well-being while using digital technologies, and understanding of the environmental impact of digital technologies and their use. Different studies carried out over the last few years have highlighted the general lack of teacher training in the e-safety area of teacher digital competence [13,16,19,23,24,25,26] and, in particular, the lack of training on digital identity, privacy, and protection and use of personal data [13,19,26]. The Spanish Data Protection Agency, aware of the doubts of school administrations and teaching staff on how to proceed when faced with certain situations involving personal data, developed a guide for schools, which was published in 2018 [27]. This guide is aimed at instructing teachers on how to make appropriate use of personal data in schools and to dispel doubts about the application of the General Data Protection Regulation [28] in the education sector. The contents addressed in this guide are of crucial importance since teachers, in order to perform their duties, need to deal with their students’ personal data and must do so with due diligence and respect for their privacy, always bearing in mind the interest and protection of minors. Despite this fact, to date no study has investigated the effectiveness of a training action based on this guide.

One possible solution for initial and continuing teacher training in digital competence that has proven to be effective is the use of MOOC-like courses [29,30,31,32,33], that is, online courses with instructional designs that allow massive participation and require little supervision. While it is true that the evidence reported to date is still scarce, the published results are very promising. Furthermore, this type of course has also been found effective for acquiring and enhancing personal competencies such as resilience, achievement drive, communication and self-esteem [34]. Successful experiences of face-to-face courses on teacher digital competence have also been reported [35]. However, it should be noted that this option is very expensive and has low capacity to reach all teachers. One of the main disadvantages of MOOCs compared to traditional face-to-face courses is their low completion rate [36]. For this reason, it is crucial to incorporate highly motivating activities in MOOCs to accompany the videos, quizzes and lessons commonly used in these types of online courses.

A form of learning applicable to both face-to-face and online teaching that is increasingly gaining more relevance is game-based learning and, especially, educational video game-based learning, i.e., learning using video games designed specifically for educational purposes. A multitude of studies published over the last few years has provided strong and consistent evidence that this type of learning can produce positive impacts on student motivation and learning outcomes at all educational levels and subject areas [37,38,39,40,41,42,43,44]. Video game-based learning attempts to take advantage of the enormous motivation that video games arouse to achieve improvements in the learning process. Video games are the world’s leading source of audiovisual entertainment. In Spain, according to the last yearbook published by the Spanish Video Game Association [45], almost half (47%) of the citizens between 6 and 64 years old play video games. If we also take into account that the people who play video games spend an average of 7.5 h a week playing them, and that approximately half of these people are men (54%) and half are women (46%), it becomes clear that the use of video games is widespread throughout society. The enormous interest in video games, coupled with the fact that there is an increasing body of evidence supporting their use for educational purposes, is driving the use and adoption of video game-based learning in all educational settings, including teacher education [46].

Several studies have examined the use of educational video games to teach issues related to e-safety to students of different educational levels and citizens without a specific profile [47,48,49,50,51,52,53], obtaining quite positive results overall. However, the existing evidence about the effectiveness of educational video game-based learning for the development of teachers’ digital competence in the area of e-safety is very limited. In this regard, the literature review conducted by Sandí-Delgado and Sanz [54] identifies a number of serious games (i.e., games and simulations whose main purpose is different from entertainment) and analyzes their possibilities to enhance the acquisition of different skills and knowledge in teachers, such as teamwork, ethical and legal aspects related to digital technologies, project management and planning, computer networks, and programming. It is worth noting that, of all the studies analyzed in this literature review, the vast majority do not report on the use of serious games by teachers, and only one examines the use of an educational video game for teacher training in the area of e-safety. This study [55] evaluated the use of an online game called “Tamagocours” to teach 25 preservice teachers how to use digital resources in educational contexts while respecting their corresponding copyright. Although the study analyzed the behavioral patterns of the players based on their interactions, it did not conduct any analysis of the instructional effectiveness of the game. Only one other study has been found in the literature that reported a learning experience based on educational video games aimed at developing the digital competence of teachers in the e-safety area. This study [56] examined the use of a game based on a simulated social network named “Social Lab” to train 70 teachers on social engineering. While the results of this study were positive, it should be noted that the evaluation was based solely on the perceptions of the participants, which were collected through a questionnaire.

In view of the current state of the art, it is evident that more research is needed on the effectiveness of educational video game-based learning to develop the different areas of teachers’ digital competence and, in particular, e-safety. This study contributes to fill this gap in the literature by analyzing an educational video game-based learning experience conducted in an online course in MOOC format in order to develop the digital competence of the participating teachers in the area of e-safety.

2. Methods and Materials

This study aims to empirically investigate the effectiveness of educational video game-based learning for teacher training in the e-safety area. Thereby, the present study aims to determine whether this novel form of learning can be a useful solution to mitigate the existing teacher training deficiencies in this area of digital competence. The following research questions were posed:

(a)

Is educational video game-based learning an effective way to develop the teachers’ digital competence in the area of e-safety?

(b)

What are the teachers’ perceptions toward the use of educational video game-based learning for the development of teacher digital competence in the area of e-safety?

To answer the above research questions, this study analyzed an educational video game-based learning experience carried out in an official teacher training course organized by the Regional Ministry of Education of Castilla y León (Spain). In this experience, an educational video game was employed with the objective of training teachers on how to make correct use of personal data in educational institutions.

Given that prior research has found educational video game-based learning to be an effective and motivational learning approach at all educational levels, and in a wide range of knowledge areas, it could be hypothesized that this learning approach would also provide an effective and motivational way to develop teachers’ digital competence in the area of e-safety.

2.1. Sample

A total of 179 teachers belonging to publicly funded nonuniversity schools in the region of Castilla y León (Spain) participated in this study. Of these teachers, 31.8% were men and 68.2% were women. In terms of age, the teachers were between 25 and 62 years old, with a mean age of 42.6 years (SD = 7.6). All of the participating teachers agreed to participate in this study.

All of the participating teachers were enrolled in an official teacher training course organized by the Regional Ministry of Education of Castilla y León. This course was offered to all teachers belonging to publicly funded nonuniversity schools in all the provinces of the region of Castilla y León. The course was an online course in MOOC format that covered various topics on e-safety, including digital identity, protection and use of personal data, privacy management and licensing of digital content. The MOOC format was chosen for running the course due to the need to reach a large number of geographically dispersed teachers, as well as to enable these teachers to learn at their own pace. Two editions of the course were run: one in 2018 and another one in 2019. On completion of the course, teachers were awarded with a certificate and three continuing education credits recognized by the Regional Ministry of Education of Castilla y León.

2.2. Procedure

Two different instruments were used to analyze the learning experience based on educational video games. First, to analyze the instructional effectiveness of the experience, the knowledge gained by the participants on the use of personal data in schools was measured by administering a pre-test and a post-test. Second, in order to collect the perceptions of the participants toward different aspects of the educational video game employed in the experience, a questionnaire was administered. Participants first completed the pre-test. After that, they played the educational video game until all the questions integrated into it were answered correctly (details about the game and the integrated questions are provided in the next section). Then, they completed the post-test. Finally, participants filled out the questionnaire.

2.3. Materials

The educational video game used was created by integrating educational resources into an existing video game, which was based on “Flappy Bird”, a mobile game that became very popular a few years ago. The educational video game was created with a web platform called SGAME (available at https://sgame.dit.upm.es (accessed on 29 July 2021)) [57,58]. The gameplay of this educational video game is very straightforward: the player controls a bird that flies continuously to the right and has to dodge pairs of pipes. A screenshot of this game is shown in Figure 1. Each time the player clicks the mouse left button or presses the space bar key, the bird flaps and moves upwards slightly. If no action is taken by the player, the bird falls due to gravity. For each pair of pipes the bird crosses without hitting them, the player scores one point. If the bird collides with a pipe, the game is halted and shows the player an educational resource randomly chosen from those available. All educational resources integrated into the game are two-slide presentations where the first slide shows a question with multiple choices (as the one shown in Figure 2) and the second one contains a “pill” of information related to that question (as the one shown in Figure 3). Thus, the player can correctly answer the questions by consuming the information pills and reasoning the answer based on the acquired knowledge. All questions and information pills were developed based on the guide for schools published by the Spanish Data Protection Agency [27] in order to explain to the participating teachers how to make a correct use of personal data in educational institutions. For creating the slideshow presentations that were integrated into the game, the ViSH Editor authoring tool [59] available in the SGAME platform was employed. Whenever the player answers correctly one of the questions that appears when the bird hits a pipe, the bird goes through the pipe and the player can continue playing. On the other hand, if the player answers incorrectly, the bird falls to the ground and the player must start a new game. When the player answers a question correctly, the educational resource containing the question is never shown again. However, if a question is answered incorrectly, the player will have another chance to answer it in subsequent plays. As soon as players answer correctly all the questions integrated into the educational video game, they are informed that they have successfully completed the activity and that they can stop playing.

The teachers who participated in the game-based learning experience analyzed in this study played the educational video game described above and learned how to make correct use of personal data in educational institutions by consuming the information pills and answering the questions integrated into the game. These information pills and questions were presented to teachers gradually during playtime, as they were shown one by one to each teacher each time he/she failed to get the bird to dodge a pair of pipes.

The educational video game employed in the game-based learning experience had to be authored specifically for this study, since no other video game that covers the targeted learning objectives existed. The SGAME platform was chosen as a game authoring tool because it allows creation of educational video games by easily integrating tailored educational resources into premade games, which are termed game templates. As previously indicated, in this study, the SGAME platform was used to create the educational video game by integrating information pills and questions on the use of personal data in schools into a premade video game based on “Flappy Bird”. There were two main reasons why this game based on “Flappy Bird” was chosen as a game template among all those offered by the SGAME platform. On the one hand, it is extremely easy to learn how to play this game. The usability of the video game was considered a crucial factor for the success of the game-based learning experience, since the video game was expected to be played by teachers with different profiles, including nongamers and casual gamers with low gaming skills. On the other hand, the game template was chosen with the aim of presenting the integrated educational resources to the teachers at an appropriate rate in relation to the time required for the consumption of these resources, achieving in this way an appropriate balance between learning and fun in the educational video game.

2.4. Instruments

The pre-test and the post-test were comprised of the same set of 10 true/false questions, which were designed to assess the teachers’ knowledge of the use of personal data in educational institutions in accordance with the General Data Protection Regulation [28]. These questions asked about the concepts, rights and obligations regarding data protection and data treatment processes described in the guide for schools published by the Spanish Data Protection Agency [27]. All of these questions were different from those integrated into the educational video game analyzed. The pre-test and the post-test were scored from 0 to 10. The participating teachers were given 10 min for completing each of the tests so they had 1 min for each question. In order to prevent the teachers from memorizing the answers of the test, no feedback was given to them after completing the pre-test, although they could access the solution of the test after the end of the game-based learning experience. Regarding this issue, it should also be indicated that the participating teachers did not know that both tests had the same questions until they took the post-test.

The questionnaire administered to collect the opinions of the participating teachers toward the educational video game employed in the experience included six five-point Likert-type questions (1 completely disagree–5 completely agree) and two closed questions: one asking about the overall opinion on the game, and another one asking teachers whether they would recommend the game to a colleague. In order to increase the likelihood of obtaining honest answers, all teachers completed the questionnaire anonymously.

Section 3.2 presents the items of the questionnaire together with its results. The reliability of the questionnaire administered to the participating teachers was checked by using Cronbach’s α [60]. The calculated Cronbach’s α was 0.94, indicating a very high reliability of the questionnaire.

2.5. Data Analysis

The Shapiro-Wilk test of normality was performed to assess the assumption of normality for the pre-test scores and the post-test scores. The results of this test showed that these variables were not normally distributed. Therefore, the Wilcoxon Signed-Ranks Test for paired samples was used to compare the difference between pre-test and post-test scores. To determine the magnitude of this difference, the size of the effect was measured using the biserial correlation coefficient (r). According to Cohen’s guidelines [61], 0.1 ≤ r < 0.3 constitutes a small effect size, 0.3 ≤ r < 0.5 constitutes a medium effect size, and r ≥ 0.5 constitutes a large effect size. The results of the questionnaire administered to the participating teachers were analyzed by using the mean (M) and standard deviation (SD) descriptive statistics.

3. Results

3.1. Knowledge Acquisition

Table 1. Results of the pre-test and the post-test (N = 179).

Pre-Test (0–10)		Post-Test (0–10)		Wilcoxon Signed-Rank Test
M	SD	M	SD	Effect Size (r)	p-Value
5.0	1.3	6.9	1.8	0.46	<0.001

shows the results of the pre-test and the post-test completed by the 179 participants. According to the results obtained in the Wilcoxon signed-rank test, the difference between the post-test and pre-test scores was statistically significant at a level of significance of 0.001. The obtained results show that the difference between the scores achieved by the participants in the post-test and those achieved by them in the pre-test was statistically significant with a medium to large effect size (r = 0.46). These results demonstrate that the learning experience based on educational video games strongly and positively impacted the participating teachers in terms of knowledge gained about the use of personal data in educational institutions.

3.2. Teachers’ Perception

The results of the questionnaire answered by the 179 teachers who participated in the video game-based learning experience reported in this article are shown in

Table 2. Perceptions of the participating teachers (N = 179).

	M	SD
What is your overall opinion on the educational video game? 1 (Horrible)–5 (Excellent)	3.4	1.0
The educational video game helped me learn.	3.6	1.2
The educational video game was attractive and motivating.	3.2	1.3
The educational video game made learning fun.	3.2	1.3
The educational video game was easy to use.	3.0	1.3
The educational video game was properly integrated into the virtual platform that gave me access to it.	3.7	1.3
I would like to play educational video games like this again.	3.4	1.3
	Yes	No
I would recommend the educational video game to a colleague	79.3%	20.7%

.

The average overall rating of the educational video game employed was 3.4 on a scale of 1 to 5, which indicates that the participating teachers were, overall, moderately satisfied with the training intervention. The high level of acceptance of the educational video game is also reflected in the fact that almost 80% of the participating teachers indicated that they would recommend the game to other teachers, and that most of them were in favor of using similar video games in the future for the development of their digital competence. The results also indicate that participants slightly agreed on average that the video game was beneficial for their learning, that it was motivating and made learning fun. However, the item related to the usability of the educational video game had an average rating of 3.0 out of 5. About one out of three teachers rated this item below three, indicating that they experienced some difficulties when playing the game. Regarding the integration of the educational video game into the platform used to deliver the online course in MOOC format, most participants were satisfied.

The results of the questionnaire show that, in general terms, the teachers’ perceptions toward the educational video game-based learning experience were positive, since they had a good opinion of the educational video game used and, to some extent, perceived it as motivating, fun and beneficial for their learning. However, these results also indicate that approximately one third of the teachers found the video game difficult to use.

4. Discussion and Conclusions

This study examined the application of educational video game-based learning for the development of teacher digital competence in the area of e-safety. Specifically, this study analyzed the effectiveness of this strategy for teachers’ acquisition of knowledge about the use of personal data in educational institutions. The results obtained provide strong empirical evidence that educational video game-based learning is an effective and viable option to train teachers on issues pertaining to the e-safety area of teacher digital competence. In addition, these results also show that this new form of learning is well received by teachers.

Although it is reasonable to think that the use of educational video games could be perceived negatively by teachers as being too childish, or not serious enough to be used in their professional training, the results obtained in this study refute this hypothesis, since the teachers considered that the video game used was didactic, motivating and fun. Nonetheless, the results also reveal that a notable percentage of teachers found the video game hard to use or not particularly motivating for their learning. This fact suggests that there is a teacher profile for whom video game-based learning may not be suitable.

The results of the pre-test completed by the teachers participating in this study evidence the current lack of teacher training on the protection and use of personal data, a finding that was highlighted in previous research [13,19,26], as well as by the Spanish Data Protection Agency [27]. This study provides stronger evidence of this fact, as this is based on measures of teachers’ actual knowledge on this matter rather than on their own perceptions.

The present study provides, for the first time, measures of the knowledge acquired by teachers on the protection and use of personal data as a result of participating in a learning experience based on educational video games. Although previous research [55,56] has reported the use of educational video games to develop teacher digital competence in the area of e-safety, such research did not provide measures of the actual knowledge acquired by the teachers, or any other measures of the actual learning gains experienced by them. The evidence reported by Gordillo et al. [56] was based exclusively on the perceptions of the participants, while the study conducted by Sanchez and Emin-Martinez [55] did not analyze the effectiveness of the video game from a pedagogical point of view. Therefore, the present study makes an important and novel contribution to teacher education research.

Previous research examined the use of online courses in MOOC format for initial and continuing teacher training in digital competence [29,30,31,32,33], obtaining very positive results in terms of effectiveness and teacher acceptance. However, none of these investigations addressed the use of educational video games in this type of courses. The present study shows that MOOC-like courses are a suitable environment for the development of teachers’ digital competence in the area of e-safety by means of educational video games.

Regarding the limitations of this study, it is worth pointing out that, although the reported results provide evidence on the instructional effectiveness of educational video game-based learning in terms of knowledge acquisition, the effectiveness of this learning approach to promote highly practical skills related to teacher digital competence was not assessed. Therefore, future studies should explore this issue.

A very interesting line of future research would be to study how the incorporation of learning experiences based on educational video games influences the completion rates of MOOCs, which are usually very low compared to those of face-to-face courses [36]. Future studies should also analyze the effectiveness and acceptance of these experiences for teacher training in other topics and areas of teacher digital competence. Moreover, further research is needed to investigate the influence of teacher characteristics on these factors, for instance the influence of liking games, gaming habits or the player type (e.g., “killer”, “achiever”, “socializer” or “explorer” according to the Bartle’s taxonomy [62]). Findings on this issue could allow to determine when the use of educational games is the preferable option to make up for the training deficiencies of a particular teacher and, if not, to offer that teacher another type of activity as an alternative. Finally, it is worth mentioning educational escape rooms [63,64]; a new type of educational games that are gaining much popularity and are increasingly attracting the attention of educators and researchers. In this regard, it would be of great interest to examine the use of this new type of educational games in MOOCs for initial and continuing teacher education.