Assessing the Efficacy of Reflective Game Design: A Design-Based Study in Digital Game-Based Learning

Abstract

Reflective learning is widely recognised as a highly effective approach to learning. It involves learners in a process wherein they reflect on their past experiences to improve their skills and knowledge. In recent years, there has been growing recognition of the capacity of games to facilitate experiential learning. Therefore, this research aimed to evaluate the effectiveness of integrating reflective design principles into digital game-based learning (GBL). The focus was on assessing the reliability and appropriateness of reflective game design, as well as learner perceptions. This study adopted a participatory design approach, ensuring the active involvement of the target audience across all extensive phases. The research consisted of three main phases. Firstly, reflective learning was integrated into game design to develop the Reflective Game Design (RGD) framework. This development was guided by insights gathered from participants through both a user survey (N = 101) and semi-structured interviews (N = 15). Subsequently, a GBL digital game was developed based on the RGD framework in co-design sessions (N = 6). Finally, a prototype evaluation (N = 56) was conducted to assess the feasibility and practicality of implementing reflective game design within digital GBL contexts. Overall, the RGD approach and the game prototype received positive feedback in terms of engagement, content clarity, the incorporation of reflective game elements, and player immersion. This study contributes to the literature by providing principles and guidelines for RGD, thus offering valuable insights for researchers, educators, and game designers looking to create effective educational games.

1. Introduction

Game-based learning (GBL) has garnered widespread recognition as an effective educational and training tool [1]. A survey conducted by the “Games and Learning Publishing Council” shed light on the integration of digital games into teaching, revealing that 55% of the nearly 700 educators surveyed have incorporated games into their curriculum on a weekly basis [2]. Over the past decade, the realms of academia and edutainment have witnessed a surge in interest surrounding serious games, GBL, e-learning, and gamification [3]. These concepts have considerable overlap and often share elements with one another. In practice, differentiating between games and “artefacts with game elements” can be a challenging endeavour, as illustrated in Figure 1, which highlights the intricate relationships among these educational approaches. It is worth noting that these concepts often overlap, blurring the lines between traditional games and artefacts with game elements. GBL is an active educational approach that integrates elements and principles commonly found in games, such as gamification, learning objectives, and pedagogical methods, into the learning process. According to Prensky [4], Digital GBL has the potential to enhance and facilitate learning by adding an element of excitement and boosting student motivation and engagement.

In GBL, the concept of “play” assumes a pivotal role. Play allows learners to acquire knowledge and skills by actively engaging with and interpreting their environment, which encompasses both physical and social contexts [1]. GBL leverages gaming technologies and techniques to create a dynamic, enjoyable, and interactive virtual learning environment. This environment is designed to foster situated experiential learning, a process wherein learners gain knowledge by immersing themselves in real-world scenarios and actively participating in problem-solving and decision-making activities [5].

1.1. Learning Theories in GBL

In GBL, learning theories like behaviourism, constructivism, and cognitive theory serve as frameworks for enhancing learning experiences. Over the past decade, there have been significant developments in the field of GBL, with a growing emphasis on integrating learning theories to better understand how games can effectively engage learners. Educators and game developers can integrate these theories to create educational games that are both immersive and effective in facilitating learning [6]. The “Adaptive Digital GBL” framework [7] initially focuses on key pedagogical elements, psychological requirements, cognitive development, and learning behaviours. This approach allows the platform to gain a comprehensive understanding of learners’ capabilities, allowing the games to be customised to cater to the specific learning needs of the target audience. As GBL has expanded into a vast and diverse field with various approaches and elements, there is still a noticeable gap in the literature concerning its fundamental dimensions. This information is crucial for comprehensive evaluation. To address this need, Tahir and Wang [8] conducted a directed content analysis and developed a hierarchical framework known as LEAGUE. This framework encompasses six core dimensions: Learning, Environment, Affective-Cognitive Reactions, Game Factors, Usability, and User. LEAGUE serves as a comprehensive and practical guide for designers, researchers, and developers in the educational gaming industry. Furthermore, Flynn, et al. [9], delved into the impact of fundamental learning theories on the design of computer and video games. Initiating their investigation with a comprehensive analysis of prominent learning theories, including learning paradigms and hierarchical learning theory, they posited that a correlation exists between these theories and the framework of video game design. The pattern in their findings reveals a significant connection between the evolution of video game design over the past approximately twenty-five years and the inherently hierarchical nature of learning theories. Consequently, the need arises for educational games to be categorized based on their adherence to specific criteria, thus clarifying their appropriateness for various types or areas of learning when they are employed for educational purposes.

1.2. Reflective Learning in GBL

Designers and educators strive to optimise GBL by integrating learning and pedagogical theories. A notable theory that has garnered acclaim in enhancing education is reflective learning theory. This approach involves students reflecting on their prior experiences to augment their learning and skill development. John Dewey introduced the idea of reflection as part of the learning process and discussed its effects on skill development [10]; further, he explained reflection as an activity that includes revisiting and reassessing previous beliefs intentionally and proactively. As reflective learning is a useful form of learning, Sengers, et al. [11] were the first to present the term “Reflective Design” in the context of technology. Further, they presented generic reflective design principles and approaches that technology designers use to rethink designing metaphors and values to encourage users to engage in the same practice more skilfully. Thus, reflective design practices assist both designers and learners in a continuing reflection on the development of technology in terms of its relationship to personal life experiences.

Over the past decade, it has been observed that games offer experiential learning and are reflective in nature, making them the ideal medium through which to engage students in reflective learning [12]. This characteristic makes games a powerful tool to help us learn through reflection. Over the last few years, reflection has become an essential part of interaction design in human-centred design, i.e., reflection within the technology [11] and reflective game design [13]. In addition, according to Khaled [13], “Games are reflection machines” because they naturally encourage us to think. Reflective game design offers new prospects for research, i.e., exploring how educational games are a suitable tool to facilitate reflection and how they can significantly influence users’ learning outcomes and behavioural changes [13]. Moreover, digital games possess distinct traits such as freedom [14] and elements of fantasy and curiosity [15], which can help promote stealth reflective learning. Furthermore, digital games encourage interactive learning, overcome disengagement, and promote authentic practice [16]. Therefore, serious games are a suitable mechanism for triggering and supporting reflection [13]. Khaled [13] proposed a list of principles that emphasise qualities of reflective design and discourage certain conventionally accepted design elements. In a survey conducted by Shaheen and Fotaris [12], 86.5% of the responses supported the concept of reflection through digital games. According to the survey responses, Heads-Up Displays (HUDs), such as on-screen scores, maps displaying player location, and progress bars received high recognition as practical game elements that facilitate reflection and reflective learning. Social collaboration also garnered significant attention, with positive responses from 60.7% of respondents, highlighting that multiplayer games are the prevailing platform for social discourse. Furthermore, during the semi-structured-interview phase of the same study, one of the participants stated, “I think the best practice which compels player for reflective behaviour is giving him loss/punishment over his mistakes. Just the way it is in real life. Humans learn from their mistakes. Mistakes that gave them a loss”.

1.3. Research Gaps

While games inherently possess qualities that promote reflection, only a few of the previously mentioned digital GBL frameworks employ active-reflection methods or practices to effectively engage players in the learning process. Sengers, et al. [11] offered broad principles and guidelines for reflective design with the aim of assisting technology designers in re-evaluating their design strategies and creating designs that hold significance for users. It is important to note, however, that these guidelines were directed more toward general technology applications rather than being specifically tailored to the nuances of digital game design. A literature review reveals that only Kiili (2005) incorporated reflective feedback into their Experiential Gaming Model, even though her primary emphasis was on utilizing flow theory to construct a GBL framework. Furthermore, the development tool known as ctrl + R has been devised to stimulate the development of fresh and reflective ideas among game developers by presenting them with eight random questions on digital “cards”. This tool functions by posing random questions to designers, potentially facilitating the creation of games that foster reflective thinking. Nonetheless, the extensive potential of integrating reflective learning into GBL has not been fully explored.

The remainder of this paper is structured as follows: Section 2 presents the overview of the methodology. Section 3 is based on research Phase 1, i.e., reflective design principles and the RGD framework. Section 4 focuses on Phase 2, which is the design of a GBL based on the RGD framework. Section 5 provides an in-depth description of Phase 3, including the results obtained. Finally, Section 6 discusses the results of this research and highlights potential directions for future research.

2. Methodology

2.1. Research Objectives

This research aims to assess the effectiveness of reflective design in terms of its usability, reliability, immersion, engagement, and appropriateness within the context of game-based learning, while also examining how these ideas are received by learners. Therefore, in order to achieve this aim, this ongoing research advises the following objectives:

• To integrate reflective learning into game design and create a Reflective Game Design (RGD) framework.
• To design a digital GBL based on the proposed RGD framework using principles of participatory design.
• To evaluate a functional prototype to assess the feasibility and applicability of reflective game design within the context of digital GBL.

Hence, the primary goal of this research is to incorporate reflective learning principles into digital educational digital games (GBL) and demonstrate their potential to yield substantial benefits.

2.2. Study Design

This study aimed to assess the efficacy of designing a reflective game by involving the target audience in an iterative process. Therefore, this study falls under the umbrella of design research; its process activities are adopted from an educational research design proposed by Van den Akker, et al. [17] and Reeves [18]. Reeves [18] presented the design research approach from a technology-focused perspective. The primary advantages of a design research approach as follows:

• The identification and analysis of the research problem.
• The creation of a prototype based on existing design principles.
• The evaluation and refinement of the prototypes and the design principles until satisfactory outcomes are reached.

Figure 2 represents the illustration of the design research approach used in this study. In this design research approach, researchers, and practitioners, i.e., participants, work in collaboration to identify the learning solution, develop a design based on existing principles, and then iteratively refine it by testing until satisfactory design outcomes have been reached.

According to Reeves [18], design research is not an activity a researcher can conduct in isolation from practice. Therefore, collaboration with practitioners is required to achieve satisfactory and practical solutions. The following three phases have been adapted from Reeves [18] to carried out the research with the collaboration of the target audience:

• Phase 1 involves delving into the problem space and defining the research problem through collaboration among researchers, user participants, and practitioners.
• In Phase 2, low-fidelity prototypes were developed during co-design sessions with the intention of submitting them to an iterative refinement process.
• In Phase 3, users’ feedback was incorporated as designers developed high-fidelity prototypes, culminating in design evaluation conducted through a comparative study.

In order to fulfil the goals of this design-based research, a blend of quantitative and qualitative methodologies (mixed methods [19]) was employed. The incorporation of both qualitative and quantitative data enhances the evaluation by balancing the shortcomings of one type of data with the strengths of the other. Consequently, the pragmatic paradigm [20] is the most appropriate research framework for designing and conducting this study. Furthermore, this research is grounded in the “Human-Centered Design approach” [21], which underscores the integral involvement of users in iterative design. Users play a crucial role throughout all phases of design, development, evaluation, and design refinement. As articulated by Reeves [18], design research is not an isolated endeavour, but requires collaboration with practitioners to achieve practical and effective solutions.

The study was conducted in accordance with the Declaration of Cross-School Research Ethics Committee C by the Ethics Committee of the University of Brighton (Ref:2022-9375).

2.3. Participants’ Involvement in Research

Considering the widespread popularity of digital games, particularly among young adults and children, this research explores the advantages of integrating reflective learning into GBL by examining the perspectives of individuals aged 18–24 years old on reflective game design (RGD). The inclusion criteria for participants were a background as professional gamers or any gaming experience. According to Prensky [22], it is likely that in no previous era has technology been so readily accessible at such a young age. Young adults have become familiar with interacting and communicating in a connected world at all times [23]. Young adults have more gaming experience than older adults; therefore, this study uses their addiction to and experience with digital devices and social media to promote their well-being.

2.4. Data Collection and Analysis Methods

The data collection in this study strictly adhered to rigorous anonymisation protocols. Personal or confidential inquiries were carefully avoided during interviews and group sessions. Participants were furnished with a consent form that included an information sheet, and they were required to read and sign it, demonstrating their clear understanding of the voluntary nature of their involvement. In each phase of the study, data collection was planned carefully, with a strong focus on ethical considerations. Ethical approval was sought and obtained from the Brighton Research Ethics Application Manager (BREAM) to ensure that human subjects were handled appropriately and in accordance with ethical standards. In the preliminary research phase focused on users, participants with gaming experience were purposefully chosen to maintain a specific demographic focus and avoid unnecessary diversity. The recruitment survey was thoughtfully designed, incorporating a screening question to categorise participants based on their interest in gaming. Data were gathered through user surveys, focus-group discussions, semi-structured interviews, and co-design sessions at different phases of the study. The recruitment process followed a sequential approach, beginning with the administration of a survey. Upon survey completion, participants were given the opportunity to opt in to semi-structured interviews, focus groups and co-design sessions.

To analyse the data, both quantitative and qualitative methods were used. The user survey was analysed using descriptive statistics such as means, variance, and standard deviations. For the comparative study, skewness and kurtosis values were considered, revealing that the survey data did not follow a normal distribution. Therefore, the Wilcoxon signed rank-sum test [24] was employed to obtain reliable comparison data. All interview discussions were transcribed for further analysis using the qualitative analysis tool NVivo software v1.7. Initially, transcriptions were coded by structure, allowing interviews to be conducted until “category saturation” was achieved [25]. Hence, data collection ceased when analysis stopped resulting in the development of new codes [26].

3. Phase 1—Reflective Game Design (RGD) Framework

In order to gather foundational data, a systematic review of the use of reflective design features in GBL [27] was carried out. Findings were divided into the following two categories:

3.1. Reflective Learning Principles

Bolton and Delderfield [28] briefly explained that trust is foundational in reflective learning, fostering confidence in our practice and reflective abilities. This trust allows critical engagement with our experiences, encompassing examination, questioning, exploration, and experimentation. Sengers, et al. [11] also provided general guidelines to encourage designers to incorporate reflective practices into the process of designing technology. Moreover, reflective learning places a strong emphasis on self-respect, valuing our beliefs, actions, feelings, values, and identities, promoting self-kindness, and acknowledging the validity of our experiences. It also involves reflexivity and self-reflection. Practitioners are encouraged to generously invest time, energy, and commitment in personal and professional development, both inward and outward. Genuine reflection promotes authenticity and sincerity in our developmental efforts. Additionally, positive regard and empathy are pivotal, fostering non-judgmental acceptance of ourselves and others and facilitating deeper connections through understanding of perspectives and emotions.

Based on the values mentioned above, the reflective learning principles in game design encompass the following:

• Emotional Connections: Create opportunities for players to connect with their emotions by providing chances for them to reflect on their in-game experiences and emotional responses. This aim can be achieved through mechanisms like perspective/reflexivity and journaling.
• Narrative: Encourage players to share their experiences through narratives, allowing them to relate their personal stories or contrast them with those told within the game.
• Mindfulness: Integrate mindfulness practices, such as meditation and visualisation, into the game to prompt players to focus on the present moment and their inner thoughts and feelings.
• Reflection: Offer players opportunities to reflect on their experiences, thoughts, and emotions through activities like self-reflection, journaling, or other reflective practices.
• Emotional Intelligence: Promote the development of emotional intelligence among players by educating them about emotions, empathy, and emotional regulation.
• Empathy: Create in-game opportunities for players to understand and connect with the emotions of others through interactions, empathy exercises, and related activities.
• Personal Growth: Foster personal growth in players by encouraging them to identify and pursue personal goals, such as enhancing self-awareness, emotional intelligence, and emotional regulation.
• Player-Driven Content: Empower players to generate and share content, such as personal stories, emotional reflections, and artwork, to stimulate reflection on their in-game experiences and emotional responses.

These principles collectively create a framework for effective reflective learning. By embracing trust, self-respect, responsibility, generosity, genuineness, and empathy, individuals can engage in meaningful self-exploration and personal growth while also developing a deeper understanding of others and their experiences.

3.2. Reflective Game Elements

The findings of the systematic review [27] also revealed that games offer experiential learning and are reflective in nature, making them the ideal medium into which to integrate reflective learning. The research findings also revealed existing game elements that are reflective in nature.

Figure 3. Examples of reflective game elements, (a) Process Displays, i.e., oxygen level, CO₂ level etc., [29], (b) Process Prompts showing useful information [29], (c) Process Model example showing master solution and common errors for comparison [30].(d) Multiplayer game with audio communication among players [31].

Figure 3. Examples of reflective game elements, (a) Process Displays, i.e., oxygen level, CO₂ level etc., [29], (b) Process Prompts showing useful information [29], (c) Process Model example showing master solution and common errors for comparison [30].(d) Multiplayer game with audio communication among players [31].

Generally, game features can be categorized into the two types of reflection outlined by Schon [32]: “reflection in action” and “reflection on action”. For instance, “reflection in action” occurs while a player’s actions are unfolding during gameplay in a GBL context. On the other hand, “reflection on action” is a form of post-action reflection that takes place after an activity has been completed. It allows the learner to comprehensively reflect on their actions, identify mistakes, and consider feedback. Post-activity reflection significantly influences a user’s overall learning process, contributes to skill improvement, and enhances learning experiences. In addition, Lin, et al. [33] identified four Reflective Design Features:

• Process Displays: These visual aids make implicit learning processes explicit, allowing learners to assess their progress and reflect on ongoing actions (see Figure 3a).
• Reflective Prompts: These visual aids integrated into the user interface, these prompts explain and evaluate learners’ actions before, during, and after problem-solving (see Figure 3b). They encourage learners to articulate solutions and offer direct feedback.
• Process Models: These benchmarks present standardized process steps, enabling learners to compare their progress to their goals and relate to their own progress. For example, they can set learning objectives within simulations (see Figure 3c).
• Social Discourse: This collaborative space within the game enables learners to explore and reflect on various perspectives. It facilitates feedback on problem-solving and learning steps (see Figure 3d), enhancing the reflective learning experience.

3.3. Reflective Game Design (RGD) Framework

The findings were used to formulate the Reflective Game Design (RGD) framework [34]. This framework is modelled after Kolb’s learning cycle [35] and maps game-design activities onto experiential learning theory, as depicted in Figure 4. The RGD framework provides a mechanism through which to incorporate reflective learning into GBL approaches. The aim is to provide learners with reflective feedback at each stage of the RGD framework.

The RGD framework is an iterative model, in which each iteration improves learning with reflective observation and authentic feedback. Reflective feedback is a central part of this framework, as shown in Figure 4b. Thus, this framework embeds reflective learning elements into each stage with experiential learning methods. Reflective learning principles and game-reflective features were considered fundamental building blocks within each stage. Each stage is briefly explained as:

• Gameplay: The first stage is “Gameplay”, which enables learners to interact with a game through game elements (game rules, game mechanics, challenges, stories, feedback, levels, etc.) to give them authentic practice and concrete experience of a learning objective. A typical example of gameplay can be seen in Figure 4.
• Formative Assessment: It ensures learning and critical reflection through ongoing reflective feedback and encouraging players to examine their skills in gameplay. The link between abstract conceptualisation from Kolb’s learning cycle [35] and reflective assessment lies in the cognitive process of translating theoretical knowledge or abstract concepts into practical understanding and application. Abstract conceptualization involves grasping complex ideas or theories, while reflective assessment entails critically examining and evaluating one’s experiences and understanding. In essence, individuals engage in abstract conceptualization to form a theoretical foundation, and reflective assessment serves as a means to introspectively analyse how these abstract concepts manifest in real-world scenarios, fostering a deeper and more nuanced comprehension of the subject matter.
• Level Up: This stage involves active experimentation with a learned skill in a new level or situation. Level design is an important aspect of game development, and as players progress through each level by applying skills they have acquired, the process enhances their genuine learning and practical experience.
• Reflective Feedback: This stage is an integral part of the RGD framework. Games can provide both reflection in action and reflection on action. For example, reflective feedback is used to highlight the player’s code, line by line, visualise the result of the player’s code during playback (Figure 5), and visualise the student’s actions to help them reflect on and examine their process. The result is improvement in learning skills.

3.4. Reflective Game Design Idea Reception

As digital games are popular among people of all ages, particularly young adults and children, user research was designed to explore the advantages of explicitly incorporating reflective learning into digital game-based learning (GBL) by investigating young adults’ perceptions of reflective game design (RGD). User research based on the user survey (N = 101) and semi-structured interview (N = 15) conducted by Shaheen and Fotaris [12] showed that 86.5% of participants accepted the idea of reflective game design, indicating a high level of interest in this approach to learning. In the second phase, fifteen in-depth interviews were conducted to explore further participants’ perceptions and acceptance of reflection in games. The thematic analysis of the interviews revealed common trends in regard to using reflective game practices to design a new GBL approach. Participants noted that reflective game design can foster deeper learning, promote problem-solving skills, and enhance motivation to learn. Furthermore, the findings suggested that digital games implicitly promote reflective learning by encouraging critical thinking, self-awareness, problem-solving skills, and motivation. Additionally, reflective learning provides immediate feedback to students, promoting self-directed learning. Allowing students to reflect on their gaming experiences can make digital games more immersive, leading to deeper learning.

4. Phase 2—GBL Using the RGD Framework

The objectives of this phase were to use the RGD framework to design and develop a GBL. In this study, a predetermined GBL concept centred around “Increasing Self Awareness in Young Adults” was employed.

During the initial phase, content matter experts (CME) carefully developed learning objectives (LO), which were subsequently used as a foundation for designing corresponding instructions. The next step was to involve the target audience at each phase of the research. Game design was conducted through co-design sessions with participants. This study opted for an online co-design session (OCD) for the participatory research design due to the geographic distribution of participants [37]. Because the participants could be reached remotely, the researchers primarily leveraged the Zoom and WhatsApp platforms to conduct the first Co-Design session. The structure of the first co-design session was adapted from Agbo, et al. [37] approach and modified for this study.

The content of the co-design session was extracted from the primary research [12], i.e., user surveys, focus-group discussion and interview data, and secondary research, i.e., the literature review. Co-design sessions were held iteratively to allow the game to be tested and refined by user testing. A visual representation of the process is shown in Figure 6.

In total, six participants who were short-listed based on the primary research took part in the OCD sessions, and each session lasted 1–2 h. Afterwards, the meeting continued with all participants in groups on the WhatsApp platform. The WhatsApp groups were created to facilitate collaboration during co-design activities and to allow participants to discuss designs and ideas. Additionally, to ensure effective collaboration on the assigned tasks, the researcher regularly used the participants’ messages and content to gain insight into how to motivate them.

All conversations and design decisions were recorded via Google Docs and Google Jam-boards in the shared Google Drive. In addition, several participants made sketches to show their ideas, some of which are shown in Figure 7. Two sessions were held each week, and this schedule was maintained for two months.

4.1. Design Process

As outlined in the Reflective Game Design (RGD) framework (Figure 4b) the design of each game activity consists of four distinct phases:

• Gameplay with Narrative and Game Mechanics: This stage involved crafting the gameplay experience, integrating the narrative, and incorporating the intended learning objectives (LO).
• Formative Assessment Design: In this stage, formative-assessment game activities were designed to evaluate whether the LO had been attained.
• Utilizing LO in Subsequent Levels: The LO achieved in the previous phases were applied in the design of subsequent game levels to ensure authentic practice.
• Incorporating Reflective Feedback: Reflective design features and feedback were added to the end of all developed game activities and assessments to enhance the overall learning experience.

The iterative design process offers a distinct advantage during the third stage, wherein a fully developed activity can be produced. Subsequently, the addition of reflective feedback to the following stage makes it possible to generate another design prototype, allowing for continued refinement and enhancement of the activity. This iterative approach enables the creation of multiple design iterations, each building upon the previous one. Hence, upon the conclusion of the design process, two distinct prototypes were developed: (1) A game design without reflective feedback (non-RGD), and (2) A game design with reflective feedback (RGD). These prototypes serve as two distinct versions of the game design, one with integrated reflective feedback and the other without it, allowing for comparative assessment and evaluation.

As depicted in Figure 7, the co-design sessions followed an iterative approach. At the end of each design activity, a low-fidelity prototype such as a paper prototype was created to evaluate the design with the involvement of the target audience. Subsequently, any insights and feedback gathered during this testing phase were integrated into the design, leading to refinements and improvements in the overall design.

4.2. Game Design Decisions

After the co-design sessions, the decisions regarding game design were finalised, prioritising achievement across four fundamental self-awareness domains: private self-awareness, public self-awareness [38], goal-oriented self-awareness, and emotional intelligence. The chosen game genres included casual, adventure, and puzzle.

The game narrative was based on a home renovation that starts from the outside of the house. A player character receives social pressure in the form of questions (illustration is shown in Figure 9a), which then turn into daemons and chase the player. The daemons represent emotions and are absorbed into the character. The player character enters the house, and when the character steps into the living room, daemons representing the family’s emotions (not feeling good enough, fear of failure, low confidence) attack them and are absorbed, making them feel more overwhelmed (depicted as black). The terrified character starts running, and upon entering the bedroom, they see themselves in the mirror. At that point, daemons (anxiety, low self-esteem, self-doubt) emerge from the character and destroy the house. Terrified, the player character rushes downstairs to enter a basement, where they find themselves surrounded by the daemons. On the verge of collapse due to terror, fear, and anxiety, the player character attempts to hide in a corner. The daemons are destroying everything around them, and dust covers the environment. At that time, they hear a soothing and reassuring voice, personifying consciousness in the form of the guardian angel. The guardian angel informs the player that they must solve puzzles based on clues distributed in the house to recognise themselves and thus fight the daemons to save their house.

4.3. Iterative Low-Fidelity Prototype Testing

The objective in creating the iterative low-fidelity prototype was to offer GBL to the participants and gather their feedback within the framework of participatory design. In order to meet objectives, specific goals for the low-fidelity prototype were established. These included the following: (1) Assessing the initial reception of the game concept, which was presented as a “game to increase self-awareness using reflective design”; (2) Gathering feedback regarding the game’s storyline and activities; (3) Evaluating the participants’ level of engagement and immersion within the game; and (4) Seeking feedback on the aspects of reflective game design.

A paper prototype was created based the initial game concept, narrative, two interactive activities, and formative assessment. Details of the game activities are presented in

Table 1. Game activities used in prototype development.

#	Name	Detail	Reflective Feedback
1	Mindful Activity (Illustration is shown in Figure 9e)	Guided 4-7-8 breathing exercises that entail breathing in for 4 s, holding the breath for 7 s, and exhaling for 8 s. This activity has four rounds, and the player will be able to skip the activity. A diary will be provided as an achievement to complete this activity. The diary will serve to log to players’ activities and track their progress throughout the game.	In-action reflection: The activity will start with information on the 4-7-8 breathing exercise and how it helps players calm down. Audio-guided instructions and visual feedback on the countdown and other related information. Selection of the player’s feelings before and after the breathing exercise. On-action reflection: The player will be asked to take a minute to think about the effect of breathing on changing one’s feelings. The players’ feelings will be recorded in a diary (a personal journal, as illustrated in Figure 9d).
2	Interactive Activity (Illustration is shown in Figure 9f)	Word-search puzzle activity involving the selection of three emotions that a player will have encountered in the story and that the player can relate to themselves.	In-action reflection: Hints will be given when the player clicks on a word. Visual feedback will appear when the player finds a word, giving the player more information about the emotion they found. The player will be able to get help from the emotion wheel in the diary. On-action reflection: The player will be asked whether they can relate to or reflect on these emotions in their daily life. The player’s responses will be recorded in the diary.
3	Formative assessment (Illustration is shown in Figure 9c)	Pick three feelings from the wheel of emotions. This formative assessment appears at several stages of gameplay.	In-action reflection: Visual feedback will be provided on the selection of a feeling. On-action reflection: The player’s responses will be recorded in the diary for further reflection on their emotional state.

.

4.4. Phase 2—Data Collection

Data collection involved two primary methods: observational sheets and conducting semi-structured post-interviews. In the initial round of paper prototyping, 25 participants were actively engaged, while 15 participants tested the paper prototype. Figure 8 showcases a selection of these prototype designs, offering a visual depiction of the testing process and its results.

Our comprehensive approach data collection involved semi-structured interviews to obtain detailed and in-depth insights. This approach involved observations, post-play semi-structured interviews, and focused group discussions. The observation and scripts for the semi-structured interviews were designed around four main goals: (1) evaluating engagement with the prototype, (2) gauging understanding of game objectives, (3) capturing general feedback, and (4) gathering suggestions for improvement. The interviews were carefully conducted, with a duration ranging between 10 and 15 min. The interview process occurred in an atmosphere of friendly discussion, with the aim of establishing a comfortable environment that encouraged participants to openly share their perspectives and experiences.

4.5. Phase 2—Results

During interviews and focus group discussions, participants expressed satisfaction and enthusiasm, actively sharing their ideas for improving the game. The interview transcripts were initially subjected to structure coding, following the methodology proposed by Braun and Clarke [39], using NVivo software v1.7. A thorough reading of the transcripts facilitated the identification of essential sections of text that captured the qualitative richness of the phenomenon under investigation, aligning them with relevant themes and issues in the data and research questions [40]. Memos were diligently recorded to highlight intriguing aspects of the data and emerging impressions that had the potential to form the basis of thematic patterns across the dataset. In order to derive the qualitative inquiries and patterns from the interviews, three types of coding were used [40]: (1) descriptive coding, (2) topic coding, and (3) analytical coding. Descriptive coding was used for quantitative data about participants’ daily gaming habits. Topic coding was used for the findings regarding prototype testing. In total, two low-fidelity prototypes underwent testing in the initial iteration, involving a sample size of N = 25. The outcomes were systematically recorded and subsequently employed in the development of the second iteration, which underwent evaluation with N = 15 participants. The corresponding user responses have been documented in

Table 2. Responses to testing of the low-fidelity prototype.

Iteration	# of Participants	Response
1	25	Positive responses: The first iteration received generally positive feedback. Participants seemed happy and accepted the idea of the game. They were engaged with the story and game flow. Constructive response: Participant provided constructive responses. Examples include the following: There should not be too many HUDs.∙ There should not be a lot of text or narrative text on the screen. A few participants were unhappy about the leader board because the game is centred around self-care, so players should not be compared with others. The colours should become brighter as the game progresses. There should be something at the end of the game, such as an online blog with updates on self-care or self-awareness, A few players criticise the HUD health bars and metres on the grounds that they are confusing and they should not turn red when the player character is anxious.
2	15	Positive responses: Players generally characterised the second iteration as easy to follow and less complicated and reported that the activities were engaging. Constructive response: Participants were concerned about the animations in the story. When they engaged with the paper prototype, they imagined and thought out loud about the story animation, and their ideas were recorded for future implementation. Participants wanted to save their journal or diary and so suggested making a downloadable pdf of their diary and activities.

.

Following qualitative analysis, the topic codes were generated and are presented in

Table 3. User responses from the post-gameplay focus-group discussion and semi-structured interviews (Total N = 40 (1st iteration N = 25, and 2nd iteration N = 15)).

Codes	Response
Idea Reception and Gameplay	All participants reacted positively to the game idea and were immersed in the game’s story. They were delighted to encounter a game that explored self-awareness. Some of the responses included: “Innovative—I haven’t come across a game like this before”, and “this game is just the start of the journey towards self-care”. In terms of gameplay, approximately 85% of the participants found it easy to follow, clear, and infused with elements of curiosity. They thoroughly enjoyed playing with the prototype.
Reflective Design	At the outset, participants were unfamiliar with reflective designs. They were first prompted with reflective activities, such as “think about your breathing exercise” and were subsequently asked to explain the differences between simply carrying out the activity and engaging in thoughtful reflection afterwards. They noted a significant distinction between just performing an activity and contemplating the activity they had performed. Approximately 60% of the participants requested the opportunity to perform the activity again because they had ideas for improvement following their reflections. This outcome was a positive indicator of progress in reflective design.
Immersion, Attention, Engagement	Participants displayed signs of immersion, attention, and engagement during gameplay. They showed an emotional connection to the story and characters, looked curious, and expressed interest in the game.
Suggestions for improvements	The participants expressed their delight with the concept and were incredibly eager to provide feedback and suggestions for enhancement. They discussed the inclusion of additional story elements, such as more “daemons” representing social pressures such as “study pressure”, “body shaming”, and “social media pressure”. Other suggestions were: “There should be no end of this game, once the game would finish then there should be an online blog called “Wall of Kindness”, where people share their experiences of self-care”. “There should not be a voice-over, because it would distract from immersion”. “Reflective diary (a game element), should be able to download at the end of the game”.

, which displays both the extracted codes and participants’ responses.

5. Phase 3—Efficacy of the RGD Framework

The primary aim of this research is to assess the effectiveness of the RGD framework. As described in the previous section, the GBL design followed an iterative process, resulting in two prototypes: one without reflective design, referred to as non-RGD, and the other with reflective design, known as RGD. To determine the efficacy of the RGD prototype and identify the more effective design, user research was conducted with the aim of collecting data on the following parameters:

• Understanding of the content and objectives.
• Clarity and enjoyment of gameplay.
• Relevance of content to the game.
• Emotional engagement and immersion.

Both the non-RGD and RGD prototypes were implemented as functional prototypes. These implementations were carried out using the Unity Game engine, and the prototypes were deployed on a webGL website to facilitate distribution for evaluation purposes. An illustration of the functional prototype is shown in Figure 9.

To assess the prototypes and gauge the effectiveness of reflective design, a comparative evaluation approach was employed. In this evaluation, participants were tasked with playing both prototypes, allowing for a direct comparison of their experiences and effectiveness between games. This study design allowed for direct comparison within the same group of individuals. In order to avoid bias, participants interacted with RGD designs before they interacted with the non-RGD designs. Both functional prototypes ended with a short survey comprising 10 quantitative items with an optional open-ended writing field to express the reason for their choice.

5.1. Phase 3 Data Collection

The web-based high-fidelity prototypes consisted of a game narrative and three game activities (detail of which are presented in Table 1), followed by a short survey. The survey questionnaire encompassed questions pertaining to comprehension of game objectives, gameplay enjoyment, game visuals, reflective feedback, and suggestions for improvement. Additionally, each question included an optional open-ended field, providing participants the opportunity to elaborate on their thoughts. During the initial distribution of the survey, a total of 56 responses were obtained for each prototype. These 56 responses for each prototype, 122 responses in total, were believed to be suitable for further analysis. The analysis of the high-fidelity prototype can be divided into two distinct parts for systematic examination and interpretation.

5.2. Phase 3—Results

The most crucial aspect of the result analysis involved comparing the findings from two surveys: the survey responses following playtesting of the RGD prototype and those from playtesting of the non-RGD prototype. A comparative analysis aimed to illuminate key differences and similarities in participant feedback. Two types of analysis were performed, with the first being descriptive statistical analyses (see

Table 4. Comparison of scores from the online survey (non-RGD N = 56, RGD N = 56).

#	Topic	Likert Scale. 1 … 5	Non-RGD			RGD
			Mean	Var.	Dev.	Mean	Var.	Dev.
1	Understanding of the Content and Objectives		3.38	0.7	0.84	4.04	0.76	0.87
2	Enjoyment of Gameplay		2.77	0.72	0.85	4.12	0.75	0.86
3	Emotional Engagement and Immersion		3.54	0.63	0.8	4.12	0.70	0.86
4	Visuals of gameplay		3.73	0.27	0.52	3.92	0.71	0.84
5	Confusion in the Game		2.54	0.94	0.97	4.16	0.93	0.97

) such as means, variance, and standard deviations. The analysis revealed that the survey data did not follow a normal distribution based on skewness and kurtosis values, leading to the employment of the Wilcoxon signed rank-sum test [24] to obtain reliable comparison data. The results of the Wilcoxon signed rank-sum with an alpha value of 0.05 and critical values of −1.95996 (lower) and 1.95996 (upper), are presented in

Table 5. The paired Wilcoxon signed-rank test with N = 56, alpha value of 0.05, and critical values of −1.95996 (lower) and 1.95996 (upper).

Topic	Z	Test Statistics (W, (W−, W+))	p-Value
Understanding of the Content and Objectives	−3.4705	57, (57, 378)	0.0004068
Enjoyment of Gameplay	6.0255	28, (28, 1247)	1.686 × 10−9
Emotional Engagement and Immersion	3.2275	73, (73, 362)	0.001249
Visuals of Gameplay	1.8596	89, (89, 211)	0.06295
Confusion in the Game	5.7933	11, (11, 1024)	6.901 × 10−9

. Data analysis yielded the following results:

5.2.1. Understanding of the Content and Objectives

Participants generally demonstrated good understanding of the content and objectives of the game prototype. The difference (see Table 4) in mean ranks between gameplay for the non-RGD (mean rank = 3.38, N = 56) and RGD (mean rank = 4.04, N = 56) prototypes indicated that most participants found the latter to be clearer and easier to comprehend. This finding suggests that reflective feedback provides more clarity and understanding regarding the game’s learning objectives. Furthermore, some participants mentioned moments of confusion, particularly regarding the last step involving choosing three emotions in the word-search activity (see activity detail in Table 1), which has been duly noted to inform further improvements.

For the comparative analysis, the null hypothesis (H_o) was that there is no difference between gameplay experiences with the non-RGD and RGD prototypes. The results of the Wilcoxon signed-rank test (refer to Table 5) revealed a significant and substantial difference between the non-RGD game prototype (median= 3.5, N = 56) and the RGD game prototype (median = 4, N = 56), with a Wilcoxon Z of 3.5, p-value = 0.0004068, α = 0.05, and an effect size (r) of 0.7. As the p-value < α, the null hypothesis (H_o) was rejected. These findings, with a positive Z value, indicated that values in the RGD prototype tended to be greater than those in the non-RGD prototype, suggesting that the RGD prototype received more positive responses.

5.2.2. Enjoyment of Gameplay

Based on the mean rank difference between the prototypes (see Table 4), wirg the non-RGD prototype (mean rank = 2.77, N = 56) receiving lower scores than the RGD prototype (mean rank = 4.12, N = 56), it is evident that a majority of participants enjoyed playing the RGD prototype. Feedback emphasized that participants found elements such as the breathing exercise (Table 1, Figure 9e), narrative (Figure 9a), and emotional engagement activities engaging and enjoyable.

In comparing degrees of enjoyment of gameplay, the null hypothesis (H_o) was that there is no difference in enjoyment level between experiences of the non-RGD and RGD game prototypes. The results of the Wilcoxon signed-rank test (refer to Table 5) revealed a significant and substantial difference between the non-RGD game prototype (median = 3, N = 56) and the RGD game prototype (median = 4, N = 56), with a Wilcoxon Z of 6, p-value =1.686 × 10⁻⁹, α = 0.05, and an effect size (r) of 0.9. Thus, the null hypothesis (H_o) was rejected, and the positive Z value indicates that RGD gameplay offers a more enjoyable experience than non-RGD gameplay.

5.2.3. Emotional Engagement and Immersion

Responses regarding emotional engagement were mixed, with some participants mentioning emotionally engaging moments and others providing brief responses. Several aspects contributed to emotional engagement, including the storyline, breathing exercises, the “finding emotions” activity, and the impact of certain narrative elements. A few participants mentioned that engaging aspects included the breathing exercises and the “finding emotions” activity. A discrepancy in the mean rank difference between the two prototypes (see Table 4), with non-RGD gameplay (mean rank = 3.54, N = 56) scoring below RGD gameplay (mean rank = 4.12, N = 56), indicates that the reflective feedback in RGD helped participants to immerse and emotionally engage with the game.

The null hypothesis (H_o) in this case posited that there is no difference in participants’ “Emotional Engagement and Immersion” levels between the non-RGD and RGD prototypes. The results of the Wilcoxon signed-rank test (refer to Table 5) indicated a significant and substantial difference between the non-RGD game prototype (median = 4, N = 56) and the RGD game prototype (median = 4, N = 56), with a Wilcoxon Z of 3.2, p-value = 0.001, α = 0.05, and an effect size (r) of 0.6. These findings suggest that values in the RGD prototype tended to be greater than those in the non-RGD prototype. Consequently, the null hypothesis (H_o) was rejected, and the positive Z value indicates that participants found RGD gameplay to be more emotionally engaging and immersive compared to non-RGD gameplay.

5.2.4. Visuals

The visual aspects received positive feedback. However, the small size of the difference (see Table 4) in mean ranks between non-RGD (mean rank = 3.73, N = 56) and RGD (mean rank = 3.92, N = 56) gameplay suggests that the reflective feedback did not have much effect on the visual quality. Participants generally rated the visuals positively. Some participants provided suggestions for enhancing visuals, such as incorporating more playful graphics and infusing genuine emotions. One participant mentioned that it was a good representation of emotional chaos.

The null hypothesis (H_o) in this case was that participants will not perceive either prototype as visually more appealing. The Wilcoxon signed-rank test (refer to Table 5). showed a non-significant, moderate difference between the non-RGD game prototype (median = 4, N = 56) and the RGD game prototype (median = 4, N = 56), with a Wilcoxon Z of 1.9, p = 0.063, α = 0.05, and an effect size (r) of 0.4. As the p-value was greater than the chosen significance level (α), H_o was not rejected, indicating that reflective feedback did not have much effect on visual appeal.

5.2.5. Confusion in the Game

In regard to the RGD (mean rank = 4.16, N = 56) prototype (see Table 4), most participants reported that gameplay was “somewhat clear” or “very clear”. On the other hand, in the non-RGD (mean rank = 2.54, N = 56) prototype, many participants reported feeling somewhat confused during the game, especially concerning word finding and moments of uncertainty. Hence, the results clearly indicate that adding reflective feedback made the game clearer.

The null hypothesis (H_o) in this case is that participants will not perceive any difference between the prototypes in terms of the clarity of gameplay. Results from the Wilcoxon signed-rank test (refer to Table 5) illustrated a significant and substantial difference between the non-RGD game prototype (median = 2, N = 56) and the RGD game prototype (median = 4, N = 56), with a Wilcoxon Z of 5.8, p-value = 6.901 × 10⁻⁹, α = 0.05, and an effect size (r) of 0.9. This result indicates that values for the RGD tended to be greater than those for the non-RGD. As the p-value < α, H_o was rejected. The positive Z value provides clear evidence that adding reflective feedback made the gameplay clearer and less confusing.

5.2.6. Improvement Suggestions

The open-ended questions from the survey yielded valuable suggestions for improvement. The open-ended questions from the survey were organized into topic codes and are presented in

Table 6. User survey open-ended responses.

Topic	Responses
Game idea	“The game’s theme of battling inner demons resonates with real-life struggles, emphasizing how we often stumble and rise again. It could delve deeper into the relatable fear of confronting these inner demons, mirroring moments when we hide behind curtains in our daily lives”, “Anything which emphasises that Consciousness angel is actually yourself”.
Reflective elements	“Diary was a good part”, “there should be more related to diary like picture or doodle”, “I like thinking about my emotions and relate them to current situations”.
Game continuity	“Adding more activities to relax and divert the attention from the anxiety would be great”, “I want to see rest of the game”.

along with participants’ responses. These suggestions encompassed enhancing the visuals for the diary and breathing exercises, improving graphics, providing clearer instructions for the last step, and incorporating more activities. Furthermore, participants recommended enhancements in dialogue and graphics quality, along with the addition of a fitting soundtrack. Expressing a desire for more content, participants inquired about future developments. Additional suggestions involved incorporating more relaxing activities to divert attention from anxiety and ensuring that the user interface was intuitive and accessible.

In summary, the response from participants was generally positive. Understanding, enjoyment, and relevance to the game theme remain strong points, with some areas for improvement in visuals and clarity. Participants appreciated the element of feedback within the game and expressed a desire to see more content. As the survey consisted of open-ended questions wherein participants could share their experiences, a few responses are provided below:

6. Discussion and Future Work

This research aimed to assess the effectiveness of reflective design in terms of its usability, reliability, and appropriateness within the context of game-based learning, while also examining how these ideas are received by learners. The in-depth analysis of the user data yielded intriguing findings, underscoring the significant potential of games as a means of facilitating reflective learning. Overall, the “Reflective Game Design” GBL with a focus on “Increasing Self-Awareness in Young Adults” was positively received in terms of engagement. This positive response encourages further research to explore the potential of reflective game design for enhancing learning retention rates.

The successful completion of the first phase led to the assembly of reflective game elements and the establishment of the RGD framework. This framework serves as the foundation for integrating reflective design principles into the development of any GBL experience. The RGD framework provides a comprehensive perspective on integrating reflective learning through in-action and on-action reflection within in-game activities. Additionally, these findings could support game designers, educators, and other stakeholders in incorporating reflective learning through the utilization of reflective game elements. The second and third phases of the research entailed more elaborate and interactive engagement with the target audience. This stage necessitated close collaboration with the participants and involved an iterative approach, a notable departure from the prior phase.

The results of the second phase revealed that the target audience, i.e., young adults, could relate to the idea of games providing implicit reflection. They acknowledged having played games that triggered a gameplay experience in their brains, leading them to reconsider and improve their overall gaming experience. The semi-structured interviews and focus groups yielded substantial positive results, revealing the participants’ enthusiasm through their suggestions. These suggestions were incorporated into the third phase through the high-fidelity prototype. In the third phase, the iterative nature of the RGD framework resulted in the development of two prototypes: an non-RGD prototype and an RGD prototype. A comparison of survey responses revealed that the participants reported more positive engagement, immersion, and clarity of game objectives for the RGD prototype.

The outcomes of the third phase of the study aligned with the findings from the second phase, which involved user surveys, focus-group discussions, and interviews. These earlier phases had already showed that the design of the game, aimed at improving game design by adding reflective feedback, was reliable. The results of the third phase further confirmed this finding. According to the data, the reflective game design (RGD) effectively engaged participants, allowing them to become emotionally immersed in the experience. Moreover, participants, who belong to the target age group, were able to relate to the narrative, as was clearly reflected in their responses gathered through both RGD and the survey. This heightened engagement also translated into a deeper understanding of the game’s concepts and an increased enjoyment of gameplay.

Additionally, in terms of visuals, there was a minor difference between the prototypes, opening up new perspectives on how visuals can be improved through reflective feedback. All participants agreed that feedback in the game, including guidance through activities, enhanced their overall experience.

One limitation of this study lies in its exclusive focus on participants with gaming experience. Consequently, the findings may apply solely to individuals who identify as gamers. To address this limitation and broaden the scope of this research, researchers should considered undertaking a future study that incorporates participants without gaming experience, allowing for a more comprehensive exploration of the experience.

Finally, this research primarily focused on testing the effectiveness of RGD in terms of engagement, content clarity, understanding of instructions, the incorporation of reflective game elements, and player immersion. However, in the context of GBL, an essential aspect to consider is the learning rate. It is imperative that the learning experience goes beyond superficial or trivial understanding. In the future phases of this research, the GBL approach based on RGD will undergo further evaluation. Specifically, it will be subjected to rigorous testing against predefined learning objectives to assess its impact on learning retention. This assessment will provide a more comprehensive understanding of the game’s educational effectiveness and its ability to facilitate long-term knowledge retention among participants. By explicitly incorporating reflective learning into digital GBL, developers can make educational games that are more engaging, immersive, and effective in promoting critical thinking, STEM literacy, self-awareness, problem-solving skills, and motivation among students.