Game-Based Social Learning for Socially Sustainable Water Management

Abstract

An important aspect of the social sustainability of a proposed solution is acceptance by societal stakeholders. Acceptance is determined by the extent to which the solution matches with stakeholder perspectives on the problem and preferred ways to deal with it. Social learning can contribute to the social sustainability of water management strategies by achieving a convergence in perspectives among societal stakeholders. Serious games have proven to be effective in generating this type of social learning outcomes, but the underlying mechanisms are still unclear. This article aims to clarify how a multi-player serious game on river management (Sustainable Delta) supports social learning among participants with initially diverging perspectives. Based on a conceptual framework for game-based social learning, hypotheses and expectations were formulated and tested with quantitative and qualitative analyses of game sessions. Convergence of perspectives was observed in 10 out of 12 gaming sessions, but could not, or could only to a limited extent, be explained by the presumed learning support mechanisms in the game’s design. This underlines the importance of opening up the black box of serious games to determine how and why they work. If this is neglected, there is a clear risk that the design of games will be based on wrong, untested assumptions and will be less effective in supporting social learning and social sustainability.

1. Introduction

A solution is believed to be sustainable when it is environmentally sound, economically viable and socially acceptable [1]. Hence, an important aspect of the social sustainability of a proposed solution is acceptance by societal stakeholders. Acceptance is determined by the extent to which the solution matches with stakeholders’ perspectives on the nature of the problem, the preferred solutions and the responsibilities of the actors involved [2]. These perspectives are based on knowledge, values and interests, and may therefore differ considerably among stakeholders. In environmental and resource management, effective implementation of solutions often relies on broad acceptance and even active support by a wide range of societal stakeholders [3]. Therefore, natural resource management has a long-standing interest in stakeholder approaches that support the development of a common understanding of the problem and the acceptance of jointly identified solutions [4,5]. Social learning is seen as an important mechanism to achieve this convergence in problem perspectives [6,7]. A disadvantage, however, is that social learning is a time- and resource-intensive form of stakeholder engagement, which creates barriers to large-scale implementation [8,9]. Therefore, there is a need to make the social learning process more efficient, for example by using support tools [9,10,11].

A specific type of tool that could support social learning is serious gaming, which is receiving increasing attention in this respect [12]. Serious games are generally defined as games that have a primary purpose other than entertainment, such as educating or training the players [13]. In the context of socially sustainable management of natural resources, the focus is on ‘collaborative serious games’ in which sustainable management strategies are explored with stakeholders in a coordinated effort to solve the problem [14]. Collaborative serious games have proven to be effective in generating a wide range of social learning outcomes, and their use is growing [14]. Mostly, this happens as part of a larger stakeholder engagement process, labeled ‘game-based governance approaches’ [15] or ‘learning-based interventions’ [16]. Recently, calls have been made to shift research from the evaluation of whether games are effective in supporting social learning, to theory-based analysis of why and how these games work [12]. Based on a tested conceptualization and better understanding of the mechanisms at work, more effective games or game-based approaches may be designed [15].

In this article, we respond to these calls and focus on the learning support mechanisms in Sustainable Delta, a serious game on water management, for which we reported successful social learning, i.e., convergence of players’ perspectives, in a previous publication [17]. Interestingly, it is to date still one the few collaborative serious games for which this has been reported as a social learning outcome [14]. With the aim of better understanding how the game generates social learning outcomes, we tested theory-based hypotheses concerning the learning support mechanisms with quantitative and qualitative analyses of game sessions. In particular, two mechanisms potentially underlying the supporting role of the game are investigated: the ‘feedback function’, which confronts the players with the consequences of their choices, and the ‘platform function’, which allows the players to jointly reflect on their perspectives. In the following sections, we first present a conceptual framework for game-based social learning, briefly describe the Sustainable Delta game and explain our analyses. Next, we present the results of a quantitative analysis of 12 game sessions and a qualitative analysis of a subset of three game sessions. Finally, we discuss the main findings of our study, options for improving game design, and the limitations and dilemmas of theory-based assessment of games. We conclude with an outlook on what is needed to enhance the contribution of game-based social learning to socially sustainable water management.

2. Methods

2.1. A Conceptual Framework for Game-Based Social Learning

Social learning is a widely used term, and even in the more delimited area of natural resource management, confusion about its meaning continues. A major reason for this is that the social nature concerns both the learning process and the learning outcomes (“learn together to manage together” [7]), and as a consequence the two have often been mixed up [18]. A first step in presenting a conceptual framework for game-based social learning must therefore be a clear distinction between the setting, the process, the outcomes and the impacts of social learning [11,16,19]. The setting or context is made up of a wide range of factors influencing the learning process. In the case of game-based learning, this also includes the properties of the game [15]. The learning process concerns stepwise changes in knowledge, skills or attitudes of the individual participants based on communicative interactions between them. The end results of this process are termed the learning outcomes: cognitive or relational changes at group level, such as a common understanding of a problem or more trust among the participants. The impacts are then the real-world changes resulting from these learning outcomes, for example changes in natural resource management and governance systems.

Often both cognitive and relational learning outcomes are required to bring about such real-world changes [7], but here we will focus only on the cognitive dimension of social learning and how games can support this. Cognitive social learning in the context of natural resource management concerns changes in the perspectives of stakeholders on the nature of the problem, the preferred solutions and the responsibilities of the actors involved. The desired learning outcome is a convergence of individual perspectives towards a common group perspective which can serve as a basis for concerted or collective action to deal with the problem [20,21,22]. To study this, a perspective can be conceptualized as a set of beliefs individuals hold concerning a problem. This includes both causal beliefs, e.g., about the causes of the problem or the effectiveness of solutions, and normative beliefs, e.g., about the desired situation or acceptable interventions. By ‘mapping’ these beliefs before and after a social learning session, it can be determined whether the beliefs of the participants have become more similar, providing evidence of social learning [23].

To study how convergence of perspectives can be supported, the learning process must be further conceptualized. A learning theory which appears well-suited for this purpose is experiential learning theory (ELT) [24]. The central notion in ELT is the experiential learning cycle. In this cycle, the learner moves from experiencing the effects of actions to reflecting on these effects, and from reflecting to (re)forming the mental model (or set of beliefs) concerning what is observed. Adapting beliefs is more likely to occur when the experienced effects do not match with the learner’s expectations. The learner may then decide on new actions based on the adapted set of beliefs, and the experiences that follow from these actions start another learning cycle. Social learning has been defined as an interactive process of shared, experiential learning, amplified by facilitated communication and dialogue [25]. Given the joint experiences and the joint reflection on the meaning of these experiences, stepwise reconceptualization and change in the participants’ beliefs is expected to move in a similar direction, resulting in a convergence of perspectives [26]. In other words, individual perspectives may change when expectations are not met by observations, and this change may be convergent at group level, when the members of the group make their expectations explicit, exchange their views and reflect jointly on new information and possible discrepancies with initial expectations (‘surprises’) [23]. In this framework, two major ways in which simulation games may support social learning are: (1) providing the players with ‘feedback’ on (jointly decided) actions with simulated effects, and (2) offering the players a ‘platform’ to jointly reflect on the effects of actions and discuss possible consequences for their perspectives [9,10,14,16] (

Table 1. Game-based social learning framework: major learning support functions of collaborative serious games.

Social Learning Support Function	Mechanism
Feedback	Provides the players with ‘feedback’ on (jointly decided) actions with simulated effects
Platform	Offers the players a ‘platform’ to jointly reflect on the effects of actions and discuss possible consequences for their perspectives

).

2.2. The Game: Sustainable Delta

Sustainable Delta is a multi-player, role-play simulation game, where participants have to manage a river in a sustainable way. The goals of the game are to learn about the consequences of a long-term perspective for river management, and how learning together can result in a convergence of perspectives on the problem and preferred solution strategies (i.e., social learning). The game can be played for training purposes in a setting not directly connected to a real-life river management problem, or as part of a participatory approach to an actual river management issue. In the first case, the players learn about social learning by experiencing it, in the second the players’ social learning may contribute directly to solving the real-world problem that was the focus of the game.

In Sustainable Delta, the river system and its responses to management measures are simulated over a period of 100 years into the future with a meta-model. The (validated) meta-model is based on simplified cause–effect relations and response curves derived from complex hydrological and impact models, and fed with data from long-term climate change scenarios. Climate conditions are expressed in time series of river discharge, including yearly peak discharges and the yearly duration that discharge falls below critical levels. In four rounds, two teams of players have to select and negotiate a strategy, comprising of two river management measures to be implemented. The meta-model simulates the effects of the chosen measures over a period of 25 years, while taking changing climatic conditions into account. A wide range of possible measures is available to the players, ranging from ‘educating people about water safety’ to creating more ‘room for the river’, i.e., large-scale spatial adaptation (Appendix A). The effects of the chosen measures, which the players are confronted with in the next round, are expressed by a number of indicators: dike rings flooded (number), urban area flooded (km²), damage (€), navigability of the river (% time), area for nature (km²) and ecological diversity (index). A full description of the game and its design can be found in Valkering et al. [27], whereas Haasnoot et al. [28,29] provide more details on the underlying simulation model.

A unique feature of Sustainable Delta is the attention to the perspectives of actors that underlie support for (and opposition to) river management strategies [30]. This feature is integrated into the flow of the game:

• Before the game, the players fill in a Perspective Scoring Table (PST, Appendix B), which yields a set of endorsed beliefs representing their perspective on river management [23]. The players are then divided into two teams with initially diverging perspectives, based on the similarity of their individual PST scores (i.e., the beliefs endorsed by these participants). The teams receive an enlarged print of the PST, with pins representing the team perspective.
• Each round, the teams have to select two measures from a deck of option cards to manage the river sustainably for at least the next 25 years. Sustainability includes here human safety, nature and biodiversity, and economic interests. The two teams are asked to negotiate and agree on two measures that will be implemented.
• In the next round, the players are confronted with the effects of these measures over the past 25 years. Strictly speaking, this not only includes the simulated causal effects of the measures under a scenario of climate change, but also contains a stochastic element in the modeling of dike breach.
• After a discussion of these effects, the teams are asked to review their team perspective based on their interpretation of the events, and when deemed necessary, revise one of more of the endorsed beliefs by re-adjusting the pins on their PST.
• Then they are asked to choose again two measures, given their updated perspective on river management and the state of the river.

These steps are repeated until 100 years are simulated. The game sessions are concluded with a debriefing discussion on the game itself, the dynamics of the session and the lessons learned. For each session, a log is kept of the measures chosen and implemented, the simulated effects on the river system, and the (changes in) team perspectives.

The development of the game was commissioned by Deltares, the largest knowledge institute on water management in the Netherlands, and the game is available on their website (https://www.deltares.nl/en/software/sustainable-delta-game/, accessed on 21 April 2021). The game has been shown to be effective in generating social learning outcomes, defined as a convergence in perspectives on river management among the players. These effects were found both under more controlled conditions [17] and in a real-life problem context [31].

2.3. Analysis of Game-Based Social Learning

2.3.1. Hypotheses and Expectations

Based on the design of the Sustainable Delta game and the conceptual framework for game-based social learning, which emphasizes the ‘feedback’ and ‘platform’ functions of games in joint experiential learning cycles, we formulated two hypotheses concerning how this game supports social learning, here defined as convergence of team perspectives: (I) The effects of chosen river management measures determine changes in team perspectives on river management, and (II) (Changed) team perspectives on river management determine their choice of river management measures. From these two hypotheses, we derived five specific expectations to guide the quantitative and qualitative analysis. For the quantitative analysis, we expected: (1) a significant relation between effect of measures and change in perspective, (2) a significant relation between perspective and choice of measures, and therefore also (3) a significant relation between effect of measures and choice of measures (in the next round). For the qualitative analysis, we expected that: (4) changes in perspective are motivated primarily by the effects of chosen measures, and (5) the choice of measures is motivated primarily by the (changed) team perspective. Figure 1 shows how these hypotheses and expectations relate to the design of the Sustainable Delta game and the ‘feedback’ and ‘platform’ functions that may support social learning (Table 1).

2.3.2. Game Sessions

We analyzed 12 game sessions to test the hypotheses and expectations derived from the conceptual framework for game-based social learning. These sessions had a similar set-up (following the steps described in Section 2.2), but took place with different groups of participants: professionals, scientists and/or MSc students in environmental and water management (

Table 2. Analyzed sessions of the Sustainable Delta game, specifying players’ backgrounds and percentage convergence ¹.

Session	Players	Players’ Background		Convergence
1	10	professionals, scientists	Deltares—Institute for applied R&D in water management	4% (1/24)
2	10	professionals	Rijkswaterstaat—Government service for roads and waterways	8% (2/24)
3	10	scientists	University of Twente—Water Management Group	4% (1/24)
4	12	professionals, scientists	Participants in ‘Deltas in Times of Climate Change Conference’	0% (0/24)
5 *	10	professionals	Delta Programma—National implementation program for water management	0% (0/24)
6 *	9	professionals, scientists, students	Participants in ‘Dealing with Uncertainties in Climate Adaptation Course’	8% (2/24)
7	10	students	MSc course ‘Coastal zone and river management’	17% (4/24)
8	21	students	MSc course ‘Environmental assessment and management’	12% (3/24)
9	17	students	MSc course ‘Environmental assessment and management’	8% (2/24)
10 *	10	students	MSc course ‘Integrated water management’	4% (1/24)
11	15	students	MSc course ‘Climate and Adaptation’	4% (1/24)
12	16	students	MSc course ‘Climate and Adaptation’	4% (1/24)

¹ Convergence of perspectives was based on the difference between the number of beliefs about which there was agreement between the two teams at the start and at end of the game (numerator). The denominator is the total number of beliefs that could be endorsed (8 topics × 3 options, see Appendix B). * Session included in the qualitative analysis.

). The selection of these groups of participants was based on the relevance of their areas of expertise or study and their willingness to participate in testing the game. Before the start of each session, participants were informed that the session data would be used for research and what the research entailed. To ensure anonymity, participants were not asked for personal details. The male:female ratio was not recorded for each session, but was about 3:2 (mean of 8 sessions). The median number of participants per session was 12 (range: 9–21), and participants were evenly distributed over two teams. The length of the game sessions was about 3 h, and to avoid running late, time keeping per step was relatively strict. To enable quantitative analysis and hypothesis testing, which requires a relatively large number of comparable sessions, we chose a game setting not directly connected to real-life problem contexts. This means that the river to be managed was fictitious, i.e., not identical to a particular existing river, and that the players were not actors or stakeholders in the management of that particular river. These more controlled conditions for testing and research purposes differed from real-life applications of the Sustainable Delta game, where the game is tailored to a specific river and problem context, with real actors as players [31].

2.3.3. Quantitative Analysis

The relationships between ‘effect of measures’, ‘change in perspective’ and ‘measures chosen’ per round of the game were analyzed with the statistical software package SPSS (https://www.ibm.com/products/spss-statistics, accessed on 21 April 2021) using the logged data from the 12 game sessions (Table 2). See Appendix C for an overview of the variables from the game sessions included in the analysis. We used a Chi-squared test in the case of nominal variables, Pearson’s correlation test with ordinal variables, and Pearson’s correlation test combined with an independent samples t-test in the case of scale variables, while controlling for effects of game session, team and round using ANOVA. Specifically, we tested for significant (p < 0.05) relationships between ‘effect of measures’ (for 7 types of effect) and ‘change in perspective’ (type of belief changed, number of beliefs changed), and between ‘effect of measures’ and ‘measures chosen’. To assess the relationship between ‘perspective’ and ‘measure chosen’, we designated perspectives (usually one, sometimes two) for the different measures (Appendix A), based on theoretical expectations concerning this relationship (for details, see [30]). We then compared the designated perspective with the team perspective at the moment the team chose a measure. This comparison resulted either in a match or not. If all suggested measures in one game session for one team matched with the designated perspectives, the match would be 100%.

2.3.4. Qualitative Analysis

In three of the 12 game sessions (Table 2), we audio-recorded all verbal communication among the players for the entire length of the session. The three sessions were chosen to cover the various backgrounds of the participants. The recordings were transcribed and subjected to a detailed content analysis. For each round in each of the three game sessions, we analyzed for both teams the discussions relating to changing (or not) the team perspective, identifying the arguments for any change in beliefs and any remarkable characteristic of the discussion. For the discussions relating to the choice of measures, we identified the arguments for the choice of the first as well as the second measure. A detailed description of the changes in the team perspectives during these three sessions can be found in Van der Wal et al. [17]. Due to the structured nature of the game and associated team discussions, identification and classification of arguments (‘effects of measures’, ‘team perspective’, ‘other’) proved straightforward and not subject to variable interpretation.

3. Results

3.1. Quantitative Analysis

3.1.1. Relationship Between ‘Effect of Measures’ and ‘Change in Perspective’

During the 12 game sessions, teams changed their beliefs for all topics in the team perspective (

Table 3. Number of times beliefs were changed (per topic) during 12 game sessions ¹.

Topic of Belief	Number of Changes
I. River priority function	13
II. Trust in technology	8
III. Climate change (trend)	13
IV. Economic growth (trend)	3
V. Flood safety strategy	12
VI. Principle of spatial planning	13
VII. Responsibility of actors	2
VIII. Basis of decision making	7

¹ The maximum possible number of changes per topic is 96 (12 sessions × 2 teams × 4 rounds).

). Beliefs concerning the priority function of rivers (I), the severity of climate change (III) and principles of river management (V, VI) were most often subject to change. However, we did not find any significant directional relationship between ‘effect of measures’ and ‘change in perspective’ (Figure 1, expected relation 1). This suggests that for the players, the effect of the chosen measures was not, or at least not consistently, a major reason for the changes they made to their team perspectives.

3.1.2. Relationship Between ‘Effect of Measures’ and ‘Measures Chosen’

We found several significant relationships between the effect that measures had and the measures that teams chose in the following round (Figure 1, expected relation 3), but there were no significant effects of the factors ‘session’, ‘team’ or ‘round’ on any of the tested relationships. When a relatively large number of dike rings was flooded, the measures ‘Room for the river’, ‘Dike raising’, ’Medium sized boats’ and ‘Educating people’ were significantly more popular than other options (p = 0.006). Note that a high number of flooded dike rings usually resulted in decreased shipping suitability of the river, which explains the choice of smaller boats. Furthermore, the more urban area was flooded, the more the teams opted for the measures ‘Artificial mounds’, ‘Educating people’, ‘Room for the river’, ‘Climate dikes’ and ‘Smaller boats’ (p = 0.000). Comparable relationships were found for damage as an effect of measures. When total damage and agricultural damage were relatively large, the teams tended to choose ‘Small or medium sized boats’, ‘Educating people’ or ‘Adapting the trigger for alarm’ (p = 0.000). In the case of no or a small flooding event, teams were generally in favor of ‘Dredging’, ’Floating houses’, ‘Room for the river’ or not taking any measures at all (not significant).

Table 4. Preferred response measures after large and small flood events. Associations between events and responses were significant, unless indicated otherwise (NS = not significant).

Large No. Flooded Dike Rings	Small No. Flooded Dike Rings NS	Large Urban Area Flooded	Small Urban Area Flooded NS	High Damage (Euros)	Low Damage (Euros)
Room for the river Dike raising Smaller boats Education	Dredging Floating houses Room for the river No measure	Room for the river Climate dikes Smaller boats Education Artificial mounds	Dredging Floating houses Room for the river No measure	Trigger for alarm Smaller boats Education	Upstream cooperation Dike rings Floating houses Dredging No measure

summarizes the associations between the magnitude of flood events and the response measures the teams preferred in the following round of the game.

3.1.3. Relationship between ‘Perspective’ and ‘Measure Chosen’

We found that team perspective and measure chosen matched 49.2% of the time, per team, per game session, with a standard deviation of 18.6%. This indicates that there is no clear relation between the perspective of a team and the river management measures it chooses (Figure 1, expected relation 2).

3.2. Qualitative Analysis

3.2.1. Motives for Changes in Perspectives

Effects of chosen measures (floods) were frequently a dominant motivation to change the team perspectives, in particular in sessions S5 (professionals) and S10 (students). This was not the case, however, when these effects were deemed ‘bad luck’, ‘not negative enough’ or ‘within the range’. In session S6 (semi-professionals), the selected measures were an important motivation for both teams to change their perspectives ‘post hoc’, to ensure that perspectives and measures chosen matched. In all sessions, an additional motive for implementing changes in perspectives was the need to complete the establishment of the team perspective after the first round, which should have been done before the start of the game. In addition to these motivations, we observed a variety of other factors that hampered a clear connection between the effects of chosen measures and changes in perspectives (Figure 1, expected relation 4):

• There was very limited time to discuss the implications of the reported effects and review the team perspective (2–5 min);
• There was a strong tendency to move quickly from reviewing the team perspective to discussing the new measures, in response to the reported effects;
• The relevance of reviewing the team perspective each round when playing the game was not clear to all players;
• Meaning and implications (e.g., in terms of associated measures) of the beliefs in the team perspective were not clear to all players;
• The initial team perspective was often already quite ‘broad’ to cover the individual perspectives of team members, and therefore there was little need to change the perspective.

3.2.2. Motives for Selecting Measures

In all three sessions, the effects of measures chosen in the previous round were a dominant motivation to choose certain types of measures. The teams responded to problematic effects (flooding, dike failure, limited shipping time) with clearly associated measures (e.g., room for the river, dike raising, dredging, smaller boats). In the absence of urgent problems, the teams preferred low-cost, low-impact or low-support measures (e.g., upstream cooperation, more nature area), or even no new measures at all. The specific measure (e.g., scale of dredging, height of dike raising) was determined by additional arguments, such as effectiveness and feasibility (especially in session S5, professionals), cool, innovative character (especially in session S6, semi-professionals) and cost. Other more mundane arguments were curiosity (‘see what it does’), compromises between team members and time pressure (‘just choose one’). Some motives were typical of the second measure, including cost (if the first measure was particularly cheap or expensive), complementarity and risk management (measures from ‘opposite’ strategies to spread the risk) and consistency (both measures must fit in the same strategy).

The team perspective was rarely the point of departure in selecting measures, but it was among the additional arguments (Figure 1, expected relation 5). However, the players often perceived the association between team perspective and preferred measures differently from the game designers (as indicated in Appendix A). Furthermore, in session S10 (students), no references at all were made to team perspective in the discussions on measure selection. In the other two sessions, references to the team perspective were nearly all made during the first round, only once during the second round, and not during later rounds, suggesting that in the course of the game the participants got more and more focused on responding to the effects of their previous measures when selecting new measures.

4. Discussion

4.1. Major Findings and Conclusions

In response to calls for theory-based analysis of how serious games can enhance social learning, we focused our study on the learning support mechanisms in Sustainable Delta, a serious game on water management, which has been shown to be effective in generating social learning outcomes [17,31]. Based on social learning theory and a conceptualization of how the game supports social learning, two hypotheses were formulated as well as five associated expectations to guide the analysis. The quantitative analysis of 12 game sessions showed that only the expectation of significant relations between the effects of measures and the choice of measures in the next round was met. No significant directional relations were found between effects of measures and changes in team perspective, or between team perspective and the measures chosen by the teams. The results of an in-depth qualitative analysis of a subset of three game sessions corroborate these findings and explain why significant positive relationships were or were not found in the quantitative analysis. The effects of measures chosen in the previous round were a dominant motive in choosing new measures, although many additional arguments played a role as well. Contrary to the expectations, the team perspective did not function as a linking pin between the effects of measures chosen in one round and the choice of new measures in the next. Its function in the game appeared to be not so well understood by the players, and during game sessions players were given insufficient time to pay proper attention to the team perspective. As a consequence, the effects of chosen measures were not the major motive for changing team perspectives, and the team perspective was only a minor argument in the choice of measures. In effect, the teams played a simplified version of the game: measures were chosen, the reported effects of these measures on the river system were discussed, and depending on the nature of the effects a new set of measures was chosen.

The conclusion is that the social learning outcomes, in terms of convergence of perspectives, which were observed in 10 out of 12 gaming sessions, were not, or were only to a limited extent, generated according to the presumed mechanisms of ‘platform’ and ‘feedback’. This shows that the effectiveness of a game in generating desired (social) learning outcomes is no proof that the assumptions on which the design was based are correct. It also underlines the importance of opening up the black box of serious games to determine how and why they work, as was called for by various authors. If this is neglected, there is a clear risk that the design of games will be based on wrong, untested assumptions and will be less effective in supporting social learning and social sustainability.

4.2. Options to Improve Game Design and Effectiveness

Despite not working as intended, the Sustainable Delta game nevertheless generated social learning outcomes. However, in the 12 game sessions analyzed in this study, the convergence of perspectives observed was quite modest, on average 6%, and might have been much higher if the connections between effect of measures, perspective change and choice of measures were stronger. The observations made in the qualitative analysis point to various options to improve the design of the game in this respect: pay more attention in the briefing (before the game starts) to the role, meaning and implications of the team perspective, and allow teams more time during the game to discuss the implications of the reported effects and review the team perspective.

The successful application of the Sustainable Delta game in a real-life context studied by Lawrence and Haasnoot [31] suggests another route to enhancing the effectiveness of the game in generating social learning outcomes. In this case, game sessions were embedded in a much larger process of reconsidering the flood risk management strategy for the Hutt River in New Zealand, with real actors as players (water engineers, spatial planners and politicians). Here, the game functioned as a linking pin between a phase of creating interest in a new approach to water management and a phase of real-life experimentation with this new approach, with a knowledge broker playing an active role in each phase. An adapted version of the game was used, visually as well as content-wise tailored to the local situation and without the step of reviewing the team perspective in each round. Instead, more time was allowed for exchange of views and arguments between the teams, and for a briefing before as well as an extensive debrief after the session. The debrief was a discussion, facilitated by the knowledge broker, about the experiences during the game, the insights acquired and how the participants could apply these new insights in practice. The same elements—part of a wider approach, a real-life context with real actors as players, a simplified game tailored to a real-life situation, active facilitation, and a debrief with ample time for discussion—are also found in the case studied by Meinzen-Dick et al. [32], one of the few other successful examples of games with convergence of stakeholder perspectives and behavioral change as social learning outcomes [14]. Also Medema et al. [9] stress in their literature review the importance of embedding game-based social learning in an ongoing process of stakeholder interactions through, e.g., participatory game design and development, facilitated interactions, post-game discussions, and learning outcomes that players can directly use and apply in their day-to-day activities and institutions.

4.3. Limitations and Dilemmas in Theory-Based Analysis of Collaborative Serious Games

In response to calls for rigorous theory-based assessments of how serious games support social learning, we analyzed sessions played with the Sustainable Delta game in a similar set-up and disconnected from real-life problem contexts. Organizing game sessions in this way enabled us to collect data from a relatively large number of comparable sessions, which allowed for hypothesis testing and quantitative analysis. This contrasts with the single-case study approach and mostly self-reported learning effects in the analysis of Lawrence and Haasnoot [31] of the real-life application of the Sustainable Delta game, which provided convincing, but methodologically weak evidence of social learning mechanisms and outcomes. The rigor and strict focus of our analysis comes at a cost, however. We could not study the many other characteristics and properties of serious games that may co-determine social learning outcomes [15], nor could we pay attention to social-relational learning outcomes which are also important for socially sustainable water management, and for which collaborative serious games are a promising support tool as well [9,33]. Furthermore, as the game sessions we analyzed were one-off events and not embedded in a wider, real-life context and stakeholder approach, we also could not study important aspects such as retention of learning outcomes after the session or transfer of learning outcomes to real-life settings [14,15]. Probably the most pressing dilemma we encountered, when comparing our study with the real-life case study of Lawrence and Haasnoot [31], is that the game context that lends itself best to a rigorous analysis appears at the same time to be less effective in generating social learning outcomes. However, the complementary insights derived from both studies also suggest that a way to overcome this dilemma may be found in combining both ‘more controlled’ and real-life cases.

4.4. Outlook: Serious Games and Socially Sustainable Solutions

Many of the issues in the sustainable management of water and other natural resources can be qualified as ‘wicked problems’. This type of problems is not only very complex, in terms of causes and effects, but is also characterized by normative disagreement among stakeholders concerning the desired situation and the acceptability of solutions [34]. Neglecting the normatively contested nature of problems can result in deep societal controversy and fierce opposition to proposed solutions [2]. Joint learning of stakeholders can help to reduce disagreements, also in the case of wicked problems [2,34]. However, the few serious games identified in an extensive review by Den Haan and Van der Voort [14] capable of supporting convergence of stakeholder perspectives in real-life cases did not address value-based disagreements. Both in the case of Lawrence and Haasnoot [31] and the case of Meinzen-Dick et al. [32], the observed convergence in perspectives was based on changes in the underlying causal beliefs, i.e., in a better understanding of how the water system behaved and responded to interventions. As pointed out by Sabatier [35] in his work on belief systems, this type of beliefs is more susceptible to change, whereas people’s core norms and values are quite stable, and thus unlikely to change in the context of one or more game sessions. Normative beliefs concerning the seriousness of a specific problem and the preferred solutions, however, are more pliable than these deep core beliefs [35], and could change in the course of a social learning process. According to Cuppen [36], this requires ample attention for explicating and discussing diverging and conflicting perspectives of the participants, followed by a dialogue which focuses on specific solution options for a specific problem. As our study showed, a tension can be expected between this need for ample reflection on perspectives and the fast-paced action-reaction orientation of most simulation games. As Aubert et al. [15] noted, the wicked nature of many water governance issues thus poses a serious challenge for game designers. However, to better support the development of socially sustainable solutions, addressing this challenge should rank high on the research agenda of game-based social learning.