Using Agent-Based Models to Generate Transformation Knowledge for the German Energiewende—Potentials and Challenges Derived from Four Case Studies

Abstract

The German Energiewende is a deliberate transformation of an established industrial economy towards a nearly CO₂-free energy system accompanied by a phase out of nuclear energy. Its governance requires knowledge on how to steer the transition from the existing status quo to the target situation (transformation knowledge). The energy system is, however, a complex socio-technical system whose dynamics are influenced by behavioural and institutional aspects, which are badly represented by the dominant techno-economic scenario studies. In this paper, we therefore investigate and identify characteristics of model studies that make agent-based modelling supportive for the generation of transformation knowledge for the Energiewende. This is done by reflecting on the experiences gained from four different applications of agent-based models. In particular, we analyse whether the studies have improved our understanding of policies’ impacts on the energy system, whether the knowledge derived is useful for practitioners, how valid understanding derived by the studies is, and whether the insights can be used beyond the initial case-studies. We conclude that agent-based modelling has a high potential to generate transformation knowledge, but that the design of projects in which the models are developed and used is of major importance to reap this potential. Well-informed and goal-oriented stakeholder involvement and a strong collaboration between data collection and model development are crucial.

1. Introduction

Climate change is spurring the transformation of society and leading us towards reductions in emitted greenhouse gases (GHG), especially carbon dioxide (CO₂). Governments around the world have become active to achieve lower levels of CO₂ emissions. This endeavour especially affects countries such as Germany where primary energy is heavily sourced from fossils. The German Energiewende is a deliberate transformation of an established industrial economy aiming at a nearly CO₂-free energy system (defined as 80–95% reduction in CO₂ emissions by 2050) while phasing out nuclear energy by 2023 [1].

Over the last twenty years, the climate system, and therefore the necessary set of GHG reduction targets, have become much better understood. At the same time various energy models have been developed as a basis for long-term energy development scenarios illustrating potential pathways for the German energy system towards the ambitiously set CO₂ reduction targets [2]. However, the established scenarios illustrate merely techno-economic pathways, designed around the optimal deployment of technical measures such as clean energy production technologies or energy efficiency measures.

The energy system is, however, a complex socio-technical system, characterised by fast and considerable changes in its structure, a high number of variables, an increasing decentralization of decision making among an increasing number of actors, and a rising complexity and numerous uncertainties [3,4]. Therefore, in order to transform it to a climate friendly or sustainable status, we need to not only consider technology and prices, but to focus explicitly also on the role of the actors involved [3,5,6,7]. Behavioural and institutional aspects must be given due consideration, as it is the actors themselves that make the decisions to adopt or to use the technologies that the scenario studies of the energy system focus on [8]. The knowledge to do this is still underdeveloped due to focusing on techno-economic energy systems analysis in the past [7].

Therefore “transformation knowledge” is limited to date. It is defined as “knowledge on how to shape and implement the transition from the existing to the target situation” [9]. Transformation knowledge complements knowledge about clear targets and knowledge about the current state of the system (called “target knowledge” and “systems knowledge”, respectively; [9]). Transformation knowledge is concerned with established technologies, regulations, practices and power relations, and includes knowledge about different technical, social, political, economic, cultural, and legal methods and means that facilitate the transformation process from this current situation to the target situation [10]. Transformation knowledge is required to give actors on various levels the orientation for taking action.

Holtz et al. [11] have argued that models are useful tools to study processes of long-term fundamental change in socio-technical systems such as the energy system, because models are explicit, clear and systematic, allow for inferences about the dynamics of the system, and facilitate systematic experiments that might not be possible otherwise. Köhler et al. [12] have reviewed a range of existing modelling approaches with regard to their ability to address a number of key characteristics of such transitions.

We expect agent-based modelling (ABM) to be a method that can contribute well to exploring transformation knowledge since it provides the opportunity to include the boundedly rational and socially influenced behaviour of actors into simulation models in a bottom-up approach. It therefore enables explorations of the future course of complex systems, such as the energy system, while considering that this future course is influenced by many behavioural and institutional aspects. This complements other bottom-up scenario-driven approaches for describing the energy future, such as normative and descriptive scenarios.

While existing studies (see next section) have established the potential to study aspects of the Energiewende with agent-based models and have produced valuable insights, the overall impact of ABM on the Energiewende remains negligible up to now.

In this context, our aim in this paper, is to carve out requirements for and characteristics of agent-based model studies to produce transformation knowledge in the context of the Energiewende. We propose that such insights might contribute to larger impact of ABM in the future.

We will investigate these issues through an in-depth analysis of the experiences from four recent applications of agent-based models on transformation processes related to the German Energiewende. Those models were developed within three research projects that were part of a research programme that aimed at gaining transformation knowledge in the sense described above. We therefore presume that these model projects constitute a valuable set of cases to draw on for achieving our research aims.

In our paper, we briefly outline existing work of ABM applied to energy systems before developing an analytical framework for the analysis of the four models. The analytical framework is then applied to each of the models. Based on the results, we discuss the potential and challenges of ABM studies to generate transformation knowledge and carve out requirements for and characteristics of ABM studies that are crucial for producing transformation knowledge before giving some final conclusions.

2. Brief Review of ABM of Energy Systems

Sensfuß et al. [13], Pfenninger et al. [4] and Ringler et al. [14] provided reviews that include or focus on agent-based models of the energy system or relevant subsystems. Sensfuß et al. [13] reviewed agent-based modelling of the electricity markets. They concluded that agent-based simulations had developed from simple models to ones capable of dealing with multiple markets and time scales, and that despite remaining challenges, agent-based simulation appears promising to be useful as a test bed for the electricity sector. Pfenninger et al. [4] concluded that methods from complexity science (including ABM) have been successfully applied in energy research, but that a need for a deeper and more fundamental treatment of complexity remains with much potential for additional research. Ringler et al. [14] reviewed the use of ABM for analysing smart grids from a systems perspective, which is still a limited field of research. They assessed ABM to be a valuable approach to analyse opportunities and challenges in smart grids and markets. However, they also warned that, when addressing new issues with explorative research designs, model calibration and validation are particularly demanding, and that harmonization and consistency are essential to increase comparability between models.

Much analysis of energy markets and systems using agent-based models initially focused on wholesale markets and did not consider social aspects of the energy system explicitly [15,16,17]. Agent-based models that include social science insights to study energy markets, energy technologies and energy systems in general are relatively recent developments, but are increasingly recognized for being supportive for policy design and evaluation as well as system design and infrastructure planning [18]. Although being a recent development, they can draw on experiences and insights from the broader social simulation field that has addressed aspects, such as learning, opinion dynamics, social norms and the diffusion of innovations [19,20,21].

Until now only a limited number of studies using ABM with an explicit focus on the German energy system have been carried out. Early studies focused on the newly liberalised market: Bower et al. [16] used an agent-based model to analyse the effect of power generators’ strategic behaviour on prices in the liberalised electricity market. Scheidt [22] aimed to carve out the factors that influence volatility of spot-market prices. Müller et al. [23] focused on increasing consumer prices in the German retail market. With an increasing share of renewable electricity and concerns about climate change, studies turned to investigate those issues: using the PowerACE model, Sensfuß et al. [24] have studied the price effect created by renewables on the German electricity market, and Weidlich et al. [25] studied the effect of CO₂ emissions trading on the electricity market. Reeg et al. [26] analysed support schemes for the integration of renewable electricity into the electricity system, focussing on power-plant operators and intermediaries. More recently, agent-based models have also been used to investigate spatial aspects of the energy system: Sorda et al. [27] studied the effects of support schemes on the diffusion of and electricity generation from combined heat and power biogas plants. Lauf et al. [28] assessed the spatial allocation of renewable energy sources under different policy scenarios. Ernst and Briegel [29] and Krebs [30] used a spatially explicit and psychologically grounded agent-based model to study the diffusion of green electricity in Germany.

3. Analytical Framework

To date, there does not exist an established framework for identifying whether models help in creating transformation knowledge or not. Therefore, in order to carve out what has to be taken into account when designing model studies that will create knowledge on how to shape and implement the transition from the existing to the target situation, we first have to further clarify the concept. To specify the term and concept “transformation knowledge” more precisely, we developed a framework that considers four issues that are of particular interest:

• Transformation knowledge includes being able to link interventions to (expected) impacts. Assessing the impact of interventions in complex systems such as the energy system is, however, challenging. We therefore investigate the explanatory approach of the selected agent-based models; i.e., how they (aim to) improve our understanding of policies’ impacts on the complex energy system.
• In order to unfold transformative impact, the knowledge derived from agent-based models needs to be usable by practitioners who are responsible for implementing policies and other measures to foster the energy transition. We therefore assess the usefulness of knowledge derived from the selected agent-based models for practitioners.
• Conforming to requirements of “evidence based policy” or more nuanced demands for “good governance of evidence” [31] policy makers, who are accountable to citizens, need to be informed by transformation knowledge that is scientifically valid. We therefore investigate the validity of understanding derived by the selected agent-based models.
• Agent-based models unfold larger transformative impact if insights gained from them are of a more general nature or transferable from the initial case-study to other cases. We therefore evaluate the generality of insights derived from the selected agent-based models; i.e., whether these insights can be used beyond the initial case-study.

As a next step we operationalize these four issues for application to the case studies. We discuss in the following these four issues in turn in the context of ABM more generally, thereby also providing a brief introduction to ABM for readers who are not familiar with it. While doing so, we briefly outline what each issue means in the context of the Energiewende; based on that, we specify the information that needs to be provided for each model study to assess its ability to generate transformation knowledge.

3.1. Explanatory Approach

ABM allows for the study of complex systems from a bottom-up perspective and has been applied in a broad range of disciplines, ranging from molecular self-assembly, biology and ecology to economics, sociology, anthropology, and cognitive science [32]. Agent-based models define elements—the agents—that interact with each other and their (passive) environment. Agents may be heterogeneous in their attributes and behavioural rules. When applied to social systems, at their core, agent-based models consisting of agents that represent actors that interact with each other and their environment such as technical artefacts. Simulation of agents’ interactions over time allows for the generation of emergent phenomena on the level of a group, organisation or other collection of actors—be they spatial or temporal patterns or characteristic statistical distributions of variables of interest—based on the boundedly rational behaviour and social interactions of these actors [33,34,35]. As such, ABM provides a means to identify possible explanations for emergent system-level behaviour by identifying underlying mechanisms [36]—i.e., by providing a causal reconstruction of the processes that account for the emergent phenomenon [37]. Policy impacts can thus be interpreted in ABM studies as emergent effects from policies through their effects on the behaviour and interactions of actors.

Understanding the impact of interventions in the complex energy system requires not only a consideration of the type and strength of the intervention, but also of the state of the targeted system. To outline the explanatory approach of the agent-based models presented in the next section for understanding of the impacts of interventions for shaping and implementing the energy transition, we will describe three aspects: (1) the emergent phenomenon of interest that each model aims to explain; (2) the system representation including its main elements and processes, and how these elements and processes are involved in mechanisms that generate the emergent phenomenon of interest; (3) interventions that can be analysed with the model and how these affect the actors.

3.2. Usefulness

It is central for the usefulness of results for practitioners that the model answers a question that is relevant for them. For gaining transformation knowledge, relevant questions are typically concerned with the effects of interventions; in particular, which (type of) intervention is most adequate to support the transition from the existing to the target situation.

Models in general, and agent-based models in particular, are built for a purpose and cannot (easily) be used for a different purpose. It is therefore essential to make sure that the model purpose matches with the problems of the target group of stakeholders. This is typically best achieved through the involvement of stakeholders in the model design process [3,38]. The flexibility of the ABM method thereby facilitates a vast range of model designs—i.e., the method does not per se impose limitations to tailoring the model purpose to stakeholders’ needs. There are, however, delicate interdependencies between a model’s purpose, its design and resulting complexity, data availability, and validity. This makes some conclude that “developing good multi-agent models is still something of an art, rather than a science” [39] (p. 8). Consequently, agent-based models cannot answer each and every question of practitioners to any degree of precision. Instead, the models’ purpose and design need to find a match between practitioners’ needs and what is manageable from the modeler’s point of view.

The communication of model results to relevant stakeholders that are able to implement the proposed measures is of central importance for knowledge gained during a project to become transformation knowledge which is useful for practitioners. Communicating simulation results to stakeholders, however, is difficult in a variety of ways. Models include a lot of knowledge from various sources and many different assumptions, all of which influence (to various degrees) the model results. To fully understand a model’s results therefore requires devoting a considerable amount of time to the study of the model itself. Limited understanding of the model may reduce the trust of stakeholders in the model, especially if the results do not match their expectations.

At least two different strategies exist to mitigate these difficulties. Some researchers have developed agent-based models together with stakeholders, and the model design and the whole modelling exercise have been tailored towards a close collaboration with stakeholders (e.g., [40,41,42]). This requires extensive time commitments and engagement from stakeholders, and the resulting models have to remain modestly complex. In such cases, the stakeholders might fully understand the model and (ideally) are willing to accept its results as they understand how they were derived. If agent-based models become large and complex software modules, in order to represent large and complex systems to a sufficient degree of detail, it cannot be expected that stakeholders fully understand the models, nor that their results are readily accepted. Nevertheless, a strategy for communicating results from a model is to refrain from explaining the full model, and to instead extract core mechanisms that generate interesting results, forming isolated explanations of those mechanisms and their consequences within the broader context of the model.

For agent-based models to produce useful transformation knowledge in the context of the Energiewende, they need to ensure that their object of research is of interest to stakeholders capable of acting on the energy system. They need to represent interventions and their effects; it must be possible to communicate the results of these studies to the stakeholders who are able to implement the respective interventions.

In order to evaluate the usefulness of the models presented in the next section, we therefore (1) analyse the relevance of the studied emergent phenomenon for the Energiewende; (2) outline whether and which interventions have been made explicit in their design and (expected) effects; (3) discuss how the results have been communicated to practitioners with the powers to implement those interventions.

3.3. Validity

Agent-based modelling research started with abstract models to showcase and explore fundamental principles (e.g., [37]). This established its potential (e.g., [43]). Since then, it has increasingly been recognized that empirical embeddedness of agent-based models, including calibration and validation of models with empirical data, matters for drawing policy relevant conclusions. Calibration refers to setting the value of model parameters in a way that the model fits to the empirical data. Validation means testing that the model captures the real system well and is fit for purpose. Calibration and validation may well be interconnected, depending on the approach chosen for using empirical data (see, for example, [44]). Validation is also different from verification: the process of ensuring that the model performs in the intended manner by its designers and implementers (e.g., the program code does not contain errors) [45]. All three steps, calibration, verification and validation are important modelling steps. In the context of our framework, we focus on validity.

Traditionally, validation is understood as testing whether the model captures reality sufficiently well through comparison of the model and its results with empirical data [44,46] This includes demonstrating the correspondence between model entities and processes that constitute the micro-foundation of the model with real-world entities and processes. It furthermore includes to demonstrate the ability of the model to replicate emergent empirical patterns on the system level. Conventional validation is, however, often challenging for agent-based modelling. Degrees of freedom for designing agent-based models are large, and knowledge is often limited when it comes to specific questions of model design or might even be contradictory if different schools of thought propose different theories. Furthermore, social science theories often require a “translation” or interpretation when it comes to the details of implementation in the formal model language, which adds additional ambiguity and subjectivity. A certain level of subjectivity with regard to the model structure chosen by the model builder therefore is unavoidable for ABM. A lack of available data to parameterize the model often results in a number of free parameters, which can lead to the effect that a broad range of behaviour can be produced with an appropriate choice of parameter values. Therefore, empirical data that matches output variables of the model is typically needed for calibrating free parameters, and often little or no independent data are left for rigorous validation.

Scholars have argued that validation should be more broadly understood as the process of assessing the usefulness of a model for its intended purpose, and that this may be achieved in different ways than the conventional validation perspective proposes (e.g., [47]). Over time, a range of conventional to more innovative strategies for validation have been developed that are sensitive to the type of model developed and its intended use [44,47,48,49]. Those approaches require either data from a set of comparable cases, historical time series, the involvement of stakeholders and experts, or compare models to each other.

We argue that modelling the Energiewende is indeed at the most challenging edge of the spectrum of validation, as we are not concerned with historical developments but the (unknowable) future, and the structure of the energy system changes at a fast pace which entails that historical data from a structurally similar system is not always available. Furthermore, the Energiewende is advanced in Germany compared to most other countries worldwide, and therefore no comparable cases exist that could be referred to [50]. Furthermore, as outlined in the introduction, only a small number of models exist so far that could be used for comparison. As a consequence, involvement of experts and stakeholders as well as own data collection become important strategies for model calibration and validation.

To assess the validity of the insights generated by the models presented in the next section, we will discuss (1) the overall approach adopted for calibration and validation (i.e., how it is assessed whether the model is fit for its purpose); (2) the theoretical basis; (3) the empirical data; (4) the involvement of stakeholders.

3.4. Generality

For social science in general and for ABM in particular a tension exists between understanding a particular case well and covering a wide range of empirical phenomena that share some common features. In order to describe this tension, Boero and Squazzoni [51] have distinguished between “case-based models” and “typifications”, while Janssen and Ostrom [52] introduce the dimension of “context” versus “generalization”. The ABM method does not impose any restrictions concerning the level of abstraction of the analysis. Which abstraction level is particularly useful for generating transformation knowledge is still an open question—i.e., it is debated which amount of “context” needs to be included in models for producing practically relevant knowledge and what can be represented in a more abstract way for the sake of better transferability [53]. Furthermore, agent-based models can be developed for a variety of empirical scales, ranging from small groups up to the supra-national level, and is therefore applicable to a broad range of research questions that relate to various empirical scales. As outlined above, we use the hypothesis that the possibility for generalization of results from ABM studies beyond the initial case study helps to unfold larger transformative impact for the Energiewende.

To assess the possibility to transfer insights from the models presented in the next section to other, similar cases or the Energiewende as a whole, we check (1) the models’ empirical scale and (2) their level of abstraction. (3) We discuss the transferability of insights through carving out in which way the details of context considered in the model (if any) matter for the results.

4. Case Studies

In this section we apply the analytical framework developed in the previous section to four agent-based models that were developed within the three projects “EnerTransRuhr” (http://wupperinst.org/p/wi/p/s/pd/464/), “SW-Agent” (http://www.sw-agent.de/) and “Resystra” (http://www.resystra.de/) that were part of the research programme “Environmentally and socially compatible transformation of the energy system” (Umwelt- und gesellschaftsverträgliche Transformation des Energiesystems“, http://www.fona.de/de/) funded by the German Ministry for Education and Research (BMBF) within its long-term funding priority Social-Ecological Research (SÖF, https://www.fona.de/en/society-social-ecological-research-soef-.html) as part of its Research for Sustainable Development (FONA) framework program. Each project ran for three-years. For each model, we give a brief introduction and discuss the four issues identified.

Table 1. Overall summary and assessment of case 1 (insulation) with regard to explanatory approach, usefulness, validity and generality.

Explanatory approach	Spatially and socially disaggregated scenarios of homeowners’ insulation activity emerge from homeowners’ decision-making, their socio-geographic distribution, their social network, and the structural conditions of buildings.
Usefulness	The model has clear policy messages for the problem of increasing homeowners’ renovation activity in Germany, which is of high relevance for the increase in energy efficiency, a core strategic objective of the Energiewende [1]. The communication of the results to stakeholders has, however, been marginal.
Validity	The model has a sound micro-foundation; in particular, the core of the model, the decision-making process of homeowners, was derived from a broad review of empirical knowledge and from empirical data. The model could be calibrated to replicate historical data. However, no validation with independent system level data was possible, and stakeholders have not been involved in model design and validation.
Generality	The main conclusions regarding homeowners’ decision-making processes and recommended approaches to increasing their insulation activity are transferable to other cities in Germany and temperate climate countries

,

Table 2. Overall summary and assessment of case 2 (retail market) with regard to explanatory approach, usefulness, validity and generality.

Explanatory approach	Dynamics of the composition of an electricity retailer population, of sales margins, and of the electricity prices for households emerge from the competition for household customers by electricity companies on a computer generated landscape. Households are imbued with separate preferences for price and regionality.
Usefulness	The model provides insight into the role of regional firms in the liberalized market and the effects and unintended side-effects of liberalization. It provides relevant insights for the goal of the Energiewende to safeguard competitiveness [1]. The model also provides a sound basis to study combined effects of regionality and “green” electricity provided by regional electricity producers in future work. The model has raised the interest of electricity retailers for future practical applications.
Validity	The model has a strong empirical basis, in particular with regard to consumer preferences. The inclusion of stakeholders in model design enhances the validity of the model structure. The model was able to provide insights into empirical developments of retail prices and the markups underlying them. Validation with independent data was, however, not possible.
Generality	Although specific to Germany in its application and calibration, the structure of the model can apply to describing the effects of introducing a freely competitive, liberalized retail market for electricity into a region with a previously traditional regional-monopolistic market structure.

,

Table 3. Overall summary and assessment of case 3 (feedback products) with regard to explanatory approach, usefulness, validity and generality.

Explanatory approach	The spread of the feedback device CO2 meter and of energy-efficient ventilation behaviour emerge from three interconnected processes: (1) the diffusion of the CO2 meter through a population of households, (2) the direct impact of the CO2 meter on ventilation behaviour of adopting households, and (3) a spreading of energy-efficient behaviour via social influence.
Usefulness	The model provides an assessment of strategies for a roll-out of feedback products through analysing the system level effect on energy savings. We conclude that the model results are useful for practitioners because they give clear advice for cheap and implementable options for increasing energy efficiency, which is a core strategic objective of the Energiewende [1]. The communication of the results to stakeholders has, however, been marginal, as there was no project time left when results were available.
Validity	The model has a solid micro-foundation and could be calibrated to historical data. However, validation with independent system level data could be extended, and stakeholders have not been involved in model design.
Generality	The model is calibrated to the city of Bottrop. Results are, however, shown to not be sensitive to the detailed urban structure or the exact location where marketing is taken out. The insights from the model are therefore transferable to other cities, and in principle applicable to other behaviour-changing feedback devices; assuming their characteristics for diffusion being similar.

and

Table 4. Overall summary and assessment of case 4 (power-to-fuels) with regard to explanatory approach, usefulness, validity and generality.

Explanatory approach	The economic uptake potential of power-to-fuel applications emerges from the interactions of power-to-fuel operators, customers and subsidizers. In general, the model revolves around investment decisions made by the variety of operators, and purchasing decisions made by the individualized customer groups, all within a dynamic context of evolving regulation and technological progress.
Usefulness	Synthetic fuels are an important technological option to expand the use of renewable energies, which supports the respective strategic objective of the Energiewende [1]. The model provides simulations of future synthetic fuel market shares in correlation to changing regulatory framing conditions. The impact of single policy adjustments on the capacity growth could successfully be demonstrated. Although it can be concluded that the results can be of high relevance, especially for political decision makers, the results of the simulations were not discussed with stakeholders.
Validity	The basic setup and scenarios of interest were developed according to stakeholders’ input and feedback, and the model was calibrated to a replication of historical data. Validation with independent system level data was not possible, and some parameter values remain unknown and add some degree of subjectivity to the results.
Generality	The model is tailored to the German energy system and takes a national level perspective. Some technological specification is kept on an abstract level what enables an easy replacement by new information for other systems.

summarise and assess each case.

4.1. Case 1 (Insulation): Increasing Homeowners’ Wall Insulation Activity

In the EnerTransRuhr project, an agent-based model was developed in order to (1) increase the understanding of homeowners’ insulation activity (systems knowledge) and to (2) assess current and novel approaches aiming at an increase in homeowners’ insulation activity (transformation knowledge). “Homeowners” thereby refers to owner-occupier households in detached and terraced residential buildings. The model is described in detail by Friege [54]. It builds on a previous version [55] and a systematic secondary literature review [56].

Explanatory approach: The emergent phenomenon of interest generated by the model is a spatially and socially disaggregated scenario of homeowners’ wall insulation activities for a period of ten years. The model’s main actors are the homeowners; their decision-making process being at the core of the model. Homeowners are spatially distributed in a virtual city, and connected through social networks that are defined based on social homophily and spatial distance. To represent the decision-making of the homeowners, a logistic regression model was derived from empirical data. The main explanatory variables of this logistic regression model are: period of occupancy, attitudes towards insulation, the structural condition of the walls, and the homeowner’s age. Homeowners move out of buildings stochastically dependent on their current age, and are replaced by newly initialized homeowners. When a new homeowner moves into a building, the period of occupancy is set to zero. Attitude formation is assumed to be prone to social influences from the network of social contacts. The structural condition of the wall declines over time, while the homeowners’ age increases. Overall, the spatially and socially disaggregated scenarios of interest emerge from the homeowners’ decision-making process, their socio-geographic distribution, their social network, and the structural conditions of buildings at the beginning of the simulation. The model can be used to study the emergent effects of interventions that interfere with susceptible variables of the decision-making process of homeowners. The potential effect of providing information was studied by improving the attitudes of modelled homeowners towards insulation. A regulatory instrument was simulated by obligating homeowners who newly move into a house to carry out wall insulation within one year.

Usefulness: Several studies, exploring possible ways for reaching temperate climate countries’ climate protection targets through the development of climate protection scenarios recommend increased efforts towards improving existing building insulation [57]. The recommendations were accepted and partly picked up by some governments’ energy policies through including corresponding targets in their national energy concepts and introducing policy measures aiming at achieving those targets. However, currently there is a large gap between the needed and the current wall insulation rate. Policymakers have three options for increasing homeowners’ insulation activity: economic means, information and regulations. According to Friege and Chappin [56], the empirical data confirm that homeowners’ financial resources play no decisive role in their decision to carry out insulation projects. This casts doubt on the usefulness of widely implemented financial incentives aiming at an increase in homeowners’ insulation activity, and highlights the importance of complementary interventions based on information and regulation. The potential of information instruments was explored in the simulation model through improving the homeowners’ attitudes towards insulation. The results show that the potential is comparatively low, whereas it is advantageous—given a certain overall maximum effort—to focus the policy over a short period of time and to distribute the effort over many homeowners. According to model results, tightening regulatory instruments seems to be the best option: obligating new homeowners to insulate the wall within the first year after moving in would step into the window of opportunity associated with the change of house ownership, because many homeowners take advantage of this occasion to conduct extensive renovations anyway, but today many of those do not combine this with energetic renovation. Obligating new house owners to insulate walls within the first year after moving in has the potential to increase the total insulation rate by 40% in Germany for at least ten years (simulated time horizon). This does not yet correspond to doubling today’s insulation rate of about 1% per year needed to achieve the German government’s target of an 80% reduction in energy demand in the housing sector by 2050, but it would constitute a major step towards achieving that goal.

The structure and results of the model were presented to and discussed in the course of a workshop with representatives dealing with activities designed to increase the energy efficiency of buildings in the cities Bottrop, Dortmund and Oberhausen. Since the research has been issued by national ministries, we further expect that its results will be fed into the decision-making process on designing future policies aimed at an increase in homeowners’ insulation activity.

Validity: The overall approach followed for calibration and validation was deriving the micro-foundation of the model from existing literature and in particular from empirical data; calibration of the model to fit Germany’s current annual wall insulation rate. A systematic secondary literature review [56] revealed existing knowledge on the decision-making process of homeowners, and identified potential research gaps. The results show that literature on energy-efficient renovation “lacks a deep understanding of the uncertainties surrounding economic aspects and non-economic factors driving renovation decisions of homeowners” [56] (p. 196). An online survey with 275 usable replies was carried out to fill these knowledge gaps. Regression analysis of the survey results revealed the above mentioned main explanatory variables. The analysis indicates that knowing these variables is sufficient for correctly predicting about 80% of homeowners’ wall insulation decisions. Parameter values for social network construction were derived from the empirical data and according to [58]. Attitude formation through social influence was modelled based on the results of the online survey and Latane’s theory of social impact [59].

For model parameterization, geospatial information system (GIS) data on the location of houses in the city of Bottrop, a city in West Germany, was used. The data provided also included information on each building’s construction year. As older buildings are less likely to be already insulated, the buildings’ construction year was used to estimate their insulation status at the beginning of the simulation. Based on findings from the online survey, a direct relation between the homeowners’ attitude towards insulation and the insulation status of their buildings was assumed and used to distribute the attitudes towards insulation among the modelled homeowners. The homeowners’ age was distributed according to overall statistics of Germany. The homeowners’ overall level of renovation activity in the simulation model was calibrated to fit Germany’s current annual wall insulation rate of 1.4% per year [60]. This was necessary (1) due to the generally higher renovation activity of the homeowners taking part in the survey, whose activity was used to derive the decision-making models, and (2) given the fact that Bottrop’s existing building stock is older than the German average.

Generality: The empirical scale of the model is that of a city. The model is based on a worldwide literature review and was refined with data collected within an empirical study conducted in Germany. Information on buildings’ ages and locations is specific for the city of Bottrop, a city in Western Germany located in the Ruhr Area. In particular, Bottrop’s existing building stock is older than the German average. Information on homeowners was derived from national data. The model was calibrated to fit Germany’s current annual wall insulation rate. The model parameters that stem from the city of Bottrop (buildings’ ages and locations) have some effect on the magnitude of results, but not on the mechanisms identified and the qualitative effects of the analysed interventions. We therefore assume that the main conclusions regarding homeowners’ decision-making processes and recommended approaches to increasing their insulation activity are transferable to other cities in Germany and temperate climate countries such as in northern Europe.

4.2. Case 2 (Retail Market): The Effect of Regional Preferences on the Electricity Retail Market

The second case study was conducted in the SW-Agent project and analysed the retail market for electricity in Germany. The project highlighted the effect played by regional preferences of households which lead them to purchase from nearby suppliers. ABM was chosen as a methodology in the project potentially suited to studying the apparent preferences of households for local suppliers in addition to their preference for low prices. The model is described in detail by Yadack et al. [61].

Explanatory approach: The emergent phenomena of interest are the dynamics of the composition of the electricity retailer population, of sales margins, and of the electricity prices for households over time. The model simulates competition for household customers by electricity companies on a computer generated landscape. The generated landscapes include rural, urban and metropolitan regions and associated population densities. Both utility companies and households are decision-making agents in the model. Households are imbued with separate preferences for price and regionality (represented by geographic distance) as well as a threshold for becoming dissatisfied with their electricity provider, and choosing their preferred electricity company among those who offer in their region. Multiple electricity retailers compete in the simulation to win contracts to deliver electricity to customers by adjusting their tariff price, and through focusing expansion of their activities on relatively dissatisfied clusters of households. For the latter, electricity retailers choose to locate new generation and sales activities close to those relatively unsatisfied consumers, offering a regional electricity product. Explanatory variables in the model are, in addition to these agent-specific values for price and preference-weightings, the number of firms and households, and their geographic distribution (e.g., rural versus urban) in a given simulation run. Different simulation runs produce different development scenarios, depending on these influences and on initial values of market prices. The model can be used to study the interaction between the effects of geographic distribution of firms and households in communities and the regional preferences of households. The regulatory instrument first implemented and studied in the model was the liberalization of the retail market and the response of the household and firm agents to this regulatory instrument.

Usefulness: In a freely competitive retail market, electricity users are free to choose any electricity supplier, independent of the firms’ and power plants’ physical locations. In such a market, competition is expected to be intense, as electricity suppliers all sell a perfectly replicable and equivalent commodity. Through economies of scale and the need to leverage digital technologies, smaller, regional and in many cases community owned electricity utilities could be expected to be in danger of being priced out of the market entirely. Local utilities in Germany have, however, proven to be competitive despite competition from large players, multinationals and internet-based new entrants to the sector. According to the German Association of Energy and Water Industries, since liberalization of the German retail market in 1998 up to 2015, prices for electricity (excluding taxes and surcharges) have increased for households by 7%. The same value for industrial electricity customers has decreased by 23% [61]. This indicates asymmetric benefits of liberalization for the household electricity users and industrial electricity users, and in particular, leaves open the question as to which factors mediate the intensity of retail competition in the household electricity market when competition has so clearly intensified in the retail market for industrial electricity customers. The results of Monte Carlo studies with the developed agent-based model showed that regional preferences play a significant role in mediating the intensity of competition between electricity suppliers in a liberalized retail market. Sales margins calculated into electricity tariffs increased with increasing regional preferences. The development of these sales margins over time was also shown to be a non-linear process, and highly dependent on regional clustering of households and the threshold for dissatisfaction with one’s electricity provider. Scenarios in the simulations highlighted not only the dynamics behind price developments, but also individual firms’ strategies when their local customers fall into the scopes of low-price competitors. That local, municipally-owned electricity utilities could in the real world affect their own local residents’ attitudes toward electricity (for example through diverting portions of profits toward community projects such as playgrounds and/or other community initiatives) suggests and supports the claim that these local utilities have been able to apply a certain degree of competitive advantage in regional and social factors to defend their positions as competitors in the retail electricity market—despite liberalization and intensifying competition in price. One useful lesson for policy makers is that social factors such as regionality can affect the development of market prices as shown by the model. Future policies aimed at increasing competition in electricity markets need to account for such factors if the response of markets is to be anticipated correctly. Failure to do so could result in even more difficult corrective measures later (e.g., discussion of subsidies and fee exceptions for large industry’s electricity costs in Germany). In concluding experiences from stakeholder workshops, it could be argued that communication of the insights gained from the model to industry stakeholders was more effective than the communication to policy stakeholders. In particular, electricity retailers showed interest in practical applications of the model for individual case studies—this being due to their exposure to competitive retail market pressures which the model also simulates.

Validity: The main strategies for achieving validity were empirical data collection and the involvement of stakeholders to assure the correspondence of the model’s micro-foundation with real world entities. The development of the model was imbedded in the larger scope of the project “SW-Agent”, and as part of this project, was informed by both stakeholder workshops with utility companies and with empirical data from household surveys. To set up the model, a workshop of industry professionals was organized to draft an initial structure of the model, which was then completed and implemented in close cooperation with the utility company. The structure of households’ preferences in the model was inspired by a synthesis of work on national surveys of electricity customers of a utility company based in southern Germany [62]. Over 4000 households (from approximately 27,000 contacted households) responded to an online survey in January 2014. Respondents were presented with fictive electricity tariffs in a choice-based analysis of their preferences and were asked to choose between various alternatives. The attributes of the presented tariffs included type of firm (including whether the firm was a local utility), renewable or conventional electricity, a level of service support, and price, and the respondents’ answers were able to be collated with empirical data on location, and social demographic responses in the survey. The survey data showed a strong preference of households for local municipality-owned utilities. These granular empirical data were applied to the development of the abstracted simulation environment, and was used to structure and quantify preferences in the simulation. Population densities of different types of regions were based on public data from the German Federal Statistics Office. The pattern of suppliers’ markups to electricity prices demonstrated in the model is reflected in empirical data on real prices in Germany since market liberalization in the electricity sector.

Generality: The model itself, and especially the way in which household agents make decisions, was based on empirical observations at the micro-level of German households, complemented by statistical data for Germany. The computer generated landscapes are generic. Although specific to Germany in its application and calibration, the structure of the model can apply to describing the effects of introducing a freely competitive, liberalized retail market for electricity into a region with a previously traditional regional-monopolistic market structure.

4.3. Case 3 (Feedback Products): Evaluation of Marketing Strategies for Energy Consumption Changing Feedback Devices

The third case study, which was also conducted in the EnerTransRuhr project, used ABM to evaluate marketing strategies for feedback devices. Numerous approaches exist to give feedback to energy consumers, e. g. email, online platforms, or installed in-home displays [63,64,65]. One example of a feedback device is a so-called “CO₂ meter”, which shows the indoor air quality–measured by CO₂ level–via the colors of a traffic light. This was shown to be effective in convincing households to practice energy-efficient ventilation of rooms. In order to assist the large-scale leveraging of the potential of feedback devices, this case study addressed the question of which method of managing the rollout of feedback technology is most effective at creating additional energy-efficient heating behaviour, using the CO₂ meter as a proxy. The model, building on an earlier version [66], is presented in detail by Jensen and Chappin [67].

Explanatory approach: The emergent phenomenon of interest is the spread of the CO₂ meter and of energy-efficient ventilation behaviour. The effect of the CO₂ meter is assumed to be the result of three interconnected processes: (1) the diffusion of the CO₂ meter through a population of households, (2) the direct impact of the CO₂ meter on ventilation behaviour of adopting households, and (3) a spreading of energy-efficient behaviour via social influence. The basic model components are household agents that are assigned to three different consumer groups that differ with respect to their attitude towards eco-friendly household products. Households are spatially distributed in a virtual city, and connected through social networks that are defined based on measured homophily and spatial distance. The agents make choices on whether to adopt the CO₂ meter and whether to adopt energy efficient ventilation behaviour. These choices are both modeled such that they can be influenced by other agents in their local social network. The model was used to assess strategies for a roll out of the CO₂ meter that differ with respect to initial attributes of the initial adopters (consumer group, centrality in social network).

Usefulness: Providing feedback to energy consumers about their energy consumption behavior can help them to save energy. Feedback about behavior was found to decrease energy consumption by up to 20%, with an average of about 10% [64,68]. Further, achieving this behavioural contribution to energy efficiency is comparably cheap and simple. In simulations, marketing strategies for feedback devices successfully resulted in additional adoption of energy-efficient ventilation behavior, particularly when: (1) the use of feedback devices was incentivized economically by giving away or lending out some devices for free; (2) the social influence of well-connected opinion leaders (i.e., households that are particularly influential for others) was leveraged to promote the devices; (3) households of higher social status were targeted primarily by marketing (Jensen and Chappin 2017). The tested marketing strategies rank differently when compared based on their effectiveness than if compared for cost-efficiency. Regarding effectiveness, Jensen and Chappin [67] found the lending out of devices to be the best performing marketing strategy. The most cost-effective measure is raising awareness among households of higher social status. The issue of communicating simulation results was approached through identification of stakeholders on city level that are best able to reach the most promising target groups identified in the various strategies and who at the same time might be interested to implement the respective measure (e.g., lending out devices). Discussions with relevant stakeholders have been initiated towards the end of the project (when model results were available), but the process fizzled out in an early stage when the project ended.

Validity: The overall approach chosen for calibration and validation was pattern-oriented modelling [48]. Following this approach, model parameters are accepted if model output matches simultaneously multiple defining characteristics of the observed behaviour of the real system. The Theory of Planned Behavior [69] was used as theoretical framework to structure the actors’ decision making and to combine intrinsic motivation and social influence. The behaviour diffusion process was parameterized from survey data [66]. As no data for the diffusion of feedback devices were available, data on the diffusion of water-saving showerheads [70] were used as a proxy; arguing that these are similar with respect to characteristic that are decisive for production diffusion [66]. The impact of feedback devices on the energy demand of adopting households was derived from data from field-tests of the CO₂ meter. The agent population (consumer groups) and the social network was parameterized using empirical data on households and buildings. Most of these data were gathered in the city of Bottrop (Germany). Using commercial high-resolution marketing data, about 32,000 modeled households were mapped to the 3 consumer groups that differ in their preferences of technology adoption.

Generality: The empirical scale of the model is that of a city. The model was calibrated to the city of Bottrop with regard to agents’ consumer groups and local social network data. However, results of the models were shown to be generalizable to other German cities: a sensitivity analysis showed that model implications are specific to the composition of consumer groups in a simulated area. However, results are not sensitive to the detailed urban structure or the exact location where marketing is taken out [67]. The insights from the model are in principle transferable and applicable to other behavior-changing feedback devices; assuming their characteristics for diffusion being similar.

4.4. Case 4 (Power-to-Fuel): Evaluation of Pathways for An Economic Integration of Power-to-Fuel Concepts

In the fourth case study, which was conducted in the RESYSTRA project, an agent-based model was developed to evaluate regulatory pathways for an economic integration of renewable synthetic fuels into the German energy system [71]. A full documentation of the model is provided by Brand et al. [72].

Explanatory approach: The emergent phenomenon of interest is the economic uptake potential of power-to-fuel applications. Power-to-fuel operators, customers and subsidizers were implemented as agents in a newly developed agent-based model. In order to represent the whole supply and demand volumes while keeping the model complexity manageable, each group of similar real-world actors was represented by a single agent in the model. These agents were set into relation with one another via their participation in assigned energy markets. In general, the model revolves around investment decisions made by the variety of operators, and purchasing decisions made by the individualized customer groups, all within a dynamic context of evolving regulation and technological progress. In addition, the model includes technology classes for the production of hydrogen, methane and liquid fuels, (installed) technology instances and business cases as passive objects. Finally, the regulatory and economic context for the interaction of agents and objects is represented in the model’s environment. The model is able to simulate the effects of changes to economic variables such as taxes, fees and subsidies, as well as to rules and regulations.

Usefulness: The Energiewende process is increasingly facing challenges posed by the volatile fluctuation of electricity generation from wind and photovoltaic power. In accord, solutions for energy intensive applications like industrial high temperature processes and mobility concepts as aviation, shipping and heavy transport are required to achieve international climate protection targets. Power-to-fuel concepts represent an innovation path that might become the missing link as (excess) electricity can be utilized for a dynamic hydrogen and synthetic hydrocarbon production [73,74]. The technological readiness level of several power-to-fuel concepts allows for an upscaling progress to a large scale industrial application. However, due to high investment costs, disadvantageous framing conditions and low prices for fossil fuels the existing pilot projects are, except for small niche markets, not yet economically feasible and financially dependent on funding [75,76]. Based on stakeholder proposals four different options for political decision makers were considered to increase the market share of synthetic fuels and their effects were simulated with the model: (1) Increasing the cost for fossil fuels through the implementation of CO₂ taxes/high EU ETS (European Union Emissions Trading System) certificate prices; (2) less regulatory barriers for operators; (3) stronger subsidization of new projects in the early adoption phase; (4) implementation of strong emission reduction quotas for all industrial sectors. The simulations revealed that even increasing market prices by up to 4% per year for fossil gas and oil hardly create business cases for power-to-fuel operators without any further regulatory interventions in the energy markets. Modifications of regulatory framework conditions in the national energy laws—e.g., an exemption for operators from specific fees and taxes regarding input electricity for their production processes—appeared to be one of the key parameters for larger investments in power-to-fuel projects. However, the creation of future business cases that are independent from subsidization requires strict emission reduction restrictions for industrial companies as this will raise their willingness-to-pay for CO₂-neutral fuels in order to avoid penalties. A slow overall capacity development within the next decades could be the consequence of a lack of strong political climate protection measures.

The regulatory modifications analysed in this study have been part of debates between stakeholders and policy for years [77]. Recent development indicates that political decision makers are considering modifications in the national energy law. However, for a large-scale integration of system innovations such as power-to-fuel and a consequent emission reduction in the overall energy system, it requires additional strong political signals.

Validity: The main strategies for achieving validity were drawing on existing case-specific conceptual work and the involvement of stakeholders to assure the correspondence of the model’s micro-foundation with real world entities. The successful integration of power-to-fuel approaches into energy systems is dependent on a variety of social, technical, economic and ecological influences. Hemmelskamp defines and structures such influencing interactions with the multi-impulse-hypothesis as “impact factors” [78]. This hypothesis was used to identify and characterize important stakeholders that may influence the innovation process of power-to-fuel. Rational choice theory was chosen as a basic theoretical framework. For an explicit characterization of agents, extensive literature reviews were supplemented by expert knowledge gained in a series of interviews and workshops. In a total of three workshops with renowned researchers and industrial company representatives, information of stakeholders from the field was firstly structured in a mind map and the stakeholders were introduced into the method of ABM. Based on the revealed insights and open questions that appeared to be most prominent, a basic model framework was constructed within a scientific workshop. In the third workshop, the model framework was discussed and further adjusted based on new insights from stakeholders once again. Finally, additional information—e.g., on investment strategies was subject to discussions of researchers with individual stakeholders. Those sources highlighted that differentiated customer and operator groups as well as public subsidies were key to answer the research question. The simulations enabled for a quantitative assessment of complex interactions between decision makers and stakeholders who adapt to changing framing conditions. The model’s results were further validated by the model’s ability to replicate the power-to-fuel capacities in the real-case between 2009 and 2016.

Generality: The scope of the model is on a national level with regard to the German energy system. This is a logical consequence of the central model content, which covers a possible expansion of a system innovation such as power-to-fuel. To this end, it was necessary to implement energy and cash flows of energy markets, which extend beyond regional, smaller energy systems. Additionally, all legal decisions influencing the spread of power-to-fuel are implemented on a national level. However, geographic dispersion at the same time is not part of the model environment since local conditions have less impact at this early state of development, and this increases the degree of abstraction of the model. Some technological specification is also kept on an abstract level as, for example, there is no distinction between different types of electrolysers. These generalizations enable a better transferability of the generated knowledge as the information regarding markets and technologies might be easily replaced by new information for other systems.

5. Discussion of Potential and Challenges

Based on the experiences with the four agent-based models described above, we now discuss the potential of ABM to generate transformation knowledge, and the challenges for designing agent-based model studies to achieve this. For doing so, we follow the structure of our analytical framework.

5.1. Explanatory Approach

ABM can improve our understanding of the energy system through providing a formal language to consistently integrate knowledge from various sources and disciplines. In particular, ABM allows for the inclusion of bounded-rational and socially influenced behaviour as well as institutional factors. This fulfills the recent need to integrate primarily quantitative physical understanding of energy systems with more qualitative understanding of the social aspects of these complex systems by incorporating social and institutional elements at the technology-policy-behaviour interface [3].

Simulations performed with agent-based models then facilitate an enhanced understanding of emergent phenomena that are difficult to understand without considering social factors; such as the low insulation rate in case 1 (insulation), and the competitiveness of small regional electricity utilities during market liberalization in case 2 (retail market). This is in line with the findings of Klein et al. [79], who analysed the benefits of ABM in energy system analysis and assessed ABM to be a bridge between scientific fields which allows to relate, for example, decision models from psychology with structures of electricity markets and physical properties of renewable energy technologies. They conclude that ABM makes it possible to draw attention to emergent phenomena such as market disequilibria or system failure that are impossible to study with the more dominant optimization models. Additionally, Hansen et al. [80] in their review of agent-based modelling in the context of socio-technical energy transition studies found that the ability of ABM to address complexity is the most prominent reason for choosing ABM as a research method.

Simulations furthermore allow to assess the relevance of each of a range of potentially important factors through studying their (emergent) effects. For example, in case 2 (feedback products), simulations allowed us to investigate the interplay and the effects of product diffusion and behaviour diffusion. In case 4 (power-to-fuel), interdisciplinary knowledge (social, techno-economic and regulatory impact factors) was firstly gathered in a qualitative model. However, with regard to the studied research question, the individual “magnitude of impact” of those single factors was completely unknown. Being translated all at once into an agent-based model enabled a quantitative evaluation of these impacts on the innovation system and in this way made it possible to provide guiding orientation for the transformation process.

Finally, the impacts of specific policies can be studied. Typically, simulations with agent-based models do not provide exact numbers for policies’ impacts, but they help to understand why certain policies unfold a stronger or different effect than others through making the underlying mechanisms operating in the system visible. For example, in case 1 (insulation) it became apparent that increasing homeowners’ information about and attitude towards wall insulation (alone) has limited effect as homeowners do not enter deliberation about renovation as long as they perceive the condition of their walls to be good enough. In case 4 (power-to-fuel), it could be shown that an increase in industrial companies’ willingness-to-pay for CO₂-neutral fuels through emission reduction restrictions is required to achieve an economic uptake potential of power-to-fuel products that does not depend on subsidies. Klein et al. [79] report similar findings, and propose that, especially in the context of energy transitions, it is indispensable to take the actors’ perspectives into account for evaluating the effectiveness and efficiency of policies. They looked at policies that aim at incentivizing new investments in low-carbon technologies and flexibility options at low costs, and concluded that it is essential to look at actors’ income, stranded assets and the risks actors are exposed to.

5.2. Usefulness

All the presented cases deal with topics that are relevant for the Energiewende and help to fulfill its goals and objectives. Case 1 (insulation), case 3 (feedback products) and case 4 (power-to-fuel) also study and compare the effects of various specific interventions that could be used to steer the Energiewende; while case 2 (retail market) in particular highlights the relevance of a sound systems knowledge to understand (and avoid) unintended side-effects of political measures (in this case, liberalization).

In case 2 (retail market) and case 4 (power-to-fuel) stakeholder workshops were conducted to ensure the match of the model purpose with questions of the stakeholders, and in case 1 (insulation) the result were presented to stakeholders on the local level.

Despite the efforts to ensure the practical relevance of the research, there were difficulties to reach those stakeholders that are in charge to implement the proposed measures. The regulation suggested in case 1 (insulation) to introduce an obligation to insulate walls when moving to a new house, the design of liberalized power markets in case 2 (retail market) as well as the measures suggested in case 4 (power-to-fuel) (e.g., emission restrictions, fees and taxes) all need to be implemented by policy makers on national or even EU levels. Those stakeholders have not been involved in the projects, and therefore their awareness of the project results is likely limited. This is a main drawback of the analysed model studies, regarding their transformative impact.

The main recipients of the results of these projects can be expected to be participants of the respective national discourses, who might pick up project results and make use of them during debates. Thus, project results might influence policy making indirectly through providing arguments to be used in national discourses that precede policy making. In order to be useful for these discourses, it becomes important that key messages of a model study can be communicated and used independent of the model itself; i.e., that core results and mechanisms can be extracted from and comprehended without the model.

The following gives an example of such an abstraction from the model: In case 3 (feedback products) the model highlighted that effects of promoting feedback devices on the city level by the year 2030 are lower than can be expected by simply upscaling effects from single households through multiplication with the number of households in the city. The speed of dissemination of these products can robustly be estimated to be so slow that full penetration is not achieved by 2030. A large number of households already practice energy efficient ventilation behaviour, meaning that no additional effect can be expected in these households. Finally, it can be expected that energy efficient ventilation behaviour would also continue to spread (albeit more slowly) even if the CO₂ traffic light device was not introduced. These insights into the low effect at the city level became apparent through the integrated consideration of different factors in the model but appear to also be comprehensible in the absence of a model. Similar storylines can be derived from the other models. However, this has not been done systematically to communicate model results to essential target groups.

5.3. Validity

The flexibility of ABM facilitates the integration of knowledge from various theoretical and empirical sources, and prevents relevant knowledge from being neglected just because it is not within the frame of the chosen theoretical model, as might be the case for more rigid approaches. For example, in case 1 (insulation), the decision making process of homeowners was specifically designed to capture those characteristics of homeowners that appeared to be most relevant, based on an extensive survey of empirical studies. They are furthermore put in relation with the technical properties of different refurbishment options, a building typology, the current state of refurbishment, spatial structures of cities, and policy measures. In case 4 (power-to-fuel) the model was designed to depict specific interests of different industries which lead to representative investment decisions. A further benefit of the ABM approach for case 4 (power-to-fuel) was the possibility to include technical learning effects which correspond to global installation rates, and to evaluate their feedback on business cases. This was possible through representing all relevant market participants and the related consequences of their actions on other actors and techno-economic variables.

The cases presented above show that the principal uncertainty and subjectivity related to model design that arise from limited knowledge and ambiguity and subjectivity when choosing and operationalizing social science theories cannot be erased. However, it can be reduced through different practical strategies. Strategies to confirm that the model design considers the most relevant aspects include extensive literature surveys (case 1: insulation and case 4: power-to-fuel), discussions of model designs in interdisciplinary project teams (all cases), and stakeholder interviews and workshops (case 2: retail market and case 4: power-to-fuel).

The second main challenge for validity discussed in the analytical framework section arises from the fact that empirically based agent-based models are typically data intensive. In each of the four cases, both sources from the literature and own data collection (e.g., stakeholder dialogues, surveys, living labs) were used to gather the required information. Data availability and quality of the data have had a large influence on the models and their results, and data collection was directly aligned with the design of the models. Despite the intensity of data gathering efforts, some parameters could not be defined precisely for a range of practical reasons. Sensitive data (especially economic data) from companies is often not available, and this lack of important data has to be compensated by assumptions which increase the error of the model results. In some cases, the gathered values for certain model parameters differed dramatically from data source to data source, which makes data more difficult to handle and interpret correctly. In case 4 (power-to-fuel), for example, there was a lack of sound investment figures for implemented technologies. An extensive literature survey showed that specific investment costs differed by up to a factor of 4 for the same technology, depending on the literature source or manufacturer information. In a major model variable, such inconsistencies may dramatically falsify the results. Thus, the modeller has to choose whether a source is more trustworthy than another one or to use average values. However, both choices are accompanied by significant uncertainties. The design of surveys meant to be used in modelling studies presents another particular challenge for ABM research as the model defines data needs but the information retrieved from the collected data may, in turn, influence the model design, which then may lead to further data needs. For example, case 2 (retail market) focused on the importance of social preferences in the context of electricity retail markets and the model design was inspired by ex-post analysis of the survey results of utility customers and their preferences. These examples show that the alignment of data collection and model development becomes of great importance for developing valid models.

The limitations arising from lacking data can (partly) be compensated through sensitivity analysis in which parameter values are varied and the effect of parameter variation on results is examined. In all case studies, some sensitivity analysis was conducted. For example, in case 2 (retail market), a sensitivity analysis was carried out to test the effect of both wholesale market prices and customer preferences on the level and development trends in retail market prices. In case 4 (power-to-fuel), a sensitivity analysis on the impact of future fossil energy prices (which determine demand potentials for alternative fuels but are highly uncertain) was applied to examine their direct influence on market integration potentials.

We conclude that ABM provides the possibility for valid model structures that include the most relevant variables and mechanisms of the represented socio-technical system. However, in order to achieve this, a sound, systematic process of model building and testing, including stakeholder involvement and empirical data collection, is essential.

5.4. Generality

As the cases illustrated, results from ABM studies are often generalizable and transferable to other geographical areas or to similar technologies. In all cases, some mechanisms were identified that are independent of the specific empirical application case (if any was chosen). For example, the mechanism that leads to a low insulation rate in case 1 (insulation) is not specific for the city of Bottrop. Additionally, in some cases, it became clear which parameters need to be adapted when repeating the evaluation at a different location or for a slightly different product. An example is case 3 (feedback products) in which the sensitivity analysis showed that quantitative results are sensitive to the composition of the consumer group, but not to the specific distribution of consumer types in the urban area. Data on the composition of the consumer group, therefore, need to be collected and included in the model for applications of the model to different cities, while a detailed map of locations of consumer types is not required.

It also became clear during the case studies, that the extent to which insights are generalizable beyond the initial case study depends on the causal mechanism operating in the system, and the extent to which the most important explanatory variables and processes are idiosyncratic for the analysed case. This is not known beforehand, as understanding causal mechanisms only develops during the project. However, the ABM method facilitates clarification of the causal mechanisms and, through sensitivity analysis, allows for assessing which insights are transferable to cases beyond the initial case study.

The findings about the generality of results suggest that—although the presented cases focus on the German Energiewende—the insights generated may well be applicable to other European countries or to energy policy on a European level, or even world-wide. In order to assess transferability of insights, the soundness of the assumptions for the most important explanatory variables and processes needs to be checked for the “target” situation.

6. Crucial Aspects for Production of Transformation Knowledge

The above discussion revealed that agent-based models offer high potential as tools for generating transformation knowledge. Their ability to provide a bottom up representation of aspects of the complex socio-technical energy system based on empirical knowledge allows, in turn, for tailoring models towards the inclusion of the empirically observed most important factors, for orienting models towards the questions that really matter for stakeholders, and for understanding policies’ impacts as result of changing actor behaviour. We have seen in the discussion above that, to achieve this, the design of the model must be well aligned with stakeholder involvement and the collection of data. Therefore, we conclude that the design of projects in which agent-based models are developed and used is of major importance to reap their potential.

The four discussed different agent-based models are not representative for all potential applications related to transformations of energy systems, and each model study has its unique merits and limitations. Yet, we consider this number high enough to draw some tentative conclusions on characteristics that are crucial for ABM studies to successfully produce transformation knowledge. Those are summarized in

Table 5. Recommendations for the design of projects with the goal to create transformation knowledge.

Characteristic of Project	Relevance for Creation of Transformation Knowledge
Stakeholder involvement	Well-informed and goal-oriented involvement of stakeholders for definition of useful research goals, data collection, validation, and dissemination of insights to actors in charge
Flexible project design	Adaptation of stakeholder involvement and data collection to enhanced understanding and changing model designs; Adaptation of dissemination strategies to enhanced understanding of key actors
Long project duration	Several loops of stakeholder involvement, data collection and model revision to account for evolving understanding and ensure validity of model and well-directed dissemination

.

6.1. Stakeholder Involvement

The identification of relevant target groups and the communication of project results to those groups is of major importance for the produced knowledge to unfold transformative impact. Ideally, the target groups for policy messages derived by the model exercise should be involved in the project to ensure that the model answers a question that is relevant for these stakeholders and to increase understanding of the model and trust in model results. Stakeholder involvement is also important for the validity of models, as practitioners with systems knowledge are typically indispensable for filling knowledge gaps in the scientific literature and for providing specific information about the target system. It is important to be aware that the specific stakeholders to be included for the different goals might differ; in particular stakeholders who can provide data might differ from those in charge to implement measures.

An important issue for project design, therefore, is the well-informed and goal-oriented involvement of stakeholders. The literature on participatory modelling [38], stakeholder involvement in sustainability science [81], and more generally transdisciplinary approaches [82,83] are helpful starting points to achieve this.

6.2. Flexible Project Design

However, the cases show that planning the involvement of stakeholders during project design is no guarantee for successful production of transformation knowledge, as the measures identified to be appropriate and therefore relevant target groups to be involved may change when knowledge of the case increases. For example, case 1 (insulation) had initially set out to identify means to use networks between homeowners to foster insulation activity. This could have been done by local stakeholders involved in the project. However, increased understanding gained during the course of the project revealed that using networks—e.g., through information campaigns—has limited effects, and other measures (regulations) appear to be more effective. Those measures, however, can only be implemented by stakeholders on the national level (who were not involved in the project). Therefore, the possibility to adapt stakeholder involvement to increased understanding of the system would be ideal, from the modelling perspective.

Applicable data are essential for calibration and validation. A main observation in the case studies is that the understanding of the system develops during the course of the project, and therefore data needs could not be known exactly before or at the beginning of the project. This is the case, in particular, if stakeholders are involved in defining a model’s purpose and therefore its structure. In this case, data collection cannot follow a predetermined design. Similarly, the relevance of stakeholders for knowledge acquisition might change during the course of the project. Flexibility in the project design to adapt data collection and stakeholder involvement to increasing understanding and changing model designs would therefore be ideal from the modeller’s point of view.

A promising but resource-intensive option for model validation and calibration is to couple simulation experiments with real life experiments. Real life experiments conduct interventions in a real system of a limited scope to study its effects [84,85,86,87]. One example would be a publicly funded programme for marketing and evaluating the use of energy consumption changing feedback devices as described in case study 3 (feedback products). In principle, virtual experiments with a simulation model can help to select most promising candidates for interventions in real life experiments, while the results from real life experiments can validate the simulation model, or support its improvement. The merit of real life experiments over other methods of data collection, such as workshops, interviews and surveys, is that data for emergent properties of the complex socio-technical system can be collected. A major challenge is, however, that it takes a considerable amount of time (typically months or years) for the emergent effects to become visible, and long-term monitoring programs are time consuming and expensive beyond the usual project scope. Furthermore, the intervention to be studied is only one out of typically manifold influences on the system, and therefore isolating its specific effect is a challenge. Despite these challenges, we believe that the coupling of simulation experiments with real life experiments warrants further attention and trial.

The assessment of transferability of case study results requires an understanding of the mechanisms operating in the model. This implies that the transferability of results cannot be known exactly before or at the very beginning of the project, as the understanding of causal mechanisms operating in the system is limited at that point of time. It is therefore difficult for project designers to advocate for transferability of their “exemplary case” to other cases when applying for the project. Which target groups beyond the case study area should be targeted during the dissemination of project results is therefore difficult to plan when the project proposal is written. Ideally some flexibility to adapt dissemination activities to the understanding gained during the project should be allowed for in the project design.

6.3. Long Project Duration

Besides an increasing flexibility in project design, the duration of the research project also plays a large role in the context of ABM. Longer lasting projects, starting from 5 years, would have the benefit that stakeholders could be involved in several loops of data collection and model creation, which would help for purposeful design, calibration and validation of these models in line with increasing understanding. Additionally, real-life experiments require much time (see above), but would be very helpful—e.g., if the goal involves refining an already developed agent-based model with data from experiments.

7. Conclusions

We had set out to identify requirements for and characteristics of agent-based model studies that support the generation of transformation knowledge for the Energiewende and for energy transitions. Agent-based models are in principle well suited to produce transformation knowledge. They allow for bottom up representation of the complex socio-technical energy system based on empirical knowledge. This allows for tailoring the model towards inclusion of the empirically observed most important factors, including boundedly rational behaviour of actors, social influence and institutions. The flexibility of the ABM method furthermore facilitates the orientation of model design towards the questions that really matter for stakeholders. The bottom up nature of agent-based models provides the possibility to identify potential policy impacts as emerging from their effects on the behaviour and interactions of actors. The transferability of insights to applications beyond the initial case study depends on the causal mechanisms operating in the system, and how specific these are for the modelled case. The ABM method helps to clarify this.

We conclude that the availability of systems knowledge and data are essential to achieve valid models, but the satisfaction of data needs of agent-based models is typically challenging. Well-informed and goal-oriented involvement of stakeholders and a strong collaboration between data collection and model development is most important to reap the potential of ABM in real-world projects. Therefore, the design of projects in which agent-based models are developed and used is of major importance for the success of modelling exercises. We argue that the “ideal project” involves well-informed and goal-oriented inclusion of stakeholders, and provides flexibility to adapt data collection, the inclusion of particular stakeholders and dissemination strategies for increasing understanding of causal mechanisms operating in the system and changing model designs. Projects with a duration of five years or longer that allow for involvement of (potentially changing) stakeholders in several loops and for conducting real life experiments in interaction with simulation experiments would be most beneficial.

As of yet, research using ABM has not become mainstream, despite its relevance in studying crucial elements, typically underemphasized by the more traditional approaches for energy system modelling. Facing the huge challenges regarding actors and their behaviour, emerging not only from the Energiewende, but also for a sustainable transition in other sectors like mobility, industry or agriculture and the interconnections between them, would justify including calls for ABM in future research programmes.