Living Labs for Public Sector Innovation: An Integrative Literature Review

Abstract

The public administration literature and adjacent fields have devoted increasing attention to living labs as environments and structures enabling the co-creation of public sector innovation. However, living labs remain a somewhat elusive concept and phenomenon, and there is a lack of understanding of its versatile nature. To gain a deeper understanding of the multiple dimensions of living labs, this article provides a review assessing how the environments, methods and outcomes of living labs are addressed in the extant research literature. The findings are drawn together in a model synthesizing how living labs link to public sector innovation, followed by an outline of knowledge gaps and future research avenues.

1. Introduction

The public sector, in both theory and practice, has experimented with new forms of governance, such as networked governance and New Public Governance (Hartley 2005; Osborne 2006). This has implications for the way public sector innovation is understood and organized as an open innovation practice, aiming at bringing citizens, users and external stakeholders into the development of future public services (Sørensen 2016; De Vries et al. 2016; Chen et al. 2019). Currently, the notion of living labs has emerged as an approach to enable such co-creation processes. Living labs as phenomena evolved originally, and mainly, within information and communication technology fields, but have spread into more general applicable areas such as service innovation, urban planning, and more recently, into public welfare services (Niitamo et al. 2006). Living labs are described as experimental settings for public innovation different from the traditional, more controlled, internally driven environments of public innovation (see, for example, Ruijer and Meijer 2020). They are seen as a “collaborative platform for research, development, and experimentation in real-life contexts, based on specific methodologies and tools, and implemented through specific innovation projects and community-building activities” (Gascó 2017, p. 91). They involve users as co-creators of innovation (Schuurman and Tõnurist 2017). Innovation can be understood as the development of something new and its realization in practice (Fuglsang 2010).

The existing literature provides a fruitful basis for understanding the potentials of living labs because it contains theorization and conceptualization of the phenomenon (Schuurman and Tõnurist 2017); it explores the methods and processes of living labs (Dekker et al. 2020); and it reports findings from empirical studies (Criado et al. 2021; McGann et al. 2019; Olejniczak et al. 2020). However, the literature on living labs in a public sector context is still fragmented (Greve et al. 2020), and its linkages to public sector innovation are not well explained. Therefore, a more integrated understanding of the living lab phenomenon in the context of the public sector is needed—especially to better understand the relevant dimensions of living labs and how they add to or enable public value creation through innovation processes.

Systematic and hermeneutical reviews of living labs do exist (e.g., Følstad 2008; Dutilleul et al. 2010; Gascó 2017; Tõnurist et al. 2017; McGann et al. 2018; Hossain et al. 2019; Dekker et al. 2020; Greve et al. 2020). The main foci of the existing reviews have been on the concept, theoretical foundations, possible outcomes, and on living labs as a research approach. These reviews are helpful for advancing research knowledge in this field, although they shed light on different aspects of living labs, leading to a somewhat fragmented knowledge base. As such, there is need for an integrative review addressing how the different aspects of living labs come together, leading to a more holistic understanding of the phenomenon. Moreover, while the literature points to interlinkages between co-creation, innovation and living labs in the public sector, these interlinkages require more thorough exploration. Specifically, there is a need for a deeper understanding of the connections between living labs, co-creation and innovation in the public sector. More knowledge is needed regarding what environments for experimental governance they provide, what methods they apply for the development of service relations, and what public value creation they entail (see also Dekker et al. 2020; Hansen and Fuglsang 2020).

To identify these key elements of living labs in the context of public innovation, and thereby bring clarity to what constitutes a living lab in this specific setting, we have conducted an integrative literature review. Integrative reviews are useful when assessing the literature on a phenomenon that cannot be clearly defined and demarcated (Snyder 2019).

The review reveals what the extant research literature on living labs states about: (1) the environment living labs provide for public innovations; (2) how methods are understood and approached in the context of living labs; and (3) what the intended outcomes and public values of living labs might be in a public sector context.

By analyzing the existing literature on living labs based on these three review questions, we explore which dimensions the research on living labs in the public sector has covered. We draw together the findings in a model depicting the connections between the different dimensions of living labs, and we use this to highlight the knowledge gaps and propose avenues for further research. The review contributes by conceptualizing these dimensions of the living lab construct relevant for public innovation while also outlining dimensions that need to be further explored. The findings from the literature review may provide guidance to public sector organizations and their collaborating partners on how to create environments for the co-creation of innovation through living labs. As such, the main contribution is both the mapping and discussion of relevant dimensions of living labs in a public innovation context and the identification of relevant questions for future research.

The article is structured as follows: first, the review method and process are described; subsequently, a thorough analysis of the living lab literature is presented, structured around the three review questions; finally, knowledge gaps in the existing literature are discussed and future research avenues are proposed.

2. Methodology

To answer the review questions, an integrative literature review was conducted. This is a subcategory of systematic reviews, and closely related to semi-structured reviews (Jesson et al. 2011; Snyder 2019; Torraco 2016). Systematic reviews originate from clinical research in which the object of a review generally can be clearly defined, and exclusion and inclusion criteria of relevant studies are linked to the assessment of the methodological quality of eligible studies. The social sciences tend to deal with topics that are more elusive and eclectic, implying that the systematic review principles from clinical research are not fully applicable. Integrative reviews are a response to this: they follow the principles of systematic reviews when it comes to ensuring systematization and transparency in searches, screening, and assessment processes, but they allow for more flexibility regarding selection and inclusion of literature. They may include and thereby integrate findings from different kinds of studies (qualitative, quantitative and theoretical) to gain a more holistic understanding of a given phenomenon. Thus, integrative reviews are not mainly focused on assessing and systematizing existing knowledge of a topic; they can also aim to develop a better understanding of a concept or phenomenon. This tends to require iterative processes, in which the selection, reading, interpretation and analysis of literature is more intertwined compared to pure systematic reviews. The iterative approach is necessary because the process of systematizing the research literature intersects with the process of understanding and defining the phenomena (Jesson et al. 2011; Snyder 2019; Torraco 2016). This is the case with “living labs” which are an elusive concept, defined and understood differently in different research streams. The review process revealed the need for clearer conceptualizations of living labs, which can help clarify and structure its different dimensions. Based on this rationale, our review includes a mapping of the existing literature, an analysis of how the literature covers different dimensions of living labs, and subsequently identifies research gaps and further research avenues.

2.1. Strategies for Searching, Scanning, and Selecting Literature

The strategy for searching, screening and assessing the literature base followed the systematic approach of the PRISMA reporting checklist (Figure 1), as proposed by Moher et al. (2009). The PRISMA model ensures methodological rigor and transparency, in line with the principles for systematic reviews. However, because we found diverse conceptualizations and understandings of living labs in the literature, an integrative analytical approach was used for the subsequent analysis. This means that besides mapping the literature and identifying research contributions and gaps in the existing literature, the literature is critically reviewed and synthesized to suggest ways to expand the prevailing theoretical foundations.

The publications for the present review were searched in the databases Scopus and Web of Sciences, using the search string ‘living lab’ in the title, abstract and key words. The search included all papers on living labs published until mid-2021. To include the academic literature and current discussions on the living lab phenomenon across contexts and sectors, only publications in English were chosen from peer-reviewed journals. As a result of this first screening, conference papers and grey literature were excluded. Interestingly, it turned out that only 25 publications were published before 2010, and most of them were in English. From this initial research, 668 publications were identified. To ensure broad coverage and to validate the identification of key publications in the field, an additional search was conducted in Google Scholar. This led to another 914 publications. After removing duplicates, a total of 1619 publications were included in the screening based on the pre-selected eligibility criteria by reading the full abstract and, in some cases, skimming the full text.

Searching for and selecting literature involved some demarcation challenges because the concept of living labs intersects with interrelated concepts such as policy labs and innovation labs. When reviewing the literature, we found that these are terms which tend to refer to similar kinds of phenomena even though differences between the concepts are also discussed (see, for instance, Schuurman and Tõnurist 2017). Thus, a broadened search string including intersecting terms such as policy and innovation labs allowed us to strengthen the review by obtaining an overview of the different kinds of ‘lab’ constructs in the public sector, and hence validate our more demarcated review, focusing explicitly on living labs—this enabled us to explore this specific concept more in depth. We found this strategy to be purposeful given the complexity and ambivalence of the living lab concept. However, we also acknowledge the need to understand the living labs concept in relation to intersecting terms. We return to reflections on this in the concluding section.

2.2. Eligibility Criteria

The studies from the search were included in the integrative review when they met all the following inclusion criteria: empirical publications and/or conceptual publications on living labs as specific approaches to innovation; publications concerned with citizen and/or public servant engagement; and public service innovation targeting development of public services. The main exclusion criteria were studies on living labs that did not target public service and public value explicitly or implicitly (e.g., living labs in a purely private context), and studies on living labs or innovation labs that were not based on citizen or public servant participation.

Hence, publications were excluded if citizens or public servants were not actively participating in public service innovation processes and/or if the studies were anchored in engineering and technical perspectives. The review process revealed that living labs as innovation approaches are applied in varied public sector contexts; hence, there seemed to be no most common ‘subject’ area. Thus, based on the eligibility criteria, a final number of 59 publications were included in the qualitative analysis. As such, the final, quite limited, number of publications reflect that living lab as a concept within public sector innovation research and practice is still only emerging. The selection process is presented in Figure 1.

2.3. Analysis and Synthesis Strategy

To ensure consistency in the reporting, data extraction sheets for the 59 publications were developed to note the author(s) and title for each article, publication year, country, method, main theme, main concepts used, definition of living lab, geographical context, and sector context. In addition, the main content of each scientific paper, its methodology and findings, themes, and key contribution to the field were documented. Based on the extraction sheet, the articles were first categorized individually by two scholars and subsequently discussed within a group of nine scholars. The overall content of the publications was initially organized into seven major themes: living lab definitions, living lab frameworks, practices of living lab organization, interaction with users/citizens, innovation as a process or outcome, living lab methodologies, and assessment and legitimacy of living labs. This gave a preliminary overview of the main themes of the publications.

In order to develop a more integrated framework, we organized it into a conceptual model that described the contribution of living labs in terms of: (1) an environment for innovation, i.e., the arenas for empowered participation that living labs create for innovation; (2) methodology, i.e., the methodologies of innovation applied through living labs; and (3) intended outcomes (effects) of living labs in terms of public value. This conceptualization helped us to explore how living labs affect the innovation process. In traditional models of public innovation within Traditional Public Administration or New Public Management (Hartley 2005), innovation is an internally driven, top-down approach targeting internal efficiency. However, we assumed that living labs broaden the environment, engage external stakeholders and target a more differentiated set of outcomes. This rearranges the original seven themes into a more condensed model. Developing models or frameworks that provide more holistic understandings in an emerging field is generally the purpose of reviews that address relatively new topics (Snyder 2019; Torraco 2016).

3. Results

The results are based upon and structured around the three review questions of the study, concerned with: living labs as organizational environments; methodologies of living labs; and public values co-created in living labs.

3.1. Living Labs as Environments

The publications describe living lab as not one, but several types of innovation environments that can be organized with different degrees of ‘realism’ and different types of ‘interaction’ with users. In terms of realism, the environments can be: (1) semi-realistic; (2) real-life environments; or (3) platform/network environments. In terms of interaction with users, living lab environments may entail: (1) observing users; (2) co-creating innovations with users and other stakeholders; (3) co-researching with users and stakeholders; and (4) democratizing innovation. Interaction with users can be indirect, direct, or beyond the user perspective (i.e., other stakeholders). The living lab environments are described as experimental and structured environments for innovation; thus, one type of environment is not, by definition, more structured or formal than the other. Each publication often critically discusses several of these living lab environments in order to document strengths and weaknesses.

Table 1. Types of living lab environments—examples from the literature.

Realism of Environment Interaction with Users	Semi-Realistic Environment	Real-Life Environment	Network and Platforms
1. Observing users	“Living Labs are environments for innovation and development where users are exposed to new ICT solutions in (semi)realistic contexts” (Følstad 2008).	“…in vivo monitoring of a ‘living’ social setting generally involving experimentation of a technology” (Dutilleul et al. 2010).	“Living labs typically refer to co-creation and appropriation of innovations by users, often in a (online or offline) community setting, and also involving business stakeholder” (Ballon and Schuurman 2015).
2. Co-creating innovation with users and stakeholders	“Innovation labs have thus far focused on the ideation and genesis stage of innovation, and then let go of the project afterwards (Schuurman and Tõnurist 2017).	“Living labs are driven by two main ideas: (1) involving users as co-creators of innovation outcomes on equal grounds with the rest of participants and (2) experimentation in real-world settings” (Gascó 2017).	“Living labs are both a physical space where, and a methodology through which, stakeholders, particularly users, participate in the development, testing and evaluation of a product or a service assisted by experts, using an open-driven approach to innovation” (Nesti 2017).
3. Co-researching with users and stakeholders	“MindLab’s way of working is based on the laboratory idea, where new methods and approaches to strengthen citizen involvement—where possible across the three Ministries—are examined and the applicability of potential solutions is tested and developed” (Carstensen and Bason 2012).	“Living labs are a research and design methodology applied by research institutes in cooperation with public and private partners for developing and testing innovations in co-creation with users in real-life settings” (Dekker et al. 2020).	“The concept of Living Lab can be interpreted and used as a human-centric research and development approach in which IC innovations are co-created, tested, and evaluated in open, collaborative, multi-contextual real-world settings” (Ståhlbröst 2008).
4. Democratizing innovation	---	---	“…an open innovation milieu where new constellations, issues and ideas evolve from bottom-up long-term collaborations amongst diverse stakeholders” (Björgvinsson et al. 2012).

provides examples of how these approaches are combined in the reviewed publications.

Below, the three types of living environments described in the literature are depicted, including the type of interactions with users and stakeholders these environments imply.

3.1.1. Semi-Realistic Environments

In the early literature, living labs were defined as artificial homes where visitors were exposed to new technology and their reactions were observed by researchers (Eriksson et al. 2005; Grisenti et al. 2021; Nesti 2017). Some publications describe these living labs as ‘showrooms’ or ‘testbeds’ (cf. Følstad 2008). The users were mainly passive recipients of innovations.

A different perspective of semi-realistic environment is expressed in publications on ‘innovation labs’ (Carstensen and Bason 2012; Schuurman and Tõnurist 2017). These are described as relying on expert innovation teams who explore opportunities for innovation by consulting with users and stakeholders. Innovation labs are seen as safe spaces at a distance from (but still close to) everyday routines; thus, operational priorities can be somewhat relaxed, and more risks can be taken (Carstensen and Bason 2012). The purpose of such labs can also be to co-research innovation with users.

3.1.2. Real-Life Environment

Many authors refer to living labs as real-life environments (Gascó 2017; Eriksson et al. 2005; Ståhlbröst 2008). Living labs are “offering the most realistic environment possible to allow ‘sense-making’ processes to take place through experiential learning leading to better understanding of product/service adoption behaviours by users” (Lehmann et al. 2015, p. 1093). There seems to be wide-ranging agreement in the literature that living labs can be understood as sites that are reminders of real-life settings, which refers to the ‘living’ in the living lab (Schuurman and Tõnurist 2017; Følstad 2008); living labs turn real-life settings into experimental sites (Ballon and Schuurman 2015). The purpose of these real-life settings can be to explore opportunities for co-creation with end users through direct interaction with users (Eriksson et al. 2005; Gascó 2017). Real-life settings can involve a vision of meeting users or citizens on “equal grounds” (Gascó 2017). Users and user groups can influence innovations through testing/evaluating new products and services, or at the strategic level through co-designing new government and community systems, or new business models for innovation (Lehmann et al. 2015).

A real-life environment is also conceived as a research and design context where innovation experts develop and test innovations together with users through direct interaction (Dekker et al. 2020). In this case, living labs generate actionable and situated knowledge, where the researchers are the main actors. Users help to evaluate innovations (Dekker et al. 2020).

3.1.3. Networks and Platforms

The literature also describes living labs as networks or platform settings for innovation. They derive their realism from focusing on systemic aspects of innovation beyond the perspective of single users. The distinction between living labs as real-life environments and as networks and platforms is blurred; however, there is a tendency in the literature to increasingly stress the ecosystem context of innovation beyond the user perspective, i.e., a living lab is not an isolated space disconnected from its environment. This means that the two forms are often described as combined. Gascó (2017) stresses that living labs are collaborative platforms for research, development, and experimentation. Leminen et al. (2016) argue that a new generation type of living lab can be identified that takes a broader perspective on living labs as structures for collaborative innovation, moving emphasis from real-life environments to networks or platforms that involve many stakeholders.

Furthermore, different publications also describe living labs as an ecosystem environment (Gascó 2017), open innovation networks (Leminen et al. 2012), public open innovation intermediaries (Gascó 2017; Hernández-Pérez et al. 2020), knowledge systems (Lehmann et al. 2015), and multi-stakeholder organizations (Schuurman and Tõnurist 2017). These publications have the common notion of living labs as moving beyond the direct interaction with users. Another take on living labs as broader platforms is that they enable citizens to gain more democratic control with innovation (Björgvinsson et al. 2012).

In summary, the reviewed literature reveals that there is not one but at least three living lab environments in the literature. These imply varied types of direct and indirect interactions with users, or direct interaction beyond the interaction with users. Most of the publications in the sample stress that living labs are real-life environments for direct interactions with users, but an increasing number of publications include direct interactions with other stakeholders as well.

4. Methodologies of Living Labs

The issue of methods was approached in various ways, and the reviewed literature could be divided into four broad camps. First, there are publications that take a meta-perspective on the phenomenon of living labs, which conceptualize, define, theorize and develop typologies of living labs in relation to interlinked concepts and methods (e.g., Cossetta and Palumbo 2014). Secondly, there are empirical studies of concrete living labs, which are based on surveys, interview studies and case studies (e.g., Gascó 2017). Such publications analyze and understand living labs from an outside perspective or a reflexive stance, and seek to report on their structures, activities, methods, etc. In these studies, the methods for studying living labs are separated from the methods of living labs. A third category of studies is based on the use of living labs to carry out empirical studies and experiment with innovation (e.g., Baccarne et al. 2016). In these studies, there are no clear distinctions between research methods and the methods of the living labs; the purpose is not necessarily to produce knowledge about the nature or character of living labs, but to use living labs as research sites to produce innovation and new knowledge. Thus, these studies report primarily on the methods and findings of the phenomena under scrutiny, but they may also generate knowledge on living labs by conveying how living labs work as methods for certain research-based decisions on innovation. Finally, parts of the living lab literature seek to model living labs and their methods by outlining the components that living labs ought to comprise (e.g., Liedtke et al. 2012).

Table 2. Overview of how methodologies of living labs are addressed and understood in the literature.

Research Approaches to Living Labs	Understandings of Living Lab Methods in the Literature	Examples from the Reviewed Literature
1. Living lab methods through theorization and conceptualizations	Living labs are perceived as an overarching concept covering different kinds of user-centered research methods or methods for co-creating decisions with citizens and other stakeholders. They are also perceived as a specific approach or step within broader (innovation) methodologies.	Schuurman and Tõnurist (2017) Dutilleul et al. (2010) Leminen and Westerlund (2017) Hansen and Fuglsang (2020)
2. Living lab methods assessed through empirical research (research on living labs)	Reviews of empirical studies of living labs show that living labs are eclectic phenomena referring to a wide range of activities, processes and methods for making decisions about innovation. A shared trait in the empirical research is that living labs are somehow perceived as providing models for various forms of co-creation for innovation.	Dell’Era and Landoni (2014) Kanstrup (2017) Gascó (2017) Leminen et al. (2012)
3. Living labs as research methods (research in living labs)	Research set in living labs, or which uses living lab research methods, perceives living labs as approaches to research and innovation with the involvement of users and citizens, or as a research infrastructure for user-centered research involving sensing, prototyping, and validating complex solutions in real-life contexts.	Buhr et al. (2016) Edwards-Schachter et al. (2012) Keijzer-Broers et al. (2015) Dekker et al. (2020)
4. Modelling living labs (presenting methods and models for living labs)	Reflects varied understandings of living lab methods, such as an approach to collaborative/participatory governance and infrastructure for testing, experiments and research in real or semi-real environments. Methods may be outlined as vague guidelines or more specified protocols.	Liedtke et al. (2012) Reiter et al. (2014) Gatta et al. (2017) Gago and Rubalcaba (2020) Grisenti et al. (2021)

gives an overview of these different understandings of living lab methodologies found in the literature.

4.1. Multi-Stakeholder Approach as Method

An important aspect of a living lab methodology is how the interaction and collaboration between stakeholders is organized, which is evident in cases depicted by partnerships (Edwards-Schachter et al. 2012; Nyström et al. 2014; Äyväri and Jyrämä 2017; Schliwa and McCormick 2016; Gascó 2017). In some publications, this is conceptualized as ‘the four Ps’, i.e., a public–private–people partnership (Edwards-Schachter et al. 2012; Veeckman et al. 2013), whereas others refer to the quadruple/quintuple helix model of open innovation (Haider et al. 2016; Cossetta and Palumbo 2014; Baccarne et al. 2016; Keijzer-Broers et al. 2015). Finally, some authors mainly refer to partnerships as cross-sectorial collaboration (Gatta et al. 2017; Nesti 2017). In summary, most living lab cases rely on methods to orchestrate and facilitate interaction between business, research, public administration and civil society/users.

There are differing perceptions of how the user or citizen is engaged in living lab activities: on the one hand, the literature emphasizes engagement through democratic ideals (Björgvinsson et al. 2012; Cardullo et al. 2018); on the other hand, there is a more individualistic view that emphasizes the expression of subjective needs (Äyväri and Jyrämä 2017; Edwards-Schachter et al. 2012). Dell’Era and Landoni (2014) refer to these different approaches as either an expert mindset or a participatory mindset.

4.2. Design Methods and Living Lab Research

Some publications pair the living lab methodology with design approaches such as human-centered design (Reiter et al. 2014), whereas others see design methods such as design thinking and participatory design as the core methods applied within a living lab structure (Poldma et al. 2014; Björgvinsson et al. 2012). Consequently, process models for living labs and innovation lab projects intersect with process models for design thinking (Carstensen and Bason 2012). Overall, examining living lab methods shows that these are largely anchored in different design methodologies and a myriad of concepts are used interchangeably. There are some efforts to bring about more clarity, for instance, by mapping the domain landscape of a living lab in relation to diverse design approaches (Pallot et al. 2010). However, because living lab research and practices operate in highly interdisciplinary or even transdisciplinary environments, it may be difficult to agree on a clear shared vocabulary.

In summary, a key point that prevails across the literature is that almost no living lab uses a single method or has developed radically new methods for co-creation and user involvement. On the contrary, it is emphasized that the ‘newness’ of living labs lies in the transdisciplinary approach, i.e., the openness toward methods and the way they are used in combination with practical and professional knowledge as tools of innovations. Hence, it is essentially the eclecticism that forms a defining trait of living lab methods. Moreover, living labs are based in underlying democratic ideals, framed, among others, as co-creation (Carstensen and Bason 2012; Buhr et al. 2016; Hakkarainen and Hyysalo 2016).

5. Outcomes and Value of Living Labs

The third review question focuses on the types of values that are developed by living labs in the public sector. The process or application of living lab methods can in itself generate public value, for example, by being more democratic or by strengthening a feeling of belonging to a community. Public value is generally defined by Moore (1995) as the contributions which public managers make, especially in the ways they apply entrepreneurial activities to contribute to society or the public in general. In the review of the existing living lab publications, four types of public values were identified: (1) administrative values that focus on the improvement of administrative processes (Alford and O’Flynn 2009); (2) citizen values that aim to improve the relationship between public administrations and citizens (see, for example, Bryson et al. 2014); (3) societal values that improve transparency, accountability and responsibility for the sake of the larger society (see, for example, Jørgensen and Bozeman 2007; Stoker 2006); and (4) economic values that improve how public administrations deliver services, save costs, and generally become more efficient and effective (see, for example, O’Flynn 2007). Each will be discussed in turn.

5.1. Administrative Values

Administrative values include those that support the improvement of administrative processes, reduce administrative burden and generally contribute to a more responsive and agile public administration. One of the main administrative values of living labs is learning about and trying out new modes of work practices that would be too disruptive to test during real-life operations of an organization and can here be tested in a safe lab environment (Tõnurist et al. 2017). This includes, for example, testing technological solutions in living labs where experimentation is allowed (Følstad 2008; Kanstrup 2017). This can lead to the initiation or fostering of organizational change (Carstensen and Bason 2012; Schuurman and Tõnurist 2017) which would not have been possible without the experiments and trials performed in the living lab context. The results are easier to demonstrate to top management and will lead to a higher top-management backing than trial-and-error tests in live operations. Processes and services can therefore be tested and improved, and might then add administrative value by changing work practices.

In living labs, public administrations can experiment with new forms of collaborative processes (Meijer and Bolivar 2015; Niitamo et al. 2006; Windeløw-Lidzélius 2018; Leminen et al. 2016; Dezuanni et al. 2018; Baccarne et al. 2016; Leminen et al. 2012; Leminen and Westerlund 2017). This might involve networking with otherwise disconnected internal peers or external stakeholders. As an example, Gascó (2017) shows how living labs serve as intermediaries between different sectors. The focal organization gains access to users and can involve them at the frontend of an innovation acquisition (Salminen and Konsti-Laaks 2010). Networking with different types of stakeholders then leads to new administrative knowledge, and public sector organization might have additional opportunities to attract human capital who might have otherwise not have seen the public sector as an attractive employer (Meijer and Bolivar 2015).

5.2. Citizen Values

The second type of public values that living labs contribute are so-called citizen values. These are values that support the relationship between public administrations and one of their core constituents: citizens. Given the administrative approaches discussed in the previous section, one characteristic is predominantly present in living approaches: user- or citizen-centricity. For the design of any public service or product, the main consideration is not the product itself, but the users and their needs (Redström 2006). Through methods such as citizen participation elements, the living lab steps away from traditional policy- or administration-centric approaches and starts to design processes and products from a citizen or user viewpoint.

Several authors point to citizen-centricity as a major value of the approaches used in living labs (Bergvall-Kåreborn and Ståhlbröst 2009; Almirall et al. 2012; Kanstrup 2017; Lehmann et al. 2015; Dell’Era and Landoni 2014; Schliwa and McCormick 2016). Hence, citizen-centricity is not just a method for designing specific services; it can be considered a mindset in itself, supports the changing relationship with citizens, and therefore becomes a value in itself. As a result, the public value is the creation of knowledge about citizen needs by bringing citizens into the organization and learning from them. During the experiential phases, knowledge from citizens and their needs is extracted and increases the knowledge base of the organization. In addition, it changes the relationships with citizens by creating a new form of connection which is built on partnership, balances out distance between government and citizens, and might, in turn, lead to higher levels of trust as a public value. Other authors in this space label these as co-production processes (Nesti 2017) or service-dominant logic (Äyväri and Jyrämä 2017). Citizen centricity also contributes to additional public values, such as user-driven innovation (Cossetta and Palumbo 2014), increased empathy for citizen values through participatory methods (Björgvinsson et al. 2012), and increased inclusiveness (Martinez et al. 2016). These values might lead to higher citizen satisfaction and lower numbers of complaints or dissatisfaction with government—if they were to be measured.

However, there is also substantial criticism for these approaches, as stated by Cardullo et al. (2018): being citizen-led or citizen-engaged in a living lab does not necessarily confer notions of citizenship, rights to the digital city, or guarantee new digital urban commons. Additional research is therefore necessary to dive deeper into the measurement of the outcomes of living labs to understand whether they are indeed contributing to the production of citizen values.

5.3. Societal Values

Living labs are, by definition, designed to create public sector innovation as their main contribution to society. Public innovation itself is a relatively fuzzy term, although it has a positive connotation and is therefore assumed to create public and societal value (Dekker et al. 2020; Følstad 2008; Evans et al. 2015).

Additional societal values include the increase in democratic and societal values (Ståhlbröst and Holst 2017) by allowing stakeholders access to government, by being more inclusive, and by valuing external input by non-experts. Therefore, some studies also see living labs as tools for the democratization of innovation. Björgvinsson et al. (2012) stress how living labs enable different voices to be heard in struggles and conflicts about technological change and social development.

Other authors see societal value in the potential of living labs to develop disruptive innovations, especially in public health (Hesseldal and Kayser 2016). Government-led living labs are independent of market forces and can therefore close the gap between competing government and market interests to build solutions in areas where the market itself might fail.

However, there are also critical voices which show that better understandings of social needs are necessary in order to produce more direct societal value (Franz 2015).

5.4. Economic Values

Economic values produced by living labs are rarely identifiable and are also rarely reported in the reviewed literature. However, it is anticipated that some of the practices introduced by living labs might lead to increased effectiveness and efficiency compared to previous approaches toward innovation creation.

However, some of the literature hints at possibilities of economic value creation. Living labs are set up by definition to tackle wicked problems. These are societal issues that cannot be addressed by a single actor, are too complex to solve with the traditional means, or cannot be efficiently solved by the market. For example, Steen and Van Bueren (2017) evaluated 90 sustainable urban innovation projects in Amsterdam, and showed how otherwise wicked global problems can be solved together in these urban living lab arrangements.

Others focus on the creation of new products and services that, in turn, can decrease costs or even increase revenue. What this might look like is highlighted by Hakkarainen and Hyysalo (2016), who point to the fact that new products and processes are developed. Using open innovation approaches and bringing together otherwise disconnected actors leads to innovations that, by definition, are superior to previous products or processes. The authors show that otherwise undetected user needs are uncovered through the methods applied in living labs and in the open innovation process, which then lead to new processes and products. Similarly, Angelini et al. (2016) showed how in the interplay between different types of actors, heterogeneous knowledge is combined to create new products and processes.

The following table (

Table 3. Overview of public values co-created by living labs.

Types of Public Values	Public Values Co-Created by Living Labs	Examples from the Reviewed Literature
1. Administrative values	Improved administrative processes through safe experimentation Access to otherwise inaccessible knowledge New forms of collaboration	Baccarne et al. (2016); Dezuanni et al. (2018); Leminen et al. (2016); Leminen et al. (2012); Leminen and Westerlund (2017); Meijer and Bolivar (2015); Niitamo et al. (2006); Windeløw-Lidzélius (2018); (Tõnurist et al. 2017)
2. Citizen values	Citizen centricity Increased empathy for citizen needs Increase in inclusiveness and access to public services Citizen satisfaction Improved relationship between government and citizens Increase in public trust	Bergvall-Kåreborn and Ståhlbröst (2009); Almirall et al. (2012); Cardullo et al. (2018); Dekker et al. (2021); Kanstrup (2017); Lehmann et al. (2015); Dell’Era and Landoni (2014); Redström (2006); Schliwa and McCormick (2016)
3. Societal values	Disruptive public sector innovation Democratization of public sector innovation	Björgvinsson et al. (2012); Criado et al. (2021); Dekker et al. (2020); Følstad (2008); Evans et al. (2015); Franz (2015); Hesseldal and Kayser (2016); Ståhlbröst and Holst (2017)
4. Economic value	Cost savings that are otherwise unfeasible New product and process developments that are more effective and efficient Solving wicked societal problems which are otherwise not solvable	Steen and Van Bueren (2017); Ruijer and Meijer (2020); Hakkarainen and Hyysalo (2016); Angelini et al. (2016)

) synthesizes the types of public values created by living labs.

In summary, living labs have the potential to generate different types of public values. Here, we have identified four types of public values which were derived from the environment, methods, aims and goals mentioned in the living labs literature. There is room to measure and thereby specify the types of values that are actually generated. At this stage, measurement, especially, is rarely mentioned in the literature, and—similarly to the innovation literature—the mere presence of living labs is considered a positive feature to generate public value.

6. Integrative Findings

The research questions for the article were the following: (1) What kinds of environments do living labs provide for public innovations?; (2) How are methods understood and applied in the context of living labs?; and (3) What are the intended outcomes and/or values of direct innovation in living labs in a public sector context?

The questions guiding the review were based on a basic understanding of innovation as something that emanates from an environment, uses methods and entails certain intended outcomes (Fuglsang 2010). This is a simple way of describing actor-to-actor relationships, practices, and purposes of innovation, and stems from the idea that public innovation is an interactive and collaborative process (Torfing 2019), that day-to-day practices of problem-solving relate to systemic aspects of innovation (Bloch and Bugge 2013), and that public innovation often has a normative content because public managers want to ‘do things better’ (Arundel et al. 2019).

Public innovation traditionally has been driven by politicians and policymakers and has been implemented through internal processes, often with the purpose of internal efficiency in mind, whereas our review shows that the specific form of public sector innovation in the shape of living labs provides new foundations for innovation. Specifically, by assessing the environments, methods, and outcomes of living labs, we suggest that they contribute to broadening the space, expanding on methods and differentiating the outcomes of innovation. We conceptualize these dimensions of living labs and their interconnections in Figure 2 below, followed by a brief description of each dimension.

6.1. Broadening the Space

The literature review revealed that living labs constitute different kinds of environments for activities, such as ideation, experimentation and testing. The environments of living labs are described as constituting a semi-realistic context, a real-life context and a network/platform context. In addition, interactions in the context of living labs are described as enacted in different forms, such as observing users, co-creating with users and stakeholders, co-researching with users and of democratizing innovation. As such, the differing environments and organizational setups can be seen as specific forms of inclusive arenas for input to public sector innovation processes in which citizens and other stakeholders are brought together in new arenas. Living labs thus recognize the need to accommodate a broader actor-to-actor approach to innovation, ranging from a relatively narrow approach to an open eco-system approach with the involvement of a broad range of stakeholders. Interaction with users can be indirect, direct, and beyond the perspective of the single users through engaging many stakeholders in interactive processes of innovation. This lays the groundwork for open and interactive settings of public sector innovation; living labs may, as such, challenge the more internal and closed innovation processes characterizing the conventional innovation model in the public sector.

6.2. Expanding on Methods

This review shows that living labs contribute to the expansion of the repertoire of methods applied for renewal and innovation in the public sector. However, it is difficult to capture precisely what living lab methods comprise, and we have shown through the literature research that the issue of methods is addressed differently. The identified vagueness regarding living lab methods is not necessarily problematic. On the contrary, if the methods of living labs could be clearly defined and described, living labs would not be providing spaces that enable playful and experimental approaches to public sector innovation. The creativity and innovativeness of living labs therefore lies in their ability to continuously experiment with and develop methodologies, which implies that the issue of methodology remains somewhat fluid and vague.

Nevertheless, the methods of living labs can still be studied and analyzed, and there is need for research assessing the role and outcomes of different kinds of methods. Existing living lab research does this, showing, for instance, how methods of living labs involve means for ensuring the involvement of citizens, service users, and other stakeholders at the micro level. Moreover, the literature shows that the methods of living labs are closely intertwined with design methodologies such as human-centered design approaches, design thinking, participatory design and service design. In summary, this shows that living labs provide methodological means that enable more citizen-centered, open, and bottom-up approaches to public sector innovation. However, there are still knowledge gaps and needs for further research on the methods of living labs, which we reflect on in the concluding discussion below.

6.3. Differentiating the Outcomes

We have identified four different types of public values as potential outcomes produced by living labs through the application of experimental methods: administrative, citizen, societal and economic values. Each of these values can be either tangible or intangible. Tangible public values are easy to measure, such as the number of new services designed with user needs in mind or the value of costs reduced because of innovative practices introduced by living labs. Intangible outcomes are much more difficult to measure, or even to identify. They need to be derived from the types of goals and aims, as well as the types of interactions between different stakeholders. These therefore result in indirect outcomes, such as perceived citizen satisfaction or increased transparency. Such types of outcomes are usually assumed, but rarely measured.

To summarize, Figure 2, which reflects a simplified model of innovation dependent on actor-to-actor relationships, innovation practices and intended outcomes helps us to conceptualize how living labs connect to changing premises of innovation in a public context. Living labs set the stage for new approaches to public innovation by changing actor-to-actor relationships, by expanding on the repertoire of methods, and by addressing outcomes in new ways. They disrupt traditional top-down, internally driven and efficiency-oriented innovation practices, and seek to replace them with inclusive, experimental and iterative approaches to innovation. This also involves approaching outcomes of living labs along a broader set of values, or value dimensions.

7. Concluding Discussion: Knowledge Gaps and Future Research

Our analysis of the literature on living labs indicates that living labs contribute to broadening the space for innovation and experimentation, expanding on the repertoire of methods used, and providing means to rethink and differentiate between various forms of outcomes of public sector innovation. Living labs seem to disrupt traditional forms of public innovation by giving vastly more attention to varied forms of co-creation of innovation, especially by the direct or indirect involvement of users and other stakeholders in experimental forms of innovation. Thus, these changes are significant, and living labs entail a deeper involvement of managers, employees and users with experimental innovation activities, i.e., an increased willingness to invest resources in innovation activities, as well as an ability to develop and distribute certain innovation tasks across actors and extracting value from them. Adding these activities to everyday practice may create a more challenging environment for public managers and employees with more tasks in a more experimental setting, although these living lab activities may also be constrained by extant institutional practices and values (Nesti 2017; Tõnurist et al. 2017).

As such, living labs seem to contribute a renewal of structures and models for innovation in the public sector. However, the various forms of expansion that living labs entail can also be described in terms of a heightened institutional complexity (Greenwood et al. 2011) because employees must handle demands stemming from different government paradigms. Living labs can be seen as positioned within New Public Governance (NPG), or networked governance, as indicated in the Introduction, and public sector organizations also adhere to traditional public administration, New Public Management (NPM), and other post-NPM paradigms (Christensen and Lægreid 2011). This complexity indicates that those participating in or managing living labs are likely to face contradictory demands such as adhering to traditional hierarchies and line management of PSOs and engage in networks and horizontal structures at the same time. Thus, although living labs entail various forms of expansion of innovation models in the public sector, this expansion is not without dilemmas when it comes to the daily practices and operations of living labs. Consequently, the literature review through illuminating such tensions draws attention to the importance of managing living labs carefully in relation to various tensions and complexities (Smith et al. 2017).

Moreover, although the findings of this review indicate that living labs represent new ways of approaching innovation in the public sector, we found that the different dimensions of living labs have been unequally attended to in the research literature. Thus, there are still considerable knowledge gaps in this field. We discuss these gaps and suggest directions for future research that might contribute to closing these gaps.

The literature on living labs suffers largely from fragmentation and lack of conceptual clarity. The lack of clarity is a doubled-edged sword. On the one hand, there is a need to maintain the playfulness and creativity of living labs so as to experiment with environments, methods and outcomes of innovation relevant to public innovation. On the other hand, when there is no single accepted definition of living labs, and the literature does not clarify the boundaries of living labs vis-à-vis other structures and models, it seems challenging to develop and profile the approach in ways that bring about substantial change (Van Der Sloot and Lanzing 2021). Thus, there is a risk that the term ‘living lab’ is used as a cosmetic label for initiatives that lack substance and fail to adhere to the visions of living labs. In this manner, living labs have to manage a paradox of being experimental, creative and playful on the one hand, and strive for some conceptual clarifications in order to institutionalize an approach and more clearly relate to intersecting concepts and phenomena on the other hand.

Our review contributes to this by examining what the existing literature can tell us about the environments, methods and outcomes (values) of living labs. As a result, we provide a holistic understanding of living labs in the public sector by addressing these three crucial and interconnected dimensions. However, this research also identified knowledge gaps related to the three dimensions. We discuss these knowledge gaps next and outline how they provide avenues for future research.

Our review shows that living labs create environments or spaces that support new ways of working with learning, experimentation, and innovation in the public sector. However, living labs still represent a relatively new phenomenon, and extant research shows mainly how living labs provide potential new environments for innovation. It is not clear how and whether living labs manifest as visible environments, and we do not know whether these environments sustain over time or if they disseminate to a broader range of public sector settings. Therefore, there is need for research examining how living lab environments are created and recreated, because living labs only exist as environments for innovation if they are continuously recreated by people.

We were not able to identify how the methods and outcomes of living labs could be transferred into routines or standard operating procedures of public administrations. This is an especially large knowledge gap, given that employees and managers in the public sector tend to be torn between a range of obligations and commitments related to their regular jobs, which compete with expectations to participate in innovation processes fostered through living labs. Thus, understanding the conditions for living labs’ sustainability and the transferability of practices into the routines of public administrations will require new research into what may hinder people from participating, such as time pressures, occupational identity, bureaucratic structures, and administrative procedures.

Similarly, the interest of citizens, service users, and other stakeholders to take part in co-creating innovations through living labs may fade over time if they cannot identify added value or tangible outcomes of their engagement and contributions. Although an increasing number of private firms and public sector agencies are focusing on co-creation and open innovation processes, there is also the increasing competition of people’s attention and participation. Living labs represent a relatively new and refreshing way of facilitating co-creation which may attract participation, but we need to know whether this will sustain over time. Thus, studies examining the conditions for creation and re-creation of living labs as vibrant environments for innovation in the public sector represent an important avenue for future research. This should include studies of institutional designs and leadership approaches that support the methods and approaches of living labs to sustain and stay relevant over time.

Next, although the existing literature provides interesting insights on the methods of living labs, the literature is also discordant because it approaches living lab methods from very different vantage points (for an overview, see Table 2). This is, in some ways, a strength because it elicits different kinds of insights on living lab methods. As such, we found that the method dimension, especially, is quite well covered in the existing research. On the one hand, there is research that systemizes and categorizes living lab methods at a relatively high level of abstraction (see, for instance, Leminen and Westerlund 2017). On the other hand, there is research depicting the more detailed practices involved in applying living lab methods in specific case contexts, which provide a closer and more practice-oriented understanding of living lab methods (see, for instance, Keijzer-Broers et al. 2015; Buhr et al. 2016). There is also more prescriptive literature, which tries to model living lab methods. Despite this varied and relatively rich insights on methods that we found in the literature, a major unresolved issue is how living lab methods relate to research and researcher roles. This review touched on this issue by differentiating between the different ways researchers address living lab methods, although this is a topic that deserves more thorough and careful attention. Some studies assume living labs to be a social science methodology to be applied by researchers (see Dekker et al. 2020). Living labs can be set up and enacted by researchers to bring about innovation or societal change, and thus the researcher role resembles action research (Buhr et al. 2016). However, this is just one of several ways of enacting researcher roles in relation to living labs. Identifying and discussing different researcher roles and the different roles of research in living lab settings is greatly needed. This should also cover analysis of how different roles link to epistemological positioning and different scientific paradigms.

Finally, our review shows that there is need for research on the outcomes of living labs, because there is no consistent focus on what the potential tangible and intangible results of establishing public sector living labs are (Dekker et al. 2021; Ståhlbröst 2012). We propose that a (public) value framework can support evaluations and assessments of living lab outcomes. A (public) value framework would help to articulate and differentiate between different kind of outcomes and facilitate the evaluation of outcomes across contexts. This review provides a steppingstone for studies to further examine the differentiated outcomes of living labs in the future.

To summarize, based on these discussions of the knowledge gaps we have identified from our integrative review, we propose three major avenues for future research: First, how are living labs created and recreated as environments for public sector innovation over time?; Secondly, what are the linkages between living labs and the role of research and researcher roles?; Thirdly, what are the outcomes of living labs in the public sector and how may a public value framework facilitate the evaluation of outcomes?