Measuring Progress in Sustainable Food Cities: An Indicators Toolbox for Action

Abstract

Despite the growth of urban food policies across the globe, a key challenge remains around measuring the impact of these initiatives in building more sustainable and just cities. The literature identifies as the main barriers to progress food system assessments the lack of clear definitions of sustainability, insufficient data, the low applicability of global conceptualisations to local conditions and vice versa and low levels of actor involvement. We aim to address these gaps by co-developing a sustainability assessment framework to evaluate food systems performance in UK cities. The framework emerging from this collaborative process overcomes key limitations of previous exercises by providing a place-based and participative definition of sustainability aligned with global conceptualisations, building on the experiences and needs of a wide range of practitioners and taking a holistic but non-prescriptive approach to understanding food system outcomes. However, its application to the city of Cardiff reveals new challenges, mostly regarding the need to incorporate underrepresented stakeholders as well as account for multiscalar food system interdependencies and their positive but also negative impacts. Results show the need to embed critical perspectives in sustainable food assessments that actively expand their transformative capacity by developing further inclusive, participatory, place-based and whole-systems approaches.

1. Introduction

In the context of rapid urbanisation, global resource depletion, climate change and vast socio-economic inequalities [1,2]; the development of more sustainable and just cities has become a priority on the agenda of governments, international institutions and practitioners around the world. In order to transform urban spaces, cities are mobilising food as a vehicle to deliver health, social, economic and environmental benefits [3]. As exemplified by the more than 170 signature cities of the Milan Urban Food Policy Pact, an ever-increasing number of cities are using the convening power of food to bring together civil society, policy makers and the private sector to build more sustainable urban environments.

However, and despite the amount of urban food initiatives taking place, a key challenge remains around measuring the impact of these policy processes and specific projects in building more sustainable and fair cities. Scholars identify four main barriers to progress sustainable food systems conceptualisations and related assessments: (i) the lack of a clear definition of what is (and is not) sustainable; (ii) the low applicability of global conceptualisations to local decision-making communities; (iii) constraints caused by the accessibility of qualitative and quantitative data, and (iv) the diversity of approaches applied at the local level which prevents aggregation of results and measurement of global progress [4,5]. Furthermore, many existing initiatives have focused either on national and international food systems evaluations which struggle to incorporate local-based specificities and contradictions, or on specific initiatives and places, with limited capacity to compare across sites. In other occasions, these exercises fail to address the systemic and interconnected nature of sustainability and food insecurity [6,7]. Nonetheless, numerous voices call for a holistic systems approach as the most effective strategy to deliver good food for all in a more sustainable manner [8,9]. Sustainability assessments are complex, and tend to be conducted by experts, however, sustainability transformations involve changes in a range of practices and therefore require a diversity of actors to participate.

In this paper, we aim to address these gaps by discussing the co-development of a sustainability assessment framework to evaluate food systems performance in UK cities. For that purpose, we have conducted a participatory action research process with the Sustainable Food Cities (SFC) Network—a network of over 55 cities in the UK that are building local partnerships and delivering food strategies guided by an SFC Award framework (http://sustainablefoodcities.org/awards) covering all issues of the food system. This is creating a unique community of practice driving food system change and a step forward in scaling up and out urban food strategies in a national context. The project aimed to develop a set of indicators that helps councils and community food initiatives to make the case for and measure their progress towards more sustainable and secure food systems. This exercise constitutes a crucial step at a time of austerity to maximise the efficiency of resource investment and improve sustainability and food security outcomes. The framework proposed here―and associated toolkit―builds on place-based experiences of cities developing sustainable food activities; incorporates the visions, needs and challenges of a wide range of practitioners; and, at the same time, takes a holistic approach to understanding food system outcomes. The resulting sustainability assessment framework aims to provide a template to evaluate progress in key sustainability dimensions, identify problematic areas and inspire action across civil society, policy makers and the private sector.

The remainder of the paper is organised as follows. First, we discuss the results of a systematic literature review to provide an in-depth analysis of sustainable food assessments, including their benefits and limitations. Second, we describe the methodological design of this action-research process. Section 3 presents the sustainability assessment framework, including its main components and how they work together. Section 4 discusses the results and insights gained from applying this framework to the city of Cardiff. We conclude by reflecting on how this framework addresses key barriers to progress sustainable food assessments and raises new challenges and research questions.

2. Measuring Success: Literature Review on Sustainability and Food System Indicators

We have used a systematic literature review method to present the current state of art in developing indicators to measure sustainability in food systems. “Sustainability”, “food systems” and “indicators” were the keywords selected to search for relevant academic literature in Scopus. The search, which was restricted to the titles of the articles, their abstracts and the keywords identified, released 68 academic articles. The review of these papers was complemented with the analysis of a vast body of “grey” literature that we identified through a Google search (also using “Sustainability”, “food systems” and “indicators” as key words) or that was cited in the academic literature that we reviewed. We also included review papers on measuring sustainability in cities and food security indicators. All material was then analyzed to identify the main focus of food system assessments, their benefits and limitations. Inevitably this systematic review process also has limitations. For example, many studies that are focused on a particular topic, such as urban agriculture, did not surfaced in the search process. This is due to the methodology design which aims to capture more holistic assessments of sustainable food systems. See Supplementary A for a compilation of indicators used in the literature.

2.1. What Is Assessed?

Indicators have been defined as “a way to measure, indicate or point to with more or less exactness”, “something used to show the condition of a system” [10]. In the urban sustainability realm, indicators “provide information on the state, dynamics, and underlying drivers of urban systems, with specifications of time dimension, limits, or targets associated with them” [11]. A recent review of sustainability assessments in the food sector revealed the diversity of purposes that these tools serve, including: research, providing policy advice, farm monitoring, farm advice, certification, self-assessment, landscape planning or consumer information [12]. They also differ in their level of assessment―from focusing on the farm level to specific products―and scale, for example assessing regional, national or global dynamics (ibid). Non-academic institutions have been particularly active in developing some of these frameworks. Of particular importance are the Sustainability Assessment of Food and Agriculture systems (SAFA Guidelines) developed by FAO [13] which aims to harmonize existing approaches within a common framework that comprises four dimensions of sustainability―Good Governance, Environmental Integrity, Economic Resilience, and Social Well-Being―in order to evaluate the sustainability of farms or businesses along the food value chain.

An increasing number of exercises are evaluating food system’s performance in relation to different concepts, not only sustainability, such as sustainable nutrition security [8], food security [9], resilience of the global food system [14], agroecology [15], sustainable intensification [16] or sustainable diets [17]. While some of these more holistic accounts of food systems provide specific indicators [8], many initiatives just highlight areas of concern or elements to take into account.

Nevertheless, most of the papers reviewed fail to provide a comprehensive evaluation of the food system. On the one hand, an important part of this literature concentrates on just one activity or aspect of the food system for example the sustainability of phosphorous [18], farm workers’ health [19], organic agriculture [20], home gardens [21], meat consumption and production [22], the performance of a specific product [23] or obesity [24]. On the other hand, many works focus on just one dimension of sustainability, such as studies on planetary boundaries [25], food ecological footprints [26,27,28], or nutrition functional diversity [29]. Indeed, health and nutritional aspects are increasingly considered as a key topic in these frameworks [30], although not always in relation to all the other sustainability or food security dimensions. This is illustrated for example by Seconda’s et al., [31] assessment of the sustainability of the Mediterranean diet through an individual behavior approach or Röös’ et al. [32] assessment of the environmental impact and nutritional quality of diets.

A significant breadth of work on assessing sustainable food systems relates to the re-spatialisation of foodscapes and the development of more localized and proximate relationships in a specific area. The indicators proposed here range from food miles to the area of land actively farmed or the resilience of foodsheds to climate change challenges [33,34,35]. This literature is by and large linked to the Alternative Food Networks scholarship [36]. As part of this body of work, different exercises have been conducted to evaluate the impact of specific initiatives on different sustainability dimensions from community supported agriculture to food initiatives in schools [37,38,39]. For example, Peano et al. [40] developed a set of sustainability indicators and applied it to a Slow Food project which focuses on developing farm-to-market systems for local, high-quality, sustainable products. Another relevant example is FOODSCALE, a method that provides a holistic sustainability assessment of public food procurement [41].

Finally, there are few contributions around urban food system assessments. To date, an important part of the urban food literature has concentrated on evaluating the contribution of urban agriculture to sustainability [42,43,44] This expanding body of work evaluates the environmental, social and economic dimensions of sustainability of urban food production, with a particular focus on developing quantitative assessment methods. For example, this includes determining whether cities can become self-reliant on food and its associated sustainability and health benefits [45] or quantifying the contribution of home gardening to urban food security and its environmental impact [21]. The growing field of urban agriculture studies is actively contributing to expand our understanding of sustainable food cities. However, it is paramount to acknowledge that cities rely on food produced in different places and therefore sustainability assessments present complex geographies. In this line, more comprehensive approaches to study urban food system―from production to consumption and waste ―need to be further explored. Recent works are taking this more holistic perspective of urban food systems. For example, Landert et al. [46] propose a framework to evaluate urban food system governance. Efforts have also been directed towards assessing and planning city-region food systems [47]. This joint effort by practitioners and academics shows the complexity of defining common indicators for different contexts as well as the lack of reliable data. Notwithstanding, Prosperi’s et al. [48] recent review shows that there is an important body of work on sustainability and cities that can inform exercises to assess sustainable urban food systems (see for example [6,49].

2.2. What Is the Logical Framework?

The selection of sustainability indicators generally follows a logical or conceptual framework and relies on a series of criteria for checking the availability and quality of the data. Building on Huang’s et al. [11] review of urban sustainability assessments, Prosperi et al. [48] identify three common types of conceptual frameworks used in food studies:

-

The driving force-Pressure-State-Impact-Response framework (DPSIR) where the indicators are identified and clustered according to pressures or/and driving forces, system state or/and impacts, and responses. An example is Johnston’s et al. [17] examination of the determinants of sustainable diets and definition of a causal model and framework from which to build indicators.

-

Theme-based frameworks, which cluster indicators around four dimensions of sustainability (environment, economy, society, and governance). There are many examples in the literature that depart from these dimensions [46] and even expand them, for instance adding cultural and food quality aspects [36]. In the case of Nasir et al., [50] they develop an index for measuring sustainability in food systems using three dimensions of food security: accessibility, availability and utilisation. By and large, all the exercises that aim to have a holistic perspective include social, health and wellbeing, economic, environmental and governance dimensions in their framework grouping them in different ways (see [48,51] for recent reviews).

-

Goal-oriented approaches, which define an overall goal, desired outcomes and indicators to measure progress. Examples include Seekell’s [14] attempt to define global food system resilience indicators or the definition of sustainable nutrition outcomes of food systems, and the identification of 7 metrics and associated indicators to achieve them [8].

These approaches result in different types of indicators which, according to the literature [13,52], can be by and large classified as:

-

Performance-based: aimed at measuring performance and setting up sustainability indicator targets and benchmarks to motivate the agents in the food system to contribute to the transformation.

-

Values-based: aimed at communicating and mediating sustainability values to enable coordinated and cooperative action to transform the food system.

-

Reflexive-based: grounded in both knowledge, the limits of knowledge, and values.

-

Practice-based: focus on prescribing the necessary tools and systems required to implement good practices and therefore are process rather than outcome oriented.

2.3. How Are Indicators Selected?

The systematic review shows that indicators are selected by combining methods such as literature reviews, available data, benchmarks obtained from good practice and interviews with experts [8,53]. The criteria to select indicators is a key aspect of these exercises, juggling aspects such as using publicly available data that allows cross-comparison or develop fine-grained analysis at the local level [4]. The SMART (Specific, Measurable, Accessible, Reliable, Timed) criteria constitutes a common departing point. However, Li-Yin Shen et al.’s [53] review of urban sustainability indicators concludes that the selection of indicators should be done with the understanding of the needs of the end-users. To this aim, Landert et al. [46] narrowed their selection from all food-related processes in the urban area to those processes that could be influenced by institutions. This was to ensure that the assessment results yielded practical suggestions for policy makers.

2.4. What Are the Benefits and Limitations of These Exercises?

Sustainability assessment exercises by and large contribute to a better understanding of the performance of specific initiatives and systems, as well as monitor and effectively influence the path towards sustainability [4]. For example, these assessments can constitute a way of measuring urban sustainability but also contribute to making the city’s sustainable development more visible and transparent; support comparisons, evaluation and prediction of trends; help construct and harmonise data banks; provide decision-making with relevant information, stimulate communication and promote citizen empowerment and participation [6]. However, key challenges emerge when considering the integration of frameworks across scales and thematically, as well as critical reflections around whose knowledge, values and goals are reflected in these exercises, as discussed below.

National and international exercises such as the Food Sustainability Index that rely on widely available data have the capacity to shape our understanding of problems and approach to solutions. Nevertheless, it is increasingly challenging to align national and international exercises with “local-level specificity, as even within nations, different localities may experience significantly different cause-and-effect relationships between the ecological and social variables unique to each context” [4] (p. 3). Local level indicators therefore need to be co-produced with local actors and remain context-dependent. However, efforts to align local exercises with more widely used frameworks are still relevant. This is the case of the use of SAFA guidelines to develop an urban food governance framework for Basel [46] or the establishment of connections between indicators used by different North American cities in their urban food strategies and the Sustainable Development Goals [54]. Despite these efforts, aggregation still constitutes a key challenge. For example, Tanguay et al. [5] study the application of Sustainable Development Indicators (SDI) for 17 western cities. They find a lack of consensus in several steps in the creation of SDI that stems from the ambiguity in the definitions of sustainable development, objectives for the use of such indicators, the selection method and the accessibility of qualitative and quantitative data.

The review of the literature also highlights the importance of developing holistic accounts of sustainability. Currently, most urban sustainability assessment methods fail to demonstrate sufficient understanding of the interrelation and interdependence of social, economic and environmental considerations [6]. The separation of these dimensions of sustainable urban development hinders the ability of most systems to accurately understand broad movements towards sustainability [7,42]. In this regard, sustainability pillars still constitute the main common framework. While some exercises reviewed here promote more complex conceptualizations of food systems and sustainable development interactions, they recognize the importance of working through key sustainability dimensions in order to align with existing discourses and improve communication [55].

Finally, critical reflection on how knowledge is codified and used in sustainability assessment exercises is paramount to design tools to transform the food system. Working through sustainable food challenges and trade-offs necessarily requires addressing power imbalances which entails taking a participatory perspective, from problem definition to the identification of goals and development of solutions [4,52]. Participatory processes can contribute to overcome data gaps, bridge different types of knowledge or resolve conflicts over values. However, putting in practice the principles of sustainable assessment is highly complex, since many sustainability attributes are not (yet) measurable and “hard” methodologies need to be complemented by “soft” methodologies to identify critical issues and trade-offs across different types of stakeholders [56,57,58]. To this effect, there is a recent change in indicator-related exercises, from the study of expost evaluation of cities’ problems to focus on the exante stage where indicators can be operationalized as planning tools and become instruments of policy [6]. This coincides with an overall approach of providing not only assessment tools but means for improving the sustainability of urban food systems.

This literature review highlights key definitions and pointers that guided the design of the methodology and framework described in the subsequent sections. The review also provides the grounds for a critical analysis of this framework’s contribution to current debates as discussed in the concluding section.

3. Methodology

Building on the lessons learned from the literature review, we designed a participatory-action research process to co-produce a place-based, holistic and action-oriented assessment framework with the aim to evaluate current foodscapes, provide tools to guide action and improve sustainability outcomes in specific places. The process was led by academics and SFC practitioners and involved different stakeholders to ensure a collective identification of goals and criteria to select indicators. The project was articulated around the following steps (see Figure 1).

First, the review of both academic and “grey” literature led to the identification of indicators used to measure the environmental, social and economic sustainability of urban food systems. The results of this literature review guided the design of four participative workshops enrolling 99 practitioners from 41 different places to define a vision for a sustainable food city as well as identify the most relevant indicators to guide action towards that goal. The workshops were organized geographically (Cardiff, London, Edinburgh and Liverpool) to grasp spatial issues that might affect involvement with urban food policies at the national level and maximize city governments’ and civil society organisations participation in the project.

A third step consisted in synthesizing the results of the workshops and verifying the feasibility and coherence of the selected indicators based on the available literature and insights from practitioners. For that purpose, a draft toolbox was widely disseminated. This included conducting a webinar with 42 attendees. A meeting with 17 academics and practitioners was also organised in London to gather extra feedback. In total more than 70 participants provided comments on the draft toolbox.

Finally, we applied the framework to the cities of Cardiff and Bristol, actively involved in the SFC Network. This step included gathering information from different institutions and databases produced in the last five years to feed into the indicators toolbox and gain additional insights. Only the application of the framework in Cardiff is presented here. These activities resulted in the development of a conceptual framework and toolbox to measure progress towards sustainable food cities as described in Section 4 below.

4. Co-Developing a Place-Based and Systems Approach to Assess Sustainable Food Cities

The literature review shows the usefulness of theme-based frameworks, particularly structured around the sustainability dimensions, which allows to establish linkages with other exercises and improves communication with a broad audience. We therefore used sustainability as the theme-based framework and combined it with a logical framework consisting of goals, outcomes and indicators that facilitated action (see

Table 1. Definitions of the conceptual framework.

Term	Definition	Example
Goal	An overarching aim	Heathy cities
Outcome	A state or position which is reached in order that the goal is achieved	Low incidence of diet-related illnesses
Indicator (outcome-based)	A measure of progress towards delivery of an outcome, that is, a change in a relevant and measurable parameter	Decrease in the incidence of diet-related illnesses
Activity-based indicators	Activities that can potentially contribute to improve indicators	Increase portions of vegetables in school meals

). From this starting point, and to co-develop a place-based and sustainability food systems framework, we started by asking practitioners in four workshops what does a sustainable food city mean to you. This open question is particularly important since sustainability is a contested concept constructed by multiple actors that hold different values and ethical perspectives shaped by specific socio-economic contexts and places [59], and therefore questions such as how sustainability outcomes are prioritized and by who emerge as key elements in defining what the right courses of action are judged to be [60]. Responses were clustered around key sustainability dimensions, however, in every workshop this classification was open to contestation and consequently evolved, resulting in four broad themes: economy, health and social wellbeing, environment and governance.

Participants were asked to define a broad goal under each topic and then to identify key related outcomes. These outcomes helped to select and ground potential indicators. This required a discussion on what constitutes an indicator and what are key selection criteria (see Supplementary C). The process of defining outcomes and indicators revealed close linkages between the four sustainability dimensions and therefore the need to consider them in conjunction. We developed this toolbox further by providing a set of activities―and associated evidence―to help cities progress on these indicators.

The analysis of the workshops’ results led to the development of a sustainability assessment framework to measure progress in sustainable food cities. This framework consists of four thematic dimensions and associated goals, a selection of outcomes expressed through thematic (outcome-based) indicators, a broad range of activity-based indicators to guide action, and a set of evidence that links the impact of specific activities to the broader sustainability outcomes as well as illustrates best practice through case studies (see Figure 2). Below we present the process of developing this framework, as well as how its different elements work together.

4.1. Sustainability Dimensions and Goals

The framework includes three thematic dimensions: economic, environmental and health and social wellbeing. A fourth transversal element, governance encapsulates systemic changes highlighting the normative and therefore political dimension of the sustainable food city agenda. Under each dimension we defined goals by synthesizing the vision for a sustainable food city expressed by workshop participants. These dimensions and associated goals are intimately related to each other and reflect a holistic approach to food system reform to deliver long-lasting change (see [61]). These relationships are expressed more clearly through the levers for change (see examples below).

4.2. Outcomes-Based Indicators

The framework grounds these goals by selecting a range of outcomes-based indicators (see

Table 2. Sustainability dimensions, goals and outcomes-based indicators.

Dimension	Governance	Health and Wellbeing	Economy	Environment
Goal	Ensure long-term success of city-scale food programmes by setting up cross-sectoral food partnerships that are embedded in high level city governance structures and deliver integrated food strategies and action plans covering all key food issues.	Improving physical and mental health and wellbeing by reducing food poverty; improving access to culturally acceptable, affordable healthy food for all; promoting healthy diets; and increasing participation in food related physical and social activity.	Creating new and sustainable jobs and businesses as part of a vibrant, culturally diverse and prosperous local food economy that provides fair and equitable economic benefits to all actors involved in both local and global supply chains.	Reducing the negative ecological and ethical impacts of the food system from production, processing and distribution to consumption and waste, including GHG emissions, soil and water degradation, biodiversity loss, waste and poor animal welfare.
Outcomes-Based Indicators (Examples, see full list in Supplementary C)	Establishment of a local food partnership Adoption of a food strategy and action plan covering all food issues	Decrease in the number of people requiring emergency food aid Decrease in the number of people overweight or obese	Increase in the number of jobs in the local food economy Increase in the amount of money circulating in the local food economy	Decrease in greenhouse gas emissions associated with the food system Decrease in the consumption of meat and meat-based products

). This selection draws first on the indicators identified by participants, second on the expert feedback on the draft toolbox, third on the indicators systematically collated in the literature (see Supplementary A) and fourth, on the criteria identified by workshop participants as important in the selection of indicators (see Supplementary B).

An additional element that emerged through expert consultation was the importance of aligning these outcomes-based indicators with the priorities and language of local actors. For example, a public health officer should be able to identify shared goals and outcomes within the ‘Health & Social Wellbeing’ dimension (Table 2) such as the decrease in the number of people overweight or obese. This alignment facilitates collaboration between different stakeholders and allows for the measurement of the collective impact of diverse actors working together.

4.3. Outcomes-Based Indicators

The framework grounds these goals by selecting a range of outcomes-based indicators (see Table 2). This selection draws first on the indicators identified by participants, second on the expert feedback on the draft toolbox, third on the indicators systematically collated in the literature (see Supplementary A) and fourth, on the criteria identified by workshop participants as important in the selection of indicators (see Supplementary B).

4.4. Activity-Based Indicators and Associated Evidence

Finally, the framework identifies a set of activities that help to support progress towards achieving the outcomes-based indicators. The activities have been largely inspired by the SFC Award framework for action and are classified on the one hand according to their contribution to the three thematic dimensions that make up a vision for a sustainable food city, and on the other hand in relation to the type of tools and mechanisms employed to deliver them. This is to improve the practicality of the toolbox and facilitate its local application.

Whilst the outcomes have been classified for practical reasons according to the main health and wellbeing, economic or environmental dimension they contribute to, the levers for change and associated activities draw attention to the interdependent and the interconnected nature of food systems. Hence, the conceptual framework conveys the need for a systems, place-based and partnership approach to deliver different sustainability outcomes. This is done by mobilising diverse levers and implementing a range of activities which have been collectively identified as underpinning a sustainable food city. In complex and adaptive systems such as the food system, this place-based and systems approach ensures adequate monitoring so that the unintended consequences of action in one area are accounted for and mitigated if necessary [8].

The framework also includes a wide range of academic evidence, grey literature and case studies that underpin with evidence the capacity of each activity to progress towards the outcomes-based indicators and goals identified in this framework (see

Table 3. Example of activity-based indicators and evidence.

Dimensions	Governance
	Health and Wellbeing	Economy	Environment
LEVERS FOR CHANGE   (Example of ACTIVITY-BASED INDICATORS under lever for change: Infrastructure & planning)	The Council works to prevent the development of food deserts (where people cannot access affordable healthy food within 500 m) and food swamps (where the high street is dominated by fast food outlets).	The Council maps redundant retail and brownfield sites and makes them available to new food enterprises, for example through use of meanwhile and special leases and business rates reductions and holidays.	The Council maps green and brownfield sites that could be used for food growing, composting and local food processing and distribution and makes them available to local communities.
	The Council/city protects and/or re-establishes vital local sustainable food infrastructure, such as Grade 1 and 2 agricultural land, local processing and wholesale businesses, food hubs and distribution networks.   EVIDENCE (The evidence and case study presented here are to illustrate the functioning of the toolkit. Each activity has its own range of related evidence and case studies.)   Grey and academic literature Ref 30. Campaign to Protect Rural England. 2012. ‘Mapping Local Food Webs Toolkit’. CPRE. The Food Webs Toolkit stresses the importance of maintaining and building strong local food infrastructures to create new jobs, and small businesses, to ensure that more money is spent and stays in the local economy, to reduce food miles and food waste and to secure better access to fresh, healthy and affordable food. Case study CS18. Brighton & Hove City Council holds in public ownership 11,923 acres of ‘downland’ farmland. A City Downland Advisory Board has been established to develop policy which supports a viable local farm economy, diversification such as eco-tourism reconnects farmers and city residents, and promotes sustainable food production.

).

In order to test the usefulness of this tool, the framework was applied in Cardiff and Bristol. Due to space limitations, in the section below we present the results of this exercise for Cardiff.

5. Application of the Framework in Cardiff

The framework was applied to Cardiff through the collection of secondary data. Data gaps were discussed further with key stakeholders from the city council (business development, waste management and public space/allotments officers), public health, the local food partnership and two main civil society organisations through 8 semi-structured interviews plus targeted requests for information. The application of the framework served the purpose of connecting different actors and linking their actions, as recognised by interviewees who for example asked for additional information on programmes they were unaware of. This exercise shows three main sets of results.

First, the framework allows for the characterisation of Cardiff’s foodscape, providing information on its evolution and the possibility of comparing key trends on food sustainability with other cities as well as national and international dynamics (see Supplementary C). The governance indicators provide a qualitative account of how the city has developed a cross-sectoral partnership and related policy activities, including commitment from key stakeholders, number of organisations involved in the partnership, the delivery of a city-wide action plan and the active participation of the partnership in key policy spaces―including Wales and UK fora―which result in embedding food in multiple urban plans and programmes.

In terms of health-related outcomes, by and large Cardiff is experiencing negative trends, for example the percentage of adults eating five portions of fruit and veg showed a decrease from 36% in 2014/15 to 32% in 2016/7; similarly the percentage of 4/5 year olds recorded as obese or overweight increased from 26.1% to 27.1% for the same period. There are some positive trends around increase in dental health and a reduction of one point in terms of percentage of population suffering from income deprivation.

Accessing secondary data on the economic and environmental dimension of the local food system proved extremely difficult. On the one hand, this is due to the lack of specific data on the food system, for example the living wage foundation does not include a classification of food businesses. On the other hand, data collection is costly and individual studies that shed light on a particular topic do not necessarily track their evolution. However, the compilation of activity-based indicators under the different levers for change and goals allows us to understand some of the initiatives and progress taking place in the city that so far are not monitored. The lack of data was complemented with interviews. In this regard interviewees highlighted a general increase in interest in sustainable and local food “led by the consumer” (interviewee 4) with an “increase in spend on food and drink offer which the council needs to respond appropriately to maintain employment levels in and around retail centres” (Interviewee 7). Local businesses are also increasingly “offering more than the living wage” although there are people “on zero-hour contracts and have no idea of the implications in terms of tax and national insurance” (Interviewee3). Nevertheless, some initiatives are struggling. For example 14 food co-ops offering affordable fruit and veg across the city have closed in the last year (interviewee 5).

The environmental dimension shows positive trends for key measurements such as the reduction of CO₂ emissions per capita or a 17% increase in the percentage of household and non-household waste prepared for reuse, recycling or composting in Cardiff; but there are important data gaps. Interviews on the environmental dimension of the city’s food system showed an increasing level of activity around tackling food-related waste, including plastics, through training, restrictions and campaigns. However, it also surfaced contradictions for example “recycling targets are weight based, if everyone stopped producing food waste we wouldn’t be getting the recycling in” (Interviewee 2) and specific targets will not be met; also there are unforeseen consequences of new projects that “are trying to do the "right thing" and move to bio-degradable packaging but it is actually worse for us, because our processors won’t accept it, (…) so it ends up in landfill” (interviewee 2). Food growing is the other key issue policy makers and civil society organisations have acted upon to address the environmental dimension of sustainability. Key informants record an overall increase in the number of sites and public engagement on food growing across the city. However, the lack of reliable measures together with changing levels of activity across sites shows once more the difficulty of monitoring progress.

The second set of results revolves around how the framework exposes which goals and tools are addressed by partnerships as well as key gaps. For example, in terms of activity-based indicators, Cardiff is clearly focusing its efforts on improving governance and health and wellbeing outcomes, but also on developing activities that simultaneously address more than one sustainability dimension (see

Table 4. Results of application to Cardiff.

Types of Activity-Related Indicators		N Activities Achieved	N Cross-Cutting Activities Achieved	Total Achieved	Potential N of Activities (with Cross-Cutting)	% Achieved
By Outcomes/Sustainability Dimensions	Health	5	4	9	25	36%
	Economy	2	5	7	25	28%
	Environment	3	4	7	22	32%
By Levers for Change *	Partnership & collaboration	2			3	67%
	policies and strategies	4			7	57%
	Infrastructure and planning	1			9	11%
	Public services and support	4			9	44%
	Knowledge and awareness	4			12	33%
	Market-based mechanisms	1			7	14%

* There are no cross-cutting activities by levers of change.

).

Figure 3 shows that most of the activities implemented relate to establishing partnerships and collaborations, developing policies and strategies and setting up public services and related support programmes. There is a lack of activity around other key levers for change such as market-based mechanisms and infrastructure and planning.

Finally, the application of the framework demonstrated the usefulness but also the limitations of such exercise. While indicators where collectively selected by more than 100 practitioners, using metrics that are accessible, reliable, comprehensive and inexpensive to collect remains a key challenge. Furthermore, places measure similar processes differently―as revealed through the application of the same framework to Bristol―which complicates comparability across a country or internationally. As illustrated above in the case of food waste, some indicators present conflicts in terms of how they actually represent progress towards sustainability. It is paramount to acknowledge these trade-offs and use criteria for the selection of indicators that will be most meaningful for end-users. In this case, rather than developing a perfect assessment framework, the aim was to collectively develop a flexible toolbox that provides a common narrative to map and encourage action towards sustainability in the city. In the case of the UK, some of the outcome-based indicators are more accessible than others; for example, it is easy to monitor the decrease in the number of overweight or obese people, and more difficult to measure changes in the number of jobs in the local food economy.

The analysis also reveals that despite progress in terms of the number of activities implemented in the city, there are multiple processes at play that can hinder progress towards more sustainable foodscapes. For example, despite the commitment of the City Council and the push towards incorporating healthier options in school meals, vending machines and the high street, Cardiff is the UK city with the highest number of unhealthy hot food take-aways per capita ((See local campaign Peas Please on providing more healthy food (http://foodcardiff.com/update-on-peas-please-and-veg-power/) and also reports on increasing number of unhealthy take-aways (https://www.walesonline.co.uk/whats-on/food-drink-news/fast-food-outlets-cardiff-mapped-14864764)). These overlapping dynamics highlight the need to measure progress and related activities that contribute towards sustainability goals, but also to monitor negative trends and developments that might be concurrent to these developments. This also calls for a cautious approach to attributing causality between specific actions and sustainability outcomes.

6. Conclusions: Assessment as a Tool for Food System Transformation

This paper presents an innovative participative, systems and place-based approach to sustainable food system assessments that fosters city-to-city collaboration, provides sources of evidence to make the case for a holistic intervention in the food system and aligns goals and practices of different actors shaping the urban foodscape. This co-produced framework ultimaltely offers tools for practitioners to foster effective collective action across sectors towards sustainability. At the same time, the framework allows us to understand unique challenges faced by different cities and efforts to overcome them, providing a non-prescriptive and more grounded sustainability assessment of food systems. That is, rather than focusing on specific activities implemented or snapshots of data, this framework starts to capture the range of challenges cities face as well as the various assets and capabilities they possess to build their own sustainability pathway.

Consequently, the framework proposed overcomes key limitations of previous sustainability assessment exercises identified in the literature review, by providing a place-based and participative definition of sustainability aligned with global conceptualisations, widely applicable and relevant for local communities. However, a key challenge remains in accessingrelevant data. In this framework, data gaps are partially overcome by mapping activity-based indicators which constitute a proxy to understand progress towards sustainability in the local foodscape. The framework so far has only been applied to Cardiff and Bristol, however, there is potential for data gaps to be filled as more cities conduct this exercise and identify new or alternative data sources or methods to capture these. Furthermore, there are a number of exercises in place developing urban food indicators (see for example the Milan Food Policy Pacy Monitoring Framework), as well as quantitative academic contributions particularly around urban agriculture [43,44,45]. While in this paper we have relied on key literature (see Section 2), in order to continue advancing in the development of sustainability assessment frameworks, it is paramount to develop mechanisms that facilitate the crossfertilisation of different exercises, associated data and their related conceptual and practical advances. While we have demonstrated that participatory, place-based and systems approaches to food assessments are necessary to understand urban food dynamics and design effective interventions, the development and application of this framework also reveals two new challenges. First, the extensive participatory process tailored to SFCN members and allies failed to include the diversity of knowledge, practices and needs in the food system; particularly of underrepresented stakeholders, such as those experiencing food poverty and representatives of food businesses. Experiences of food poverty are seldom recognised in assessment exercises, indicators or mainstream sustainability discourses, and therefore their absence might limit the transformative capactity of these frameworks. Within the same city there are a myriad of processes taking place.Consequently it is paramount to reflect this diversity by breaking down measurements in terms of neighbourhoods, gender, age, income and ethnic background.

Secondly, the SFCN framework takes a flexible approach to the establisment of boundaries and scales, indeed the network includes regions, counties, cities, towns and boroughs. The framework therefore adapts to these administrative boundaries, taking a losely defined ‘local food system’ approach. However, this framework does not account for the interdependencies at play in the food system, that is, how proximate and distant foodscapes condition how food is produced, transformed, consumed and disposed of in a particular place. Indeed, developing more sustainable foodscapes is a collective endeavour that requires cross-sectoral, cross-scalar and translocal efforts.

Our analysis calls for a more critical approach to sustainable food assessments that actively expands their transformative capacity by developing inclusive, participatory, place-based and systems approaches. Future transformative frameworks should actively address how they can reify, shift or empower particular framings and practices around food and sustainability; as well as explore further tools for policy makers and practitioners to support the creation of translocal alliances that collectively address multiscalar, synergistic and conflicting processes currently shaping our unsustainable food system.