Development and Application of Technical Key Performance Indicators (KPIs) for Smart Water Cities (SWCs) Global Standards and Certification Schemes

Abstract

Smart water cities (SWCs) use advanced technologies for efficient management and preservation of the urban water cycle, strengthening sustainability and improving the quality of life of the residents. This research aims to develop measurement and evaluation tools for SWC key performance indicators (KPIs), focusing on innovative water technologies in establishing unified global standards and certification schemes. The KPIs are categorized based on the stage at which water is being measured, namely the urban water cycle, water disaster management and water supply and treatment. The objective is to assess cities’ use of technologies in providing sufficient water supply, monitoring water quality, strengthening disaster resilience and maintaining and preserving the urban water ecosystem. The assessment is composed of a variety of procedures performed in a quantitative and qualitative manner, the details of which are presented in this study. The developed SWC KPI measurements are used to evaluate the urban water management practices for Busan Eco Delta City, located in Busan, South Korea. Evaluation processes were presented and established, serving as the guideline basis for certification in analyzing future cities, providing integrated and comprehensive information on the status of their urban water system, gathering new techniques, and proposing solutions for smarter measures.

1. Introduction

Urban water management plays a crucial role in ensuring an adequate and clean water supply for cities. There has been an increase in the management’s responsibility to adopt more sustainable approaches in the management of urban water due to external factors such as climate change and population growth; therefore, there has been the adaptation of advanced technologies and more robust management models due to increased water service demands.

The use of Information and Communication Technology (ICT) is often presented as a powerful strategy for smart water development, allowing the better management of natural water resources and facilitating positive progress. The integration of ICT in urban water management is integral in addressing the complexities of water management ranging from various categories, including water source, supply, stormwater, wastewater, reuse, disposal, environmental flow, water conservation and more [1]. The combined effect of urbanization in response to population growth in cities affects the natural landscape, affecting the quality and quantity of usable water from water sources [2].

Key performance indicators (KPIs) are sets of values that are used to measure the performance of the criteria. The assessment of different technological human interventions and their impact throughout the urban water cycle helps the assessment of the extent to which a city can be considered a “smart water city”. The concept of smart water cities (SWCs) pertains to urban municipalities that integrate advanced technologies and implement data-driven solutions to optimize the efficiency, sustainability, and resilience of their water management system [3]. Such cities leverage the use of ICTs and innovative tools in the monitoring, analysis, and management of various aspects of the urban water cycle. The proposal for SWCs offers hope and progress in time of rapid urbanization and growing water challenges such as water scarcity, water pollution, flood, drought and more.

While previous standards and certification schemes have dealt with urban sustainable growth and development since the early 1990s, certain deficits have been identified. Despite their usefulness, these standards primarily focus on measuring access to drinking water and sanitation but lack a comprehensive assessment of smart urban water management. The deficiencies cited include a limited number of indicators in examining diverse water functions, insufficient measurement of smart technology implementation, inadequate preparation for climate change, limited application of low-impact development and green infrastructures and insufficient evaluation of the quality and effectiveness of observation-measuring instruments. These standards are reviewed based on the performance of the standard measures such as sustainability, smartness and resilience and mainly focus on the local level, primarily understanding the city as a unit. These standards are as follows: United 4 Smart Sustainable Cities (U4SSC) [4], ISO 37120 Series (Sustainable Cities and Communities) [5], OECD (Organization for Economic Co-operation) Smart City measurement framework [6], CITYKeys Smart City Index [7], LEED for Cities and Communities [8], Arcadis Sustainable Cities Water Index [9], KWR Blueprint Approach [10] and AWS International Water Stewardship Standard [11].

Upon analysis, these standards are recognized as valuable tools for measuring various urban aspects; however, the examination revealed certain deficiencies in the context of smart water management. The identified deficits include (1) indicators examining the diverse functions of water in the city being limited. The existing standards predominantly focus on almost exclusively measuring the accessibility to drinking water and sanitation, overlooking other vital functions; (2) the standards lack comprehensive indicators measuring the implementation of smart technologies in urban water management. This limitation hinders the understanding regarding whether the cities are incorporating smart technologies effectively in their smart water management practices; (3) the standards inadequately address the cities’ preparedness for climate change, specifically concerning issues such as the excess and deficiency in the water supply; (4) insufficient emphasis on the application of low-impact development and green infrastructures, which play a crucial role in enhancing the sustainability of the urban water cycle (5) and the lack of evaluation on the quality and effectiveness of observation-measuring instruments used in urban water management. These identified deficits emphasize the need for a more comprehensive and inclusive approach in the development of standards that can effectively analyze the city’s urban water management, allowing integral information for policy makers and providers to define the priorities in urban water management at the municipal level.

The goal of this research is to assess the technological solutions implemented by cities to strengthen their water resource management. To achieve this, the study involves an examination of the existing global standards and frameworks for smart water cities, aiming to identify common water challenges and, finally, develop a robust smart water city key performance indicators (KPIs) evaluation methodology that can be applied to examine the technological approaches developed by cities to monitor and mitigate water-related issues.

The smart water city project aims to develop an instrument for measuring and evaluating smart urban water management in cities around the world to form a global standard and certification scheme for smart water cities. The instrument will serve to examine and compare urban water management for urban water resources in different countries. The global standard and certification schemes will provide integrated and comprehensive information on the urban water status and can offer guidance to cities and communities to become smart(er). This global standard and certification scheme will contribute to the development of capacities and decision-making functions of local water providers and communities, as well as for authorities.

The developed methodology for assessing the performance of smart water cities is implemented in the examination of technologies and practices employed in Busan Eco Delta City located in Busan Metropolitan City, South Korea. The evaluation results derived from the assessment can serve as valuable resources for cities worldwide, aiding in identifying deficiencies in their urban water management strategies and offering potential solutions for implementation. At the same time, this presents an excellent opportunity to acknowledge and showcase the innovative efforts of the cities in providing unique solutions to their water problems. This recognition can serve as a guide for other cities to emulate successful practices and apply tailored solutions within their urban context.

2. Developing Technical Key Performance Indicators

For this study, the indicators designated to evaluate the characteristics of urban water and the strategies implemented by the city to optimize the water management, showcasing features recognizable as a smart water city, shall be referred to as technical key performance indicators (KPIs). The development of these KPIs follows the methodology outlined in the figure below (Figure 1).

Firstly, a comprehensive review and compilation of the existing international standards for smart water cities is conducted. Simultaneously, analysis of the current state of water management in cities located in both progressive and developing countries is undertaken (Details of this review can be found in the Smart Water Cities Phase 1: Identifying Smart Water Cities Report 2021 [12]). The prevalent water technologies used and the recurring challenges in urban water management are considered. Subsequently, an assessment is made regarding the availability of the data needed for the evaluation, both for developed and developing countries. The KPIs are then systematically categorized into either technical or water governance indicators. Governance indicators pertain to regulations and operational practices in urban water management, as well as the effective implementation of water policies. In this evaluation, water governance indicators shall not be prioritized, and the primary focus shall be applied to the smart water technologies, contributing to their role in the city water cycle management. Recurring KPIs from established global standards are retained based on their significance, and new indicators are developed based on the overall assessment of the inadequacies in the current water management of cities. The final phase of the methodology involves formulating the calculation methods and evaluation processes derived from an extensive literature review, urban water reports, guidelines, research articles and related sources. The developed methodology is used to calculate the city’s standings in the application of Information and Communication Technologies (ICTs) and its efficiency in urban water management. This assessment aims to determine whether the city can be quantified as a smart water city.

2.1. Categories

The technical KPIs evaluate the city’s proficiency in leveraging technologies in enhancing water disaster resilience, mitigating risks, ensuring adequate water supply, monitoring, and maintaining the quality of potable water and wastewater and sustaining the balance of the urban water ecosystem. This research delineates three primary categories in the assessment to distinguish the technological water interventions based on the functions they fulfill in the city water management. These categories are named the urban water cycle, water disaster management and water supply and treatment.

2.1.1. Urban Water Cycle

The urban water cycle category is composed of technical indicators that evaluate the city’s ability to facilitate water data and information management related to precipitation, ground surface, infiltration, and urban water bodies. The indicators in this category assess how the natural hydrological cycle is being preserved and maintained, despite the ongoing development of artificial structures as a result of dynamic urbanization.

The indicators in this category evaluates the capacity of the cities to (1) monitor the processes of water circulation within the city by consistently observing the spatial and temporal characteristics of the hydro-meteorological and urban surface data; (2) ensure the quality of the observation data through high-frequency monitoring; (3) guarantee the accuracy of observations through consistent data quality assurance, minimizing missing or erroneous data, and conducting instrument calibrations; (4) implement developments and infrastructures to assist the natural flow of the water circulation, [including, but not limited to, low-impact developments (LIDs), green infrastructures (GIs) and nature-based solutions (NBSs)]; (5) preserve the health of the urban water bodies and aquatic ecosystems; (6) apply modern technological developments in the collection, storage and transmission of water cycle data; (7) implement real-time measurements, automations and facilitate easy access to data and (8) build infrastructures that enhance urban water amenities for ecological health and aesthetic purposes.

2.1.2. Water Disaster Management

The water disaster management category outlines the measures cities undertake to manage the impact of water-related disasters and adaptation strategies implemented to prepare for the anticipated effects of climate change. Recognizing and addressing the city’s vulnerability to water disasters through building and strengthening resilience is proven to be more cost-effective than mitigating losses and sustaining during disaster events.

These water disaster management indicators evaluate the city’s (1) exposure and vulnerability to water-related disasters, specifically urban flooding and drought events; (2) capability to provide accurate weather-related forecasts and adequate early warning to residents; (3) ability to identify and fortify urban risk areas; (4) utilization of real-time and state-of-the-art measuring devices for disaster monitoring and prevention; (5) application of ICT-based technologies in flood and drought management; (6) capability to store excess water and provide an alternative water supply source in case of water deficits; (7) development of city-scale climate adaptation action plans and (8) efforts conducted to save and reduce the municipal-level energy usage, including the utilization of renewable energy, usage of eco-friendly facilities, zero-emission devices and more.

2.1.3. Water Supply and Treatment

Finally, smart water cities must be evaluated based on their capacity to provide sufficient and high-quality water supply to urban residents, as well as their proper handling of sanitation and wastewater treatment at the municipal level. This category highlights the assessment of the distribution, treatment and management of potable water and wastewater. The water supply and treatment KPIs provide an assessment of the city’s proficiency in the following aspects: (1) constant monitoring and maintenance of the standard water supply and wastewater quality, (2) provision of an alternative source of water supply, (3) sufficient distribution of the water supply and wastewater services to the urban population, (4) detection of deteriorating supply and sewage pipe networks, along with the provision of adequate maintenance, (5) management and operation of city-scale water purification and wastewater treatment plants, (6) implementation of real-time water flow and water quality monitoring or smart meter reading, (7) utilization of ICT-based technologies in remote control and monitoring, (8) the application of advanced water treatment technologies in treatment facilities, (9) assurance of the quality of the drinking water, (10) the establishment of proper treatment regulations for sewage waste discharge and (11) efficient reuse and constant monitoring of the quality of treated wastewater.

2.2. Key Performance Indicators

The smart water city technical evaluation categories are further broken down into 11 subcategories, classifying indicators based on specific functions that different water technologies fulfill at the city-level water management. These subcategories are named precipitation, surface water, stream water level, stream water quality, groundwater level and groundwater quality for the urban water cycle category; flood, drought, and climate change for the water disaster management and water source, drinking water treatment, water distribution, wastewater treatment and water reuse for the water supply and treatment category, respectively. The details of the indicators for each subcategory are listed in

Table 1. Smart water city evaluation main categories, subcategories, and respective key performance indicators.

1. Urban Water Cycle
	Subcategory	Sustainability	Smartness
1.1	Precipitation	Precipitation station density Precipitation observation frequency Precipitation missing and error data	Precipitation data automation and quality control ICT-based Precipitation data collection process Precipitation data accessibility
1.2	Surface water	Impervious surface percentage Urban stream biodiversity Stream waterfront facilities	LID and green infrastructures
1.3	Stream water level	Stream water level station density Stream water level observation frequency Stream water level missing and error data	Stream water level data automation and quality control ICT-based stream water level data collection process Stream water level data accessibility
1.4	Stream water quality	Stream water quality station density Stream water quality observation frequency Urban stream water quality error and missing data Stream water quality standard	Stream water quality data automation and quality control ICT-based stream water quality data collection process Stream water quality data accessibility
1.5	Groundwater level	Groundwater level station density Groundwater level observation frequency Groundwater level missing and error data	Groundwater level data automation and quality control ICT-based groundwater level data collection process Groundwater level data accessibility
1.6	Groundwater quality	Groundwater quality station density Groundwater quality observation frequency Groundwater quality missing and error data Groundwater quality standard	Groundwater quality data automation and quality control ICT-based groundwater quality data collection process Groundwater quality data accessibility
2. Water Disaster Management
	Subcategory	Sustainability	Smartness
2.1	Flood	Flood casualty index Flood property index Flood risk area index Levee structure and maintenance	Flood hazard map analysis Integrated disaster information center Urban flood prediction and early warning
2.2	Drought	Drought damage index Recent drought occurrences	Drought hazard mapping Drought information and emergency water supply facilities Drought prediction system
2.3	Climate change	City-scale climate adaptation planning	Renewable energy usage
3. Water Supply and Treatment
	Subcategory	Sustainability	Smartness
3.1	Water source	Water source monitoring frequency Water source availability	Water source data automation and quality control ICT-based water source data collection process Water source data accessibility
3.2	Drinking water treatment	Drinking water quality compliance Drinking water quality monitoring frequency	Drinking water treatment data automation and quality control ICT-based drinking water data collection process Drinking water data accessibility Advanced drinking water treatment process
3.2	Water distribution	Water supply network distribution Aging water supply pipe status Revenue water percentage Water storage effective capacity	Water supply data automation and quality control Water supply network maintenance Smart water metering Water supply data accessibility
3.4	Wastewater treatment	Sewage pipe network distribution Aging sewage pipe status Sewage water quality monitoring frequency	Sewage water treatment data automation and quality control Separated sewage network Sewage pipe network maintenance Sewage water treatment process
3.5	Wastewater reuse	Wastewater reuse and recycle	Sludge waste recycle

.

For each subcategory, the Smart Water City Index selected indicators that can be categorized into two types: sustainability and smartness based on the types of water management they measure. Sustainability refers to the city’s capacity to provide basic necessities in the sustainable management of the urban water system. In the technical aspect, sustainability indicators provide a fundamental framework in preserving and improving the quality of life of citizens, ensuring the stability of the urban water system. It emphasizes enhancing the integrity of the urban water environment, addressing not only the present needs but also preparing for future water challenges. Indicators of this type seek to measure the performance of these functional purposes at different stages of urban water management. Smartness, on the other hand, defines the city’s ability to leverage more sophisticated forms of technologies to achieve and enhance the sustainability of the urban water management system as a solution tool to resolve urban water problems. Smartness indicators are comprised of advancements in the provision and operation of modern technology in terms of data, infrastructure and services. Data indicators provide an evaluation for the management and security of fundamental hydro-meteorological data for optimal urban smart water management. The indicators under infrastructure assess the ICT-based infrastructure solution tools for urban water system functional awareness and solving urban water-related problems. Meanwhile, service indicators analyze the smart city’s capacity to provide decision support services, enhancing the efficiency and competitiveness of water services in improving the quality of life of the citizens.

2.3. Scoring and Evaluation

The technical KPIs are scored based on the performance of the city examined, with each indicator serving as a measure of the city’s ability to manage and preserve the urban water cycle. The main objective is to enhance sustainability and contribute to the improvement in quality of life of the urban residents [3]. The cities shall be evaluated on whether they satisfy such criteria. These KPIs are scored on a scale from 0 to 4, where 4 indicates excellent performance in the variable measured by the indicator, 3 corresponds to a “good performance”, 2 is a “moderate” performance, 1 is a “poor performance” and 0 is a “bad” performance.

The assessment and rating of the water management demands an in-depth understanding of the local water systems and their implementation. The primary challenge in evaluating urban water management lies in the need for a comprehensive understanding of these systems and their functionalities. Achieving an overall assessment requires expertise from various city water departments. This ensures that the evaluation accurately reflects the existing water management practices, establishes realistic standards, and considers the attainability of necessary water data.

2.3.1. Evaluation Guidelines

The indicators undergo evaluation using either quantitative or qualitative methods. To define the actual values for each indicator, various strategies are implemented [13]:

• Ratio calculations: Indicators are assessed as a ratio to a certain established reference value.
• Reference range: Indicators are assessed based on a specific range from an established reference (journal articles, technical reports, website data, established guidelines, etc.), which is specific to the indicator.
• Standards: Indicators are assessed based on the presence (full score) or absence (zero score) of a certain standard. In these cases, evidence of existence shall be required, such as documentation, reports, photo, etc.
• Survey questionnaires: Indicators are assessed based on survey questionnaires confirming the present establishment of certain standards.
• Comparison with other cities: Indicators are assessed as a comparison to the average performance of progressive cities/mega cities.
• Expert opinion: In addition, certain criteria lacking established standards from the literature shall be evaluated based on experts’ opinions. Further evaluations of the KPIs were collected from specialists from different fields within the water sector to identify the appropriate evaluation methods for these particular indicators.

2.3.2. Developed KPI Calculation and Evaluation Process

Following the evaluation guidelines, methods are devised to calculate the performance of the city, assessing its potential to be classified as a smart water city. These methods are categorized into quantitative and qualitative approaches, depending on the nature of the available data. The list of KPIs, along with the corresponding evaluation type, evaluation method and the literature basis for calculation, is presented in

Table 2. Evaluation type, method and corresponding reference for each technical indicator.

	KPIs	Evaluation Type *	Evaluation Method	Based on
1.1a	Precipitation station density	Qn	Number of rainfall stations per city area	WMO (2020) [14]
1.1b	Precipitation monitoring frequency	Qn	Frequency interval at which rainfall is being recorded	Christiano et al. (2017) [15]
1.1c	Precipitation missing and error data	Qn	Percentage of missing rainfall values over total observation	Ocampo-Marulanda et al. (2021) [16]
1.1d	Precipitation data automation and quality control	Ql	Status of automation and quality assurance for rainfall recording instruments	Developed
1.1e	ICT-based precipitation data collection process	Ql	Availability of alternative ICT-based rainfall data collection	Maswanganye et al. (2018) [17]
	Precipitation data accessibility	Ql	Public access to rainfall data	Developed
1.2a	Impervious surface percentage	Qn	Percentage of impervious surface over city area	Liu et al. (2014) [18]
1.2b	Urban stream biodiversity	Qn	Percentage of conserved area over city area	Developed
1.2c	Waterfront facilities	Ql	Existence and function of waterfront facilities	Developed
1.2d	LID and green infrastructures	Ql	Status of city application of LID and green infrastructures	EEA (2022) [19]
1.3a	Water level station density	Qn	Number of stream water level stations over stream total extent	WMO (2020) [14]
1.3b	Water level observation frequency	Qn	Frequency interval at which water level is being recorded	Liu et al. (2021) [20]
1.3c	Water level missing and error data	Qn	Percentage of missing water level values over total observation	Tencaliec et al. (2015) [21]; Mfwango et al. (2018) [22]
1.3d	Water level data automation and quality control	Ql	Status of automation and quality assurance for water level recording instruments	Developed
1.3e	ICT-based water level data collection process	Ql	Status of ICT-based stream water level data collection	YSI Incorporated (2022) [23]
1.3f	Water level data accessibility	Ql	Public access to stream water level data	Developed
1.4a	Water quality station density	Qn	Number of stream water quality stations per city area	WMO (2020) [14]
1.4b	Water quality observation frequency	Qn	Frequency interval at which water quality is measured	Corragio et al. (2022) [24]
1.4c	Water quality error and missing data	Qn	Percentage of missing water quality values over total observation	Developed
1.4d	Water quality standard	Qn	Stream water quality status compared to standard	US EPA (1999) [25]
1.4e	Water quality data automation and quality control	Ql	Status of automation and quality assurance for water quality recording instruments	Developed
1.4f	ICT-based water quality data collection process	Ql	Status of ICT-based stream water quality data collection	Developed
1.4g	Water quality data accessibility	Ql	Public access to stream water quality data	Developed
1.5a	Groundwater level station density	Qn	Number of groundwater level stations per city area	Kwater (2017) [26]
1.5b	Groundwater level observation frequency	Qn	Frequency interval at which groundwater level is being recorded	IGRAC (2008) [27]
1.5c	Groundwater level missing and error data	Qn	Percentage of missing groundwater level values over total observation	Developed
1.5d	Groundwater level data automation and quality control	Ql	Status of automation and quality assurance for groundwater level recording instruments	Developed
1.5e	ICT-based groundwater level data collection process	Ql	Status of ICT-based groundwater level data collection	Developed
1.5f	Groundwater level data accessibility	Ql	Public access to groundwater level data	Developed
1.6a	Groundwater quality station density	Qn	Number of groundwater quality inspections per city area	Developed
1.6b	Groundwater quality observation frequency	Qn	Frequency interval at which water quality is being measured	Barcelona et al. (2002) [28]
1.6c	Groundwater quality error and missing data	Qn	Percentage of missing groundwater quality values over total observation	Developed
1.6d	Groundwater quality standard	Qn	Groundwater quality status compared to standard	Developed
1.6e	Groundwater quality data automation and quality control	Ql	Status of automation and quality assurance for groundwater quality recording instruments	Developed
1.6f	ICT-based groundwater quality data collection process	Ql	Status of ICT-based groundwater quality data collection	Developed
1.6g	Groundwater quality data accessibility	Ql	Public access to groundwater quality data	Developed
	KPIs	Assessment	Evaluation Method	Based on
2.1a	Flood casualty index	Qn	Number of recent flood-related casualties	Developed
2.1b	Flood property index	Qn	Amount of recent flood-related damages over city GDP	Kwater (2017) [26]
2.1c	Flood risk area index	Qn	Percentage of flood-prone area over city area	Zhu et al. (2020) [29]
2.1d	Levee structure and maintenance	Qn	Percentage of completed levee maintenace over urban stream extent	Kwater (2017) [26]
2.1e	Flood hazard map analysis	Ql	Status of city-scale flood hazard mapping	Developed
2.1f	Integrated disaster information center	Ql	Status of city-scale disaster management information system	Developed
2.1g	Urban flood prediction and early warning	Ql	Status of city-scale flood prediction and early warning	Developed
2.2a	Drought damage index	Qn	Percentage of population affected by recent drought events	Developed
2.2b	Recent drought occurences	Qn	Frequency of recent drought events	Developed
2.2c	Drought hazard mapping	Ql	Status of city-scale drought hazard mapping	Developed
2.2d	Drought information and emergency water supply facilities	Ql	Status of drought information and alternative water supply	Developed
2.2e	Drought prediction system	Ql	Status of city-scale drought prediction system	Developed
2.3a	City-scale climate adaptation planning	Ql	Status of city-scale climate change adaptation planning	Developed
2.3b	Renewable energy usage	Ql	Status of usage of renewable energy	Developed
	KPIs	Assessment	Evaluation Method	Based on
3.1a	Water source monitoring frequency	Qn	Frequency at which water source data is recorded	Developed
3.1b	Water source availability	Qn	Percentage of available water supply over city water supply consumption	Developed
3.1c	Water source data automation and quality control	Ql	Status of automation and quality assurance for water source recording instruments	Developed
3.1d	ICT-based water source data collection process	Ql	Status of ICT-based water source data collection	Park et al. (2022) [30]
3.1e	Water source data accessibility	Ql	Public access to water source data	Developed
3.2a	Drinking water quality compliance	Qn	Drinking water quality status compared to standard	Seoul Metropolitan Government (2022) [31]
3.2b	Drinking water quality monitoring frequency	Qn	Frequency at which drinking water quality is being monitored	Developed
3.2c	Drinking water treatment data automation and quality control	Ql	Status of automation and quality assurance for drinking water quality monitoring instruments	Developed
3.2d	ICT-based drinking water treatment data collection process	Ql	Status of ICT-based drinking water data collection	Developed
3.2e	Drinking water data accessibility	Ql	Public access to drinking water quality data	Developed
3.2f	Advanced drinking water treatment process	Ql	Application of advanced drinking water treatment process in the Purification plants	University of California (2022) [32]
3.3a	Water supply network distribution	Qn	Percentage of population with access to water supply over total population	Developed
3.3b	Aging water supply pipe status	Qn	Percentage of deteriorating water supply pipes over total pipe extension	Seoul Metropolitan Government (2017) [33]
3.3c	Revenue water percentage	Qn	Percentage of city water consumption over drinking water supply	Klepka et al. (2015) [34]
3.3d	Water storage effective capacity	Qn	Percentage of daily maximum water intake over maximum water storage capacity	Kwater (2017) [26]
3.3e	Water supply data automation and quality control	Ql	Status of automation and quality assurance for water supply monitoring instruments	Developed
3.3f	Water supply network maintenance	Ql	Application of ICT-based technology in water supply pipe maintenance	Developed
3.3g	Smart water metering	Ql	Applicaiton of smart water metering	Developed
3.3h	Water supply data accessibility	Ql	Public access to water supply data	Developed
3.4a	Sewage pipe network distribution	Qn	Percentage of population with access to wastewater distribution over total population	OECD (2022) [35]
3.4b	Aging sewage pipe status	Qn	Percentage of deteriorating sewage pipes over total pipe extension	Developed
3.4c	Sewage water quality monitoring frequency	Qn	Frequency at which wastewater quality is being monitored	Song et al. (2022) [36]
3.4d	Sewage water treatment data automation and quality control	Ql	Status of automation and quality assurance for sewage water monitoring instruments	Developed
3.4e	Separated sewage network	Qn	Percentage of separated sewer system over total sewage pipe extension	Developed
3.4f	Sewage pipe network maintenance	Ql	Application of ICT-based technology in sewage pipe maintenance	Developed
3.4g	Sewage water treatment process	Ql	Application of advanced wastewater treatment process in the Sewage plants	Water Corporation (2023) [37]
3.5a	Wastewater reuse	Qn	Percentage of recycled water over total water usage	Developed
3.5b	Solid water recycle	Qn	Percentage of sludge materials being recycled	Developed

Note(s): * Ql and Qn in the evaluation type pertains to qualitative and quantitative assessments, respectively.

.

Calculation methods are systematically compiled for each indicator based on an extensive literature review. For quantitative indicators, the computational formulas and standards are derived from the existing guidelines found in research articles, reports and frameworks established by credible organizations. For example, the recommended rainfall observation coverage density for urban areas according to the WMO (2020) [14] is at least one rain gauge for every 10–20 km² surface area. An example of the evaluation form for smart water city assessment can be seen in Figure 2. These types of indicators determine the percentage or ratio of a certain value and its correlation to the recommended value from the established literature. This involves the use of numerical data or statistical analysis, focusing on the quantifiable measures to analyze the performance of the city.

For the qualitative indicators, however, a set of criteria is presented to assess the performance of the city based on the existence or establishment of certain elements, such as the application of particular technologies, implementation of specific systems, adherence to standards and more. These indicators are designed to interpret non-numerical data and categorical variables. The formulation of these guidelines is derived from recommendations provided by experts from various water sectors. An example of the developed evaluation form for the smart water city assessment under a qualitative approach can be seen in Figure 3. This approach seeks to comprehensively evaluate the aspects that may not be easily quantified, utilizing expert-derived criteria to assess the city’s performance in smart water management.

A comprehensive set of 78 key performance indicator evaluation methods has been formulated to assess the efficiency of the city’s urban water management across various stages of the water system. For a complete version of the evaluation forms, kindly refer to the attached Supplementary Materials. Weighing values are assigned to each indicator, subcategory and category based on their significance in the urban water cycle. Finally, the final scores are presented, providing a clear depiction of the strengths and weaknesses in the city’s urban water management system. This assessment serves as a valuable tool for identifying potential improvements in the future.

3. SWC Pilot Testing

A pilot evaluation was undertaken to assess the feasibility of the new initiative on a smaller scale level, aimed at determining the city’s strengths, as well as the potential issues in their management of the urban water. The selection of the pilot city was conducted on the basis of (1) the relevance of the evaluation to identify the needs and challenges of the city, highlighting its goals and priorities, (2) the commitment of the local stakeholders to ensure the active participation in the evaluation and (3) the feasibility of implementing the assessment to the selected city, particularly considering the availability of the relevant data needed for the evaluation.

The smart water pilot city evaluations have been executed for a city, determining the applicability of the developed smart water city evaluation methodology. The pilot city assessments were conducted on the urban water management of Busan Eco Delta City, located in Busan Metropolitan City, Republic of Korea.

3.1. Busan Eco Delta City (Busan Metropolitan City), Korea

The smart water city technical evaluation is applied to Busan Eco Delta City (BEDC), located in Busan, Republic of Korea (Figure 4). BEDC is a new urban development in the western part of Busan, poised to accommodate approximately 76,000 residents. The new urban district has a total surface area of 11.8 km², dedicated to prioritizing building for housing, commerce, logistics and research and development. BEDC is designed to underscore the efforts to create a new city that integrates blue and green spaces into various aspects of urban living. The city’s diverse building units serve as the testing ground for different water technologies, offering valuable insights in the practical application and adaptability of smart water technologies. The BEDC assessment can serve as a unique exploration of the smart water city evaluation, presenting an advanced benchmark aspiring to integrate cutting edge technologies into local water management.

The indicators for the technical aspect of the evaluation utilized information drawn from both BEDC and the city of Busan as a whole. Due to the fairly recent establishment of BEDC, some indicators may encounter insufficient information. Busan Metropolitan City has a total surface area of 770.9 km², serving about 3.47 million inhabitants (2022). The metropolitan city is composed of mountainous and coastal areas, with its ports ranking among the busiest ports in the world. The Nakdong River, South Korea’s largest river, flows through the city, with its estuaries leading to the East China Sea, serving as the primary water source for 94% of the city’s population. Busan Eco Delta City is designated as Busan’s eco-friendly waterfront city, situated in the junction of the West Nakdong River, Pyeonggang Stream and Maekdo River.

The data used to for the evaluation of Busan Eco Delta City were gathered from either the surrounding districts of BEDC (Gangseo, Sasang and Buk Districts) or from the broader datasets of Busan Metropolitan City. In the Republic of Korea, the majority of the water information required for the smart water city evaluation is accessible online. A comprehensive list of agencies and their corresponding data source for each indicator is presented in

Table 3. Smart water city evaluation technical KPI data sources and access for BEDC and Busan Metropolitan City.

Data	Source Agency	Data Source
Precipitation	Korea Meteorological Administration (KMA) (Open data portal) Water Resources Management Information System (WAMIS)	data.kma.go.kr (accessed on 10 April 2023) wamis.go.kr (10 April 2023)
Impervious surface	Korea Water Resource Corporation (K-water)	kwater.or.kr (24 April 2023)
Urban surface water	Busan Metropolitan City Government Busan Development Institute (BDI)	busan.go.kr (14 June 2023) Yeo et al. (2021) [38]
Stream water level	Kwater Busan Open Data Portal WAMIS	water.or.kr (14 June 2023) data.busan.go.kr (12 May 2023) wamis.go.kr (4 May 2023)
Stream water quality	Kwater Busan Metropolitan City Government WAMIS Water Environment Information Center	data.busan.go.kr (4 May 2023) busan.go.kr (22 May 2023) wamis.go.kr (22 May 2023) water.nier.go.kr (3 July 2023)
Groundwater level	National Groundwater Information Center Integrated Groundwater Services	gims.go.kr (21 June 2023) gims.go.kr/en (21 June 2023)
Groundwater quality	National Groundwater Information Center Integrated Groundwater Services WAMIS	gims.go.kr (21 June 2023) gims.go.kr/en (20 June 2023) wamis.go.kr (4 May 2023)
Flood	WAMIS Busan Public Data Portal Busan Metropolitan City Urban Flood Integrated Information Busan Metropolitan City Government	wamis.go.kr (25 May 2023) data.go.kr (3 July 2023) safecity.busan.go.kr (4 July 2023) busan.go.kr (15 July 2023)
Drought	National Drought Information Portal	drougt.go.kr (15 July 2023)
Climate change	Korea University Korea Advanced Institute of Science and Technology (KAIST)	Lee and Kim (2018) [39] Baek et al. (2016) [40]
Water source	Busan Water Authority National Water Supply Information System Kwater Information Portal Busan Metropolitan City Government	busan.go.kr/water (25 July 2023) waternow.go.kr (25 July 2023) water.or.kr (24 July 2023) busan.go.kr (24 July 2023)
Drinking water	Busan Water Authority Busan Metropolitan City Government	busan.go.kr/water (7 August 2023) busan.go.kr (7 August 2023)
Water distribution	National Water Supply Information System Korea Statistical Information Service (KOSIS) Busan Water Authority	waternow.go.kr (24 July 2023) kosis.kr (7 September 2023) busan.go.kr/water (25 July 2023)
Wastewater	KOSIS Ministry of Environment (ME) Busan Environmental Corporation (BECO) Korea Environment Corporation (KECO)	kosis.kr (7 September 2023) me.go.kr (18 September 2023) beco.or.kr (18 September 2023) keco.or.kr (25 July 2023)
Wastewater reuse	KOSIS	kosis.or.kr (7 September 2023)

.

3.2. Evaluation

The smart water city evaluation is performed for Busan Eco Delta City (BEDC) in Busan, South Korea. Scores were assigned based on the evaluation of the water data gathered from both cities. The comprehensive evaluations of the pilot city are detailed as follows.

The smart water city evaluation for Busan Eco Delta City, as part of Busan Metropolitan City, is presented for each category in Figure 5, Figure 6 and Figure 7. The horizontal axis delineates the individual key technical key performance indicators, while the vertical axis illustrates the final scoring for each KPI. The graphical representations offer an overview of the city’s performance across various water management categories, showing its strengths and areas for improvements.

Busan Eco Delta City excels in several smart water city indicators within the urban water cycle category, demonstrating the city’s strong commitment to efficient urban water data management. BEDC achieved a high score in the following: monitoring coverage; frequency; missing data percentage and data accessibility to key parameters such as rainfall, stream water level, stream water quality, groundwater level and groundwater level; the establishment of waterfront amenities; the application of LID and green infrastructures and water quality standards for stream and groundwater. This implies that the city exhibits smart water management in facilitating and monitoring the urban water data. The thorough spatial and temporal observation of the hydrological parameters, coupled with the application of ICT and green infrastructures and easy access to data, ensures a high level of awareness regarding the city’s urban hydrology. Notably, excellent scores in stream water and groundwater quality standards indicate minimal pollutants in water sources, signifying good overall urban water health. The prevalent use of nature-based solutions further underscores the city’s commitment to preserving water quality through natural means.

However, the evaluation highlights areas for improvement, particularly in addressing missing data, specifically for the stream water level and groundwater level. Mitigating missing hydrological data is crucial to avoiding biases in model simulations and ensuring accurate interpretations for future water level forecasts. Additionally, enhancing the number of rain gauge stations is recommended to monitor rainfall spatial distribution effectively, especially in densely populated areas. These improvements will further enhance the city’s smart water management capabilities and contribute to a more resilient urban water system.

The water disaster management in Busan Eco Delta City and, by extension, Busan Metropolitan City demonstrates effective preparation and mitigation strategies, evident from the full scores attained in various crucial aspects. These include the flood property index; flood risk area index; flood hazard mapping; integrated disaster information system; urban flood prediction and early warning; drought damage index; drought information and availability of emergency water supplies; drought prediction system and the implementation of city-scale climate adaptation planning, renewable energy usage and energy-saving strategies. These results underscore the city’s exceptional performance in predicting and mitigating localized flood events through the application of modern technologies. Smart water management practices are evident in the utilization of advanced technologies for flood hazard mapping, an integrated water disaster information system, city-scale flood and drought forecasting and flood early warning systems. The city’s proactive climate preparatory strategies, encompassing climate change actions, adaptation measures and a focus on alternative renewable energy sources, contribute to maintaining low risks in flood damage and flood-susceptible areas. These efforts serve as significant indicators of effective smart water management.

Despite these achievements, the analysis highlights the city’s vulnerability to climate risk events, particularly drought events and flood-related casualties recorded in recent years. While the city performs well in many water disaster management indicators, some deficiencies are noted. These include a relatively low percentage of completed flood preventive structures and a need for an enhanced usage of advanced drought forecasting and impact assessment in drought hazard mapping. The construction of levees and dams proves instrumental in reducing the risk to urban residents during flood events, while modern procedures for drought hazard development ensure the accuracy and efficiency of drought prediction. Addressing these deficiencies will further strengthen the city’s resilience to water-related disasters and enhance the overall efficacy of its smart water management practices.

Lastly, the city managed to obtain exceptional points in the water supply and treatment management, particularly in key areas such as monitoring the frequency of water sources and wastewater, application of ICT-based technologies in data collection, availability of consumable water, capacity of water treatment plants, compliance with safe drinking water standards, service coverage of water supply distribution, accessibility to water-related data, application of advanced technologies in water treatment, maintenance of pipelines, installation of smart meters, acceptable quantity of aged pipelines and a high percentage of treated wastewater being recycled. These outcomes underscore the city’s embodiment of smart water city characteristics in effectively managing the water supply, drinking water treatment and wastewater treatment. The concerted efforts of responsible agencies in facilitating the water supply contribute to the safety of drinking water and the reliability of facility services, ensuring efficient water distribution and minimizing non-revenue water, a critical factor in water conservation. However, relatively lower scores are obtained in the application of a separated sewage network and recycling of wastewater sludge materials. The implementation of a separated storm and sewage system ensures the consistency of stormwater flow during flood events, while the application of biowaste recycling helps in the reduction of wastewater byproducts that can affect the health of the ecosystem if not disposed of properly.

However, there are relatively lower scores in the application of a separated sewage network and recycling of wastewater sludge materials. Implementing a separated storm and sewage system is crucial for managing the stormwater flow during flood events and the application of biowaste recycling aids in reducing the environmental impact of wastewater byproducts. Addressing these aspects will further enhance the city’s overall water treatment and conservation efforts, contributing to a more comprehensive and sustainable smart water management system.

3.3. Overall Pilot Testing Assessment

An overall evaluation for BEDC is detailed in Figure 8. By examining each of the key performance indicators under sustainability and smartness, insights can be gathered on the effectiveness and efficiency of the pilot cities’ water management practices, contributing to a broader understanding of their capabilities in the application of advanced technologies for sustainable and resilient urban water management systems.

In terms of sustainability, Busan Eco Delta City excelled in the stream water level, climate change, water source and wastewater reuse indicators. The city demonstrated effective sustainable management by maintaining a high level of proficiency in monitoring the stream water levels through well-distributed water level stations, ensuring minimal error data, and recording frequencies. Additionally, its proactive approach to climate change actions, robust adaptation strategies, the meticulous monitoring of primary water sources and efficient recycling of treated wastewater underscored the city’s commitment to sustainable water practices. Under the smartness category, the city achieved full scores in the surface water, flood, and climate change indicators. This success is attributed to Busan Eco Delta City’s implementation of low-impact development and green infrastructures, facilitating a natural urban hydrological flow. The city’s utilization of advanced flood hazard mapping, an integrated disaster information center and urban flood forecasting and early warning systems during flood events showcased its advanced technological measures. Furthermore, the city’s endeavors to maximize renewable energy usage and employ energy-saving strategies underscore its qualification as a smart water city.

3.4. Alignment with Previous Standards

The smart water city assessment applied to Busan Eco Delta City and Busan Metropolitan City involved an analysis of the water-related aspects using the established frameworks for smart cities. These frameworks were selected for their alignment with the key themes that they examine, such as sustainability, resilience, and smartness, as well as their relevance at the local level. Particular recurring indicators include surface and groundwater quality (GCF); green space (CITYKeys, LEED and Arcadis); stormwater management (LEED); flood vulnerability status (LEED, Arcadis and GCF); climate resilience (ISO 37120, OECD, CITYKeys and GCF); the application of renewable energies (LEED); accessibility to a water supply (U4SSC, ISO 37120, LEEDS and Arcadis); water loss in the water distribution system (U4SSC, ISO 37120, CITYKeys, Arcadis and GCF); accessibility to wastewater collection services (U4SSC, ISO 37120 and LEED); compliance of the drinking water and wastewater treatment quality (ISO 37120, LEED, Arcadis and GCF); implementation of smart water meters (U4SSC, ISO 37122, OECD, LEED and Arcadis); utilization of ICT-based technologies in the monitoring of water distribution systems (U4SSC, ISO 37120 and OECD); number of water quality monitoring systems (ISO 37122); application of advanced wastewater treatment (U4SSC); recycling of wastewater and biosolids (ISO 37122, CITYKeys, Arcadis and GCF); sewage pipe monitoring using ICT (ISO 37122 and GCF) and alternative water source and water sufficiency (ISO 37123, Arcadis and GCF).

The remaining KPIs were supplemented through an extensive literature review of individual indicators, an examination of the average performance of progressive cities and surveys based on experts’ opinions. The availability of the water data was confirmed, and the computational analysis was applied for the pilot city. In conclusion, all 78 developed KPIs were assessed either quantitatively or qualitatively, following sets of standards from established references.

4. Conclusions and Recommendations

The evaluation of smart water cities underscores the crucial role of advanced technologies in the efficient management of urban water, hereby enhancing sustainability and improving the quality of life of residents. The primary objective of this research is to develop measurements and evaluation tools for SWC key performance indicators, focusing on innovative water technologies to establish unified global standards and certification schemes.

The indicators are categorized into three stages: the urban water cycle, water disaster management and water supply and treatment. These categories aim to assess the effective use of technologies in ensuring a sufficient water supply, monitoring the water quality, enhancing disaster resilience, and maintaining the overall urban water system. The developed evaluation methods and measurement tools are applied to evaluate the urban water management practices of Busan Eco Delta City in Busan, South Korea. The established evaluation processes serve as guideline basis for smart water city certification, offering a framework for analyzing future smart water cities. The evaluation would identify the city’s good water practices and, at the same time, pinpoint the areas of water management that need further improvements.

Based on the overall evaluation, the technical assessment of Busan Eco Delta City highlights the application of smart water city technologies and strategies in the management of its urban water. Smart water cities leverage the usage of advanced technologies and smart water management techniques to optimize urban water management, ensuring consistent monitoring, efficient water usage and the health of the population and the environment. Further improvements can be suggested in the following areas: Minimizing erroneous or missing data specially on the stream water quality and groundwater level observations necessitates a commitment to consistent instrument calibrations and data quality control implementations. To enhance urban water management practices, it is recommended to prioritize the installation of separated stormwater and sewage water pipe networks. This approach mitigates floodwater contamination by segregating sanitary wastewater and stormwater runoff. Furthermore, efforts should be directed towards expanding the utilization of recycled sludge materials. This can include their application as fertilizers in agriculture and as materials in cement production for construction purposes. Such measures contribute to sustainable waste management practices and resource conservation. Lastly, it is advisable to install more rainfall monitoring stations within the city to ensure the accurate recording of rainfall distribution that is critical for city-scale weather forecasting and flood early warnings. This step ensures comprehensive coverage and reliable recording of rainfall distribution, enhancing the overall resilience of urban water management systems.

The developed smart water city evaluation scheme not only emphasizes the utilization of smart water technologies in enhancing the overall urban water management but also accentuates the sustainable practices adopted by cities to safeguard the safety and improve the quality of life of the residents. Parameters such as the recommended number of the monitoring stations, recording intervals, acceptable levels of missing data, compliance to quality standards, application of flood hazard mapping tools, access to water information and implementation of recycling and reuse strategies serve as crucial measurement tools to improve sustainability, particularly for climate-vulnerable, lesser-developed cities. However, while this study evaluates the technical capacities of cities to maintain urban water sustainability, a comprehensive evaluation of water governance is also imperative. Water governance pertains to the economic, political, and administrative systems set in place by the city government to manage the urban water resources and effectively implement its policies. An assessment of the technical, as well as governance, aspects of smart water cities can provide valuable insights to serve as models for efficient management of the urban water system.

As a next step, we are aiming to further develop and refine the scoring and measuring tools to evaluate average cities in developing countries. This evaluation would examine the current practices, for both the technical and governance aspects, adopted by average cities, highlighting the smart innovations tailored to their specific urban water issues and addressing the limitations in their urban water management practices. By performing smart water city evaluations, improvements can be made towards facilitating more effective management while simultaneously sharing novel approaches implemented by smart water cities worldwide, contributing to the improvement of urban water cities in the future.