Next Article in Journal
Predicting Public Bicycle Adoption Using the Technology Acceptance Model
Next Article in Special Issue
Exploring No-Cost Opportunities for Public Sector Information Systems Energy Efficiency: A Tennessee Application
Previous Article in Journal
Antecedents and Consequences of Firms’ Climate Change Management Practices: Stakeholder and Synergistic Approach
Previous Article in Special Issue
Approaching Integrated Urban-Rural Development in China: The Changing Institutional Roles
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 7
Issue 11
10.3390/su71114537
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

From Indicators to Policies: Open Sustainability Assessment in the Water and Sanitation Sector

by
Martín Alejandro Iribarnegaray
1,
María Laura Gatto D’Andrea
1,
María Soledad Rodriguez-Alvarez
1,
María Eugenia Hernández
1,
Christian Brannstrom
2 and
Lucas Seghezzo
1,*
1
Instituto de Investigaciones en Energía No Convencional (INENCO), Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), Universidad Nacional de Salta (UNSa), Avenida Bolivia 5150, A4408FVY Salta, Argentina
2
Department of Geography, Texas A&M University, College Station, TX 77843, USA
*
Author to whom correspondence should be addressed.
Sustainability 2015, 7(11), 14537-14557; https://doi.org/10.3390/su71114537
Submission received: 21 July 2015 / Revised: 24 September 2015 / Accepted: 26 October 2015 / Published: 30 October 2015
(This article belongs to the Special Issue Government Policy and Sustainability)
Browse Figures
Versions Notes

Abstract

:
A water and sanitation sustainability index (WASSI) was developed and estimated in four cities of the province of Salta, in northern Argentina. The index was built with nine descriptors and fifteen indicators that covered all essential aspects of the sustainability of local water and sanitation management systems. Only one of the cities studied obtained a sustainability value above the acceptability threshold adopted (50 of 100 points). Results indicate that the water company needs to address some environmental and social issues to enhance the sustainability of the systems studied. The WASSI was conceptually robust and operationally simple, and could be easily adapted to the case studies. The index can be followed and updated online on a web site specially developed for this project. This website could be useful to promote participatory processes, assist decision makers, and facilitate academic research. According to local stakeholders, a more open sustainability assessment based on sustainability indices and supported by virtual tools would be relevant and highly feasible. It would help decision makers improve the sustainability and transparency of water and sanitation management systems, and promote more sustainable water policies in the region and beyond.

1. Introduction

Sustainability indicators can be used to assess products, sectors, companies, regions, and even countries [1,2]. A number of assessment methods based on sustainability indicators are now available, covering a wide variety of issues and methodological approaches [3,4,5,6,7]. The basic assumption behind the use of sustainability indicators is that they actually assess the “sustainability” of the case under assessment. However, the very concept of sustainability is complex and contested since it reflects, to a great extent, people’s worldviews, social perspectives, and interests [8]. Therefore, the use of sustainability indicators requires a degree of consensus among policy makers, scholars, and other relevant local actors on the concepts and methods that will be used for the assessment [9,10,11].
It is generally accepted that the ultimate goal of sustainability assessment is to assist in decision-making processes [12,13,14,15,16,17]. However, the fact that results from these assessments are rarely incorporated into official decision-making processes has put sustainability indicators under scrutiny [18,19]. Particularly contentious issues in the use of these indicators are concerns with their transparency, openness, and democratic character as a result of the exceedingly technical approaches used that tend to preclude the participation of end users and other relevant stakeholders [17].
The water and sanitation sector is not an exception to this general context. Sustainability indicators have been proposed to evaluate the performance and quality of Water and Sanitation Management Systems (WSMS) [20,21,22,23,24,25]. Such assessments are greatly needed to ensure the provision and sustainable management of water and sanitation, as per Goal Six of the new Sustainable Development Goals (SDG) of the United Nations [26]. As a general framework, the SDG also recommends to “build on existing initiatives to develop measurements of progress on sustainable development”. Such measurements, necessary to understanding and assessing specific management processes, are increasingly being carried out with the help of local stakeholders via participatory processes [27,28,29,30,31,32]. Essential to more genuine participation is unrestricted access to all available information on local systems. Recent advances in information and communication technologies could greatly facilitate this [33,34,35,36]. A more participatory sustainability assessment is a necessary component to improve sustainable “water governance”. Namely, the assessment needs to incorporate the full range of political, organizational and administrative processes through which community interests are articulated, their input is incorporated, decisions are made and implemented, and decision-makers are held accountable in the development and management of water resources and delivery of water services [37,38]. The concept of water governance is based on the interaction process between contributing stakeholders to define clear sustainable water policy goals and targets at all levels of government, ensuring inclusiveness of stakeholders through democratic legitimacy and fairness for society at large [39].
In this paper, we reflect on our experience working with sustainability indicators in the water and sanitation sector in the province of Salta, in northern Argentina. We developed a sustainability index in collaboration with the local water company. The sustainability index was initially tested in the provincial capital (Salta, pop. 550,000) and then validated in three smaller cities under different climatic, geographic, and demographic contexts. For each city, areas were identified where improvements and corrections were needed. To facilitate the use and dissemination of this assessment tool an online interface was developed by which companies, the government, non-governmental organizations, and end users could access real time information on the water and sanitation system, introduce new data, generate their own reports, and make suggestions or comments. Initial feedback from the water company, the regulatory body, and other local stakeholders suggests that this more “open sustainability assessment” methodology better responds to contemporary management needs and can be useful in monitoring water and sanitation systems, and promote more transparent and sustainable water policies.

2. Materials and Methods

2.1. The Water and Sanitation Sustainability Index

The water and sanitation sustainability index (WASSI) was developed as a tool to support governance processes for more sustainable water management [23,40]. The WASSI was built on the five-dimensional sustainability concept proposed by Seghezzo [11]. This idea can be represented with a new sustainability triangle formed by “Place”, “Permanence”, and “Persons”. Place contains the three dimensions of space (x, y, and z), Permanence is the fourth dimension of time (t), and the Persons corner is the fifth, human dimension (i). The corners of this sustainability triangle are closely interrelated and are difficult to deal with in a fragmented way as is generally the case for economic, environmental, and social issues. In the WASSI, each one of these three aspects was described with three “descriptors” [41] or “orientors” [1]. Descriptors could be seen as the characteristics that need to be known to get a comprehensive idea of the system under analysis. Place is related to the system interaction with the biophysical territory upon which it operates (descriptors included availability, infrastructure and equity). The permanence aspect intends to shed some light on the short, medium, and long-term aspects of the WSMS, reflecting local capacity to solve problems and improve the system (descriptors include access, planning and participation). Finally, the persons aspect put the attention on the human dimension that seems increasingly relevant in the case of water and sanitation management (descriptors include use, impact and satisfaction). Descriptors were assessed in terms of one or more “indicators” and variables were selected based on their relevance to assess the satisfaction of the descriptors (see Table 1). For comparison, the same set of descriptors was used in each of the four case studies. Indicators and variables varied slightly with the specifics of each case study, the information available, and the opinions of the scholars, policy-makers, and other stakeholders that were consulted for this assessment.
Table
Table 1. Descriptors, indicators and variables used to build the Water and Sanitation Sustainability Index (WASSI). DW = Drinking Water.
Table 1. Descriptors, indicators and variables used to build the Water and Sanitation Sustainability Index (WASSI). DW = Drinking Water.
DescriptorIndicatorShort DescriptionVariableShort Description
AvailabilityDW QuantityQuantity of drinking waterWater ProvisionWater from surface, sub-surface, and groundwater sources (L/p.d)
TrendsQuantity or quality trendsAquifer DepthYearly average increase in the depth of water wells (m/year)
DW QualityQuality of drinking waterResidual ChlorinePercentage of Residual Chlorine out of range in water samples (%)
Arsenic ConcentrationPercentage of arsenic out of range in water samples (%)
InfrastructureRisksWater safetyWater SafetyRelative risks identified by the Water Safety Plan (WSP)
LifetimeLifetime of water infrastructureInfrastructure AgeAge of water and sanitation infrastructure (year)
DiseasesEffects on human healthWater Related DiseasesOccurrence of diarrheas in children under five in critical areas
EquityCoverageSpatial differences of servicesLack of Sewage ServicesPopulation without sanitation services (critical areas/control) (%)
AccessCostsEconomic accessibilityRelative Water CostsProportion of minimum wage for water and sanitation services (%)
InformationInformation accessibilityWeb SitesAssessment of the information contained in institutional web sites
RightsAccess to basic water needsBasic Water AllowanceGuaranteed amount of water for basic needs (L/p.d)
PlanningInstitutional QualityInstitutional capacityInstitutional AssessmentAssessment in terms of funds, planning, and personnel
ParticipationInteractionsParticipation instancesParticipation EventsNumber of significant participation events per year (Events/year)
UseConsumptionWater consumptionExcess ConsumptionRelative water consumption above a given target value (%)
ImpactPollutionEnvironmental pollutionUntreated SewageUntreated/poorly treated sewage discharged into environment (%)
SatisfactionPerceptionPublic perceptionComplaintsNumber of technical servicerelated complaints (No./1000 p)

2.2. Index Calculation

For the calculation of the indicators of WASSI, quantitative and qualitative values assigned to the different variables were converted into a centesimal sustainability scale modified from Bossel [1], as follows: Value < 25 = great concern (red); 25 ≤ Value < 50 = medium concern (yellow); 50 ≤ Value < 75 = little concern (green); and Value ≥ 75 = no concern (blue). Transform functionsfrom quantitative variables into the sustainability scale were built assuming linear relationships and identifying, for each variable, two “anchor points” that represent the best and worst values in terms of sustainability [42,43]. For a more detailed description, Figure 1 shows a graphic example where a variable measured in days is defined by anchor points 5 (sustainability = 0) and 0.2 (sustainability = 100). This defines a linear function that can transform units to a sustainability scale (1.9 days to 64% of sustainability). The reference condition or “band of equilibrium” [2] was set at threshold of 50. This threshold can be adjusted or modified to better fit the needs of management or to foster a process of continuous improvement. Information was analyzed using the Simple Multiple Attribute Ranking Technique (SMART), a multi-criteria decision-making method based on the Analytical Hierarchy Process (AHP) [42,44].
Sustainability 07 14537 g001 1024
Figure 1. Graphic example describing in detail the mechanism by which variables can be converted to a comparable sustainability scale. NC = No Concern; LC = Little Concern; MC = Medium Concern; GC = Great Concern.
Figure 1. Graphic example describing in detail the mechanism by which variables can be converted to a comparable sustainability scale. NC = No Concern; LC = Little Concern; MC = Medium Concern; GC = Great Concern.
Sustainability 07 14537 g001
Information for the calculation of the WASSI was obtained by several means, including literature and press reviews and field visits. Semi-structured interviews were also held with technical staff of the provincial water company (CoSAySa—Compañía Salteña de Agua y Saneamiento S.A.), the governmental control agency of the province (ENRESP—Ente Regulador de los Servicios Públicos), and the provincial Secretary of Water Resources (SRH—Secretaría de Recursos Hídricos). It is important to note that anchor points were selected based on a literature review and several workshops held within the research group. The selection of case studies and indicators used for the construction of WASSI was held in the context of a partnership agreement between National University of Salta (UNSa) and CoSAySa. This project allowed for a close relationship with technicians and decision makers of the water company and other regional institutions. Indicators and variables of WASSI in each case study were also discussed with CoSAySa and other representatives of local institutions. These values need to be adapted to each particular setting, and they can also change over time. Some variables like websites and institutional assessments demanded qualitative evaluations conducted by the research team. The quality of the website of CoSAySa (www.aguasdelnortesalta.com.ar) was evaluated by analyzing technical criteria like quality and quantity of the information available, interaction capabilities, clarity, among others, and was then assigning a score from 0 to 100. Institutional local capacity in each case study was assessed by means of an assessment that involved criteria clustered in three main topics: funds, planning, and personnel. The variable water safety (for indicator risks) was estimated together with personnel from CoSAySa by performing a Water Safety Plan (WSP) based on the methodology proposed by the World Health Organization (WHO) [45]. Due to space limitations, a detailed description of all calculations is not feasible in this paper. More details about the calculation of indicators and variables can be found in Iribarnegaray et al. [23] and Iribarnegaray and Seghezzo [40] Information needed to estimate indicators and variables was updated to December 2014.

2.3. Case Studies

The WASSI was estimated in four cities of the province of Salta, in Northwestern Argentina, namely the capital city Salta (SAL), San Ramón de la Nueva Orán (SRNO), Joaquín Víctor González (JVG), and San Antonio de los Cobres (SAC) (Figure 2 and Table 2).
Table
Table 2. Short description of the case studies.
Table 2. Short description of the case studies.
CharacteristicCity
SALSRNOJVGSAC
Population535,30084,00013,3006100
Altitude (m.a.s.l.)11873363663775
Type of climateSubtropical and temperateSubtropical and warmSubtropical with a dry seasonArid and cold
Temperature (°C)16.922.321.57.7
Rainfall (mm)689945578117
Water sourcesDeep wells, surface and sub-surface waterDeep wells and sub-surface waterDeep wellsSmall creek
Water coverage (%)94828078
Sewerage (%)88755050
Main problemsPopulation growth; aging infrastructureInformal settlements; outbreaks of water-borne diseasesWater leakages; lack of local institutional capacitySource water contains arsenic; understaffed water office
OthersHeadquarters of CoSAySa, the provincial water companyPopulation with many basic needs despite intensive agro-industrial activitiesLocated at the center of the soybean region, in the flatland known as the “Chaco” regionStrong temperature fluctuations during the day (down to −20 °C in winter)
Sustainability 07 14537 g002 1024
Figure 2. Case studies selected in the province of Salta, in Northwestern Argentina. SAL: Salta; SRNO: San Ramón de la Nueva Orán; JVG: Joaquín Víctor González; SAC: San Antonio de los Cobres.
Figure 2. Case studies selected in the province of Salta, in Northwestern Argentina. SAL: Salta; SRNO: San Ramón de la Nueva Orán; JVG: Joaquín Víctor González; SAC: San Antonio de los Cobres.
Sustainability 07 14537 g002
These cases were selected to maximize the diversity with respect to the number of inhabitants, environmental conditions, water sources, water and sewerage coverage, and main management problems. Information and data were obtained from the National Institute of Statistics and Censuses (INDEC) (www.indec.mecon.ar), CoSAySa (www.aguasdelnortesalta.com.ar), the Ministry of Interior and Transport (www.mininterior.gov.ar), and the ENRESP (www.entereguladorsalta.gov.ar).

2.4. Virtual Interaction and Monitoring Spaces

A thorough description of the cases, methodological details, and the results obtained in this study can be found online at the project’s web site (www.isinenco.com.ar). This site, specifically developed for this project, was created both as a source of information and as a tool to facilitate the transparency and sustainability of local water management systems. The opinion of relevant local actors on the potential of a more open sustainability assessment method facilitated by online tools was solicited in open interviews. The selection of actors was based on a previous study we conducted in the city of Salta using Q methodology (see [19]), in which we identified four social perspectives on water and sanitation, namely: (a) Rights-based consumption advocates; (b) Proponents of market-based and technical water management; (c) Participatory governance advocates; and (d) State-led governance supporters. For the current study, we interviewed one actor for each social perspective identified [46,47,48]. At the start of each interview, we discussed the values obtained for the WASSI in the cities studied, focusing on the potential benefits of the method and possible obstacles for its widespread dissemination. Subsequently, interviewees were provided with a quick visit of the project web site described above. We then asked their opinion on the idea of a more open sustainability assessment in the water and sanitation sector in terms of transparency, technical quality, social acceptability, impact, and other criteria they perceived as being important.

3. Results

As revealed in Table 3, the WASSI was relatively low for all cases, with only one city (JVG) above the acceptability threshold of 50 points. The city of Salta (SAL) obtained a slightly lower value (36) than in a previous assessment (41) (see [23]), due to adjustments in some variables (relative water costs, excess consumption, and untreated sewage).
Even though all cities studied are served by the same water company, differences could be detected for some indicators. For instance, this was the case for the indicator DW quantity, which was assessed through the amount of water provided by the company per person (variable: water provision). This indicator was limiting for SAC (Table 3, line 1, column M) but not for the other three cities. For this variable, the upper anchor point of 250 L/p.d. was adopted based on the recommendation of Provincial Law N°7017/98. We also found differences for the indicator coverage (variable: lack of sewage services). With the exception of SAL, the rest of the cities lack adequate coverage of sewage services (collection and treatment) (Table 3, line 8). This indicator is usually critical in developing countries, where water companies concentrate efforts on the provision of drinking water, and where sewage services might be circumscribed to urban cores or affluent neighborhoods, leaving the outskirts of the city and other marginal areas unattended. The spatial unfairness of this situation is unacceptable in terms of sustainability [49]. When it comes to water consumption, SAL and SRNO have a daily per capita consumption that exceeds the value recommended by local regulations by more than 50% (see variable excess consumption in Table 3, line 14, columns C and D). Citizens from JVG consume less water, but they still exceed the recommended value (Table 3, line 14, column E). SAC was at the other end of the spectrum (Table 3, line 14, column F). In this city, due to severe water shortages, water consumption is even lower than the recommended value and the sustainability value assigned was therefore at the maximum (Table 3, line 14, column M).
The calculation procedure was different for specific indicators. The indicator DW quality was calculated using two possible methods, depending on the city. For SAL, SRNO, and JVG, the percentage of water samples from the grid with residual chlorine out of range (variable: residual chlorine) was used. In this case, worst and best anchor points were five and 1% of the water samples, respectively (Table 3, line 3, columns H and I). SAC was a particular case because its current drinking water source contains high concentrations of arsenic. For this city, arsenic concentration was considered a better indicator of water quality than the degree of water disinfection. As shown in Table 3 (line 4, column F), all assessed water samples contained arsenic concentrations exceeding the standard of 0.01 mg/L established as the national norm (the Argentinian Food Code).
Some variables could not be calculated in all cities due to lack of information. The indicators trends and diseases, for instance (Table 3, lines 2 and 7), could only be calculated for SAL and SRNO. Statistics for these indicators were not available in the other two cities due to institutional limitations. This situation affected the estimation of the indicator institutional quality, which was significantly higher for SAL, location of the water company’s headquarters (Table 3, line 12, column C).
Table
Table 3. The WASSI in the four cities studied. MC = Medium Concern; LC: Little Concern. DW = Drinking Water.
Table 3. The WASSI in the four cities studied. MC = Medium Concern; LC: Little Concern. DW = Drinking Water.
ABCDEFGHIJKLMNOPQR
IndicatorsVariablesValuesUnitsAnchor PointsSustainability
IndicatorsDescriptors
SALSRNOJVGSAC0100SALSRNOJVGSAC SALSRNOJVGSAC
1DW QuantityWater Provision489.3805.5386.0157.0L/p.d5025010010010043Availability597310021
2TrendsAquifer Depth1.00.0 m/y2048100
3DW QualityResidual Chlorine3.84.30.0 % samples512918100
4 Arsenic Concentration 100.0% samples500 0
5RisksWater Safety30.742.246.442.7-100069585457Infrastructure56405229
6LifetimeInfrastructure Age28.840.035.050.0year50207133500
7DiseasesWater Related Diseases0.30.3 -012728
8CoverageLack of Sewage Services84.0100.0100.0100.0%1001519000Equity19000
9CostsRelative Water Costs1.81.51.51.5%50.268747474Access68707070
10InformationWeb Sites36.736.736.736.7-010037373737
11RightsBasic Water Allowance87.779.379.379.3L/p.d550100100100100
12Institutional QualityInstitutional Assessment62.623.336.031.8-010063233632Planning63233632
13InteractionsParticipation Events0.20.40.60.0Events/year012040600Participation2040600
14ConsumptionExcess Consumption82.853.519.4−78.3%5000061100Use0061100
15PollutionUntreated Sewage41.428.231.426.0%50017443748Impact17443748
16PerceptionComplaints190.038.856.74.2No./1000 p2504248679100Satisfaction248679100
WASSI =36425537
Sustainability range =MCMCLCMC
The values of other indicators, like coverage, were very low in all cities (Table 3, line 8, columns C through F). This indicator was based on the provision of sewage services in critical areas of each city [23]. Indicators included in descriptor access (costs, information, and rights) (Table 3, lines 9, 10, and 11) are relatively independent of the location, since they are related to national and provincial policies (such as the minimum wage, water cost per cubic meter, information shared through the company’s web site, and the minimum water allowance provided for free per household). For this reason, variables have the same value in all cases (differences in the variable relative water costs originate in slightly different water costs for SAL) (Table 3, line 9, columns C through F). It is expected that these types of indicators will be more sensitive in relation to comparing the WSMS of different regions or countries.
Finally, the indicator perception, based on the number of user complaints, may be somewhat misleading. The low value obtained in SAL (Table 3, line 16, column J) does not necessarily mean that the service is worse in this city. It simply shows that either people in SAL complain more, or that the water company keeps a record of their complaints. This indicator might be determined by the availability and quality of internet services, the spread of cell phone coverage, citizens’ awareness of their rights, the degree of trust in the ability of the water company to solve problems, income, or other site-specific cultural aspects.
As shown in Figure 3, several descriptors fell below the acceptability threshold in all cities, reducing the sustainability of their WSMS.
Sustainability 07 14537 g003 1024
Figure 3. Values obtained for the descriptors of the WASSI ordered according to the averages obtained for the four cities (averages shown within bars).
Figure 3. Values obtained for the descriptors of the WASSI ordered according to the averages obtained for the four cities (averages shown within bars).
Sustainability 07 14537 g003
Equity, participation, and impact received the lowest scores when averaged for the four cities (5, 30, and 37, respectively). Conversely, availability, access, and satisfaction exhibited the best scores (63, 70, and 72, respectively). Planning, use, and infrastructure scored in the middle (39, 40, and 44, respectively). Equity, measured through the indicator coverage, was the worst-ranked descriptor of the WASSI in the cities assessed. The variable used (lack of sewage services) was very useful to show differences even within cities. In a context of intensive urban growth, sewer services are usually provided later than water services, especially in peri-urban areas, revealing deficiencies in urban planning. Risks arising from this situation are high in the most disadvantaged areas of the cities. Low scores obtained for the descriptor participation are a direct consequence of the little number of formal interactions between the water company and end users. This suggests that more efforts are needed to promote new (and more frequent) interaction points to enhance deliberation and promote agreements. All case studies showed scores below the threshold for the descriptor impact because an important percentage of the wastewater remains untreated or is poorly treated prior to its discharge into the environment.
These figures suggest that the water company is focusing its efforts only on issues directly related to the provision of drinking water and pays less attention to other environmental and social issues that also affect the sustainability of the system but are potentially less visible to managers and customers. It is possible that the incorporation of additional quality standards, which may include equity indicators, citizen involvement, and information transparency, could force the water company and other institutions to pay more attention to these aspects of the service.
Figure 4 shows in detail the average values of the descriptors for SAL (Figure 4, left), where Availability, Infrastructure, Planning, and Access are in the little concern zone. If descriptors are calculated using the worst indicator per descriptor (Figure 4, right) only Planning remains in the little concern zone.
Sustainability 07 14537 g004 1024
Figure 4. Descriptors of the WASSI calculated as averages of their respective indicators (left) and taking into account only the worst (limiting) indicator per descriptor (right).
Figure 4. Descriptors of the WASSI calculated as averages of their respective indicators (left) and taking into account only the worst (limiting) indicator per descriptor (right).
Sustainability 07 14537 g004

4. Discussion

The structure of descriptors, indicators, and variables used to build the WASSI was useful to assess the sustainability of the WSMS in each of the case studies. This structure was easily adapted to the different settings of each case study with minor adaptations at the variables level. We believe that the lower the level of the analytical category (descriptors, indicators, variables), the more flexible the index becomes. A different structure could also be used, provided that the assessment pays sufficient attention to the spatial, temporal, and personal aspects of the management system under study, as postulated by the conceptual framework adopted. In this manner, the index is both conceptually robust and operationally flexible and simple. We acknowledge that a number of different indices could be built in different places or for different periods of time. This is not seen as a drawback to the approach, but rather a reflection of the intricacies of social-environmental issues contingent to local circumstances and bound to a large degree on human subjectivity.
Lack of information is always a hindrance to building trustworthy decision-support models. Yet it should not prevent policy makers from making decisions using the best available information [50]. Establishing a working set of explicit criteria and variables is a valuable effort to initiating a more rational and potentially more participatory decision-making process. Based on this idea, our model is an attempt to capture the complexity of WSMS in situations where information is often lacking or fragmented.
Assigning numerical values to relatively subjective variables poses important methodological and political challenges. The validity of these values will be directly related to the social acceptability of the person or persons in charge of the assessment. Different stakeholders can assign different or opposing values to some of the variables used, and even select a different set of variables since these models are not objective accounts but interpretations of reality that are influenced by social contexts [51]. Model building raises many scientific and policy questions and the choices made are determined by the priorities of the participants. Because the range of possible social perspectives, stakeholders, variables, classes, and values is hypothetically very large, so are the possible outcomes that might emerge from the assessment. Therefore, the informed participation of all relevant stakeholders is necessary for planning processes to be legitimate and democratic [52].The role of scientists and experts is important in these assessment processes since they can generate missing information, propose pertinent conceptual models, help identify stakeholders, and assist during decision-making processes [53].
We believe that the selection of indicators should always follow the “principle of parsimony” to avoid over-specifying the explanatory model by using more variables than strictly necessary to describe the system under analysis [54,55]. This means that the fewer descriptors, indicators, and variables are better, as long as all essential aspects of the system are considered. Fewer indicators simplify the analysis, reduce the risk of overlaps, and increase the feasibility of the assessment. The bottom line should be that estimates should be useful in clarifying the relevant aspects of the conceptual model of sustainability that we adopted for our particular case study.

4.1. Scope and Limitations of the WASSI

The structure of nine descriptors of the WASSI follows a conceptual framework that postulates that sustainability can be understood by looking at its spatial, temporal, and personal aspects [11]. To maximize the consistency of the method, this structure has also been applied at the level of descriptors by defining three descriptors per aspect that respectively point to the spatial, temporal, and personal facets of each aspect. At the level of indicators and variables the WASSI becomes more flexible to facilitate its adaptation to the specific characteristics of each particular case. In our case, the variable aquifer depth, for instance, was adopted to describe long-term trends when groundwater was the main source of drinking water. However, another variable can be used to better reflect trends in water availability when local conditions so dictate without compromising the integrity and analytical power of the index. We believe that this flexibility in the selection of categories is one of the strengths of the WASSI, and allows for more bottom-up public participation, arguably enhancing the potential of the method to help formulate specific policies and solve practical problems [56].
Whether water and sanitation aspects need to be described by the same number of indicators and variables is subject to debate. A balanced index seems desirable, but the selection of indicators and variables is, to a great extent, case-specific and contingent upon the stakeholders and experts involved in the assessment. For instance, a water safety plan could well be replaced by a water and sanitation safety plan [57] to give values to a variable such as water safety (indicator: risks). The selection of cases is also a delicate issue.
Although we selected our cases to maximize social, demographic, economic, climatic, and geographical differences, all our case studies are embedded in the same legal and institutional framework since they belong to the same provincial state (with only slight differences in municipal legislation). Besides, water and sanitation services are provided by the same water company. Larger differences between case studies might be needed to better judge the adaptability of the WASSI to different settings.
The WASSI can also be enriched by paying attention to other sustainability-related issues such as nutrient recovery, energy efficiency, climate change, global water trade, and so on [58]. Further research is needed to fully address these issues in a comprehensive yet simple way.

4.2. Sustainability or Un-Sustainability?

For practical, historical, and epistemological reasons, it seems easier to assess the “un-sustainability” of a system (the problems that affect it) rather than its actual sustainability (its proximity to a hypothetically desirable state) [7]. Building on this idea, and on Liebig’s “law of the minimum”, it might be better to focus only on those indicators that are “limiting factors” to the sustainability of the system [1]. This focuses information retrieval and improves the efficiency of the assessment in identifying areas where improvements are needed, since corrective measures will be specifically focused on addressing the weakest aspects of the system. In the case of SAC, for instance, the presence of high concentrations of arsenic in water was so serious that a very expensive water catchment and transportation project was initiated to solve the problem.
Financial considerations aside, this intervention alone could improve the sustainability of the entire system more than any other measure or combination of measures that might enhance the value assigned to other indicators. It is clear that an index built on averages (Figure 4, left) will always be higher than one built on limiting factors (Figure 4, right). This might not be attractive for water companies, yet the latter will probably better reflect the real sustainability (or un-sustainability) of the system and may be more useful for decision making and monitoring.

4.3. From Indicators to Policies

Monitoring the quality of water and sanitation services demands ever increasing levels of technical efficiency [59]. The use of indicators is one of the ways by which sustainable governance practices can be promoted in the water and sanitation sector [60]. Indeed, achievement of the new Sustainable Development Goals (SDG) set out by the United Nations will require a combination of accurate monitoring methods (including the estimation of sustainability indicators) and adequate governance processes that include full social participation [30]. Thus far, even though there is a multiplicity of approaches and methodologies for the estimation of sustainability indicators, the impact of these indicators on concrete governance processes seems disappointingly small [16,61,62]. This disconnection may partially be due to the fact that researchers, who concentrate their efforts on the descriptive aspects of sustainability and governance, do not always satisfactorily engage with the more normative and fully political aspects of governance “for” sustainability [40,63].
It is also possible that current methods for sustainability assessment do not fully incorporate mechanisms of societal participation. Participation mechanisms need to ensure that all societal actors have the opportunity to take part in the decision-making process, including the implementation phase [64]. Most participation methodologies still seem too “reductionist” (quantitative and expert-led). A more “conversational” methodological paradigm (bottom–up, more qualitative, and participatory philosophy) might better assure success [17,65]. By combining both approaches, it is possible to acknowledge the importance of local contexts and diverse local perspectives, while maintaining the rigor provided by explicitly quantitative indicators [66]. In any case, full availability of accurate, transparent information and genuine public participation seem essential to fostering more open sustainability assessment practices that can be coupled with the deliberation processes necessary to reaching agreements for political action [62,67].
The project website that was developed to increase accessibility to relevant information on the system and facilitate public participation has recently been uploaded onto the internet. This site can be updated in real time and can also be used to add new cases from the region, the rest of the country, or from anywhere in the world. Confidential, unconfirmed, or provisional information can be safeguarded by assigning specific entry credentials to different users. This online tool can help promote participatory processes and might be useful for both decision making and academic purposes [68]. Salta’s water company is currently assessing the feasibility of incorporating such an online tool into their management processes.
As depicted in Figure 5, a more open sustainability assessment demands a smooth and constant flow of information to the stakeholders that will be in charge of the decision-making process. In an open sustainability assessment, indicators and indices (information) are essential to providing stakeholders with science-based decision-making tools [21,53]. Such governance processes could take place in specific virtual spaces by allowing two-way communication, where stakeholders agree to participate and interact to know and understand the problems of the WSMS and incorporate their own visions [69]. Websites, social media, and other virtual spaces are potentially good channels for societal interactions and can increase the availability and accessibility of sustainability information [49,70]. The use of interactive websites can also generate trust, promote social learning processes, and improve the efficiency of management policies [71]. However, these spaces can complement but not replace the institutionalized mechanisms of representative democracy, complemented by instances of face-to-face deliberative processes that could be designed and facilitated by governmental and non-governmental organizations, multilateral institutions, and companies [72,73,74]. Well-coordinated instances of deliberative and personal interaction can empower stakeholders irrespective of their social perspectives, interests, relative influence, values, or feelings [75,76]. Deliberative communication is characterized by a close relationship between institutions and stakeholders in specific, planned, and physical interaction places. To make this possible, institutions need to play a proactive role to operationalize and organize deliberative interaction processes and facilitate the formulation of policies that can move the system toward a situation of more sustainability [65,66].
Sustainability 07 14537 g005 1024
Figure 5. Elements and relationships in a more open sustainability assessment approach.
Figure 5. Elements and relationships in a more open sustainability assessment approach.
Sustainability 07 14537 g005
All stakeholders interviewed at the end of this study considered that a more open sustainability assessment approach backed by virtual tools such as the project website would be relevant and feasible in the province of Salta (Table 4).
Table
Table 4. Summary of the results obtained during the interviews (see Materials and Methods for a description of the different social perspectives).
Table 4. Summary of the results obtained during the interviews (see Materials and Methods for a description of the different social perspectives).
Stakeholders (Perspectives)Information on the Water SystemGovernment ProficiencyParticipation ProcessesStakeholders CommitmentRelevance of Virtual Spaces
ALowLow technical expertise; little decision powerPoor; should be improvedLow; little knowledge on the water systemHigh; virtual spaces useful and feasible
BSufficientEnough technical expertise; little decision powerAdequateLow; little knowledge on the water systemHigh; virtual spaces useful and feasible
CLowLow technical expertise; little decision powerPoor; should be improvedVery low; unaware of the problemsHigh; virtual spaces useful and feasible
DLowLow technical expertise; little decision powerNone; Water Company not interestedHigh; very aware of the problems in the water systemHigh; virtual spaces useful and feasible
Whether such tools would change the way things are done in water and wastewater management is debatable, since all interviewees expressed doubts about the decision making power of governmental control offices. With the exception of the stakeholder representing perspective B, they also agreed on the need to improve the level of public participation in water management. However, there is less agreement on the level of stakeholder commitment to participating in decision-making processes and on their knowledge of the water and wastewater system. As usual, real management systems cannot be easily explained, let alone governed, by resorting to simplistic models or by underestimating issues of power and economic interests. However, we believe that a thorough assessment of the sustainability of these systems, coupled with some knowledge on the social perspectives held by local stakeholders, can shed some light on issues that are usually disregarded in traditional governance schemes.

5. Conclusions

The WASSI was successfully assessed in four cities of the province of Salta. None of the cities exhibited very high levels of sustainability, with only one city above the acceptable threshold of 50 points over 100. Results suggest that the water company in charge of the WSMS in the province should direct more attention to some of the environmental and social issues that are currently not adequately addressed and are severely impeding the sustainability of the systems studied. We believe that the WASSI is conceptually robust and operationally simple and could be easily localized to the particular cases with minor adaptation.
The web site developed for this project has recently been made operational on the internet. It can be updated in real time and can be used to add new cases from the region, or from other regions. Such online tools can be useful to promoting participatory processes, assisting decision makers, and facilitating academic research in water and sanitation.
A more open sustainability assessment based on indicators and indices (such as the WASSI) can be a useful tool in assisting stakeholders and decision makers improve the sustainability and transparency of WSMS in the region and beyond. Local stakeholders support the use of an open sustainability assessment approach facilitated by virtual tools as relevant and feasible to the province of Salta.

Acknowledgments

Part of the information contained in this work was kindly provided by CoSAySa, SRH and ENRESP. All those who participated in meetings and workshops are gratefully acknowledged. This project was financed by CONICET (Project PIP 11420090100392), the Research Council of UNSa (Projects 1857 and 2031), and the National Ministry of Science, Technology and Productive Innovation (MINCyT) (Project PFIP 2008-1). Many thanks to Keith Duhaime for his grammatical input.

Author Contributions

Martín Alejandro Iribarnegaray and Lucas Seghezzo conceived and designed the study, analyzed the data, interpreted the results and wrote the manuscript. María Laura Gatto D’Andrea, María Soledad Rodriguez-Alvarez and María Eugenia Hernández performed some experiments and contributed to the analysis of data. Christian Brannstrom contributed to the discussion section of the manuscript. All authors have read and approved the final manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bossel, H. Indicators for Sustainable Development: Theory, Method, Applications. A Report to the Balaton Group; International Institute for Sustainable Development (IISD): Winnipeg, MB, Canada, 1999. [Google Scholar]
  2. Bell, B.; Morse, S. Sustainability Indicators: Measuring the Immeasurable? Earthscan Publications Ltd.: London, UK, 2008. [Google Scholar]
  3. Valentin, A.; Spangenberg, J.H. A guide to community sustainability indicators. Environ. Impact Assess. Rev. 2000, 20, 381–392. [Google Scholar] [CrossRef]
  4. López Ridaura, S. Multi-Scale Sustainability Evaluation. A Framework for the Derivation and Quantification of Indicators for Natural Resource Management Systems. Ph.D. Thesis, Wageningen University, Wageningen, The Netherlands, 2005. [Google Scholar]
  5. Ness, B.; Urbel-Piirsalu, E.; Anderberg, S.; Olsson, L. Categorizing tools for sustainability assessment. Ecol. Econ. 2007, 60, 498–508. [Google Scholar] [CrossRef]
  6. Van de Kerk, G.; Manuel, A. A Comprehensive index for a sustainable society: The SSI—The Sustainable Society Index. Ecol. Econ. 2008, 66, 228–242. [Google Scholar] [CrossRef]
  7. Walter, C.; Stützel, H. A new method for assessing the sustainability of land-use systems: Identifying the relevant issues. Ecol. Econ. 2009, 68, 1275–1287. [Google Scholar] [CrossRef]
  8. Stringer, L.C.; Dougill, A.J.; Fraser, E.; Hubacek, K.; Prell, C.; Reed, M.S. Unpacking “participation” in the adaptive management of social-ecological systems: A Critical review. Ecol. Soc. 2006, 11. Article 39. Available online: http://www.ecologyandsociety.org/vol11/iss2/art39/ (accessed on 12 April 2015). [Google Scholar]
  9. Becker, J. Measuring progress towards sustainable development: An ecological framework for selecting indicators. Local Environ. 2005, 10, 87–101. [Google Scholar] [CrossRef]
  10. Mineur, E. Towards Sustainable Development Indicators as a Tool of Local Governance; Department of Political Science, Umea University: Umea, Sweden, 2007. [Google Scholar]
  11. Seghezzo, L. The five dimensions of sustainability. Environ. Polit. 2009, 18, 539–556. [Google Scholar] [CrossRef]
  12. McCool, S.; Stankey, G. Indicators of sustainability: Challenges and opportunities at the interface of science and policy. Environ. Manag. 2004, 33, 294–305. [Google Scholar] [CrossRef] [PubMed]
  13. Gibson, R.B. Beyond the pillars: Sustainability assessment as a framework for effective integration of social, economic and ecological considerations in significant decision-making. JEAPM 2006, 8, 259–280. [Google Scholar] [CrossRef]
  14. Blühdorn, I.; Welsh, I. Eco-politics beyond the paradigm of sustainability: A conceptual framework and research agenda. Environ. Polit. 2007, 16, 185–205. [Google Scholar] [CrossRef]
  15. Rametsteiner, E.; Pülzl, H.; Alkan-Olsson, J.; Frederiksen, P. Sustainability indicator development—Science or political negotiation? Ecol. Indic. 2011, 11, 61–70. [Google Scholar] [CrossRef]
  16. Garnåsjordet, P.A.; Aslaksen, I.; Giampietro, M.; Funtowicz, S.; Ericson, T. Sustainable development indicators: From statistics to policy. Environ. Policy Gov. 2012, 22, 322–336. [Google Scholar] [CrossRef]
  17. Reed, M.; Kenter, J.; Bonn, A.; Broad, K.; Burt, T.P.; Fazey, I.R.; Hubacek, K.; Nainggolan, D.; Quinn, C.H.; Stringer, L.C.; et al. Participatory scenario development for environmental management: A methodological framework illustrated with experience from the UK uplands. J. Environ. Manag. 2013, 128, 345–362. [Google Scholar] [CrossRef] [PubMed]
  18. Shen, L.-Y.; Ochoa, J.J.; Shah, M.N.; Zhang, X. The application of urban sustainability indicators—A comparison between various practices. Habitat Int. 2011, 35, 17–29. [Google Scholar] [CrossRef]
  19. Iribarnegaray, M.A.; de la Zerda, F.; Hutton, C.M.; Brannstrom, C.; Liberal, V.; Tejerina, W.; Seghezzo, L. Water-conservation policies in perspective: Insights from a Q-method study in Salta, Argentina. Water Policy 2014, 16, 897–916. [Google Scholar] [CrossRef]
  20. Hellström, D.; Jeppsson, U.; Kärrman, E. A framework for systems analysis of sustainable urban water management. Environ. Impact Assess. Rev. 2000, 20, 311–321. [Google Scholar] [CrossRef]
  21. Molle, F.; Mollinga, P. Water poverty indicators: Conceptual problems and policy issues. Water Policy 2003, 5, 529–544. [Google Scholar]
  22. Dinar, A.; Saleth, R.M. Can water institutions be cured? A water institutions health index. Water Supply 2005, 5, 17–40. [Google Scholar]
  23. Iribarnegaray, M.A.; Copa, F.R.; Gatto D’Andrea, M.L.; Arredondo, M.F.; Cabral, J.D.; Correa, J.J.; Liberal, V.I.; Seghezzo, L. A comprehensive index to assess the sustainability of water and sanitation management systems. J. Water Sanit. Hyg. Dev. 2012, 2, 205–222. [Google Scholar] [CrossRef]
  24. Lemos, D.; Cláudia Dias, A.; Gabarrell, X.; Arroja, L. Environmental assessment of an urban water system. J. Clean. Prod. 2013, 54, 157–165. [Google Scholar] [CrossRef]
  25. Kayser, G.L.; Moriarty, P.; Fonseca, C.; Bartram, J. Domestic water service delivery indicators and frameworks for monitoring, evaluation, policy and planning: A review. Int. J. Environ. Res. Public Health. 2013, 10, 4812–4835. [Google Scholar] [CrossRef] [PubMed]
  26. UN (United Nations). The Road to Dignity by 2030: Ending Poverty, Transforming all Lives and Protecting the Planet. Available online: http://www.un.org/ga/search/view_doc.asp?symbol=A/69/700&Lang=E (accessed on 20 May 2015).
  27. Bond, A.J.; Morrison-Saunders, A. Re-evaluating sustainability assessment: Aligning the vision and the practice. Environ. Impact Assess. Rev. 2011, 31, 1–7. [Google Scholar] [CrossRef]
  28. Boyko, C.; Gaterell, M.R.; Barber, A.R.; Brown, J.; Bryson, J.R.; Butler, D.; Caputo, S.; Caserio, M.; Coles, R.; Cooper, R.; et al. Benchmarking sustainability in cities: The role of indicators and future scenarios. Glob. Environ. Change 2012, 22, 245–254. [Google Scholar] [CrossRef]
  29. Dahl, A.L. Achievements and gaps in indicators for sustainability. Ecol. Indic. 2012, 17, 14–19. [Google Scholar] [CrossRef]
  30. Griggs, D.; Stafford-Smith, M.; Gaffney, O.; Rockström, J.; Öhman, M.; Shyamsundar, P.; Steffen, W.; Glaser, G.; Kanie, N.; Noble, I. Sustainable development goals for people and planet. Nature 2013, 495, 305–307. [Google Scholar] [CrossRef] [PubMed]
  31. Plumecocq, G. Rhetoric as a means for sustainable development policy. Environ. Values 2014, 23, 529–549. [Google Scholar] [CrossRef]
  32. Bell, S. Renegotiating urban water. Prog. Plan. 2015, 96, 1–28. [Google Scholar] [CrossRef]
  33. Thomson, P.; Hope, R.A.; Foster, T. Is silence golden? Of mobiles, monitoring, and rural water supplies. Waterlines 2012, 31, 280–292. [Google Scholar] [CrossRef]
  34. Ball, M.; Rahman, Z.; Champanis, M.; Rivett, U.; Khush, R. Mobile data tools for improving information flow in WASH: Lessons from three field pilots. In Proceedings of the IRC Symposium on Monitoring Sustainable WASH Service Delivery, Addis Ababa, Ethiopia, 9–11 April 2013; pp. 1–13.
  35. Estevez, E.; Janowski, T. Electronic governance for sustainable development. Conceptual framework and state of research. Gov. Inform. Q. 2013, 30, S94–S109. [Google Scholar] [CrossRef]
  36. Nel, P.J.C.; Booysen, M.J.; van der Merwe, B. ICT-enabled solutions for smart management of water supply in Africa. In Proceedings of the First International Conference on the Use of Mobile Informations and Communication Technology (ICT) in Africa, Stellenbosch, South Africa, 9–10 December 2014; pp. 40–44.
  37. Bakker, K.; Morinville, C. The governance dimensions of water security: A review. Philos. Trans. R. Soc. Lond. A 2013, 371. [Google Scholar] [CrossRef] [PubMed]
  38. Biggs, E.M.; Duncan, J.; Atkinson, P.M.; Dash, J. Plenty of water, not enough strategy. How inadequate accessibility, poor governance and a volatile government can tip the balance against ensuring water security: The case of Nepal. Environ. Sci. Policy 2013, 33, 388–394. [Google Scholar] [CrossRef]
  39. Organization for Economic Co-operation and Development (OECD). Water Governance in Latin America and the Caribbean. A Multi-level Approach; OECD Publishing: Paris, France, 2012. [Google Scholar]
  40. Iribarnegaray, M.A.; Seghezzo, L. Governance, sustainability and decision making in water and sanitation management systems. Sustainability 2012, 4, 2922–2945. [Google Scholar] [CrossRef]
  41. Torquebiau, E. Are tropical agroforestry home gardens sustainable? Agric. Ecosyst. Environ. 1992, 41, 189–207. [Google Scholar] [CrossRef]
  42. Belton, V.; Bell, A. Comparison of the analytic hierarchy process and a simple multi-attribute value function. Eur. J. Oper. Res. 1986, 26, 7–21. [Google Scholar] [CrossRef]
  43. Singh, R.K.; Murty, H.R.; Guptac, S.K.; Dikshit, A.K. An overview of sustainability assessment methodologies. Ecol. Ind. 2012, 15, 281–299. [Google Scholar] [CrossRef]
  44. Saaty, T.L. Relative Measurement and its generalization in decision making. Why pairwise comparisons are central in mathematics for the measurement of intangible factors: The Analytic Hierarchy/Network Process. RACSAM 2008, 102, 251–318. [Google Scholar] [CrossRef]
  45. Seghezzo, L.; Gatto D’Andrea, M.L.; Iribarnegaray, M.; Liberal, V.; Fleitas, A.; Bonifacio, J.L. Improved risk assessment and risk reduction strategies in the Water Safety Plan (WSP) of Salta, Argentina. Water Supply 2013, 13, 1080–1089. [Google Scholar] [CrossRef]
  46. Calheiros, D.F.; Seidl, A.F.; Ferreira, C.J.A. Participatory research methods in environmental science: Local and scientific knowledge of a limnological phenomenon in the Pantanal wetland of Brazil. J. Appl. Ecol. 2000, 37, 684–696. [Google Scholar] [CrossRef]
  47. Rapley, T.J. The art (fulness) of open-ended interviewing: Some considerations on analyzing interviews. Qual. Res. 2001, 1, 303–323. [Google Scholar] [CrossRef]
  48. May, T. Social Research: Issues, Methods and Process, 4th ed.; Open University Press: Berkshire, UK, 2011. [Google Scholar]
  49. Pintér, L.; Hardi, P.; Martinuzzi, A.; Hall, J. Bellagio STAMP: Principles for sustainability assessment and measurement. Ecol. Indic. 2012, 17, 20–28. [Google Scholar] [CrossRef]
  50. Kemp, R.; Parto, S.; Gibson, R.B. Governance for sustainable development: Moving from theory to practice. Int. J. Sustain. Dev. 2005, 8, 12–30. [Google Scholar] [CrossRef]
  51. Elgert, L. Hard facts and software: The co-production of indicators in a land-use planning model. Environ. Values 2013, 22, 765–786. [Google Scholar] [CrossRef]
  52. Arias-Maldonado, M. Rethinking Sustainability in the Anthropocene. Environ. Polit. 2013, 22, 428–446. [Google Scholar] [CrossRef]
  53. Moreno-Pires, S.; Fidélis, T. A proposal to explore the role of sustainability indicators in local governance contexts: The case of Palmela, Portugal. Ecol. Indic. 2012, 23, 608–615. [Google Scholar] [CrossRef]
  54. Binder, C.; Feola, G.; Steinberger, J. Considering the normative, systemic and procedural dimensions in indicator-based sustainability assessments in agriculture. Environ. Impact Assess. Rev. 2010, 30, 71–81. [Google Scholar] [CrossRef]
  55. Mukherjee, R.; Sengupta, D.; Sikdar, S. Parsimonious use of indicators for evaluating sustainability systems with multivariate statistical analyses. Clean. Technol. Environ. 2013, 15, 699–706. [Google Scholar] [CrossRef]
  56. Hering, J.G.; Sedlak, D.L.; Tortajada, C.; Biswas, A.K.; Niwagaba, C.; Breu, T. Local perspectives on water. Science 2015, 349, 479–480. [Google Scholar] [CrossRef] [PubMed]
  57. WHO (World Health Organization). Sanitation Safety Planning: Manual for Safe Use and Disposal of Wastewater, Greywater and Excreta; World Health Organization: Geneva, Switzerland, 2015. [Google Scholar]
  58. Vörösmarty, C.J.; Hoekstra, A.Y.; Bunn, S.E.; Conway, D.; Gupta, J. What scale for water governance? Science 2015, 349, 478–479. [Google Scholar] [PubMed]
  59. EBC (European Benchmarking Co-operation). Learning from International Best Practices; EBC Foundation: The Hague, The Netherlands, 2014. [Google Scholar]
  60. Van Leeuwen, C.J.; Koop, S.H.A.; Sjerps, R.M.A. City Blueprints: Baseline assessments of water management and climate change in 45 cities. Environ. Dev. Sustain. 2015. [Google Scholar] [CrossRef]
  61. Hezri, H.; Dovers, S. Sustainability indicators, policy and governance: Issues for ecological economics. Ecol. Econ. 2006, 60, 86–99. [Google Scholar] [CrossRef]
  62. Newig, J.; Fritsch, O. Environmental governance: Participatory, multi-Level-and effective? Environ. Policy Gov. 2009, 19, 197–214. [Google Scholar] [CrossRef]
  63. Bosselmann, K.; Engel, R.; Taylor, P. Governance for Sustainability. Issues, Challenges, Successes; The International Union for Conservation of Nature (IUCN): Bonn, Germany, 2008. [Google Scholar]
  64. Organization for Economic Co-operation and Development (OECD). Stakeholder Engagement for Inclusive Water Governance; OECD Publishing: Paris, France, 2015. [Google Scholar]
  65. Bell, B.; Morse, S. Groups and indicators in post-industrial society. Sustain. Dev. 2011, 22, 145–159. [Google Scholar] [CrossRef]
  66. Reed, M.S.; Fraser, E.D.G.; Dougill, A.J. An adaptive learning process for developing and applying sustainability indicators with local communities. Ecol. Econ. 2006, 59, 406–418. [Google Scholar] [CrossRef]
  67. Newig, J.; Schulz, D.; Fischer, D.; Hetze, K.; Laws, N.; Lüdecke, G.; Rieckmann, M. Communication regarding sustainability: Conceptual perspectives and exploration of societal subsystems. Sustainability 2013, 5, 2976–2990. [Google Scholar] [CrossRef]
  68. Ramos, T.B.; Caeiro, S. Meta-performance evaluation of sustainability indicators. Ecol. Indic. 2010, 10, 157–166. [Google Scholar] [CrossRef]
  69. Nabatchi, T. Putting the “public” back in public values research: Designing participation to identify and respond to values. Public. Admin. Rev. 2012, 72, 699–708. [Google Scholar] [CrossRef]
  70. Isenmann, R.; Bey, C.; Welter, M. Online reporting for sustainability issues. BSE 2007, 16, 487–501. [Google Scholar] [CrossRef]
  71. Myeong, S.; Kwon, Y.; Seo, H. Sustainable e-governance: The relationship among trust, digital divide, and e-government. Sustainability 2014, 6, 6049–6069. [Google Scholar] [CrossRef]
  72. Von Korff, Y.; d’Aquino, P.; Daniell, K.; Bijlsma, R. Designing participation processes for water management and beyond. Ecol. Soc. 2010, 15. Article 29. Available online: http://www.ecologyandsociety.org/vol15/iss3/art1/ (accessed on 22 June 2015). [Google Scholar]
  73. Bryson, J.M.; Quick, K.S.; Schively Slotterback, C.; Crosby, B.C. Designing public participation processes. Public Admin. Rev. 2012, 73, 23–34. [Google Scholar] [CrossRef]
  74. Schulz, D.; Newig, J. Assessing online consultation in participatory governance: Conceptual framework and a case study of a national sustainability-related consultation platform in Germany. Environ. Policy Gov. 2015, 25, 55–69. [Google Scholar] [CrossRef]
  75. Mitchell, R.K.; Agle, B.R.; Wood, D.J. Toward a theory of stakeholder identification and salience: Defining the principle of what and who really counts. Acad. Manag. Rev. 1997, 22, 853–886. [Google Scholar]
  76. Rodela, R. Advancing the deliberative turn in natural resource management: An analysis of discourses on the use of local resources. J. Environ. Manag. 2012, 96, 26–34. [Google Scholar] [CrossRef] [PubMed]

Share and Cite

MDPI and ACS Style

Iribarnegaray, M.A.; D’Andrea, M.L.G.; Rodriguez-Alvarez, M.S.; Hernández, M.E.; Brannstrom, C.; Seghezzo, L. From Indicators to Policies: Open Sustainability Assessment in the Water and Sanitation Sector. Sustainability 2015, 7, 14537-14557. https://doi.org/10.3390/su71114537

AMA Style

Iribarnegaray MA, D’Andrea MLG, Rodriguez-Alvarez MS, Hernández ME, Brannstrom C, Seghezzo L. From Indicators to Policies: Open Sustainability Assessment in the Water and Sanitation Sector. Sustainability. 2015; 7(11):14537-14557. https://doi.org/10.3390/su71114537

Chicago/Turabian Style

Iribarnegaray, Martín Alejandro, María Laura Gatto D’Andrea, María Soledad Rodriguez-Alvarez, María Eugenia Hernández, Christian Brannstrom, and Lucas Seghezzo. 2015. "From Indicators to Policies: Open Sustainability Assessment in the Water and Sanitation Sector" Sustainability 7, no. 11: 14537-14557. https://doi.org/10.3390/su71114537

APA Style

Iribarnegaray, M. A., D’Andrea, M. L. G., Rodriguez-Alvarez, M. S., Hernández, M. E., Brannstrom, C., & Seghezzo, L. (2015). From Indicators to Policies: Open Sustainability Assessment in the Water and Sanitation Sector. Sustainability, 7(11), 14537-14557. https://doi.org/10.3390/su71114537

Article Metrics

Back to TopTop