Next Article in Journal
A Sustainable Approach for the Geopolymerization of Natural Iron-Rich Aluminosilicate Materials
Previous Article in Journal
The Cultivation of Electric Vehicles Market in China: Dilemma and Solution
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 6
Issue 9
10.3390/su6095512
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Sustainability Assessment and Indicators: Tools in a Decision-Making Strategy for Sustainable Development

by
Tom Waas
1,*,
Jean Hugé
1,2,
Thomas Block
1,
Tarah Wright
3,
Francisco Benitez-Capistros
2,4 and
Aviel Verbruggen
5
1
Centre for Sustainable Development, Ghent University, Poel 16, 9000 Ghent, Belgium
2
Laboratory of Systems Ecology and Resource Management, Université Libre de Bruxelles, Avenue Franklin Roosevelt 50, 1050 Brussels, Belgium
3
Environmental Science Department, Dalhousie University, 1355 Oxford Street, P.O. Box, Halifax, NS 15000, Canada
4
Plant Biology and Nature Management, Vrije Universiteit Brussel, Pleinlaan 2, 1050 Brussels, Belgium
5
University of Antwerp, Prinsstraat 13, 2000 Antwerp, Belgium
*
Author to whom correspondence should be addressed.
Sustainability 2014, 6(9), 5512-5534; https://doi.org/10.3390/su6095512
Submission received: 10 April 2014 / Revised: 28 May 2014 / Accepted: 11 August 2014 / Published: 25 August 2014
(This article belongs to the Section Economic and Business Aspects of Sustainability)
Browse Figures
Versions Notes

Abstract

:
Recognizing the urgent need for sustainability, we argue that to move beyond the rhetoric and to actually realize sustainable development, it must be considered as a decision-making strategy. We demonstrate that sustainability assessment and sustainability indicators can be powerful decision-supporting tools that foster sustainable development by addressing three sustainability decision-making challenges: interpretation, information-structuring, and influence. Particularly, since the 1990s many substantial and often promising sustainability assessment and sustainability indicators efforts are made. However, better practices and a broader shared understanding are still required. We aim to contribute to that objective by adopting a theoretical perspective that frames SA and SI in the context of sustainable development as a decision-making strategy and that introduces both fields along several essential aspects in a structured and comparable manner.

1. Introduction

In response to a growing environmental crisis and to vast social inequalities in global development, modern society adopted sustainable development as a leading development model [1,2,3,4] with clear “action-guiding power” [5]. In this paper, we will use the terms sustainable development and sustainability interchangeably as synonyms, although sometimes a distinction in meaning is made [2,6,7,8,9,10].
Notwithstanding its apparent vagueness and “interpretative flexibility” [10], sustainable development incorporates a more or less stable set of defining characteristics [11], which must always be respected no matter which view one adheres to [10,12]. Offering a meta-perspective [5], these characteristics can be termed principles—or rules of action—that embody sustainable development and represent the key changes needed: (1) equity (refers to justice/fairness in the way we develop and includes inter/intra-generational equity (not compromising the ability of future and current generations to meet their own needs/aspirations), interspecies equity (environmental stewardship that refers to the survival of other species on an equal basis to human survival), geographical equity (global responsibility in a spirit of shared but differentiated responsibility), and procedural equity (democratic and participatory governance); (2) dynamics (sustainable development is a process of change because the environment and society change continuously, entailing uncertainties and risks that need a precautionary approach); (3) integration (of the different sustainability principles in an harmonious manner to reconcile development objectives with environmental ones); and (4) normativity (sustainable development is a social construct and basically amounts to making normative decisions and choices, which are ultimately based on the values we maintain about the way we develop, now and in the future) [10].
Despite various political commitments and popularity of the sustainability discourse among various stakeholders its practical implementation falls short [10,13,14,15,16,17]. When it comes to addressing the implementation gap—the gap between words/discourse and deeds/implementation—sustainable development must be considered as a decision-making strategy because at the very heart of every action lies a decision [18]. We argue that sustainability assessment (SA) and sustainability indicators (SI) are decision-supporting tools that can play an important role in such a strategy by addressing three challenges: interpretation, information-structuring, and influence. However, the intrinsic holistic and multi-dimensional nature of sustainable development with its uncertainties and risks renders it assessment and measurement complex [19]. It is no coincidence that different scholars speak of “measuring the immeasurable” [19,20,21]. The rich debate and literature demonstrates that, particularly, since the 1990s, many substantial and often promising SA and SI efforts are made. However, if SA and SI and accordingly the sustainability transition are to be advanced, better SA and SI practices [21,22,23] and a broader shared understanding are required [19].
We aim to contribute to that objective by adopting a theoretical perspective that frames SA and SI in the context of sustainable development as a decision-making strategy and that introduces both fields along several essential aspects in a structured and comparable manner. The theoretical framework offers guidance for practice and should be further operationalized within a given socio-environmental context.
The literature was purposively sampled [24] and analyzed following open coding procedures [25]. The essential aspects that emerged during sampling and analysis until a sound understanding of each was reached are represented by the different sections of the paper.

2. Sustainable Development as a Decision-Making Strategy

Thus far, we have seen that sustainable development has gained broad acceptance amongst political and other stakeholders in society, but in reality, this is still mainly visible in the lip service paid to it. The popularization of the sustainable development discourse did not—yet—lead to a satisfactory implementation.
However discourses might become powerful and might initiate social learning and eventually steer society towards sustainable development [26]. Social learning can be understood as “a change in understanding that goes beyond the individual to become situated within wider social units or communities of practice through social interactions between actors within social networks” [27]. However, there is still a great deal of work to be done to move from shared preoccupation and discourse to actual implementation, as the gap between words and deeds is still large and growing [13]. Implementation means that the sustainability discourse should be translated into actions [28]. This translation exercise starts with making decisions. Therefore, if sustainable development is to be a useful and implement-able concept, it must enter the field of decision-making and must be considered as a “decision-making strategy”. Two terms need clarification: strategy and decision-making.
A strategy is a way forward to make happen a desired future, in this case the achievement of sustainable development and its objectives [18]. Whereas, a decision is a determination arrived at after consideration (Merriam-Webster English Dictionary). Peterson and Blomberg [29] define decisions as choices of solutions that end uncertainty or reduce contention or, in other words, when choices are made the results are decisions. As such, decision-making is a cognitive process resulting in the selection of a preferred option or a course of action among several alternatives [30]. Intellectually appealing, decision-making can be seen as a “rational process” were decisions are based on certain criteria or strategies and well—possibly scientifically—informed [30,31,32]. However, in reality, decision-making is often a much more “fuzzy process” that is less defined and subject to many influences [22,32,33,34]. It goes beyond the assumption that more (scientific) information will logically lead to “better” decision-making [35]. Indeed, although “evidence/knowledge-based decision-making” is an increasingly coined catchphrase, (scientific) information is just one of the factors influencing decision-makers besides more subjective factors including ideology, values, norms, interests, power relationships and institutional context [22,32,33,34]. This is particularly the case for sustainability problems characterized by divergent values and norms among decision-makers, high uncertainty about causes and solutions, and risks [36]. As such, decision-making can also be considered as a matching (as some kind of “compatibility test”) between an actor’s more subjective factors—besides (scientific) information—and the expected impact of each alternative option or course of action [37].
Ultimately, the key societal choices are in hand of various stakeholders who decide upon the preferred options or course of action for the many sustainability challenges we face. Stakeholders are individuals (citizens—every one of us) yet also include different type of organizations/groups of individuals (for example governments, companies, non-governmental organizations, citizen committees, schools, higher education), and even the world community as sustainable development is to be implemented globally.
Conceptualizing sustainable development as a decision-making strategy allows to actually “use” it, thereby moving beyond the rhetoric, and turning sustainability and its “action-guiding” power into an “action-generating” concept. To understand the linkages between sustainability and decision-making, and as such the realization of sustainable development as a decision-making strategy, at least the following three challenges should be considered: [18]
  • Interpretation (sustainability should be interpreted considering its organizing principles, applied in a given socio-environmental context);
  • Information-structuring (the inherent multi-dimensional complexity of sustainability should be structured into operational information units (for example indicators) and properly communicated in order to feed the decision-making process);
  • Influence (sustainability information should exert a real influence on decision-making and on the actual implementation of sustainable development).
SA and SI are tools to support decision-making for sustainable development that can be used in different fields/disciplines and in various socio-environmental contexts by many stakeholders. However, in practice such “managerial” tools are mostly but one single element within a tangled web of decision-making processes [38] and influences (rational and subjective). Unlike strong “technical-rational” perspectives on SA and SI, Foucauldian perspectives for example focus on the role and impact of power configurations, hegemonic regimes, and political struggles. The development and use of sustainability tools, such as SA and SI, are often contested and subject of competition, not least because they can contribute to the legitimization of governmental action [39]. There is for example a real risk of resistance vis-à-vis SA and SI as they might communicate negative information about particular stakeholders or force stakeholders to deal with sustainability issues they prefer not to address [40].
Ideally, SA and SI address the above mentioned three sustainability decision-making challenges. As such, these challenges are not only opportunities to understand the linkages between sustainability and decision-making, but also opportunities to improve SA and SI theories and practices.

3. Sustainability Assessment

SA is still a relatively new field and, according to Bond et al. [23], in a first stage of development, where early practice is being transformed to fit new situations and contexts. Increasingly SA has become associated with the broad field of impact assessment [41]. The International Association for Impact Assessment [42] defines impact assessment as “[…] the process of identifying future consequences of a current or proposed action”.
There are two views, respectively considering impact assessment or SA as the overarching field. On the one hand, impact assessment could be considered as a generic field that encompasses different, approaches and processes, some of which are well known and widely used (for example environmental impact assessment, strategic environmental assessment, health impact assessment and risk assessment) while others are more recently developed (SA) or less clearly defined. Similarly, Bond et al. [23] consider SA as a “recent framing of impact assessment”, sometimes called its “third generation” following environmental impact assessment and strategic environmental assessment. On the other hand, Hacking et al. [43] and Ness et al. [41] consider SA, and not impact assessment, as an umbrella term, including indicator development, product-related assessment (for example life cycle analysis), and integrated assessments, such as impact assessment (for example environmental impact assessment). The difference in points of view can be explained by the different fields that are involved in SA practices [23].

3.1. Definition

There is a wide variety of SA practices [23] and accordingly there are various definitions. Probably the most “all-inclusive” is the one proposed by Bond et al. [23]. They adopt a broad perspective and define it as “any process that directs decision-making towards sustainability”. Given the importance to consider sustainable development as a decision-making strategy, we prefer to highlight the main challenges of such a strategy in which the role of SA is key. Therefore and based on the work of Devuyst et al. [44], Gasparatos et al. [45], Gibson et al. [46], Ness et al. [41], Pope [47], and Bond et al. [23], we define SA as follows:
Sustainability assessment is any process that aims to:
  • Contribute to a better understanding of the meaning of sustainability and its contextual interpretation (interpretation challenge);
  • Integrate sustainability issues into decision-making by identifying and assessing (past and/or future) sustainability impacts (information-structuring challenge);
  • Foster sustainability objectives (influence challenge).
Note that, in analogy with IAIA’s view on impact assessment, SA is often considered as an “ex-ante” process aimed to predict future outcomes [41], or to assess the effects of decisions in advance and to support choice between various options [23,48]. The assessment of the effects triggered by an intervention (past outcomes) that is mostly used at the end of a policy or management cycle is then termed “evaluation” [41,48]. However, the distinction between “assessment” (ex-ante) and “evaluation” (ex-post) is not always made. Both are also considered under the same SA-umbrella [23,41,49]. The proposed definition offers the flexibility to allow both views to co-exist.

3.2. Purposes

SA serves different purposes, in a decision-making strategy for sustainable development, often simultaneously. We identify four purposes, which contribute to the realization of the three sustainability decision-making challenges:
  • Information generation for decision-making (information-structuring challenge);
  • Operationalization and forum for participation, debate and deliberation (interpretation challenge);
  • Social learning (interpretation and influence challenge);
  • Structuring complexity (information-structuring challenge).
Firstly, SA generates information for decision-makers, ensuring that a decision is taken with the best available knowledge of its full—(un)intentional—sustainability impacts. Ideally this leads to choose the “best” alternative, in terms of sustainable development. Hence SA creates choice opportunities [23] and shows pathways for action.
Secondly, SA operationalizes sustainable development by attributing meaning to the concept in a given socio-environmental context and could be a forum for debate and deliberation among various involved stakeholders [50,51]. It also serves as a vehicle to actually organize and structure stakeholder participation.
Thirdly, SA is a learning process that can lead to a shift in the involved stakeholders’/decision-makers’ sustainability knowledge, attitude and views [52]. New knowledge and pre-decision debates can provide them with new insights and perspectives [20], creating opportunities for change in the decision-making.
Fourthly, SA structures complex information. The intrinsic multi-dimensional complexity of sustainability creates an ever-growing need for information [53] and demands tools to structure it. SA provides a systematic and stepwise approach for information-structuring that allows decision-makers to deal with the complexity of sustainable development.
The appreciation of the effectiveness of SA will be influenced by the main purposes assigned to the SA initiative by different stakeholders in different circumstances. In general, four categories of SA effectiveness can be distinguished: substantive (the achievement of the intended purposes of the SA), normative (the achievement of normative goals—i.e., can stakeholders learn, improve their knowledge and change their views), procedural (consideration of SA process aspects and the establishment of SA procedures and policy), and transactive (the achievement of intended purposes with minimal resources and time or in other words efficiency) [54,55].

3.3. Methodology

In support of the different purposes of SA various methodological aspects should be considered. In 1996, an international group of SA experts developed the “Bellagio Principles—Guidelines for Practical Assessment of Progress Toward Sustainable Development”. The principles served as general guidelines for SA [48,56,57]. After more than a decade of practice and taking into account the changing SA context, the principles were reviewed and renamed into “Sustainability Assessment and Measurement Principles” (“Bellagio STAMP”) [48]. They were reduced from ten to eight, more succinctly phrased, some of the ambiguities and duplications were eliminated, and new insights were included [48]. Similar to the original Bellagio Principles, Bellagio STAMP’s first principle deals with the starting point of SA which is the establishment of a guiding vision, principles two to four deal with the content of SA, principles five to seven deal with the process of SA, and finally principles eight deals with the necessity for establishing continuity and capacity for SA [57].
However, also in the literature, different guidelines or characteristics are found. Table 1 summarizes generic characteristics of an ideal-typical SA considering the Bellagio Principles, Bellagio STAMP and the literature [12,23,46,58,59,60,61,62], grouped in four categories (fostering sustainability objectives, adopting a holistic perspective, incorporating sustainability in the assessment process, supporting decisions).
First of all any sustainability assessment should be guided by the defining principles of sustainable development. This implies that sustainability principles should not only pervade the sustainability assessment’s content—fostering sustainability objectives and adopting an holistic or integrative perspective—but also its process dealing with normativity, participation, transparency, a precautionary stance and responsiveness to change. Secondly, SA should be conducted in support of decision-making. This means assessing sustainability impacts and alternatives for decision-making, including rules for synergies and trade-offs (for concrete guidelines see [12]). This objective is often realized by means of multi-criteria decision analysis (for an overview of tools see [41]) [63,64,65,66]. Furthermore SA should be based on a conceptual sustainability framework and its indicators, it should ensure effective communications (clear language, fair and objective, visualization tools and graphics, make data appropriately available), it should be adapted to and integrated into the institutional context, it should follow an iterative assessment process, it should develop and maintain adequate capacity, and it should foster continuous learning and improvement.
Table
Table 1. Characteristics of an ideal-typical sustainability assessment.
Table 1. Characteristics of an ideal-typical sustainability assessment.
1. Fostering sustainability objectives
  • Intergenerational equity
  • Intragenerational equity
  • Geographical equity
  • Interspecies equity
  • Procedural equity
2. Adopting a holistic perspective
  • Assess the system as a whole, including it parts and their interactions
  • Assess the system considering the different sustainability objectives together (integration)
  • Assess dynamics and interactions between trends and drivers of change
  • Adopt appropriate time horizon (short, medium, and long term) and (geographical) scope
3. Incorporating sustainability in the assessment process
  • Consider the normative nature of sustainability
  • Broad participation of stakeholders, including experts, while providing active leadership to the process
  • Transparency regarding data (sources, methods), indicators, results, choices, assumptions, uncertainties, funding bodies and potential conflicts of interest
  • Avoid irreversible risks and favors a precautionary approach
  • Be responsive to change, including uncertainties and risks (dynamism)
4. Supporting decisions
  • Assessment of sustainability impacts and alternatives for decision-making, including synergies and trade-offs
  • Establish formal and transparent synergy/trade-off rules
  • Assessment is based on a conceptual sustainability framework and its indicators
  • Ensure effective communications (clear language, fair and objective, visualization tools and graphics, make data appropriately available)
  • Adapted to and integrated into the institutional context
  • Iterative assessment process, starting at the onset of the decision-making process
  • Develop and maintain adequate capacity
  • Continuous learning and improvement

4. Sustainability Indicators

SI are an essential and powerful tool in decision-making for sustainability and of any SA [22,48,67,68,69]. Agenda 21, for example states that SI “[…] need to be developed to provide solid bases for decision-making […]” [70].
Over the last two decades there have been plenty SI-initiatives by various stakeholders (for example governments at various levels, communities, businesses, higher education and universities, and non-governmental organizations), applied in different contexts around the world, for different purposes and using a multitude of methodologies [22,48,71]. Pintér et al. [48] speak suitably of a real “indicator zoo”. Nevertheless considerable improvements can be made, such as the influence on decision-making, which is largely insufficient [22,48,72,73,74,75].

4.1. Definition

In our daily life we continuously use indicators to understand and interpret the world, mostly without actually realizing it (for example traffic lights, rankings) and act upon them [69,74]. Therefore, intuitively we probably all know what an indicator is. From a theoretical perspective more clarity is useful because definitions and terminology vary considerably (for example a variable, a parameter, a measure, a value, a meter, metrics, a measuring instrument, an index, something, a piece of information, representation, a proxy [22,67,74,76,77], which is particularly confusing [78].
We propose an integrative definition, from both a system and a technical perspective (Figure 1).
Sustainability 06 05512 g001 1024
Figure 1. Schematic presentation of an indicator’s integrative definition.
Figure 1. Schematic presentation of an indicator’s integrative definition.
Sustainability 06 05512 g001
From a system’s perspective the current consensus [74] defines an indicator as “an operational representation of an attribute (quality, characteristic, property) of a system” [67]. A “system” stands for “[…] an interconnected set of elements that is coherently organized in a way that achieves something” [79]. A system can be almost everything, but three characteristics are essential: (1) elements; (2) interconnectivity; and (3) purpose [79].
From a technical perspective, an indicator is a “variable” or an aggregation/function of a number of variables [67], related to a reference value that gives meaning to the values the variables take [48,49,73,78,80,81,82,83]. As Lancker et al. [83] point out, “[…] a given indicator doesn’t say anything about sustainability, unless a reference value […] is given to it”. Rigby et al. [73] argue “An indicator devoid of context has no value. Only in the context of a pre-specified value does it acquire meaning”. The reference value could be for example a goal, a target, a norm, a standard or a benchmark [67]. Because of this essential relationship, we put it even sharper and argue that “by definition”, an indicator is related to a reference value, as the term stems from the Latin “indicare” which means pointing towards something. Actually it is the presence or absence of a reference value that distinguishes an indicator from a variable. Without being related to a reference value, we cannot speak of an indicator because in that case the variable does not point towards something (a goal, a target, a norm, a standard or a benchmark) and, as such, does not acquire meaning and does not provide information about (changes in) the state of a system. The need for a reference value and accordingly the difference between a variable and an indicator is often subject of misconception.
Hence, an indicator shows performance as a measurement of distance-to-target. It measures the distance between current or predicted values of the variable and the reference value [84,85]. Gauging also requires a unit of measurement. Variables (and their values) can be quantitative (numbers) or qualitative (for example graphics, colors, symbols). In the case of SI, the reference value is what is understood by sustainable development or in other words the value at which sustainability is achieved [80,84,86]. Of course, intermediary targets can be articulated as well.
This leads to the following integrative definition of an indicator:
An indicator is the operational representation of an attribute (quality, characteristic, property) of a given system, by a quantitative or qualitative variable (for example numbers, graphics, colors, symbols) (or function of variables), including its value, related to a reference value.
The definition and its schematic presentation are not only useful theoretically, but also practically. It is generic and can be used as a logic and systematic building framework to develop SI. In sequence, one ideally starts with the sustainability attributes of a system. Then the SI can be developed—the variables by which the sustainability attributes are to be measured and the sustainability reference values. However, in some cases a (more) pragmatic approach, in which this sequence is not (strictly) followed, is desirable (for example when existing indicators embody sustainability and can be used to define a system’s sustainability attributes and are to be integrated in the SI development process). However, in any case the various elements of the proposed definition could serve as building blocks in any SI development process.
The generic framework can easily be linked and applied to various existing and to be developed sustainability and SI-frameworks (for example driving forces—pressures—states—impacts—responses/DPSIR framework [87], the three dimensional sustainability triangle or nested sustainability representations [10]) (for an overview of sustainability and SI frameworks/models see [9,49]).
Indicators are often condensed and aggregated into a single metric, commonly referred to as “index”, in casu a sustainability index (for example the ecological footprint [88] and the human development index [89], for an overview see [49]). Gallopin [67] argues that it is not generally correct to put indices at a higher hierarchical level of aggregation than indicators. Instead, Gallopin [67] argues the distinction between indicators and indices lies in the complexity of the functions by which they are obtained and not in their assumed hierarchy.

4.2. Purposes

SI have several complementary purposes in a decision-making strategy for sustainable development and in SA, supportive to the three sustainability decision-making challenges:
  • Structure complexity and communicate information (information-structuring challenge);
  • Operationalization of sustainable development (interpretation challenge);
  • Social learning (interpretation and influence challenge);
  • Demonstrate accountability and benchmarking (influence challenge);
  • Identification of knowledge and data gaps (information-structuring challenge).
Firstly, SI communicate information in a structured manner to inform decision-making for sustainable development [22,67,76,81,90]. By doing so, SI make sustainability for a particular system observable, demonstrable and measurable. As Meadows [69] states, “We have no choice. Without them [SI] we fly blind. The world is too complex to deal with all available information”, and the attractiveness of SI lies in their ability to structure, to summarize, and condense the sustainability complexity to a manageable amount of meaningful information [19,49]. Indeed, adopting a holistic perspective, SI should bridge various environmental and societal complex fields particularly in a time of dramatic changes and transformations. Furthermore, the current flood of data and information in more and more sustainability fields, requires SI to simplify and interpret them [71].
Secondly, SI operationalize sustainable development [80,90]. The development of SI pulls the discussion of sustainable development away from abstract formulations and encourages explicit discussions on concepts with operational meaning [40,73], which is a prerequisite for the practical implementation of sustainability [91].
Thirdly, SI facilitate continuous learning among involved stakeholders [69,92,93] and their development and application could be considered as a way of social learning [94]. Shields et al. [95] argues that SI will only be effective if they support social learning. As such, SI can induce change in the mindset of decision-makers and affect decision-making and behavior. Moreover the SI development and application itself is a learning process [69]. Pintér et al. [48] argues “Changing the way society measures progress represents a key leverage point in tackling the root causes of unsustainable development”. As a consequence of the complexity of whole system’s sustainability measurements, SI should be selected, applied, revised and refined [49,69]. According to Dahl [22], the most significant effect of SI, particularly in its early adoption, can simply be to make a problem visible and to sensitize decision-makers.
Fourthly, SI can be used to demonstrate accountability to society and its stakeholders by communicating about a systems sustainability performance [51,92] (for example corporate social responsibility/accountability of businesses). They can also be used to allow benchmarking between stakeholders (for example between organizations).
Finally, SI identify knowledge and data gaps and can suggest priorities for filling these gaps [92].
In addition it is useful to be aware that in reality different types of SI “use” exist including, instrumental use (for action and problem solving/decision-making, there is a direct link between the SI and decision outcomes), conceptual use (to clarify and improve understanding of a problem or situation, over time conceptual use may subsequently induce decision outcomes), tactical use (as a delaying tactic, as a substitute for action or to deflect criticism), symbolic use (to give ritualistic assurance that those who take decisions hold appropriate attitudes towards decision-making, in other words, to measure another reality—“good decision-making”), and political use (to support a pre-determined position, it is about persuading others to a particular view of the problem and ways to solve it) [81,96].
In line with SA the effectiveness is influenced by the main intended purposes of the SI and similarly covers a broad scope. For instance, the effectiveness of SI cannot be evaluated by “technical/analytical” aspects of measurement—or quality criteria as further explained—, but rather by the way SI are successfully implemented to influence decision-making, to educate stakeholders and to demonstrate accountability [72,97]. With regard to the influence challenge Bell et al. [75] argue that the limited influence on decision-making is in part “due to a historical and continuing technical emphasis on improving measurement rather than ‘use’”. In the literature the influence challenge receives almost no attention [98]. As Rigby et al. [73] state “Much of the measurement of indicators has […] largely resulted just in the measurement of indicators. The actual operationalisation of indicators to influence or change, for instance, policy is still in its infancy”.

4.3. Methodology

The development of indicators in various fields such as economic development, social progress, quality of life, environment and natural resources, healthy communities and sustainability have influenced SI methodologies to a great extent [96]. In general two broad methodologies for SI can be distinguished, “top-down”/“expert-driven” and “bottom-up”/“stakeholder-driven” [74,75,99,100]. Bell et al. [75] also refer to them as respectively “reductionist” and “conversational”. The latter can also be referred to as “constructionist”, after the opposing reductionism and constructionism epistemologies.
Top-down approaches are characterized by quantitative indicators, which are developed by experts and with explicit, clearly stated methodologies. Bottom-up approaches instead use qualitative indicators which are developed by (local) stakeholders and with implicit, no clearly defined methodologies [75]. Of course each end of the divide has its strengths and weaknesses. Top-down SI are developed by experts and “scientifically rigorous” but such methodologies fail to engage (local) stakeholders, whereas the opposite is true for bottom-up methodologies [99].
Therefore, any combination/integration of both methodologies is not only possible, but even strongly recommended [20,22,51,67,68,69,75,93,100,101] taking into account (1) the required combination of different kinds of knowledge (for example stakeholder/citizen/lay knowledge vs. expert/scientific knowledge) regarding the (local) environment and (local) society and (2) participation as a democratic principle to include stakeholder views and (3) participation to create opportunities for learning, empowerment and ownership [51,75,93]. Indeed, while SI often fail to influence decision-making bottom-up methodologies are supposed to increase that influence [49], involving those who are intended to ultimately benefit from the SI [75]. As Reed et al. [100] states “Only through active community involvement can indicators facilitate progress toward sustainable development goals”. Meadows [69] speaks in this sense of “scientific credibility” and “political credibility”, Hak et al. [101] of “scientific relevance” and “public relevance” of SI, whereas Cloquell-Ballester [68] calls for “scientific” and “social validation” of SI.
Among scholars, the need for such “multiple perspectives” or “hybrid” SI that capture the strengths of both streams of thought is increasingly recognized [75,99,100]. While there is no consensus on how this combination should be best accomplished, and recognizing the epistemological differences of each school of thought, a number of general similar steps can be identified: (1) establish the system’s socio-environmental context; (2) set sustainability objectives and strategies; (3) identify, evaluate, select SI; and (4) apply SI [99]. The development and application of hybrid SI is increasingly documented (see for example [20,74,94,99]).
Complementing the different methodological approaches various quality criteria of SI can be found, also showing the need to integrate top-down and bottom-up approaches. Hak et al. [101], for example identified 260 different criteria, although there was substantial overlap in content and meaning. Considering principle four of Bellagio STAMP and the reviews of quality criteria of SI made by Gallopin [67], Hezri [81] and Van de kerk et al. [86] we identify the following ones, organized according Hezri’s [81] classification: robustness (scientifically credible; clear and standardized methodology to facilitate comparison; values should be measurable; data should be available/obtainable/reliable/up-to-date; sensitive to changes; practical focus such as a limited number of key issues; reference values should be included; based on models with holistic perspective; appropriateness of scale; no overlap; and independence between indicators), democratic (participation of and supported by stakeholders, experts and policymakers; openness with accessible sets of indicators, methods and explicit judgments), longevity (capacity for repeated measurement; iterative and adaptive to change; cost-effective; resource availability), and relevance (institutional capacity for data collection, maintenance and documentation; meets the needs of audience and users; presentation in understandable structure; guided by clear vision of sustainability and relevant for its sustainability attributes).
It should be emphasized that quality criteria go beyond the “technical/analytical” soundness of SI measurement and include process criteria of what might be called “managerial” soundness as well. Ultimately, the necessary qualities depend on the SI-purpose(s) [76,86,102] and in practice it is hard to meet all criteria simultaneously [76,103].
In the selection of a good set of SI there are some common pitfalls including: over-aggregation; hiding relevant more detailed information; measuring what is measurable, rather than what is important; dependence on a biased model; deliberate falsification in favor of stakeholders own interests; diverting attention from direct experience; over-confidence in indicators while they might depict a wrong picture; and incompleteness of the indicators because they are not the real system [69].

4.4. Dichotomies

SI can be classified along various dimensions of measurement, such as sustainability attributes (for example socio-economic or environmental attributes) or frameworks (for example DPSIR-indicators) [49]. At least the following dichotomies deserve attention because they touch upon several important aspects of SI, and are in practice frequently subject of discussion and controversy:
  • Descriptive vs. normative;
  • Quantitative vs. qualitative;
  • Objective vs. subjective;
  • Community vs. expert;
  • Ex-ante vs. ex-post.
Descriptive SI give a description of an actual situation, while normative SI compare an actual situation with a desired one [104]. However, taking into account the definition of an indicator this distinction is redundant and even invalid, because inherently an indicator always includes a reference value, and as such is inherently normative [78].
Quantitative SI are based on quantitative data and provide information in a quantitative—numerical—manner, while qualitative SI are based on qualitative data and provide information in a qualitative—non-numerical—manner. Traditionally, indicators are considered as quantification tools, however for sustainable development quantitative and qualitative SI are complementary, as we cannot measure human experiences in a quantitative manner alone [20,69,80]. Recognizing the attractiveness of quantified SI for decision-making [105], qualitative data and SI can often, if desired, be converted into quantitative ones [67]. However, we should be very careful with conceiving and interpreting the world solely in quantitative terms and solely decide upon them. “The fact that people consider something ugly or beautiful, harmonious or dissonant, noble or ignoble, is not to be swept away as ‘mere opinion’”. If we guide our decisions only by quantitative indicators and not qualitative ones, we will produce a world of quantity without quality. Many of our social and personal problems arise from the fact that we are well on our way to doing exactly that” [69].
Taking into account the traditional scientific objective—subjective divide, a distinction is made between objective and subjective SI. Objective SI are sensed by instruments outside the individual, such as thermometers or counters that can be verified by others. Subjective SI are sensed only from within the individual—by individual judgments—not verifiable by others through instruments but only verifiable through “subjective” explanations. Within this divide objective SI primarily measure in a quantitative way, while subjective SI primarily measure in a qualitative way [69]. However, ultimately all SI are (partially) subjective because their development is full of (implicit) subjective choices (for example the system’s attributes that are going to be measured, by which SI and by which variables, their calculation, the identification of reference values—what is sustainability?) [49,69]. In the words of Morse et al. [106] “[…]indicator development and interpretation is more art than science and the room for subjectivity in all stages of their application is large”. Meadows [69] argues: “Indicators arise from values (we measure what we care about), and they create values (we care about what we measure)”. Hence one of the challenges ahead is to develop a new set of values-based SI, measuring sustainability’s underlying ethical principles to guide the transition towards a (more) sustainable world [22]. The sustainability principles of the Earth Charter [107], probably representing the most inclusive participatory process ever associated with the drafting of an international declaration [107], which are widely recognized as a global consensus statement can easily be picked-up to take on that challenge and go ahead.
The distinction between community and expert SI is essentially about who develops the SI—stakeholders “bottom up” and/or experts “top down” [49,73,93]. “Top down” approaches enable experts to define the sustainability framework and its indicators. “Bottom up” approaches allow the participation of stakeholders in such a process [49]. As already said, “hybrid” or “multiple perspective” SI are also possible and needed.
The distinction between ex-ante and ex-post SI goes hand in hand with the ex-ante/ex-post SA-divide. Ex-ante SI provide information to assess the effects of decisions in advance and support choice between various options before practical implementation, while ex-post SI provide information after decisions are taken to assess or evaluate their practical implementation.

4.5. Interpretation

SI should be analyzed and interpreted in terms of “distance-to-target”, in relation to other indicators and taking into account the real world context. As such, the information they communicate can be turned into knowledge in order to learn about a particular system’s sustainability [108,109] (Figure 2).
Sustainability 06 05512 g002 1024
Figure 2. From data to knowledge; modified, after [110].
Figure 2. From data to knowledge; modified, after [110].
Sustainability 06 05512 g002
Indicators are derived from data [110], which estimates the value they adopt [67]. Sometimes a distinction is made between primary and secondary data. The former are the actual measurements (or observations in the case of qualitative data) of the values of the indicator’s variables. The latter are analyzed primary data (for example added, averaged, etc.) [67,111]. From a hierarchical perspective—although contested [67] (see 4.1 Definition)—, the relation between data, indicators and indices is depicted by the “information pyramid” [111] or—“iceberg” [112] (Figure 3).
Considering the real world context during decision-making is essential because SI and their building frameworks or models are in every instance a social construction, reduction and simplification of the complex reality and its many uncertainties and risks, acquired by various ways of learning [67,69,111,113]. To understand the real world, make decisions and act upon them we need to simplify [80,113], at least to some extent. However, at the same time decision-makers should need to learn and deal with the unavoidable real world complexity and its many uncertainties and risks. This can be done by complementing SI with additional information. As Dahl [22] argues “Even the best system of indicators will need to be complemented by other measures and inputs to ensure decisions in the interest of long-term sustainability”. However, the decision-making process can also be modified by including more spontaneous ex-post micro decision-making processes as decisions and consequent actions unfold to deal with the many uncertainties and risks (also referred as “governing”). This complementary to the prevailing consciously designed ex-ante ways of working (also referred to as “governance”) [114]. However, in the first place decision-makers should be aware of and recognize the reductionist and simplifying feature of SI. The trap of creating and (blindly) believing in a virtual SI reality should be avoided. This is a real risk because from a “rational managerial” perspective simplification and the reduction of ambiguities, uncertainties and risks is appealing in order to straightforward “manage and control”.
Sustainability 06 05512 g003 1024
Figure 3. The information pyramid [111].
Figure 3. The information pyramid [111].
Sustainability 06 05512 g003
It also implies that in spite of all recommendations and attempts for holistic approaches, SI will remain reductionist tools. The most extreme representation of it is found in the desire to create a single, quantifiable sustainability index—“the quest for the unicorn”. This is understandable from a managerial and competitive point of view. It simplifies complexity into a single value that readily allows comparison [80] and, whereas the growth economy has its Gross Domestic Product unicorn—certainly (still) the most popular indicator to measure progress among national leaders [22]—, sustainability proponents look for the antipode. Nevertheless (more) holistic SI-approaches are highly needed. In this sense, Gasparatos et al. [45] calls to move away from indices towards methodological pluralism. For example, the Systemic Sustainability Analysis and Imagine approach of Bell et al. [20,74,80,94] aims to combine the strength of SI supported decision-making in a (more) holistic manner with a participatory development of these indicators, recognizing underlying normative choices, which are inherent to sustainability and any development of SI. Instead, Babcicky [19] underlines the need for indexes, but at the same time argues to apply various perspectives on variable selection, re-consider weighting schemes and re-think methods to improve their quality.

5. Conclusions

To move forward in the societal transition and to become fully operational, sustainable development must be considered as a decision-making strategy by all stakeholders from the local to the global level. Doing so turns sustainability from an “action-guiding” concept into an “action-generating” concept. In decision-making for sustainable development key challenges include interpretation, information-structuring and influence. The SA and SI purposes demonstrate that they have the potential to address them.
The definitions demonstrate that SA deals with the assessment “process” while SI deal with the “technical” aspects of measurement. As such, both fields are complementary and mutually strengthening. The relative value of SA and SI in decision-making lies respectively in the contribution to the interpretation and influence challenge, and in the contribution to the information-structuring challenge. Although, as a logic consequence of shared concerns both fields deal with the three challenges simultaneously. Indeed, although their focus differs SA and SI are inherently linked. SI, whether implicitly or explicitly are part of any SA and in any development of SI a process is involved as well.
Obviously, it seems less important to which field an initiative (practitioner) or research(er) “belongs” or prefers to “belong” to. The more important is that process and technical measurement aspects are taken into consideration. Beyond their theoretical relevance the definitions are also practically useful. They highlight the fundamental elements of any SA and SI, i.e., (1) interpreting sustainability; (2) measuring impact and structuring information—it is particularly here that SI and their building blocks come in—; and (3) exerting influence on decision-making in support of sustainable development.
While the definitions point out their difference and their specific role, the similarities and high connectivity between SA and SI is probably best shown by their purposes. SA and SI: (1) operationalize sustainable development for a given system in a particular socio-environmental context and, as such, attribute meaning to the concept, going beyond abstract formulations and definitions; (2) generate and communicate complex sustainability information, in a well-structured manner informing decision-making, including to demonstrate accountability and allow benchmarking; (3) facilitate continuous (social) learning among involved stakeholders and decision-makers; and (4) identify knowledge and data gaps.
Additionally, from a methodological perspective, SA and SI share similar concerns. The proposed characteristics of an ideal-typical SA describe a broader methodological framework than SI, including the need for “top-down/expert-driven” and “bottom-up/stakeholder-driven” integration—so called “multiple perspectives”/“hybrid” approaches. Therefore, the SA-characteristics can serve as a methodological umbrella for SA and SI, which can be further complemented by SI’s quality criteria, also considering more “technical” measurement aspects (Figure 4).
Sustainability 06 05512 g004 1024
Figure 4. Merging sustainability assessment and indicator methodologies.
Figure 4. Merging sustainability assessment and indicator methodologies.
Sustainability 06 05512 g004
Because to some extent we need to simplify, SA and SI are indispensable decision-supporting tools to understand and interpret the world and to guide us towards a (more) sustainable society. However they are always a reduction and simplification of reality and should never be mistaken for it. To deal with the real world complexity, its many uncertainties and risks, there is a strong call for (more) whole-system holistic approaches. This can be done through suited—more holistic—SI approaches and adapted SA processes that include additional information for decision-making and considers more spontaneous ex-post micro decision-making processes as a complement of prevailing consciously designed ex-ante ways of working.

Acknowledgments

We acknowledge the support of the KLIMOS Acropolis Research Platform for Sustainable Development Cooperation funded by the Belgian Development Cooperation, the Flemish Interuniversity Council—University Development Cooperation and the Walloon-Brussels Academy of Research and Higher Education for this paper. We are grateful to Rodrigo Lozano and the four anonymous reviewers for their comments and suggestions for improvement.

Author Contributions

All authors contributed to the article in sequence of appearance as co-authors. All authors have read and approved the final published manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Carley, M.; Christie, I. Managing Sustainable Developmen; Earthscan: London, UK, 2000. [Google Scholar]
  2. Reid, D. Sustainable Development—An Introductory Guide; Earthscan: London, UK, 2005. [Google Scholar]
  3. Dalal-Clayton, B.; Bass, S. Sustainable Development Strategies: A Resource Book; Earthscan: London, UK, 2002. [Google Scholar]
  4. Rogers, P.; Jalal, K.; Boyd, J. An Introduction to Sustainable Development; Earthscan: London, UK, 2008. [Google Scholar]
  5. Christen, M.; Schmidt, S. A formal framework for conceptions of Sustainability—A theoretical contribution to the discourse in sustainable development. Sustain. Dev. 2011, 20, 400–410. [Google Scholar] [CrossRef]
  6. Dresner, S. The Principles of Sustainability, 2nd ed.; Earthscan: London, UK, 2008. [Google Scholar]
  7. Robinson, J. Squaring the circle? Some thoughts on the idea of sustainable development. Ecol. Econ. 2004, 48, 369–384. [Google Scholar] [CrossRef]
  8. Gibson, R. Specification of Sustainability-Based Environmental Assessment Decision Criteria and Implications for Determining “Significance” in Environmental Assessment; Public Works and Government Services Canada: Ottawa, ON, Canada, 2000. [Google Scholar]
  9. Lozano, R. Envisioning sustainability three-dimensionally. J. Clean. Prod. 2008, 16, 1838–1846. [Google Scholar] [CrossRef]
  10. Waas, T.; Hugé, J.; Verbruggen, A.; Wright, T. Sustainable development: A bird’s eye view. Sustainability 2011, 3, 1637–1661. [Google Scholar] [CrossRef]
  11. Lélé, S. Sustainable development: A critical review. World Dev. 1991, 19, 607–621. [Google Scholar]
  12. Gibson, R. Sustainability assessment: Basic components of a practical approach. Impact Assess. Proj. Apprais. 2006, 24, 170–182. [Google Scholar] [CrossRef]
  13. Lafferty, W.; Meadowcroft, J. Implementing Sustainable Development. Strategies and Initiatives in High Consumption Societies; Oxford University Press: Oxford, UK, 2000. [Google Scholar]
  14. Sneddon, C.; Howarth, R.; Norgaard, R. Sustainable development in a post-Brundtland world. Ecol. Econ. 2006, 57, 253–268. [Google Scholar] [CrossRef]
  15. Rees, W. What’s blocking sustainability? Human nature, cognition, and denial. Sustainability 2010, 6, 13–25. [Google Scholar]
  16. Quental, N.; Lourenço, J.M.; da Silva, F.N. Sustainable development policy: Goals, targets and political cycles. Sustain. Dev. 2011, 19, 15–29. [Google Scholar] [CrossRef]
  17. Fergus, A.; Rowney, J. Sustainable development: Lost meaning and opportunity? J. Bus. Ethics 2005, 60, 17–27. [Google Scholar] [CrossRef]
  18. Hugé, J.; Waas, T.; Eggermont, G.; Verbruggen, A. Impact assessment for a sustainable energy future—Reflections and practical experiences. Energy Policy 2011, 39, 6243–6253. [Google Scholar]
  19. Babcicky, P. Rethinking the foundations of sustainability measurement: The Limitations of the Environmental Sustainability Index (ESI). Soc. Indic. Res. 2012, 113, 133–157. [Google Scholar] [CrossRef]
  20. Bell, S.; Morse, S. Sustainability Inidcators—Measuring the Immeasurable? Earthscan: London, UK, 2008. [Google Scholar]
  21. Böhringer, C.; Jochem, P. Measuring the immeasurable—A survey of sustainability indices. Ecol. Econ. 2007, 63, 1–8. [Google Scholar]
  22. Dahl, A.L. Achievements and gaps in indicators for sustainability. Ecol. Indic. 2012, 17, 14–19. [Google Scholar] [CrossRef]
  23. Bond, A.; Morrison-Saunders, A.; Pope, J. Sustainability assessment: The state of the art. Impact Assess. Proj. Apprais. 2012, 30, 53–62. [Google Scholar] [CrossRef]
  24. Patton, M.Q. Qualitative Research and Evaluation Methods; Sage: Thouasands Oaks, CA, USA, 2002. [Google Scholar]
  25. Strauss, A.; Corbin, J. Basics of Qualitative Research, Techniques and Procedures for Developing Grounded theory; Sage Publications: Thouasands Oaks, CA, USA, 1998. [Google Scholar]
  26. Petschow, U.; Rosenau, J.; von Weizsacker, E. Governance and Sustainability: New Challenges for States, Companies and Civil Society; Greenleaf: Sheffield, UK, 2005. [Google Scholar]
  27. Reed, M.; Evely, A.; Cundill, G.; Fazey, I.; Glass, J.; Laing, A.; Newig, J.; Parrish, B.; Prell, C.; Raymond, C.; et al. What is social learning. Ecol. Soc. 2010, 15. Available online: http://www.ecologyandsociety.org/vol15/iss4/resp1/ (accessed on 30 July 2014).
  28. Boehmer-Christiansen, S. The geo-politics of sustainable development: Bureaucracies and politicians in search of the holy grail. Geoforum 2002, 33, 351–365. [Google Scholar]
  29. Peterson, J.; Bomberg, E. Decision-making in the European Union; Macmillan Press: Houndmills, UK, 1999. [Google Scholar]
  30. Wang, Y.; Ruhe, G. The cognitive process of decision making. Int. J. Cogn. Inf. Nat. Intell. (IJCINI) 2007, 1, 73–85. [Google Scholar] [CrossRef]
  31. Holder, J. Environmental Assessment: The Regulation of Decision Making; Oxford University Press: Oxford, UK, 2004; p. 371. [Google Scholar]
  32. Bell, S.; Morse, S. The role of sustainability indicators within evidence-based policy for sustainable development in the European Union. In Proceedings of the International Sustainable Development Research Conference, Hong Kong, China, 30 May–1 June 2010.
  33. Hertin, J.; Turnpenny, J.; Jordan, A.; Nilsson, M.; Russel, D.; Nykvist, B. Rationalising the policy mess? Ex ante policy assessment and the utilisation of knowledge in the policy process. Environ. Plan. A 2009, 41, 1185–1200. [Google Scholar] [CrossRef]
  34. Huston, A.C. From research to policy and back. Child Dev. 2008, 79, 1–12. [Google Scholar] [CrossRef] [PubMed]
  35. Runhaar, H.; Driessen, P.P.J. What makes strategic environmental assessment successful environmental assessment? The role of context in the contribution of SEA to decision-making. Impact Assess. Proj. Apprais. 2007, 25, 2–14. [Google Scholar] [CrossRef]
  36. Hoppe, R. Cultures of public policy problems. J. Comp. Policy Anal. 2002, 4, 305–326. [Google Scholar]
  37. Söderbaum, P. Issues of paradigm, ideology and democracy in sustainability assessment. Ecol. Econ. 2007, 60, 613–626. [Google Scholar]
  38. Block, T.; van Assche, J.; Goeminne, G. Unravelling urban sustainability: How the flemish city monitor acknowledges complexities. Ecol. Inf. 2013, 17, 104–110. [Google Scholar] [CrossRef]
  39. Rydin, Y. Indicators as a governmental technology? The lessons of community-based sustainability indicator projects. Environ. Plan. D Soc. Space 2007, 25, 610–624. [Google Scholar] [CrossRef]
  40. Rigby, D.; Woodhouse, P.; Young, T.; Burton, M. Constructing a farm level indicator of sustainable agricultural practice. Ecol. Econ. 2001, 39, 463–478. [Google Scholar] [CrossRef]
  41. Ness, B.; Urbel-Piirsalu, E.; Anderberg, S.; Olsson, L. Categorising tools for sustainability assessment. Ecol. Econ. 2007, 60, 498–508. [Google Scholar] [CrossRef]
  42. IAIA International Association for Impact Assessment. Available online: http://www.iaia.org (accessed on 12 March 2014).
  43. Hacking, T.; Guthrie, P. A framework for clarifying the meaning of triple bottom-line, integrated, and sustainability assessment. Environ. Impact Assess. Rev. 2008, 28, 73–89. [Google Scholar] [CrossRef]
  44. Devuyst, D.; Hens, L.; de Lannoy, W. How Green is the City? Sustainability Assessment and the Management of of Urban Environments; Columbia University Press: New York, NY, USA, 2001. [Google Scholar]
  45. Gasparatos, A.; el-Haram, M.; Horner, M. A critical review of reductionist approaches for assessing the progress towards sustainability. Environ. Impact Assess. Rev. 2008, 28, 286–311. [Google Scholar] [CrossRef]
  46. Gibson, R.; Hassan, S.; Holtz, S.; Tansey, J.; Whitelaw, G. Sustainability Assessment—Criteria and Processes; Earthscan: London, UK, 2005. [Google Scholar]
  47. Pope, J. What’s so special about sustainability assessment? J. Environ. Assess. Policy Manag. 2006, 8, v–x. [Google Scholar]
  48. Pintér, L.; Hardi, P.; Martinuzzi, A.; Hall, J. Bellagio STAMP: Principles for sustainability assessment and measurement. Ecol. Indic. 2012, 17, 20–28. [Google Scholar]
  49. Singh, R.K.; Murty, H.R.; Gupta, S.K.; Dikshit, A.K. An overview of sustainability assessment methodologies. Ecol. Indic. 2012, 15, 281–299. [Google Scholar] [CrossRef]
  50. Baber, W.F. Ecology and democratic governance: Toward a deliberative model of environmental politics. Soc. Sci. J. 2004, 41, 331–346. [Google Scholar] [CrossRef]
  51. Bebbington, J.; Brown, J.; Frame, B. Accounting technologies and sustainability assessment models. Ecol. Econ. 2007, 61, 224–236. [Google Scholar] [CrossRef]
  52. Nooteboom, S. Impact assessment procedures for sustainable development: A complexity theory perspective. Environ. Impact Assess. Rev. 2007, 27, 645–665. [Google Scholar] [CrossRef]
  53. Funtowicz, S.; Martinez-Allier, J.; Munda, G.; Ravetz, J. Information Tools for Environmental Policy under Conditions of Complexity; European Environment Agency: Copenhagen, Denmark, 1999. [Google Scholar]
  54. Chanchitpricha, C.; Bond, A. Conceptualising the effectiveness of impact assessment processes. Environ. Impact Assess. Rev. 2013, 43, 65–72. [Google Scholar] [CrossRef]
  55. Bond, A.; Morrison-Saunders, A.; Howitt, R. Sustainability Assessment—Pluralism, Practice and Progress; Routledge: New York, NY, USA, 2013. [Google Scholar]
  56. Hardi, P.; Zdan, T. Assessing Sustainable Development: Principles in Practice; International Institute for Sustainable Development: Winnipeg, MB, Canada, 1997. [Google Scholar]
  57. Hardi, P. Measurement and indicators program of the international institute for sustainable development. In Sustainability Indicators, Report of the Project on Indicators of Sustainable Development; Moldan, B., Billharz, S., Eds.; John Wiley & Sons: Chichester, UK, 1997. [Google Scholar]
  58. George, C. Sustainability appraisal for sustainable development: Integrating everything from jobs to climate change. Impact Assess. Proj. Apprais. 2001, 19, 95–106. [Google Scholar] [CrossRef]
  59. Fischer, T. Strategic environmental assessment performance criteria—The same requirements for every assessment? J. Environ. Assess. Policy Manag. 2002, 4, 83–99. [Google Scholar]
  60. Jacob, K.; Hertin, J.; Volkery, A. Considering environmental aspects in integrated impact assessment: Lesson learned and challenges ahead. In Impact Assessment and Sustainable Development—European Practice and Experience; George, C., Kirkpatrick, C., Eds.; Edward Elgar Publishing: Cheltenham, UK, 2007. [Google Scholar]
  61. Lee, N. Bridging the gap between theory and practice in integrated assessment. Environ. Impact Assess. Rev. 2006, 26, 56–68. [Google Scholar] [CrossRef]
  62. Hugé, J.; Waas, T. Converging impact assessment discourses for sustainable development: The case of Flanders, Belgium. Environ. Dev. Sustain 2011, 13, 607–626. [Google Scholar]
  63. Munda, G. Multiple criteria decision analysis and sustainable development. In Multiple Criteria Decision Analysis: State of the Art Surveys; Greco, S., Ed.; Springer: New York, NY, USA, 2005; Volume 78, pp. 953–986. [Google Scholar]
  64. Steele, K.; Carmel, Y.; Cross, J.; Wilcox, C. Uses and misuses of multicriteria decision analysis (mcda) in environmental decision making. Risk Anal. 2009, 29, 26–33. [Google Scholar] [CrossRef] [PubMed]
  65. Wang, J.-J.; Jing, Y.-Y.; Zhang, C.-F.; Zhao, J.-H. Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renew. Sustain. Energy Rev. 2009, 13, 2263–2278. [Google Scholar] [CrossRef]
  66. Dias, L.C.; Domingues, A.R. On multi-criteria sustainability assessment: Spider-gram surface and dependence biases. Appl. Energy 2014, 113, 159–163. [Google Scholar] [CrossRef]
  67. Gallopin, G.C. Indicators and their use: Information for decision-making. In Sustainability Indicators, Report of the Project on Indicators of Sustainable Development; Moldan, B., Billharz, S., Eds.; John Wiley & Sons: Chichester, UK, 1997. [Google Scholar]
  68. Cloquell-Ballester, V.-A.; Cloquell-Ballester, V.-A.; Monterde-Díaz, R.; Santamarina-Siurana, M.-C. Indicators validation for the improvement of environmental and social impact quantitative assessment. Environ. Impact Assess. Rev. 2006, 26, 79–105. [Google Scholar]
  69. Meadows, D. Indicators and Information Systems for Sustainable Development—A Report to the Balaton Group; The Sustainability Institute: Hartland, VT, USA, 1998. [Google Scholar]
  70. UN Earth Summit—AGENDA 21. Available online: http://www.un.org/esa/dsd/agenda21/index.shtml (accessed on 12 March 2014).
  71. Hak, T.; Moldan, B.; Dahl, A.L. Editorial. Ecol. Indic. 2012, 17, 1–3. [Google Scholar] [CrossRef]
  72. Krank, S.; Wallbaum, H.; Grêt-Regamey, A. perceived contribution of indicator systems to sustainable development in developing countries. Sustain. Dev. 2013, 21, 18–29. [Google Scholar] [CrossRef]
  73. Rigby, D.; Howlett, D.; Woodhouse, P. A review of indicators of agricultural and rural livelihood sustainability. Available online: http://r4d.dfid.gov.uk/Output/175219/ (accessed on 30 July 2014).
  74. Bell, S.; Morse, S. Measuring Sustainability, Learning from Doing; Earthscan: London, UK, 2003. [Google Scholar]
  75. Bell, S.; Morse, S. Breaking through the Glass Ceiling: Who really cares about sustainability indicators? Local Environ. 2001, 6, 291–309. [Google Scholar] [CrossRef]
  76. Moldan, B.; Dahl, A. Challenges to sustainability indicators. In Sustainability Indicators—A Scientific Assessment; Hák, T., Moldan, B., Dahl, A., Eds.; Island Press: Washington, DC, USA, 2007. [Google Scholar]
  77. Couder, J.; Verbruggen, A. Towards an integrated performance indicator for (energy) benchmarking covenants with industry. In The Handbook of Environmental Voluntary Agreements, Design, Implementation and Evaluation Issues; Croci, E., Ed.; Springer: Dordrecht, The Netherlands, 2005. [Google Scholar]
  78. Bakkes, J.; van den Born, G.; Helder, J.; Swart, R.; Hope, C.; Parker, J. An Overview of Environmental Indicators: State of the Art and Perspectives; UNEP/RIVM: Nairobi, Kenya, 1994. [Google Scholar]
  79. Meadows, D. Thinking in Systems; Chelsea Green Publishing: White River Junction, VT, USA, 2008. [Google Scholar]
  80. Bell, S.; Morse, S. Sustainability Indicators: Measuring the Immeasurable; Earthscan: London, UK, 2005. [Google Scholar]
  81. Hezri, A. Sustainability indicator system and policy processes in Malaysia: A framework for utilisation and learning. J. Environ. Manag. 2004, 73, 357–371. [Google Scholar]
  82. Parris, T.; Kates, R.W. Characterizing a sustainability transition: Goals, targets, trends, and driving forces. Proc. Natl. Acad. Sci. USA 2003, 100, 8068–8073. [Google Scholar] [CrossRef] [PubMed]
  83. Lancker, E.; Nijkamp, P. A policy scenario analysis of sustainable agricultural development options: A case study for Nepal. Impact Assess. Proj. Apprais. 2000, 18, 111–124. [Google Scholar] [CrossRef]
  84. Gilbert, A. Criteria for sustainability in the development of indicators for sustainable development. Chemosphere 1996, 33, 1739–1748. [Google Scholar] [CrossRef]
  85. Opschoor, H.; Reijnders, L. Towards sustainable development indicators. In Search of Indicators of Sustainable Development; Kuik, O., Verbruggen, H., Eds.; Kluwer: Dordrecht, The Netherlands, 1991. [Google Scholar]
  86. Van de Kerk, G.; Manuel, A.R. A comprehensive index for a sustainable society: The SSI—The Sustainable Society Index. Ecol. Econ. 2008, 66, 228–242. [Google Scholar]
  87. Smeets, E.; Weterings, R. Environmental Indicators: Typology and Overview; European Environmental Agency: Copenhagen, Denmark, 1999. [Google Scholar]
  88. Wackernagel, M.; Rees, W. Our Ecological Footprint—Reducing Human Impact on the Earth; New Society Publishers: Gabriola Island, BC, Canada, 2007. [Google Scholar]
  89. UNDP. Human Development Indices—A Statistical Update 2008; United Nations Development Programme: New York, NY, USA, 2008. [Google Scholar]
  90. Malkina-Pykh, I. Integrated assessment models and response function models: Pros and cons for sustainable development indices design. Ecol. Indic. 2002, 2, 93–108. [Google Scholar]
  91. Rennings, K.; Wiggering, H. Steps towards indicators of sustainable development: Linking economic and ecological concepts. Ecol. Econ. 1997, 20, 25–36. [Google Scholar] [CrossRef]
  92. Hodge, A.; Hardi, P.; Bell, D. Seeing change through the lens of sustainability. In Workshop “Beyond Delusion: Science and Policy Dialogue on Designing Effective Indicators of Sustainable Development”; International Institute for Sustainable Development: Costa Rica, Republic of Costa Rica, 1999. [Google Scholar]
  93. Fraser, E.D.G.; Dougill, A.J.; Mabee, W.E.; Reed, M.; McAlpine, P. Bottom up and top down: Analysis of participatory processes for sustainability indicator identification as a pathway to community empowerment and sustainable environmental management. J. Environ. Manag. 2006, 78, 114–127. [Google Scholar] [CrossRef]
  94. Bell, S.; Morse, S. Experiences with sustainability indicators and stakeholder participation: A case study relating to a “blue plan” project in Malta. Sustain. Dev. 2004, 12, 1–14. [Google Scholar] [CrossRef]
  95. Shields, D.J.; Solar, S.V.; Martin, W.E. The role of values and objectives in communicating indicators of sustainability. Ecol. Indic. 2002, 2, 149–160. [Google Scholar] [CrossRef]
  96. Hezri, A.; Dovers, S. Sustainability indicators, policy and governance: Issues for ecological economics. Ecol. Econ. 2006, 60, 86–99. [Google Scholar] [CrossRef]
  97. Lehtonen, M. Mainstreaming sustainable development in the OECD through indicators and peer reviews. Sustain. Dev. 2008, 16, 241–250. [Google Scholar] [CrossRef]
  98. Bell, S.; Morse, S. Groups and indicators in post-industrial society. Sustain. Dev. 2014, 22, 145–157. [Google Scholar] [CrossRef]
  99. Reed, M.S.; Fraser, E.D.G.; Dougill, A.J. An adaptive learning process for developing and applying sustainability indicators with local communities. Ecol. Econ. 2006, 59, 406–418. [Google Scholar] [CrossRef]
  100. Reed, M.; Fraser, E.; Morse, S.; Dougill, A. Integrating methods for developing sustainability indicators to facilitate learning and action. Ecol. Soc. 2005, 10. Available online: http://www.ecologyandsociety.org/vol10/iss1/resp3/ (accessed on 30 July 2014).
  101. Hak, T.; Kovanda, J.; Weinzettel, J. A method to assess the relevance of sustainability indicators: Application to the indicator set of the Czech Republic’s sustainable development strategy. Ecol. Indic. 2012, 17, 46–57. [Google Scholar] [CrossRef]
  102. Spangenberg, J.; Pfahl, S.; Deller, K. Towards indicators for institutional sustainability: Lessons from an analysis of Agenda 21. Ecol. Indic. 2002, 2, 61–77. [Google Scholar] [CrossRef]
  103. De Kruijf, H.; van Vuuren, D. Following sustainable development in relation to the North-South dialogue: Ecosystems health and sustainability indicators. Ecotoxicol. Environ. Saf. 1998, 40, 4–14. [Google Scholar]
  104. Weterings, R. Indicatoren Voor Duurzame Ontwikkeling; Raad voor Natuur- en Milieuonderzoek (RMNO): Rijswijk, The Netherlands, 1993. [Google Scholar]
  105. Rotmans, J. Tools for integrated sustainability assessment: A two track approach. Integr. Assess. J. 2006, 6, 35–57. [Google Scholar]
  106. Morse, S.; Bell, S. Sustainable development indicators: The tyranny of methodology revisited. Consilience 2011, 6, 222–239. [Google Scholar]
  107. ECI. The Earth Charter Initiative—Handbook; Earth Charter International Secretariat: San José, CA, USA, 2008. [Google Scholar]
  108. OECD. Towards Sustainable Development—Environmental Indicators; OECD: Paris, France, 1998. [Google Scholar]
  109. Levett, R. Sustainability indicators—Integrating quality of life and environmental protection. J. R. Stat. Soc. 1998, 161, 291–302. [Google Scholar] [CrossRef]
  110. Segnestam, L. Indicators of Environment and Sustainable Development; The World Bank: Washington, DC, USA, 2002. [Google Scholar]
  111. Hammond, A.; Adriaanse, A.; Rodenburg, E.; Bryant, D.; Woodward, R. Environmental Indicators: A Systematic Approach to Measuring and Reporting on Environmental Policy Performance in the Context of Sustainable Development; World Resource Institute: Washington, DC, USA, 1995. [Google Scholar]
  112. Jesinghaus, J. A European System of Environmental Pressure Indices: First Volume of the Environmental Pressure Indices Handbook: The indicators. Available online: http://esl.jrc.it/envind/theory/handb_.htm (accessed on 13 March 2014).
  113. Rosner, W.J. Mental models for sustainability. J. Clean. Prod. 1995, 3, 107–121. [Google Scholar] [CrossRef]
  114. Sanderson, J. Risk, uncertainty and governance in megaprojects: A critical discussion of alternative explanations. Int. J. Proj. Manag. 2012, 30, 432–443. [Google Scholar] [CrossRef]

Share and Cite

MDPI and ACS Style

Waas, T.; Hugé, J.; Block, T.; Wright, T.; Benitez-Capistros, F.; Verbruggen, A. Sustainability Assessment and Indicators: Tools in a Decision-Making Strategy for Sustainable Development. Sustainability 2014, 6, 5512-5534. https://doi.org/10.3390/su6095512

AMA Style

Waas T, Hugé J, Block T, Wright T, Benitez-Capistros F, Verbruggen A. Sustainability Assessment and Indicators: Tools in a Decision-Making Strategy for Sustainable Development. Sustainability. 2014; 6(9):5512-5534. https://doi.org/10.3390/su6095512

Chicago/Turabian Style

Waas, Tom, Jean Hugé, Thomas Block, Tarah Wright, Francisco Benitez-Capistros, and Aviel Verbruggen. 2014. "Sustainability Assessment and Indicators: Tools in a Decision-Making Strategy for Sustainable Development" Sustainability 6, no. 9: 5512-5534. https://doi.org/10.3390/su6095512

APA Style

Waas, T., Hugé, J., Block, T., Wright, T., Benitez-Capistros, F., & Verbruggen, A. (2014). Sustainability Assessment and Indicators: Tools in a Decision-Making Strategy for Sustainable Development. Sustainability, 6(9), 5512-5534. https://doi.org/10.3390/su6095512

Article Metrics

Back to TopTop