Urban Pluvial Flood Management Part 1: Implementing an AHP-TOPSIS Multi-Criteria Decision Analysis Method for Stakeholder Integration in Urban Climate and Stormwater Adaptation

Abstract

Cities are facing increasing pressures to enact adaptation measures due to climate change. While blue-green infrastructure has emerged as a focal adaptation technique for stormwater management, in order to craft adaptation policies cities must consider a multitude of emerging, complex, and competing stakeholder interests around multiple adaptation alternatives. However, accounting for these different interests, analyzing their diverse priorities, and maintaining a transparent decision-making process is not easily achieved within the existing policy frameworks. Here we define and present a combined multi-criteria decision analysis (MCDA) of the analytic hierarchy process (AHP) and the technique for order of preference by similarity to ideal solution (TOPSIS) methods that easily integrates and quantifies stakeholder priorities while remaining accessible for non-experts engaged in the policy-making process. We demonstrate the method’s effectiveness through analyzing opinions about stormwater adaptation in New York City across several stakeholder groups. The method succeeds in integrating quantitative and qualitative judgements, indicating stakeholder preferential differences and allowing for more inclusive policy to be crafted. It can be extended beyond stormwater to many urban climate adaptation decisions facing multi-criteria considerations.

1. Introduction

Climate change is placing enormous pressures on urban policy decision-making. Traditional governance decision-making methods are being re-evaluated with the growing importance of urban sustainability, resilience, and adaptation. While disruptive of the status quo, this shift allows cities to engage with different and larger groups of people and concepts than before.

What defines urban climate adaptation is that decisions need to be made quickly for both long-term and immediate threats, at varying geographic scales, alongside varying climate projections. Urban pluvial flood management exemplifies these difficult considerations. For long-term adaptation, projects become stalled as the main benefits from implementation are often reaped decades later providing no or limited immediate financial returns [1]. With how environmental benefits are usually quantified, it is an uphill battle to pursue future environmental investments over other immediate pressing urban issues. The idea of future proofing is also questioned as technologies may become obsolete or climate projections over or underestimate the future reality [2].

For immediate threats, urban bureaucracy still poses a barrier to adaptation. It is not a lack of best practice awareness that delays policy but rather policy systems are not designed for radical changes [3]. Policy-makers become familiar with certain methods and processes and may struggle to diverge from this path dependence in the face of emerging pressures [4]. Furthermore, communication and coordination between departments may not be streamlined despite the interdisciplinary effects of climate change. As these delays compound across the various levels of governance, policy-making may struggle unless radical change occurs.

Despite the barriers, cities are making progress incorporating climate adaptation into their decision-making framework. Environmental considerations are becoming commonplace within governance [5]. However, as with most policy a city needs both strong political systems and political will to support environmental policy [3,6]. Cities must increase their efforts to streamline policy guidance across their often-segregated departments and coordinate regulations across sometimes unresponsive state/regional and national governance. Without this inter and intragovernmental support, the adaptation decision-making process becomes more difficult [5]. Existing governance structures may not be the best suited for the complex decisions climate change requires.

Urban climate adaptation decision-making should involve many stakeholders. While every citizen will be affected by climate change, it is unrealistic to include everyone in this process. Therefore, cities must be careful to select relevant stakeholders to the decision [7]. When defining the scope of the decision, the outcome must be applicable to the relevant stakeholders so finding a balance between inclusion and decision relevance is highly important in the adaptation processes.

Relevant stakeholders often hold conflicting views further complicating the decision-making. The perceptions toward adaptation solutions are often based in existing knowledge and experiences [2,8]. Additionally, issues around the environment and climate change bring conflicting definitions, emotion, and uncertainty into the process. While green infrastructure mimicking natural process, and more specifically blue-green infrastructure concerning stormwater management, emerge as best practices, what constitutes green solutions can differ between groups and the multi-faceted aspects of these adaptation solutions create differing perspectives in best management practice [4,9]. As climate change projections adjust, the relevance of stakeholders may also need to adjust. The inclusion of different people in decision-making is imperative for successful decisions but adds a layer of complexity to the process.

Regardless of the complexity of climate change, urban decision-making and adjusting governance systems, cities continue to make adaptation decisions. Considering urban stormwater, theory indicates that rainfall intensity will increase in urban areas and best management practice is moving toward blue and green solutions [10]. However, these solutions require multi-criteria considerations and involve stakeholders outside of formal stormwater management [11]. Particularly, purely valuing blue-green infrastructure in economic terms might influence its acceptability for a given decision [12]. Considering this, our study aims to demonstrate that a multi-criteria decision analysis (MCDA), based on a combined analytic hierarchy process (AHP)- technique for order preferences by similarity to ideal solutions (TOPSIS) method, can contribute to increased stakeholder involvement and satisfaction with urban stormwater adaptation policy. While these methods have been utilized in stormwater management before, they are usually presented with a level of expertise that prevents non-experts from immediately engaging with the methodology. Additionally, the method allows for the quantification and analysis of differences between stakeholders in urban decision-making allowing for policy-makers to analyze these differences and create more inclusive policy, something that previous studies have not thoroughly explored. After defining the AHP-TOPSIS methodology, we examine New York City (NYC), where the problem of stormwater management under climate change represents a complex topic overarching several municipal departments and involving differing stakeholder opinions. This paper forms Part I of a two-part study on urban pluvial flood management.

2. Multi-Criteria Decision Analysis

MCDA tools are useful for policy-makers to visualize, quantify, and increase transparency in the decision-making process by helping to evaluate the criteria influencing the decision and the possible decision alternatives to implement. Multi-attribute utility theory (MAUT), simple multi-attribute rating technique (SMART), AHP, case-based reasoning (CBR), simple additive weighting (SAW), TOPSIS, Elimination et Choix Traduisant la Realite (ELECTRE), and preference ranking organization method for enrichment of evaluations (PROMETHEE) are some of the MCDA methods that have been successful utilized in environmental decision-making [13,14]. Each method is optimized for different types of data, required complexity of analysis, uncertainty levels, computational intensity, and experience of the decision-maker. Ultimately, there is no single best MCDA method for making decisions [11,15,16]. While it may be possible to evaluate different methods for a specific challenge, the ability to identify an optimal method is hindered when there is low knowledge of MCDA methods among the decision-makers [13].

In climate change decision-making one selects an MCDA method based on preference and perceptions of what best fits the data and can represent the requirements of the decision. Nonetheless, it is still crucial to be critical when selecting an MCDA method as it will affect the output of the criteria, alternatives, and eventual decision [11,13]. It is important to understand the method used, and how easy it is for stakeholder involved in the decision-making process to understand the method. Participants need to understand the alternatives and the criteria to conceptualize the influence their judgments will have on the decision process [17]. While it is the decision analyst’s responsibility to make sure the participants understand the alternatives and criteria, they must also be transparent in the process. A decision aid such as computer programing, forms, and surveys makes the process more transparent. MCDA methods ultimately exist to aid making complex decisions and attention should be given when selecting the appropriate method for the given problem.

Here we propose using a combined AHP-TOPSIS MCDA method for stormwater management adaptation. The AHP developed by Saaty [18] is a linear model based on pairwise comparisons while TOPSIS developed by Hwang and Yoon [19] is a compensatory method measuring the distance to an idealized solution. In this method, the AHP is used to develop the weights of the criteria and sub-criteria in the decision-making process. As the number of criteria needed in urban policy and climate change decisions becomes complex, the criteria weights are then utilized in TOPSIS to test the performance of the alternatives, reducing the number of judgements required by the decision-maker and the computational complexity of the MCDA. The strength of this method is that conceptually and mathematically it is easy for non-experts to engage with while being robust and accepted in the field. While other methods additionally reduce the computational burden on largescale pairwise comparisons, they often become complex for non-expert communities [20,21]. The method we propose is appropriate as it is approachable for various stakeholders involved in urban climate adaptation decisions. Additionally, the method allows the decision-maker to observe how different stakeholder value different criteria and policy alternatives.

The AHP is a robust tool because of its ability to handle stakeholder involvement, integration of qualitative judgements and legacy in the fields of policy, governance, and planning [14]. The AHP can directly engage with public officials familiar with the process over other MCDA methods as it is widely used. In addition, the method can easily be described to non-experts.

The AHP is capable of handling large amounts of qualitative data, important in urban environmental and climate change studies. When a large number of stakeholders with different interests are involved, the AHP allows for deep analysis of these differences [22]. This is critical when considering the integration of expert and non-expert stakeholders in decision-making. For human judgement, the optimal amount of information a person can process, particularly during the AHP is 7 ± 2 judgements [23]. If environmental judgements become too complex, particularly with multiple comparisons, an additional MCDA method can be used in conjunction. TOPSIS can integrate the criteria weights of the AHP to analyze the performance of the alternatives.

TOPSIS is based on finding an ideal alternative and measuring the distance performance of the alternatives from this. It allows for the direct comparison between cost and benefit type criteria, another important factor in environmental decision-making. We acknowledge the emergence of applying the fuzzy sets theory to the AHP and TOPSIS as a way to compensate for inherent uncertainty in climate-related decisions. However, due to the additional computational complexities and continued theoretical debate we proceed with the non-fuzzy methods.

Table 1. Recent literature on the AHP and TOPSIS methodologies within stormwater management.

Study	Year Published	Description and Context
AHP
Young et al. [24]	2010	The use of AHP in identifying stormwater management strategies in an American local municipality
Sahin et al. [25]	2013	The use of AHP in identifying stormwater management strategies across councils in an Australian state
Siems and Sahin [26]	2014	The use of AHP in identifying stormwater management strategies across councils in an Australian state.
Ebrahimian et al. [27]	2015	The use of fuzzy AHP and compromise programing in stormwater collection systems in an Iranian urban context
Alhumaid et al. [28]	2018	The use of AHP and PROMETHEE II in stormwater drainage system management in a Saudi Arabian urban context
Kordana and Slys [29]	2020	The use of AHP to evaluate stormwater management strategies in at a building in a Polish context
Yu et al. [30]	2021	The use of AHP in identifying optimal permeable pavement types for stormwater management.
TOPSIS
Jayasooriya et al. [31]	2018	The use of TOPSIS to identify green infrastructure for stormwater management in industrial sites an Australian urban area
Hager [32]	2019	The use of fuzzy TOPSIS to examine optimal stormwater management strategies in a Canadian context.
Luan et al. [33]	2019	The use of TOPSIS to evaluate green infrastructure for stormwater in a Chinese sponge city
Zeng et al. [34]	2021	The use of TOPSIS to identify green infrastructure solutions for stormwater management in a Chinese smart city
AHP-TOPSIS
Gogate et al. [35]	2017	The use of AHP-TOPSIS to identify stormwater management alternative performances in an Indian urban area
Moghadas et al. [36]	2019	The use of AHP-TOPSIS to evaluate flood risk in an Iranian urban area
Ekmekcioglu et al. [37]	2021	Fuzzy AHP-TOPSIS for flood risk mapping in a Turkish municipalities
Koc et al. [38]	2021	Fuzzy AHP-TOPSIS for stormwater management in a Turkish urban watershed.

displays recent studies from the past decade involving both the AHP and TOPSIS methodologies in a stormwater management context. These two MCDA methodologies are well established with stormwater management and have been applied to both large- and small-scale studies, with different national contexts, management alternatives, and criteria considerations. However, there are two important gaps that exist in the literature. First, it is difficult to approach the mechanics of these methods without prior familiarity with the MCDA methodologies. Additionally, many of the studies involving AHP and TOPSIS include additional computational modelling, particularly storm water management model (SWMM). While acknowledging the role these MCDA methods can have within policy formation, it is difficult for non-technical experts, many of which are still highly important decision-makers around stormwater management, to engage with the methodology. Without proper understanding of the methodology, the results of these methods can remain abstract to decision-makers thus weakening their ability to trust or defend the methods’ outputs when adopting policy for sensitive and uncertain decisions around climate change.

Second, the existing literature is focused on developing tools to find a solution to the stormwater issues the world faces while considering multiple stakeholder involved in the decision-making process. Aside from a few notable examples that consider differences between the stakeholders involved [25,26,37], however, there is little focus on how to manage the differences in opinions and how this affect the end policy decision. If differences emerge between the stakeholders involved in the MCDA it is important to acknowledge these differences so that policy can then attempt to address the disparities. This is particularly important in stormwater management and climate change adaptation as many minority stakeholders might hold crucial viewpoints and local knowledge that are not typically examined by traditional policy-making methods. It is important to consider the end goal of the MCDA but it is equally important to demonstrate how the AHP and TOPSIS methodologies can quantify stakeholder differences to allow for deeper policy discussions to occur around stormwater management.

In the proposed method we take from the existing mathematical theory to optimize the method for the nature of the stormwater adaptation process but paying particular attention to the ease of use for future decision-makers. We do not introduce new theory of calculating the AHP or TOPSIS but take from the extensive existing literature to demonstrate methods that complement each other, justifying these decisions and provide a full and comprehensive decision-making method that is readily available to be used in the field.

Criticisms

Two main technical criticisms are associated with the AHP and TOPSIS; rank reversal and consistency. Rank reversal can occur when the addition of new or duplicate alternatives or the subtraction of an existing alternative can alter the final ranking of the alternatives [39,40]. Rank reversal affects both the AHP and TOPSIS method [39,41,42]. The rank reversal phenomena is largely a result of inconsistent judgements as well as internal aggregation and normalization methods during the analysis [43]. To reduce the opportunity for rank reversal and maintain the internal mechanisms of the two MCDA methods, it is important to establish the independence of each alternative to avoid judgmental overlap or interdependencies. Furthermore, it is established that the AHP performs better when there is a small number of alternatives and a large number of criteria [13,22]. This additionally works for TOPSIS as the alternatives should be as unique and independent from one another so that the decision-maker examines the alternatives objectively and does not account for linked interdependencies in their judgments. Rank reversal in TOPSIS can further be reduced when the ideal solutions are predetermined and not reliant on the input data. The criteria should also be as independent as possible, but because they can be grouped together in sub-criteria categories, more similar criteria can still be judged objectively under a common criterion. Rank reversal is a common occurrence in both AHP and TOPSIS but in establishing a well-defined and independent set of alternatives it minimizes the chance of rank reversal with the introduction of new information.

The second criticism, consistency, influences the AHP method more than TOPSIS. There is a debate over how much inconsistency or flawed judgements stemming from human decisions the AHP can tolerate. However, as Saaty, Vargas and Whitaker [44] argue, issues regarding the AHP arise from the need to validate MCDA methods. The AHP is structured to handle the uncertainty of personal judgements and decisions without clear definitions and the results should be discussed within the context of the AHP method. While these criticisms can influence the proposed method, the method can be designed to minimize the influence of these drawbacks.

3. AHP-TOPSIS MCDA Methodology

An MCDA is generally performed in five stages (Figure 1) where we outline the proposed AHP-TOPSIS method through these stages.

3.1. Defining the Problem

The most important stage of any MCDA is to understand the nature of the problem. Like most decision-making it involves defining the problem, geographic area, pressures on the problem and for whom and what a solution is needed.

3.2. Alternative, Criteria, and Stakeholders

The criteria represent the factors of the decision and the alternatives are the potential solutions to the problem. Frequently, the criteria are derived from experience in decision-making, while the alternatives are developed from the literature [45]. While not a hard rule, it is generally preferable to verify the criteria and alternatives with experienced stakeholders or previous materials to ensure the relevance of the MCDA.

The stakeholder analysis identifies the parties relevant in the formation and implementation of policy and is integral in environmental decision-making as it helps make the decision more robust and it can adapt the decision to local characteristics [46]. Stakeholder analyses are not just useful for environmental management but are frequently used across decision areas affecting cities [46,47,48]. However, a stakeholder analysis may not capture every relevant voice and can obscure minority voices and unexpected stakeholders [46]. Ultimately, who is included and who is excluded are based on the methods used.

A simple stakeholder analysis for decisions that need to be made quickly is the interest-influence matrix to categorize the stakeholders. Participants in the analysis list stakeholders and rate their interest and influence in the decision and if desired, offer comments on the relationships between the groups [49]. The stakeholders are then categorized into four quadrants: high-interest/high-influence, high-interest/low-influence, low-interest/high-influence, and low-interest/low-influence and stakeholders can be selected as representative groups for participation in the MCDA.

No stakeholder method will be perfect in identifying everyone relevant toward the decision. As a top-down approach and depending on the participants, the interest/influence method can often identify usual stakeholders in the decision-making process and is also biased toward the opinions of those making the decision and rating the stakeholders as well as assuming how they judge other stakeholders is relevant [47]. However, other bottom-up methodologies like stakeholder-led categorization and Q as well as additional methods such as social network analysis can be time intensive and are also subject to their own flaws. There is no perfect stakeholder analysis method and ultimately every stakeholder will not be captured by the methods. However, with well-defined scopes and limitations the interest/influence method can be quickly implemented as part of the integral stakeholder for the decision-making process.

3.3. Criteria Weights and Alternative Scores

In this stage we employ the AHP-TOPSIS method. We determine the decision hierarchy of the problem to differentiate between the two MCDA methods (Figure 2), for the full equation set please refer to supplementary material.

3.3.1. AHP

We propose using the AHP to determine the criteria weights. In general, using pairwise comparisons one compares criteria $C_i$ with $C_j$ to form a square matrix of dimensions $C_{n}x{C}_n$, matrix $A=\left[a_{ij}\right]$ (Equation (1)). The matrix is reciprocal along the NW-SE axis. The matrix is then normalized by matrix column to $\widehat{A}$.

$$A=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1n}\\ a_{21}& a_{22}& \cdots & a_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ a_{n1}& a_{n2}& \dots & a_{nn}\end{array}\right]$$

(1)

where:

(i)

$a_{ij}=\frac{1}{aji}$ for $i, j=1,\dots ,n$ and $i\ne j$

(ii)

$a_{ij}=1$ for $i,j=1,\dots , n$ and $i=j$

The pairwise comparisons between criteria are made using a linguistic scale, regardless of their quantitative or qualitative natures. Saaty’s original scale is linear from values 1–9 so that the difference between each successive value is proportional in magnitude (

Table 2. Saaty (1980) linguistic and numeric scale for AHP.

Numeric Value	Description	Reciprocal Value
1	Equal Importance	1
3	Slight importance of one over another	1/3
5	Moderate importance of one over another	1/5
7	Very strong importance of one over another	1/7
9	Extreme importance of one over another	1/9
2, 4, 6, 8	Intermediate value	1/2, 1/4, 1/6, 1/8

). While other scales exist the main difference between scales is how consistent a decision-maker performs in their comparisons [50,51,52]. Saaty’s linear scale is not the best at maintaining consistency, but it is still widely used and favored in practice [50,53]. Additionally, while different scales influence the criteria weights, they rarely influence the final ranking of the alternatives [50]. Therefore, because of its integration in the non-scientific community we incorporate Saaty’s linear scale.

Next, the priority weight vector is derived from matrix $\widehat{A}$ that indicates the weight of each criterion in the matrix. Represented as $\mathit{w}$, there are several methods of calculation. Saaty supports using the eigenvector of the matrix as the priority weight vector [18,54,55]. Alternatively, Crawford and Williams [56] developed the logarithmic least squares or geometric mean method (GM) to determine $\mathit{w}$. Minimizing the logarithmic error of the weightings, Barzilai argues the geometric mean method is unburdened by scale inversion and is better at handling optimization problems and error estimations [57,58,59,60]. To calculate the GM, $\mathit{w}$ is derived from the geometric mean of the matrix rows (Equation (2)). This method describes an explicit connection between matrix inputs and the weights allowing for quick and effective sensitivity analyses [61]. Furthermore, this method remains simple regardless of the size of the matrix allowing for non-experts to quickly understand the process. Nonetheless, despite the ongoing theoretical disagreement over the optimal method to derive the priority weights, there are marginal practical changes on the end results when using both methods [62]. Because of the GM’s performance advantage as well as the direct connections to the input matrices and decision-makers ease of use, we continue with the GM.

There is an additional method of calculating the priority weight vector by using a combined weighting measure taking into account the AHP weights and an objective weight measure [63,64,65]. While these methods have not yet been widely adopted within the theoretical discussions around the AHP, they have been demonstrated to be good at removing the subjective and variable nature of the weights, thus reducing inconsistencies and the potential for rank reversals, they require an extra step of establishing the objective criteria hierarchy. In a case such as stormwater climate adaptation, a large volume of criteria may be difficult to analyze truly objectively, akin to the introduction of valuing ecosystem services within environmental economics. Furthermore, these combined methods work best when there is a high amount of accurate data regarding the alternatives and criteria [66], something that is not always available in climate decisions with various projected outcomes. While we acknowledge the promise these combined methods have in reducing the subjective nature of AHP weighting systems, they add an additional layer of technical complexity for decision-makers while attempting to derive weights objectively with often variable and uncertain indicator data; therefore, proceeding without a combined weight methodology.

$$\mathit{w}=\left(\begin{array}{c}{w}_1\\ w_2\\ \dots \\ w_n\end{array}\right)=\left(\begin{array}{c}\sqrt[n]{{\widehat{a}}_{11}\cdot {\widehat{a}}_{12}\cdot \dots \cdot {\widehat{a}}_{1n}}\\ \sqrt[n]{{\widehat{a}}_{21}\cdot {\widehat{a}}_{22}\cdot \dots \cdot {\widehat{a}}_{2n}}\\ \dots \\ \sqrt[n]{{\widehat{a}}_{n1}\cdot {\widehat{a}}_{n2}\cdot \dots \cdot {\widehat{a}}_{nn}}\end{array}\right)$$

(2)

where:

(i)

$w_i>0$ and $i=1,2,\dots n$

(ii)

${\displaystyle \sum }_{i=1}^n{w}_i=1$

Before the priority weight vector can be accepted, the decision matrix is tested for consistency. A consistency test is important due to the nature of making subjective comparisons as well as the internal structure of the AHP. How one calculates the priority weight vector determines an appropriate consistency test.

Saaty [18] developed a consistency index and ratio (CR) for the eigenvector method still widely referred to today in policies. To allow the GM to be directly comparable with these existing studies, we propose using the geometric consistency index (GCI) interpreted by Aguaron and Moreno-Jimenez [67] from Crawford and Williams [56] (Equation (3)). Here the GCI examines the “average of the squared difference between the log of the errors and the log of unity” [67] (p. 139).

$$GCI=\frac{2}{\left(n-1\right)\left(n-2\right)}{\displaystyle \sum }_{\begin{array}{c}i,j=1\\ i<j\end{array}}^{n}l{n}^2{e}_{ij}$$

(3)

where $e_{ij}=a_{ij}{w}_j/w_i$ is the error obtained when the ratio ${\mathit{w}}_i/{\mathit{w}}_j$ is approximated by $a_{ij}$.

Importantly, the GCI has a near linear relationship with Saaty’s CR for CR < 0.2 and for n ≤ 4. One can take the equivalent GCI value and compare it directly with the CR. This is useful as the CR can represent a percentage and traditionally, if the CR < 0.1 (10%), the comparison matrix is considered to have good consistency [18]. However, a CR > 0.1 does not mean that the matrix is necessarily invalidated, but critical attention should be paid to the decision matrix if it exceeds this amount. Therefore, sometimes a CR within 0.2 is still tolerable, particularly in uncertain areas such as climate change [25,68]. Here, we advocate for a CR < 0.2 or a GCI < 0.7052 (for n = 4) when considering urban climate adaptation studies. If a matrix has a greater GCI, then the analyst must either disregard the matrix or ask the decision-maker to re-evaluate their judgements. The GCI is beneficial as it is related to the GM method, directly comparable with the CR and easy to conduct.

Equations (1)–(3) are repeated for each level of the hierarchy, i.e., a new matrix is constructed for each level of sub-criteria.

3.3.2. Group Aggregation

The proposed method accounts for multiple stakeholders and thus the responses need to be aggregated together. Consistency can be used as a measure of aggregation [58,69]. Here we suggest the aggregation of individual priorities (AIP) where each decision-maker produced their own $\mathit{w}$ and GCI measure. The AIP method treats decision-makers as individuals allowing for their variation to be observed as opposed to the aggregation of individual judgements which treats decision-makers as a cohesive group aggregating them into one matrix [70]. Neither aggregation method is superior but depends on how to observe the decision-makers [71,72]. Additionally, when using AIP should each decision-maker have acceptable consistency, the final aggregated $\mathit{w}$ will be similarly acceptable [73].

As the CR can be represented as a percentage, we propose initially converting the GCI values to the equivalent CR. In practice, one cannot be more than 100% inconsistent so we propose using a consistency measure (CM) as the inverse of the CR to demonstrate consistency (Equation (4)). The CM can then subsequently be normalized by all the decision-makers to find the decision-makers’ individual aggregation weight (aiw) (Equation (5)). In a case where the decision-makers display the same level of consistency, then an equal aiw would apply for all decision-makers.

$$C{M}^k=1-C{R}^k$$

(4)

where:

(i)

$k=1, 2, \dots , r$ for the set of decision-makers

$$ai{w}^k=\frac{C{M}^k}{{{\displaystyle \sum }}_{k=1}^{r}C{M}^k}$$

(5)

where:

(i)

${\displaystyle \sum }_{k=1}^{r}ai{w}^k=1$

The aggregated priority weight vector ($\overline{\mathit{w}}$) for the entire group of decision-makers can be calculated using the weighted arithmetic or geometric mean. Both techniques do not violate the Pareto principle and are therefore viable [74,75]. However, as the nature of the AHP relies on having ratio properties, we proceed with the weighted geometric mean (Equation (6)) [71]. Additionally, this method is less likely to be skewed by outliers in the data set [74,75,76]. Finally, $\overline{\mathit{w}}$ is normalized to $\widehat{\mathit{w}}$.

$$\overline{\mathit{w}}={{\displaystyle \prod }}_{k=1}^r{\left({\mathit{w}}^k\right)}^{\left(ai{w}^k\right)}$$

(6)

Equations (4)–(6) are repeated to determine the group priority weighting of each decision matrix including each level of the hierarchy if sub-criteria are present. To condense the criteria and sub-criteria weights into the global aggregated weights of the criteria interacting with the alternatives in the decision hierarchy, one takes the weight of the aggregated parent criterion and multiplies it by the aggregated priority weight vector of the sub-criteria matrix (Equation (7)). The final global aggregated weights of the criterion that interact with the alternatives is now presented simply as the final priority weight vector $\mathit{w}$.

$${\mathit{w}}_{global sub-criteria matrix}={\widehat{w}}_i{\widehat{\mathit{w}}}_{sub-criteria matrix}$$

(7)

where:

(i)

$w_i$ is the weight of the parent criterion in the decision hierarchy from the aggregated parent priority weight vector.

3.3.3. TOPSIS

After establishing the criteria weights, TOPSIS analyzes the performance of the alternatives. As we established the decision-makers as a group in the AHP, we propose using the group aggregation method for TOPSIS provided by Shih et al. [77]. In group decision-making, certain decision-makers may provide overly strong preferences and aggregation techniques might mask this dominance leaving other decision-makers dissatisfied with the outcome. While some aggregation techniques target this phenomenon specifically [78], Shih et al. [77] demonstrate that their method is useful under different distance measurements and internal aggregation techniques.

Starting in TOPSIS, a decision-maker $k$ rates the alternatives $A_i$ to the criteria $C_j$ in a matrix $B^k=\left[f_{ij}\right]$ of dimensions $A_{m}x{C}_n$ (Equation (8)). The matrix is then normalized to matrix $Z^k=\left[z_{ij}\right]$ as the square root of the sum of the squared matrix input by column (Equation (9)). Unlike in the AHP, in TOPSIS the decision-maker is only constructing one matrix, reducing the required burden of judgements.

$$B^k=\begin{array}{cc}\begin{array}{c}{A}_1\\ A_2\\ ⋮\\ A_m\end{array}& \begin{array}{c}\begin{array}{cccc}{C}_1& C_2& \dots & C_n\end{array}\\ \left[\begin{array}{cccc}{f}_{11}& f_{12}& \dots & f_{1n}\\ f_{21}& f_{22}& \dots & f_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ f_{m1}& f_{m2}& \dots & f_{mn}\end{array}\right]\end{array}\end{array}$$

(8)

where:

(i)

A_i represents the alternative i and C_j represents the criteria j, for i = 1, …, m and j = 1, …, n

(ii)

f_ij represents the performance rating of A_i under C_j

(iii)

For k = 1, 2, …, r for the number of decision-makers

$$Z^k=\left[\begin{array}{cccc}{z}_{11}& z_{12}& \cdots & z_{1n}\\ z_{21}& z_{22}& \cdots & z_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ z_{m1}& z_{m2}& \dots & z_{mn}\end{array}\right]$$

(9)

where:

(i)

$z_{ij}=\frac{f_{ij}}{\sqrt{{{\displaystyle \sum }}_{i=1}^m{f}_{ij}^2}}$

Quantitative values are directly input into the matrix while qualitative judgements are made using a linguistic scale. Like AHP, a 1–9 scale exists for TOPSIS judgements (

Table 3. Linguistic and numeric scale for TOPSIS.

Linguistic Value	Numerical Value
Very Low	1
Low	3
Moderate	5
High	7
Very High	9

). The analyst can establish many varieties of scales but in a stochastic TOPSIS, the 1–9 scale is satisfactory [79].

At this stage, Shih et al. [77] differ from the traditional TOPSIS by delaying the addition of the criteria weights into the matrix until later in the process. The analyst now establishes the positive ideal solution (PIS) and the negative ideal solution (NIS) for each criterion, or matrix column. The PIS ($Z^{k+}$) represents the best performing alternative value in the criterion (Equation (10)) and the NIS ($Z^{k-}$) the worst performing value (Equation (11)). The PIS and NIS are influenced if the criteria is considered a cost (preferred low value) or a benefit (preferred high value).

$$Z^{k+}=\left\{\left(Max z_{ij}^k|j\in J \right),\left(Min z_{ij}^k|j\in J^{\prime }\right)|i=1,2,\dots ,m\right\}=\left\{z_1^{k+},\dots ,z_j^{k+}|j=1,2,\dots ,n\right\}$$

(10)

where:

(i)

$J$ is associated with positive criteria or benefits while $J^{\prime }$ is associated with negative criteria or costs

$$Z^{k-}=\left\{\left(Min z_{ij}^k|j\in J \right),\left(Max z_{ij}^k|j\in J^{\prime }\right)|i=1,2,\dots ,m\right\}=\left\{z_1^{k-},\dots ,z_j^{k-}|j=1,2,\dots ,n\right\}$$

(11)

where:

(i)

$J$ is associated with positive criteria or benefits while $J^{\prime }$ is associated with negative criteria or costs.

Once the PIS and NIS are established for each criterion, considering the aggregated group priority vector $\mathit{w}$ derived during the AHP, the analyst can calculate the separation measure of each alternative to the ideals. The Euclidean distance is one of several distance measures that can be considered [77,80]. However, despite the shortcomings of the measure in capturing the interdependencies between alternatives, the Euclidean distance is the traditional measure that is still applied in TOPSIS and has strong integration into the existing policy framework. The separation distance from the PIS ($S_i^{+}$) and the separation distance from the NIS ($S_i^{-}$) for each alternative is then calculated (Equations (12) and (13)).

$$S_i^{k+}=\sqrt{{\displaystyle \sum }_{j=1}^n{w}_j{\left(z_{ij}^k-z_j^{k+}\right)}^2}$$

(12)

where:

(i)

$w_j$ is the weight of criterion $j$ from the group priority weight vector $\mathit{w}$

$$S_i^{k-}=\sqrt{{\displaystyle \sum }_{j=1}^n{w}_j{\left(z_{ij}^k-z_j^{k-}\right)}^2}$$

(13)

where:

• w_j is the weight of criterion $j$ from the group priority weight vector $\mathit{w}$.

Once the separation distances are calculated for each decision-maker, the group aggregated separation distances are calculated. The aggregation can be performed but with the geometric and arithmetic mean achieving similar results [77]. Similar to the justification in the AHP aggregation, because the separation measures are achieved through ratings based on scales and the arithmetic mean’s ability to be influenced by outliers, we proceed with the geometric mean (Equations (14) and (15)).

$$\overline{S_i^{+}}={\left({\displaystyle \prod }_{k=1}^r{S}_i^{k+}\right)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$r$}\right.}$$

(14)

$$\overline{S_i^{-}}={\left({\displaystyle \prod }_{k=1}^K{S}_i^{k-}\right)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$r$}\right.}$$

(15)

In the final stage, the relative closeness to the ideal solution $\overline{C_i^{*}}$ is calculated as a measure of each alternative’s separation from the ideal positive and negative solutions (Equation (16)). The results are presented on a scale of 0–1 and the final rankings of the alternatives are listed in descending order.

$$\overline{C_i^{*}}=\frac{\overline{S_i^{-}}}{\overline{S_i^{+}}+\overline{S_i^{-}}}$$

(16)

3.4. Sensitivity Analysis

The sensitivity of the results of the AHP-TOPSIS analysis needs to be explored to understand how stable the results are. As uncertainty is prevalent in making subjective judgements, a sensitivity analysis demonstrated how reliable the results are given the criteria weights [81,82,83]. While many methods exist to examine sensitivity, we proceed with Li et al.’s [82] sensitivity test as it is computationally simple and its functioning is simple for non-experts to explore.

In general, a disturbance is placed upon one of the weights, $w_q$ where q = 1, 2, 3, …, n for the set of criteria in such $w_q$ becomes $w_q^{*}={\gamma }_q{w}_q$, where ${\gamma }_q$ is the initial variation ratio of $w_q$ and is >0. The sum of the weights must continue to equate to 1 so the weightings of the other weights are also adjusted by this variation (Equation (17)).

$$\begin{array}{ccccc}{\mathit{w}}^{\prime }& =& (& \begin{array}{c}{w}_1^{\prime }=\frac{w_1}{w_1+w_2+\cdots w_q^{*}+\cdots w_n}=\frac{w_1}{1+\left({\gamma }_q-1\right)w_q}\\ w_2^{\prime }=\frac{w_2}{w_1+w_2+\cdots w_q^{*}+\cdots w_n}=\frac{w_2}{1+\left({\gamma }_q-1\right)w_q}\\ ⋮\\ w_q^{\prime }=\frac{w_q^{*}}{w_1+w_2+\cdots w_q^{*}+\cdots w_n}=\frac{{\gamma }_q{w}_q}{1+\left({\gamma }_q-1\right)w_q}\\ ⋮\\ w_n^{\prime }=\frac{w_n}{w_1+w_2+\cdots w_q^{*}+\cdots w_n}=\frac{w_n}{1+\left({\gamma }_q-1\right)w_q}\end{array}& )\end{array}$$

(17)

where:

(i)

$w_1^{\prime }$, $w_2^{\prime }$, $w_q^{\prime }$, and $w_n^{\prime }$ are the new weights for criteria 1, 2, q, and n after the disturbance of $w_q$

(ii)

${\displaystyle \sum }_{j=1}^n{w}_j^{\prime }=1$ and $j=1,\dots ,n$

${\beta }_q$ is the unitary variation ration of $w_q$ after being altered and we can then represent ${\gamma }_q$ in terms of ${\beta }_q$ (Equations (18) and (19)).

$${\beta }_q=\frac{w_q^{\prime }}{w_q}$$

(18)

$${\gamma }_q=\frac{{\beta }_q-{\beta }_q{w}_q}{1-{\beta }_q{w}_q}$$

(19)

By establishing the parameter ${\beta }_q$, the analyst can test the variation of the weight on the criteria. For example, a ${\beta }_q$ set at 0, 0.5, 0.8, 0.95, 1, 1.05, 1.20, 1.50, 2.00 would effectively be testing the variation of a criterion when the weight is adjusted by ±5%, ±20%, ±50%, and ±100%, while the other criteria are adjusted accordingly. The resulting shifts in the $\overline{C_i^{*}}$ for each alternative can easily be visualized graphically allowing the analyst to observe the ranking changes over the criteria weight changes. By testing each criterion, one can establish the criterion that is the most sensitive to the top ranking alternative to change and the most sensitive to any alternative ranking to change.

4. New York City and MCDA Stormwater Results

4.1. Study Area

Demonstrating the effectiveness of the proposed AHP-TOPSIS method we focus on NYC stormwater management under climate change. NYC is a large coastal city with roughly 8.5 million inhabitants. Acting as a regionally, nationally, and internationally important city, the city is a leader in developing urban policy and can act as a case example for additional metropolitan areas to follow. Experiencing rapid growth in the 19th and early 20th century as well as several large public works projects in the mid-20th century, the city’s urban stormwater infrastructure is ageing and needs to be adapted to the emerging precipitation pressures of climate change. Rainfall intensity is expected to increase within the city [84]. As such NYC has begun to develop policies around stormwater management [85] and has also begun piloting specific cloudburst management neighborhood design [86].

The increasing intensity of rainfall threatens to overwhelm the local stormwater systems causing more frequent and damaging in-land flooding events while threatening the aquatic environment with combined sewage overflow events, a problem NYC has been attempting to fix. Recent rainfall events such as in July 2021, where 39.6 mm of rainfall fell within one hour flash flooding roads and causing disruptions within the underground subway network, have demonstrated the importance of adaptation to a more rainfall-intensive future [87]. However, as with many large cities, complications in policy management arise as stormwater is a complex topic that overarches several city departments including Parks, Water, and Planning with many non-governmental stakeholders also having a voice in the direction of policy management.

4.2. Defining the MCDA

To develop the criteria and alternatives we relied on the previous stormwater work by Axelsson et al. [10]. Using six dynamic and global cities from the developed world, including NYC, five policy alternatives were identified by examining 58 policy documents on stormwater, green infrastructure, and climate adaptation: 1. Grey Infrastructure Overhauls involved the (re)development of traditional concrete based stormwater systems to manage rainfall volumes, 2. Public Green Infrastructure, and 3. Private Green Infrastructure relate to the emerging best management practice of implementing green or green-blue systems for stormwater management but are differentiated by if they are financed by public or private bodies, 4. Government Streamlining which helps to merge stormwater management under one governmental agency while increasing the transparency in policy and infrastructure development and 5. Maintaining Urban Environments which ensures that the existing urban system functions and increases citizen engagement within policy management. From here in the study blue-green infrastructure is represented under the term “green” infrastructure as it connects to more general discourses around natural solutions in urban adaptation. These alternatives reflect industrial and post-industrial cities’ necessity to retrofit aging and complex layers of existing systems and infrastructure.

Using the same 58 policy documents from Axelsson et al. [10], sixteen relevant criteria were identified and organized as equal sub-criteria groupings under four principal criteria: Political, Economic, Environmental, and Social criteria. These criteria include traditional policy management considerations such as public costs and project feasibility but also introduce new and emerging criteria for stormwater management such as the ecosystem support of a project and how it can reduce urban inequalities. Please refer to supplementary material for the full list of criteria and their description as well as the full explanation of the alternatives.

In policy-making, groups who advocate and research for outcomes are influential alongside decision-makers [88,89]. Considering this we identified three groups to perform the stakeholder analysis: a green infrastructure research group (5 participants), an environmental activism group (3 participants), and a community advocate group (1 participant). With green solutions emerging as best management practice and community outreach and equity’s growing importance in planning, these three groups reflect key stakeholders in urban decision-making within water management. Due to COVID-19 restrictions, the analysis was performed via video. Each participant individually listed stakeholders involved in stormwater and rated their interest and influence on a scale of 0–10. The results were then aggregated within their group and a group follow-up discussion reflected and adjusted the placement of stakeholders. Finally, the research group re-performed the analysis to observe any changes post-discussion which results in a tighter clustering of the stakeholders. The three groups also discussed and validated the sixteen criteria’s relevance.

In total, 60 stakeholders were identified: 7 by all three groups and 14 by only two groups, in addition to some form of “citizen”. The 60 stakeholders were classified into six types: the general public, city governance, extra-city governance, advocacy/conservancy/local structuring, research/design/construction, and other.

Here, we are interested in more active stakeholders and continued with the stakeholders from city governance, advocacy/conservancy/local structuring, and research/design/construction. These types additionally have the most overlap between groups. Figure 3 demonstrates the distribution of the initial stakeholder placements and the distribution of the three types after discussion. While most remain high influence, they exert varying levels of interest.

From this, we drew upon three stakeholder groups for participation in the MCDA. City Formal Governance includes many of the departments relevant to stormwater management. Advocacy/Conservancy are vocal contributors to policy with specific focuses on aspects of stormwater. Research/Design provides the theoretical basis for many projects and is a frequent collaborator with both other groups.

4.3. Data Collection

Data were collected using an online survey where participants compared the criteria and ranked the alternatives through Zoho Survey during November–December 2020 [90]. The survey focused on the reality of NYC and not an idealized hypothetical. Initial contacts were made using the identified three stakeholder types from publicly accessible contact information and then additional contact was made through a snowball method. We collected twelve completed responses: six from governance, two from research, and four from advocacy.

As the study does not focus on a specific infrastructure project or specific locale within NYC, no indicator data were provided to the participants, i.e., monetary values regarding costs, expected times to complete projects, established environmental scoring, etc. The participants compared and scored the criteria and alternatives based on their own experience and perceptions of stormwater adaptation and policy management. By focusing on the perceptions around management and not the perceptions around tangible values, we exclude any biases toward selecting what might be perceived to be the optimal choice, for example the cheapest option, the quickest to implement, or the one labeled “most sustainable”.

The data were analyzed using Microsoft Excel to underscore the accessibility of the method. Additionally, Excel provides a visual aspect of the data manipulation. For the AHP, GCI values were included up to the CR equivalent of 0.2. If participants exceeded this threshold, they were offered an opportunity to re-evaluate the inconsistent judgements or the specific matrix was excluded from analysis. Overall, 60 matrices were constructed for AHP: five per stakeholder. Only 8 matrices were excluded.

4.4. Results

The results underscore how the AHP-TOPSIS method is effective at identifying and quantifying differences between stakeholders. This provides a basis to overcoming barriers between urban stakeholders and enact decisions acceptable to more groups. As urban decision-making on stormwater becomes more pressing, this will be beneficial to ensure new policies are effective and well received.

4.4.1. Criteria Weights

The advocacy and governance groups are closely aligned with the main criteria and sub-criteria priority weights when compared to the research group (

Table 4. The resulting criteria weights of the AHP in New York City stormwater management for the full city and by stakeholder grouping with cost criteria are emphasized in bold.

	Main Criteria	Full City	Advocacy	Research	Governance
	Political	0.335	0.401	0.187	0.342
	Economic	0.301	0.280	0.201	0.351
	Environmental	0.182	0.133	0.335	0.170
	Social	0.181	0.187	0.277	0.138
Sub-Criteria (global weights)
Political	Existing Legislative Framework	0.084	0.095	0.060	0.083
	Project Feasibility	0.102	0.165	0.035	0.088
	Jurisdiction	0.097	0.086	0.046	0.118
	Implementation Time	0.052	0.055	0.046	0.052
Economic	Public Costs	0.109	0.100	0.055	0.142
	Private Costs	0.054	0.054	0.028	0.064
	Funding Availability	0.104	0.088	0.088	0.115
	Green Industry Growth	0.035	0.038	0.030	0.030
Environmental	Stormwater Capacity	0.062	0.029	0.101	0.081
	Stormwater Quality	0.057	0.045	0.058	0.050
	Ecosystem Support	0.032	0.036	0.044	0.019
	Energy Usage	0.031	0.023	0.132	0.019
Social	Risk to Human Health and Safety	0.071	0.079	0.065	0.053
	Civic Engagement	0.029	0.033	0.049	0.019
	Reducing Inequalities	0.040	0.051	0.049	0.025
	Synergies with other Adaptations	0.042	0.024	0.113	0.040

). Political and economic interests take priority in their weightings. Inversely, social and environmental interests contribute more to the researcher’s priorities. For governance, traditional methods of politics and economic concerns may explain their priorities. Advocacy, often involved in lobbying for change places a higher magnitude of importance on politics. Political will is a large component of the effectiveness of resilience planning [9]. Researchers may be more insulated from these political pressures with research focusing on specialized aspects of stormwater.

While research can operate in more theoretical spaces, it still forms the basis of most environmental decisions. This disconnect between stakeholders can hamper the ability for effective adaptation planning. By identifying the importance of criteria for the city but also by group, we have demonstrated that the priorities of the city may mask those of influential stakeholders. To ensure that the decision is ultimately relevant for those involved, NYC governance can collaborate closely with researchers to ensure the exchange of knowledge and information. These differences may not ultimately disappear, but they should be understood and not a result of limited exchanges or misunderstanding.

4.4.2. Alternative Scores

Advocacy and research are more aligned with each other in the final performance of the five alternatives (

Table 5. The resulting TOPSIS scoring and alternative ranking in New York City stormwater for the full city and stakeholder groupings. Each color represents one of the five policy alternatives.

Alternative Ranking	Full City	Advocacy	Research	Governance
1	Governmental Streamlining 0.552	Governmental Streamlining 0.604	Governmental Streamlining 0.615	Public Green Infrastructure 0.557
2	Public Green Infrastructure 0.537	Public Green Infrastructure 0.523	Maintaining Urban Environments 0.557	Grey Infrastructure Overhauls 0.518
3	Maintaining Urban Environments 0.502	Maintaining Urban Environments 0.473	Public Green Infrastructure 0.548	Maintaining Urban Environments 0.5082
4	Grey Infrastructure Overhauls 0.477	Private Green Infrastructure 0.462	Private Green Infrastructure 0.483	Governmental Streamlining 0.5079
5	Private Green Infrastructure 0.475	Grey Infrastructure Overhauls 0.457	Grey Infrastructure Overhauls 0.421	Private Green Infrastructure 0.457

). Interestingly, despite there being an equal split between governance and non-governance participants, the ranking of the alternatives for the city overall is more aligned with the non-governance participants while governance holds more direct power in urban decision-making. Despite the frequent references to green infrastructure within stormwater and flood management discourses, only governance ranks it as the top alternative. Interestingly, governance does not place government streamlining as the top priority while the other groups do. While the alternatives are theoretically separate, the ranking of the full city indicates that securing good governance is often a desired first step before construction. Of note, while government streamlining places 4th for governance the variation between 1st and 4th position is smallest for governance indicating similar levels of alternative performance.

Grey infrastructure is another alternative of disagreement. While disliked by advocacy and research, governance provides grey infrastructure with a higher ranking. In practice, green infrastructure is not enough to handle the capacity of rain and stormwater so grey infrastructure is needed to supplement these solutions [91]. Therefore, the group responsible for implementing policy reflects a ranking focused on managing rainfall loads. Nonetheless, there is agreement among all participant groups that Private Green Infrastructure is not the best performing alternative. If governance were to implement policy based on this analysis, the other groups will not only be disappointed in the 1st position but upset at the focus on infrastructure over other softer alternatives.

The AHP-TOPSIS method again demonstrates how the differences in stakeholders can dramatically affect the policy decision. The method integrates competing voices highlighting where differences but also similarities arise. This is good for policy because local knowledge should be adapted into the decision-making framework [92]. This local expertise and environmental stewardship can bring unexpected and successful management strategies within the urban governance. Only by bringing together these policy perceptions can the differences truly be appreciated [93]. An impressive strength of this method is its ability to demonstrate these differences in quantitative terms while still providing the option to display them in visual and compelling ways. These data interpretation fits well within the quantitative-centric policy framework and engages non-experts and the public alike thus increasing transparency of the decision-making process. While there is unlikely to ever be full consensus between stakeholders, simply engaging with the process provides deeper appreciate and acceptance of policy.

By highlighting the differences, cities can also encourage behavioral changes around environmental issues by focusing on groups’ priorities and interests. By presenting environmental problems in relevant terms for a targeted audience, cities can influence urban behavior and directly engage fringe stakeholder groups who may have been excluded for the decision-making process [94]. This aids stakeholders who may lack the knowledge to make the decision but still exert varying levels of stewardship over the outcome. Inversely, minor stakeholders can also move governance toward their own goals by understanding how the existing system works underscoring that the social dimension of the city forms policy just as much as policy affects the social dimension [9]. The AHP-TOPSIS method provides exciting new ways for governance to enact more connected, localized and effective policy particularly within the complex dynamics of stormwater management.

4.4.3. Sensitivity

We tested the sensitivity of the four principal criteria for the entire aggregated city (Figure 4). For all four criteria, the top-ranking alternative, government streamlining, is relatively stable, first changing with the political criterion the weight is reduced around 50%. This demonstrates that the top-ranking alternative is unlikely to change with small adjustments in criteria weights. However, the sensitivity for the 4th and 5th ranked alternative is relatively high, with private green infrastructure and grey infrastructure overhauling swapping rankings with small percentage changes across all four criteria. Adjustments in the criteria weighting could influence a ranked policy strategy. The visualization of the sensitivity allows us to observe the relationship between the alternatives and criteria. Here, grey infrastructure is positively influenced by political bias but negatively by social bias. Policy-makers can reflect on how their decisions would have changed as the needs of the city vary through time allowing for deeper connections between present and future policies.

4.4.4. Limitations

This study does present some limitations. The worked example of examining stormwater management within NYC has not considered quantitative values attributed to the alternatives but rather relies on previous experience and perceptions. As not all stakeholders have extensive direct engagement with the stormwater decision-making process, some perceptions might be misrepresenting the reality of the alternatives. As the study has not focused on a specific site, we have only demonstrated the perceptions toward a preferred management instead of an actual preferred management. However, understanding what stakeholders perceive is useful in designing future site-specific studies to acknowledge that some management techniques should be excluded or emphasized.

The study has also not accounted for the geographic diversity within NYC. Where the stakeholders are located within the city and their previous experiences shape their responses. While this can provide us with generalized trends in NYC urban management it does not distinguish between the more and less flood susceptible neighborhood of the city.

While the study has presented the AHP-TOPSIS methodology as a user-friendly tool for policy decisions, there will always be some stakeholders who either choose not to engage with the methodology or are unable to process the methodology. While the concepts have been presented in a more general way, some of these can still pose a barrier to non-experts with the mathematical and computational prerequisites. It is recommended that an expert is always included in the use of the methodology. However, without oversimplifying the method, the way the study has presented the AHP-TOPSIS methodology has reduced the burden of complexity to engage non-expert and non-technical decision-makers with MCDA tools for decision-making.

5. Conclusions

The AHP-TOPSIS method is an effective aid for urban climate adaptation decision-makers. Building upon various aspects of the existing methodology we have combined a methodology that is effective at demonstrating differences between stakeholders. The results of the methodology, alongside the suggested sensitivity analysis, allow for simple yet engaging results encouraging the integration of the method within the existing policy framework. We have successfully presented an AHP-TOPSIS methodology that connects to the theoretical discussions of MCDA, allowing for decision-makers to understand the mechanics of the method, while still being accessible for a non-expert audience. Particularly for climate change, the method can incorporate uncertainty from many forms. Variations in climate projections, qualitative attributes, and uncertainty in personal judgements are all easily absorbed by this method. The approachability of the method for experts and non-experts alike encourages deeper stakeholder engagement with the decision-making process. A tool can only be effective if the users understand the nature of the tool.

The greatest strength of the method is to account for different stakeholder opinions while being easily adapted to local contexts. The method is responsive to the differences and similarities of different stakeholders and allows policy-makers to examine where these differences emerge, how local knowledge affects the results and to display these results in equal, quantitative terms. While stakeholder differences are well observed, the quantification of these differences connects the methodology with the existing focus on numerical decisions all while avoiding a reliance on monetary terms and incorporating emotional and qualitative judgements.

In our study in NYC, we demonstrated that not every participant prefers blue-green infrastructure for stormwater flood management, despite the alternative’s emergence as a priority in stormwater discourses. There is also disagreement over the importance of criteria between stakeholders in the city. This opens the possibility to further explore why these differences have emerged and how this will affect future stormwater policy within the city. Future work should incorporate tangible values to the alternatives to re-evaluate the ranking of the alternatives for specific policy implementation studies.

This method is contextually and geographically flexible, only requiring a shift in the framework of the decision. While presented for stormwater here, the method is adaptable for other urban adaptation problems cities face. Future work should explore how the method can be applied to these additional climate adaptation issues. The method is just one of the many policy tools available for urban climate adaptations. However, most of the traditional tools such as modelling rely on specialist knowledge exclusionary to many stakeholders engaged with policy. Future work should focus on identifying the tradeoffs between computational complexity and accuracy of the results from additional MCDA methods for climate adaptation and stormwater studies. Furthermore, future work should also consider providing more approachable descriptions of these additional methods so that policy-makers have a wider variety of tools to utilize.