A Group-Decision-Making Framework for Evaluating Urban Flood Resilience: A Case Study in Yangtze River

Abstract

Floods are among the most common and destructive natural disasters confronted by cities and are further aggravated by rapid climate change and increasing urbanization, posing a great challenge to flood risk management. To cope with uncertainty, there is a need to move towards approaches to managing flood risk by taking resilience into consideration. While the evaluation of urban flood resilience has gained much attention in recent decades, studies on quantitative measurement using multiple criteria decision making (MCDM) approaches are rare. In addition, the results determined by different MCDM methods may exhibit considerable variability. It is an intractable task to gather a group consensus from these methods. In this regard, in this paper, we propose a group-decision-making framework for measuring urban resilience to flooding, combining three stages, which are (i) normalizing the data, (ii) weighting the criteria and (iii) aggregating the results. Four objective MCDM methods—i.e., the variation coefficient method, Shannon weighting method, CRITIC and ideal point method—are proposed and treated as reliable methods. A stochastic multi criteria acceptability analysis is adopted to integrate those results into a composite resilience index. The proposed methodology is applied to the resilience evaluation problem of 41 cities in the Yangtze River basin, and the results are compared with those obtained with the four MCDM methods. It is demonstrated that our method considers all possible preferences among the results provided by various MCDM methods and is thus more robust and acceptable.

1. Introduction

Floods are among the most common and destructive natural disasters confronted by cities, causing not only major property damage but also fatalities and injuries [1,2,3]. It has been shown by the United Nations Office for Disaster Risk Reduction (UNISDR) that, during 1995 and 2015, floods were responsible for 47% of weather-related disasters, which affected over 2.3 billion people [4]. Due to rapid climate change and increasing urbanization [5,6,7], both the frequency and magnitude of flooding events have been increasing in the past decades [8,9,10,11]. According to China’s flood control headquarters, China has seen 21 large-scale floods in 2020, corresponding to a 1.6-fold increase on past years, hitting a record since 1998. As of October, floods have caused a direct economic loss of 178.96 billion CNY, while 219 people have been found dead or are missing as a result of floods. Against this background, approaches to flood risk management (FRM) must be considered.

Conventional approaches to FRM are mostly resistance-based and aim at reducing the flood hazard by dredging rivers, constructing water conservancy facilities and controlling behavior with laws and regulations [12,13]. However, these approaches are thought to be inadequate to cope with uncertainty, especially when a high-standard flooding event occurs [14,15]. In response, resilience-based strategies have been frequently considered by both practitioners and researchers [16,17,18].

The term resilience was originally introduced in the field of ecology by Holling [19], who defined it as a measure of an ecosystem’s ability to absorb disturbance and persist in the wake of a perturbation. The past two decades have witnessed an increasing application of resilience in the context of FRM [20,21,22,23,24]. Although the definitions of flood resilience in these works are different, almost all of them refer to two aspects: the capacity to reduce flood impacts on one hand and the capacity to recover from those impacts on the other [25,26].

In order to apply resilience-based FRM strategies, methods for flood resilience evaluation are essential [27]. Over the past few decades, various methods have been developed which can be generally grouped into qualitative and quantitative methods [28]. Qualitative methods are either descriptive interview methods [29,30], desk-based studies [31] or conceptual/theoretical frameworks [32,33]. Quantitative methods mainly include simulation-based approaches which simulate flooding events as a means of analyzing system performance with software [34,35], and indicator-based approaches which structure and evaluate flood resilience on the basis of multiple indicators with mathematical tools [21,26,36,37].

With indicator-based approaches, urban flood resilience evaluation problems are typically defined as ranking a finite set of cities according to their performance in terms of resilience, which is represented by multiple indicators. Multi-criteria decision-making (MCDM) has proved to be an effective tool to solve such problems [38,39]. MCDM enables the quantification of the importances of all criteria and their integration into a composite index so that decision-makers are able to identify the best alternative. As far as we know, research into the application of MCDM to flood resilience evaluation is rare. Sun et al. [9] constructed an index system consisting of five dimension sand evaluated the administrative regions around Chaohu Lake Basin in Anhui, China by means of the analytic network process (ANP). Kotzee and Reyers [17] made use of principal components analysis (PCA) and transformed 24 resilience indicators into four main components, and then integrated them into a composite index in which the weights were obtained from the varimax rotation. Moghadas et al. [37] developed a hybrid MCDM method which was a combination of the analytical hierarchy process (AHP) to weigh the selected indicators and the technique for order preference by similarity to an ideal solution (TOPSIS) to order urban districts in Tehran based on their resilience levels.

Beside the aforementioned methods, there is a large variety of other MCDM tools that are applicable for urban flood resilience evaluation problems, including the entropy weighting method [40], distance-based approach [41], CRITIC method [42], and so on. However, the results determined by different MCDM methods may exhibit considerable variability [43]. Thus, the problem that arises is to determine which method should be adopted. Furthermore, urban flood resilience evaluation is generally performed by a group of experts, and each expert may make their judgement by using one of the MCDM tools independently. This leads to another problem that reaching a rational group consensus based on the different judgements is an intractable task [44].

In this regard, in this paper, we propose a group-decision-making framework, which mainly consists of two phases, for evaluating urban flood resilience. First, each expert selects one MCDM method and conducts the evaluation of flood resilience for all cities in the study. Then, each city is represented by a set of evaluation scores from different experts in this phase. To integrate the scores into one comprehensive score, a stochastic multicriteria acceptability analysis (SMAA-2) method is adopted, which analyzes the weight space to determine possible weights, ranking each city in any position from best to worst and producing holistic acceptability indices as the final evaluations.

Compared with the previous literature on the urban flood resilience evaluation problem, this work makes two contributions.

• The proposed evaluation method considers all possible preferences among the results provided by various MCDM methods. The final result can be viewed as a group consensus of different experts. Thus, the decision bias can be largely reduced, making the results more acceptable and robust.
• The holistic acceptability index, which takes into account all the possible ranks of each city, is used to measure the performance. This scheme is completely new in the literature related to the flood resilience evaluation problem.
• We consider 41 cities located near to the Yangtze River Basin as research objects and carry out a numerical experiment. This is the first time that urban resilience to Yangtze River flooding has been explicitly quantified.

The remainder of the paper is organized as follows. Section 2 describes the research problem and presents the details of the proposed methodology. To illustrate the application of the proposed method, a case study on 41 cities along Yangtze River is conducted in Section 3 with results and discussions. Conclusions can be found in Section 4.

2. Materials and Methods

2.1. Study Area

Originating from the Tanggula Range, the Yangtze River runs through eleven provinces, including Qinghai, Tibet, Sichuan, Yunnan, Chongqing, Hubei, Hunan, Jiangxi, Anhui, Jiangsu and Shanghai. As the longest river and the largest water system in China, it is historically, economically and culturally important to the country. Periodically, the Yangtze River basin experienced large floods, and each of them led to extreme damage to the economy and environment. The recent flood occurring in the summer of 2020 has affected nearly 70 million Chinese citizens across 28 provinces which causes a heavy cost of at least 29 billion dollars.

Most of the flooding events happened in the downstream of the Three Gorges Dam, which is the lower course of Yangtze River. In 2017, the General Office of the State Council of China identified 60 cities where urban flooding had caused serious consequences in the past several years. 41 of them are located in the middle and lower reaches of the Yangtze River (shown in Figure 1), including Wuhan, Zhuzhou, Changsha, Nanjing, Hangzhou and so on. These cities belong to several economic circles, such as Yangtze River Delta, urban clusters around Wuhan in Hubei province, and the Changsha-Zhuzhou-Xiangtan city group in Hunan province, playing an important role in the development of China’s economy. In light of this, these cities are selected as research objects.

The main purpose of this paper is to assess the flood resilience of these cities based on several criteria. Let $x_{ij}$, $i=1,2,\dots ,m$, $j=1,2,\dots ,n$ denote the jth evaluation criterion which reflects city i’s resilience to urban flooding. Traditional evaluation methods always relate to the multicriteria decision making (MCDM) problem, which is commonly solved by determining the weight $w_j$ associated with each criterion j ($j=1,2,\dots ,n$) and calculating the score for city i ($i=1,2,\dots ,m$) using the following equation

$$S_i=\sum _{j=1}^n{w}_j{x}_{ij}.$$

(1)

However, different MCDM methods may generate different sets of weights and, thus, different evaluation results. Motivated by this, a group-decision-making framework is proposed which can achieve a group consensus out of selected MCDM methods by making use of a stochastic multicriteria acceptability analysis.

2.2. Group-Decision-Making Framework

The proposed group-decision-making framework consists of three stages. First, all criteria will be normalized to a common scale so as to mitigate the impact of data magnitude. Then, with normalized data, standard MCDM process will be implemented to derive the optimal weight associated with each criterion. Different MCDM methods will generate different weights which cause different results of flooding resilience assessment. Hence in the last stage, a results aggregation methodology is proposed. The details of the three stage, namely, data normalization, weights determination and results aggregation, are introduced as follows.

2.2.1. Data Normalization

A criterion is called as benefit-type if it has a positive effect on flooding resilience, or cost-type, otherwise. In this paper, two min-max normalization schemes are proposed [45] as

$$y_{ij}=\frac{x_{ij}-\underset{1\le i\le n}{min}\left\{x_{ij}\right\}}{\underset{1\le i\le n}{max}\left\{x_{ij}\right\}-\underset{1\le i\le n}{min}\left\{x_{ij}\right\}},\mathrm{for} \mathrm{benefit-type} \mathrm{criteria};$$

(2)

$$y_{ij}=\frac{\underset{1\le i\le n}{max}\left\{x_{ij}\right\}-x_{ij}}{\underset{1\le i\le n}{max}\left\{x_{ij}\right\}-\underset{1\le i\le n}{min}\left\{x_{ij}\right\}},\mathrm{for} \mathrm{cost-type} \mathrm{criteria}.$$

(3)

After normalization, all criteria have positive effect on urban flooding resilence.

2.2.2. Weights Determination

Numerous weighting methods have been proposed in the existing literature. In this paper, four popular objective methods are selected which determine the weights of criteria by making use of mathematical calculations.

Variation Coefficient Method

Variation coefficient method is based on a generic rule that criteria with greater dispersion are considered as more important, and thus, should be given larger weights [46]. For each criterion j, its dispersion is measured by the coefficient of variance which is expressed as

$$V_j=\frac{{\sigma }_j}{{\overline{y}}_j},j=1,2,\dots ,n,$$

(4)

where, ${\sigma }_j$ and ${\overline{y}}_j$ are the standard deviation and mean of criterion j, respectively.

Then, the weight of criterion j is determined by the following form

$$w_j=\frac{V_j}{{\sum }_{j=1}^n{V}_j},j=1,2,\dots ,n.$$

(5)

Entropy Weighting Method

Shannon entropy has also been used as dispersion measurement [40]. However, different from the coefficient of variance, the larger the entropy of a criterion, the smaller the degree of diversity within a criterion data set is. The weight for each criterion j is determined by following steps:

(i). Calculate the entropy value of criterion j by

$$e_j=-\frac{1}{ln\left(m\right)}\sum _{i=1}^m{y}_{ij}·ln\left(y_{ij}\right),j=1,2,\dots ,n.$$

(6)

(ii). Calculatethe measurement of dispersion for criterion j by

$$d_j=1-e_j,j=1,2,\dots ,n.$$

(7)

(iii). Then the weight for criterion j is expressed as

$$w_j=\frac{d_j}{{\sum }_{j=1}^n{d}_j},j=1,2,\dots ,n.$$

(8)

CRITIC Method

Previous two methods neglect the fact that the correlation between criteria would affect the quantity of information contained in them [47]. Such an issue has been considered by Diakoulaki et al. [48] who proposes the criteria importance through intercriteria correlation (CRITIC) method, which uses correlation analysis to find out the contrasts between criteria. The CRITIC method works as follows.

(i). For each criterion j, $j=1,2,\dots ,n$, calculate its standard deviation ${\sigma }_j$, which is defined as the contrast intensity of the criterion.

(ii). Calculate the Spearman’s rank correlation coefficient $r_{jk}$ between each pair of criteria j and k, $j,k=1,2,\dots ,n$. It is noted that smaller $r_{jk}$ indicates larger discordant of the performance of the alternatives in criteria j and k. In this manner, ${\sum }_{k=1}^n(1-r_{jk})$ quantifies the conflict created by criterion j with respect to the decision situation defined by the remaining criteria.

(iii). The quantity in criterion j, $j=1,2,\dots ,n$, denoted by $C_j$, consists of both the contrast and conflict in the structure of the decision making problem. $C_j$ is determined by the following multiplication form

$$C_j={\sigma }_j\sum _{k=1}^n(1-r_{jk}).$$

(9)

(iv). The weight for criterion j is expressed as

$$w_j=\frac{C_j}{{\sum }_{j=1}^n{C}_j},j=1,2,\dots ,n.$$

(10)

Obviously, this method gives the higher weight to the criterion which is more variable and less correlated with other criteria.

Ideal Point Method

Given a particular set of weights $w=\{w_1,w_2,\dots ,w_n\}$, an ideal point is defined as the maximum evaluation of each criterion across all alternatives under consideration. Let $Y^{*}=\{Y_1^{*},Y_2^{*},\dots ,Y_n^{*}\}$ denote the ideal point, where

$$Y_j^{*}=\underset{i}{max}\left\{w_j{y}_{ij}\right\}.$$

(11)

Let

$$y_j^{*}=\underset{i}{max}\left\{y_{ij}\right\}.$$

(12)

Then the difference between alternative i, $i=1,2,\dots ,m$ and the ideal point can be quantified by Euclidean distance as follows

$$\begin{array}{cc}\hfill D_i& =\sum _{j=1}^n{(w_j{y}_{ij}-\underset{i}{max}\left\{w_j{y}_{ij}\right\})}^2\hfill \\ \hfill & =\sum _{j=1}^n{(w_j{y}_{ij}-w_j{y}_j^{*})}^2\hfill \\ \hfill & =\sum _{j=1}^n\left[{(y_{ij}-y_j^{*})}^2{w}_j^2\right].\hfill \end{array}$$

(13)

Based on the consideration that all alternatives always try to maximize their scores on resilience, [49] proposes the ideal point method which determines the weights by minimizing the difference between all allternatives and ideal point. The method can be formulated into the following optimization problem

$$\begin{array}{cc}\hfill & min\sum _{i=1}^m\sum _{j=1}^n\left[{(y_{ij}-y_j^{*})}^2{w}_j^2\right]\hfill \\ \hfill & s.t.\sum _{j=1}^n{w}_j=1.\hfill \end{array}$$

(14)

To solve Equation (14), a Lagrange function is formed as

$$L=\sum _{i=1}^m\sum _{j=1}^n\left[{(y_{ij}-y_j^{*})}^2{w}_j^2\right]+\lambda (\sum _{j=1}^n{w}_j-1),$$

(15)

where, $\lambda$ is the Lagrange multiplier. Since the objective function in Equation (14) is convex with a non-empty convex set as its contraint, the optimal solution of $w_j$, $j=1,2,\dots ,n$ can be obtained as

$$w_j=\frac{1}{{\sum }_{j=1}^n{\left[{\sum }_{i=1}^m{(y_{ij}-y_j^{*})}^2\right]}^{-1}\times {\sum }_{i=1}^m{(y_{ij}-y_j^{*})}^2}.$$

(16)

2.2.3. Results Aggregation

Let ’V’, ’E’, ’C’ and ’I’ denote variation coefficient method, entropy weighting method, CRITIC method and ideal point method, respectively. Let $s_i^k$, $i=1,2,\dots ,m$, $k\in \{V,E,C,T\}$, denote the evaluation of flood resilience of city i using method k, which is calculated by Equation (1) where $x_{ij}$ should be replaced by its normalized value $y_{ij}$, and the weight $w_j$ is determied by method k.

Obviously, with the aforementioned four MCDM methods, for each city, four evaluation results of flood resilience can be obtained, which may be divergent and conflicting. Thus, in the third stage, SMAA-2 method is adopted to aggregate these results. Stochastic Multicriteria Acceptability Analysis (briefly, SMAA) method which has been originally developed for deterimining the preferences among the results that would make each alternative the most preferred one by searching the weight space [50]. As its extension, SMAA-2 considers all possible ranking positions, and therefore identifies good compromise alternatives [51].

To apply SMAA-2, each method is treated as an expert, and then the problem is represented as a set of m alternatives $S_1,S_2,\dots ,S_m$ to be evaluated in terms of four experts, where $S_i=\{s_i^V,s_i^E,s_i^C,s_i^T\}$. Define a utility function $u(S_i,\lambda )$ which maps different alternatives to real values by using a weight vector $\lambda$. $\lambda =\{{\lambda }^V,{\lambda }^E,{\lambda }^C,{\lambda }^T\}$ quantifies the uncertain perference among the four methods, where the joint density function $f\left(\lambda \right)$ is defined in the feasible weight space $\Lambda =\{\lambda \in R^4|\lambda \ge 0,{\sum }_{k\in \{V,E,C,T\}}{\lambda }_k=1\}$. Due to the lack of perference information, uniform distribution is adopted, that is

$$f\left(\lambda \right)=\frac{1}{vol(\Lambda )},$$

(17)

where $vol(\Lambda )$ represents the $(n-1)$-dimensional volume of the feasible weight simplex.

$$vol(\Lambda )=n^{1/2}/(n-1)!.$$

(18)

Then the rank of each alternative $S_i$, $i=1,2,\dots ,m$ is defined by the following ranking function

$$rank(S_i,\lambda )=1+\sum _{1\le l\le m,l\ne i}\rho (u(S_l,\lambda )>u(S_i,\lambda )),$$

(19)

where $\rho \left(true\right)=1$ and $\rho \left(false\right)=0$. Different weight vectors may results in different ranks of each alternative i. Define the stochastic set of favorable rank weights ${\Lambda }_i^r$ as

$${\Lambda }_i^r=\{\lambda \in \Lambda |rank(S_i,\lambda )=r\}.$$

(20)

The set ${\Lambda }_i^r$ includes all weight vectors that make alternative $S_i$ obtain rank r. Define the rank acceptability index $b_i^r$ as

$$b_i^r={\int }_{{\Lambda }_i^r}f\left(\lambda \right)d\lambda ,$$

(21)

which measures the probability of alternative $S_i$ ranking r.

To evaluate and rank the alternatives, this paper makes use of the holistic acceptability index which is computed as a weighted sum of the rank acceptabilities

$$a_i^h=\sum _{r=1}^m{\beta }^r{b}_i^r,$$

(22)

where ${\beta }^r$ are metaweights and can be viewed as the score for ranking r. Therefore, ${\beta }^r$, $r=1,2,\dots ,m$ are required to be nonnegative and nonincreasing when r increases, that is, ${\beta }^1\ge {\beta }^2\ge \cdots {\beta }^m\ge 0$. Following [45], to the compute precise values of holistic acceptability index, the metaweights are formulated as

$${\beta }^r=\frac{1}{m}\sum _{t=r}^m\frac{1}{t},r=1,2,\dots ,m.$$

(23)

3. Results and Discussions

3.1. Criteria Selection

Resilience is a complex concept and should be evaluated in terms of physical, social, economic and institutional criteria [30]. Similar to [17], the evaluation criteria are selected according to several principles including relating to flood resilience, maintaining diversity and redundancy, managing connectivity, managing slow variables and feedbacks, encouraging learning and broadening participation, as well as data availability. In total, 22 criteria were identified and then refined by removing the ones with high levels of Pearson’s correlation. The final 17 criteria are shown in

Table 1. Criteria and data sources.

Criteria	Description	Type
Age Dependency Ratio (ADR)	% Population aged under 15 and above 65	Cost
CPI (CPI)	Consumer Price Index	Cost
Figure-Ground Diagram (FGD)	Ratio of built & unbuilt space	Cost
Population Density (PD)	Number of people/km2	Cost
Unemploymet rate (UER)	% of unemployment	Cost
Access/evacuation potential (AEP)	Per capita square meters of arterial area	Benefit
Average Income (AI)	Per capita disposable income of Residents	Benefit
Average Savings (AS)	Per capita savings of residents	Benefit
Communication Capacity (CC)	% Population owning a cell phone	Benefit
Civic Involvement (CI)	Number of civic organisations per 1000 population	Benefit
Drainage capacity (DC)	Length of drain-pipe per square kilometer	Benefit
Ecological Buffer (EB)	% Natural vegetaton	Benefit
Econmoic risk-resisting ability (ERA)	Per capita property insurance	Benefit
Green Area (GA)	Per capita square meters of green area	Benefit
Medical care capacity (MCC)	Number of hospital beds per 1000 people	Benefit
Redundancy of emergency services (RES)	Number of stations, police stations & emergency operation centers per 1000 people	Benefit
Soil retention (SR)	% deep permeable soil	Benefit

.

3.2. Urban Flooding Resilience Derived from Standard MCDM Methods

The data for 17 criteria were obtained from China Urban Construction Statistical Yearbook, the Sixth National Population Census of China, as well as the Statistical Bulletin on National Economic and Social Development.

After collecting the data, the urban flooding resilience for the 41 cities was then evaluated following the steps introduced in Section 2. In the first stage, data had to be normalized using Equation (2) and (3) and the final dataset are listed in

Table 2. Data of criteria.

City	PD	ADR	UER	MCC	RES	CI	CC	AI	AS	ERA	CPI	EB	GA	FGD	AEP	SR	DC
Nanjing	0.9322	0.9600	0.7857	0.4397	0.8235	0.7236	0.6452	0.9298	0.8289	0.8014	0.4839	0.5056	0.6657	0.6828	0.4275	0.5000	0.5035
Zhenjiang	0.9188	0.8300	0.7569	0.1935	0.2353	0.4878	0.4000	0.6665	0.6195	0.3633	0.6452	0.4956	1.0000	0.6497	0.5006	0.4844	0.6652
Lianyungang	0.9344	0.2980	0.5311	0.2716	0.3529	0.2520	0.3806	0.0064	0.1342	0.2025	0.6774	0.3359	0.6031	0.7859	0.4406	0.7500	0.4487
Xuzhou	0.7587	0.3140	0.5611	0.4346	0.4706	0.5285	0.3806	0.2372	0.3156	0.2327	0.6129	0.5370	0.7026	0.7750	0.4565	0.4375	0.3669
Hefei	0.6991	0.7067	0.4816	0.3939	0.4706	0.2927	0.4645	0.4437	0.3717	0.4900	0.5161	0.3856	0.4963	0.0000	0.2514	0.8750	0.6381
Bengbu	0.8045	0.2667	0.2949	0.2343	0.3529	0.2114	0.2194	0.1714	0.0855	0.1757	0.8387	0.1810	0.4530	0.5829	0.3372	0.7031	0.3683
Huainan	0.8462	0.5480	0.3214	0.0849	0.3529	0.4797	0.1871	0.2269	0.1711	0.1103	0.9677	0.3678	0.4134	0.8409	0.2065	0.6406	0.2821
Wuhu	1.0000	0.6520	0.4136	0.2530	0.3529	0.3902	0.2968	0.3622	0.3053	0.2141	0.7097	0.2383	0.4890	0.6655	0.5106	0.8438	0.7899
Maanshan	0.6257	0.6467	0.4862	0.0543	0.2941	0.3089	0.3161	0.5493	0.3732	0.2205	0.7419	0.5636	0.6326	0.6306	0.3099	0.7344	0.7421
Xuancheng	0.7941	0.4760	0.0000	0.1851	0.2941	0.2846	0.2839	0.3161	0.2109	0.1589	0.8710	0.2839	0.5516	0.9651	0.6380	0.6250	0.6578
Chizhou	0.9637	0.4507	0.1071	0.1426	0.1765	0.3496	0.2516	0.1679	0.3053	0.1684	0.8065	0.1502	0.8260	0.9821	0.4914	0.9531	0.9507
Anqing	0.8429	0.4160	0.0760	0.0000	0.3529	0.1951	0.2194	0.1756	0.1932	0.0000	0.7742	0.6109	0.5387	0.7062	0.2974	0.6250	0.5769
Tongling	0.8163	0.6180	0.2972	0.3396	0.0000	0.2520	0.2194	0.3082	0.3673	0.1557	1.0000	0.6032	0.8803	0.8284	0.1134	0.7500	0.8294
Huaibei	0.6995	0.4667	0.3364	0.3548	0.1176	0.5203	0.2516	0.1996	0.1858	0.1793	0.9355	0.7002	0.7928	0.6957	0.2478	0.8438	0.3240
Suzhou	0.7069	0.1867	0.0945	0.0424	0.1765	0.0000	0.2581	0.1445	0.0487	0.0994	0.9032	0.3903	0.4880	0.9461	0.5541	1.0000	0.5039
Fuyang	0.8343	0.0000	0.0242	0.1715	0.3529	0.0407	0.1806	0.1429	0.0988	0.1542	0.8710	0.1011	0.5378	0.8383	0.5072	0.3906	0.3674
Chuzhou	0.9206	0.3440	0.0449	0.1426	0.2941	0.2602	0.2903	0.1687	0.1490	0.1788	0.8065	0.3702	0.5866	0.8486	1.0000	0.7500	1.0000
Luan	0.6905	0.2987	0.1290	0.0696	0.3529	0.0081	0.1871	0.1187	0.1106	0.1609	0.6774	0.3365	0.6197	0.9684	0.4521	0.7813	0.3994
Nanchang	0.2389	0.4620	0.7903	0.4160	0.4706	1.0000	0.5419	0.4362	0.5723	0.4105	0.6774	0.4009	0.3407	0.7264	0.1129	0.6406	0.3895
Pingxiang	0.4902	0.3287	0.9159	0.3328	0.1765	0.3740	0.2774	0.3081	0.1431	0.1259	0.6774	0.4630	0.2311	0.8866	0.2171	0.2188	0.0000
Ganzhou	0.5792	0.0440	0.6636	0.1817	1.0000	0.3008	0.2452	0.1944	0.1504	0.0298	0.7097	0.2295	0.3076	0.9359	0.1909	0.6563	0.4901
Jingdezhen	0.8413	0.4087	0.6417	0.3209	0.1765	0.7967	0.0000	0.3334	0.2581	0.1019	0.7742	1.0000	0.8352	0.5923	0.5624	0.3125	0.3693
Wuhan	0.5518	1.0000	0.8260	0.7946	0.9412	0.7967	0.6645	0.6004	0.6327	0.5019	0.9677	0.1336	0.2099	0.8266	0.1775	0.4531	0.7362
Huangshi	0.6793	0.5280	0.5841	0.4771	0.0000	0.3821	0.3161	0.2849	0.2330	0.1304	0.6774	0.2076	0.3444	0.0357	0.3333	0.8594	0.7737
Yichang	0.9091	0.7687	0.7419	0.5501	0.4118	0.4472	0.3935	0.2794	0.3850	0.2665	0.6129	0.3010	0.5967	0.9110	0.3974	0.5625	0.2929
Jingzhou	0.0000	0.6673	0.5818	0.2869	0.4118	0.3089	0.4000	0.2130	0.2035	0.1142	0.7742	0.0272	0.2155	0.8664	0.1159	0.6094	0.2894
Jingmen	0.8601	0.8353	0.7074	0.4109	0.1176	0.3984	0.2645	0.2532	0.3053	0.1406	0.7097	0.0000	0.3425	0.9441	0.3498	0.2031	0.6203
Enshi	0.5368	0.2320	0.4493	0.3650	0.3529	0.0732	0.2774	0.1775	0.0000	0.0754	0.6452	0.0739	0.1648	1.0000	0.0276	0.4531	0.2436
Xiangyang	0.7300	0.6413	0.6532	0.4465	0.2941	0.4309	0.3097	0.3128	0.2625	0.1499	0.6452	0.1023	0.3978	0.8907	0.2673	0.2813	0.3013
Shiyan	0.9439	0.6333	0.2719	0.8217	0.4118	0.1789	0.3290	0.1693	0.3820	0.1117	0.7419	0.0970	0.2735	0.9646	0.0928	0.4688	0.5340
Xiaogan	0.5413	0.6413	0.5000	0.1392	0.2941	0.2195	0.2000	0.2218	0.1652	0.0295	0.6452	0.0993	0.1372	0.8729	0.4231	0.4375	0.6351
Changsha	0.9946	0.7007	1.0000	1.0000	0.5294	0.5691	0.6645	0.7146	0.6873	0.6496	0.0000	0.1603	0.2431	0.2387	0.2046	0.0000	0.2983
Yiyang	0.4175	0.4633	0.8226	0.3633	0.2353	0.2195	0.2194	0.1192	0.1165	0.0909	0.7419	0.4122	0.0866	0.9063	0.1265	0.2344	0.5163
Changde	0.5216	0.5627	0.7212	0.4092	0.7059	0.6098	0.2710	0.1766	0.1593	0.1257	0.6774	0.4287	0.5083	0.9189	0.0571	0.6875	0.1519
Zhuzhou	0.9565	0.5700	0.8882	0.4703	0.4118	0.3659	0.2258	0.5071	0.3097	0.1549	0.7742	0.4654	0.4199	0.5551	0.3208	0.8125	0.4038
Zhangjiajie	0.6563	0.2800	0.5438	0.3548	0.1176	0.5447	0.2516	0.0000	0.1298	0.1304	0.7419	0.1833	0.1022	0.9892	0.3620	0.5469	0.3185
Huaihua	0.0073	0.3693	0.5910	0.4754	0.6471	0.2520	0.2387	0.0489	0.0973	0.0658	0.6774	0.1153	0.0000	0.7674	0.0000	0.8750	0.2712
Xiangtan	0.5546	0.5113	0.9574	0.4788	0.1765	0.1382	0.3355	0.3395	0.3614	0.2073	0.8065	0.3596	0.1133	0.6705	0.2873	0.5156	0.6085
Loudi	0.1935	0.2640	0.8214	0.4211	0.1765	0.1626	0.2452	0.0847	0.1431	0.0719	0.7097	0.1869	0.1326	0.6857	0.0608	0.2031	0.4458
Hengyang	0.1216	0.2500	0.7638	0.4228	0.5882	0.4309	0.2065	0.2509	0.1799	0.0724	0.5484	0.3063	0.1897	0.4391	0.1341	0.5156	0.3496
Hangzhou	0.6723	0.8467	0.6601	0.6995	0.7059	0.7073	1.0000	1.0000	1.0000	1.0000	0.5161	0.3099	0.5810	0.7015	0.1196	0.7656	0.4931

.

With the normalized data, four weights for each criterion were determined by using variation coefficient method, entropy weighting method, CRITIC method and ideal point method, respectively. The results are shown in Figure 2 and illustrate that criteria weights apparently vary across different methods. For example, criterion CPI has the highest importance with the ideal point method, but is considered as the least important by the variation coefficient method.

Such conclusion is further demonstrated by the correlation matrix shown in

Table 3. Correlation matrix among the weight vectors.

	${\mathit{W}}^{\mathit{V}}$	${\mathit{W}}^{\mathit{E}}$	${\mathit{W}}^{\mathit{C}}$	${\mathit{W}}^{\mathit{I}}$
$W^V$	1.0000
$W^E$	0.5229	1.0000
$W^C$	−0.5377	−0.6233	1.0000
$W^I$	−0.8269	−0.8532	0.5296	1.0000

, where $W^k$, $k\in \{V,E,C,T\}$ denotes the vector of weights obtained from method k. Although the correlations between $W^V$ and $W^E$ as well as $W^C$ and $W^I$ are positive, they are less than 0.5296. Moreover, both $W^V$ and $W^E$ are negatively correlated with $W^C$ and $W^I$, which indicates that the evaluation results of urban resilience generated from these methods would be different.

Afterward, we calculated the evaluation scores for all cities by using Equation (1) and ranked the cities according to these scores. The results are summarized and compared in

Table 4. Resilience scores and the rankings of cities obtained from four objective MCDM methods.

City	Ranking and Score
	Variation Coefficient Method	Entropy Weighting Method	CRITIC Method	Ideal Point Method
Nanjing	2(0.6982)	2(0.6777)	1(0.6821)	2(0.6710)
Zhenjiang	4(0.5580)	4(0.5633)	4(0.5925)	6(0.6308)
Lianyungang	22(0.3774)	22(0.3955)	18(0.4679)	19(0.5543)
Xuzhou	11(0.4457)	14(0.4539)	13(0.5001)	21(0.5463)
Hefei	9(0.4589)	8(0.4742)	21(0.4644)	35(0.4629)
Bengbu	31(0.3147)	31(0.3319)	35(0.3925)	29(0.4922)
Huainan	27(0.3483)	27(0.3669)	26(0.4404)	14(0.5691)
Wuhu	12(0.4450)	10(0.4684)	7(0.5191)	9(0.5997)
Maanshan	13(0.4428)	12(0.4632)	15(0.4969)	16(0.5626)
Xuancheng	20(0.3861)	19(0.4091)	20(0.4645)	11(0.5843)
Chizhou	16(0.4102)	16(0.4412)	11(0.5103)	4(0.6365)
Anqing	28(0.3266)	28(0.3548)	29(0.4077)	27(0.5123)
Tongling	14(0.4243)	13(0.4546)	9(0.5116)	5(0.6350)
Huaibei	17(0.4060)	17(0.4301)	17(0.4815)	12(0.5785)
Suzhou	33(0.3109)	29(0.3370)	27(0.4148)	20(0.5514)
Fuyang	38(0.2743)	39(0.2831)	39(0.3542)	32(0.4787)
Chuzhou	15(0.4174)	15(0.4484)	14(0.4993)	8(0.6014)
Luan	30(0.3149)	33(0.3301)	30(0.4048)	26(0.5125)
Nanchang	6(0.4920)	6(0.4976)	8(0.5172)	22(0.5454)
Pingxiang	29(0.3242)	35(0.3222)	32(0.3957)	33(0.4680)
Ganzhou	26(0.3502)	26(0.3674)	24(0.4513)	24(0.5296)
Jingdezhen	10(0.4561)	11(0.4660)	10(0.5115)	17(0.5585)
Wuhan	3(0.6077)	3(0.6223)	3(0.6447)	1(0.6989)
Huangshi	25(0.3621)	21(0.3964)	28(0.4093)	36(0.4480)
Yichang	7(0.4761)	7(0.4836)	5(0.5475)	7(0.6116)
Jingzhou	34(0.3079)	30(0.3326)	36(0.3812)	34(0.4656)
Jingmen	21(0.3806)	23(0.3953)	19(0.4662)	15(0.5664)
Enshi	41(0.2440)	41(0.2549)	40(0.3393)	37(0.4469)
Xiangyang	24(0.3728)	25(0.3809)	25(0.4450)	25(0.5247)
Shiyan	23(0.3754)	24(0.3948)	22(0.4620)	13(0.5723)
Xiaogan	36(0.3065)	34(0.3290)	33(0.3929)	30(0.4855)
Changsha	5(0.5434)	5(0.5148)	12(0.5047)	38(0.4346)
Yiyang	37(0.3038)	37(0.3194)	34(0.3926)	31(0.4820)
Changde	18(0.4026)	18(0.4191)	16(0.4899)	18(0.5563)
Zhuzhou	8(0.4607)	9(0.4735)	6(0.5361)	10(0.5966)
Zhangjiajie	35(0.3072)	36(0.3201)	31(0.4015)	28(0.5115)
Huaihua	39(0.2726)	38(0.2982)	38(0.3580)	39(0.4271)
Xiangtan	19(0.3922)	20(0.4053)	23(0.4617)	23(0.5333)
Loudi	40(0.2500)	40(0.2656)	41(0.3230)	40(0.3947)
Hengyang	32(0.3142)	32(0.3308)	37(0.3633)	41(0.3806)
Hangzhou	1(0.7151)	1(0.6938)	2(0.6768)	3(0.6677)

, where each cell contains both the ranking and score of one city using one particular method. For instance, in the second row and second column, $2\left(0.6982\right)$ means that Nanjing scores 0.6982 and ranks the second among the 41 cities when using variation coefficient method. According to Table 4, we have the following findings.

• Obviously, no city has the same ranking position among the four results.
• The four methods have a general consensus on the rankings of the cities which are evaluated relative high or low. For example, Nanjing, Hangzhou and Wuhan are always ranked in top three no matter what method is used.
• The rankings of cities which are evaluated in the middle are diverse. The largest difference occurs in Hefei, where the best ranking is 9 obtained by variation coefficient method and the worst ranking is 35 obtained by ideal point method.

3.3. Urban Flooding Resilience Derived from Proposed Method

To get a comprehensive score, the SMAA-2 method was employed to aggregate the evaluation results shown in Table 4. Using Equation (17) to (21), the rank acceptability index $b_i^r$ for each city i ($i=1,2,\dots ,41$) with each rank r ($r=1,2,\dots ,41$) was calculated. Due to limited space, only the index with $r\le 5$ and $r\ge 37$ is reported in

Table 5. Rank acceptability index with $r\le 5$ and $r\ge 37$.

City	Rank Acceptability Index
	${\mathit{b}}_{\mathit{i}}^{\mathbf{1}}$	${\mathit{b}}_{\mathit{i}}^{\mathbf{2}}$	${\mathit{b}}_{\mathit{i}}^{\mathbf{3}}$	${\mathit{b}}_{\mathit{i}}^{\mathbf{4}}$	${\mathit{b}}_{\mathit{i}}^{\mathbf{5}}$	${\mathit{b}}_{\mathit{i}}^{\mathbf{37}}$	${\mathit{b}}_{\mathit{i}}^{\mathbf{38}}$	${\mathit{b}}_{\mathit{i}}^{\mathbf{39}}$	${\mathit{b}}_{\mathit{i}}^{\mathbf{40}}$	${\mathit{b}}_{\mathit{i}}^{\mathbf{41}}$
Nanjing	0.1176	0.8646	0.0178	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Zhenjiang	0.0000	0.0000	0.0000	1.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Lianyungang	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Xuzhou	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Hefei	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Bengbu	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Huainan	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Wuhu	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Maanshan	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Xuancheng	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Chizhou	0.0000	0.0000	0.0000	0.0000	0.0084	0.0000	0.0000	0.0000	0.0000	0.0000
Anqing	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Tongling	0.0000	0.0000	0.0000	0.0000	0.0785	0.0000	0.0000	0.0000	0.0000	0.0000
Huaibei	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Suzhou	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Fuyang	0.0000	0.0000	0.0000	0.0000	0.0000	0.5040	0.2601	0.2030	0.0000	0.0000
Chuzhou	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Luan	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Nanchang	0.0000	0.0000	0.0000	0.0000	0.0011	0.0000	0.0000	0.0000	0.0000	0.0000
Pingxiang	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Ganzhou	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Jingdezhen	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Wuhan	0.0542	0.0072	0.9386	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Huangshi	0.0000	0.0000	0.0000	0.0000	0.0000	0.0297	0.0000	0.0000	0.0000	0.0000
Yichang	0.0000	0.0000	0.0000	0.0000	0.7305	0.0000	0.0000	0.0000	0.0000	0.0000
Jingzhou	0.0000	0.0000	0.0000	0.0000	0.0000	0.0018	0.0000	0.0000	0.0000	0.0000
Jingmen	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Enshi	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.1023	0.1379	0.6810	0.0788
Xiangyang	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Shiyan	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Xiaogan	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Changsha	0.0000	0.0000	0.0000	0.0000	0.1815	0.0014	0.0000	0.0000	0.0000	0.0000
Yiyang	0.0000	0.0000	0.0000	0.0000	0.0000	0.0464	0.0000	0.0000	0.0000	0.0000
Changde	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Zhuzhou	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Zhangjiajie	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Huaihua	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.4733	0.5267	0.0000	0.0000
Xiangtan	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
Loudi	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0916	0.9084
Hengyang	0.0000	0.0000	0.0000	0.0000	0.0000	0.4167	0.1643	0.1324	0.2274	0.0128
Hangzhou	0.8282	0.1282	0.0436	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000

.

Finally, using Equations (22) and (23), we calculated the holistic acceptability index $a_i^h$ for each city i and used the index as the resilience score. The rankings based on the holistic acceptability index are illustrated in Figure 3 against the rankings based on the four MCDM methods.

Comparison between the Proposed Method and the Standard MCDM Methods

Compared with the four standard MCDM methods, our proposed method advances in the following aspects.

First, as is discussed in Section 2.2.3, the rank acceptability index $b_i^r$ in fact represents the probability that city i ranking in position r. Thus, the proposed method does not only rank the cities based on a comprehensive resilience evaluation, but also generates the distribution of each city’s ranking. For example, as is shown Table 5, the first ranking support for Nanjing, Wuhan and Hangzhou are $11.76\%$, $5.42\%$ and $82.82\%$, respectively.

Second, Figure 3 shows that the group-decision-making method balances the results obtained from the four MCDM methods and achieve a group consensus. For example, although Yiyang is ranked quite low with ideal point method, it is ranked relatively high by other three MCDM methods. Then the ranking from our methodology is also high which is accordance with the majority of MCDM methods. When the results are diverse, such as Hefei, our methodology will choose a moderate value, like an expectation of the four results, that is neither too high nor too low. This suggests that decision bias has been largely reduced, making the results more acceptable and robust.

Third, to further prove the robustness of our method, another experiment was conducted on five datasets, each of which contains 10 cities randomly sampled from the original data without replacement. The sampled data are shown in

Table 6. Sampled data.

Data 1	Data 2	Data 3	Data 4	Data 5
Suzhou	Anqing	Hefei	Maanshan	Wuhu
Hengyang	Xuzhou	Hengyang	Yichang	Xiaogan
Hangzhou	Loudi	Huaihua	Zhangjiajie	Zhenjiang
Chuzhou	Hangzhou	Xuancheng	Zhuzhou	Maanshan
Huaibei	Chuzhou	Xiangyang	Nanchang	Xuzhou
Shiyan	Tongling	Suzhou	Huainan	Anqing
Huangshi	Bengbu	Tongling	Luan	Lianyungang
Shiyan	Yiyang	Nanjing	Tongling	Huainan
Enshi	Enshi	Xuzhou	Huaibei	Wuhan
Jingmen	Changde	Shiyan	Xiangtan	Zhangjiajie

.

For each dataset, flooding resilience of cities was evaluated and ranked using all the methods mentioned before. If a method is robust, then the relative ranking of each city in the sampled data should be concordant with the result shown in Figure 3. Therefore, this paper adopts concordance index to measure the robustness of each method. The index was introduced by Harrel et al. [52] who defined it as the proportion of concordant pairs divided by the total number of possible evaluation pairs. The higher the index, the more robust the method is. For example, using variation coefficient method and the whole data, rankings of the cities in Data 1 are {4, 17, 33, 15, 25, 21, 41, 23, 32, 1} according to Table 4. However, when applying variation coefficient method solely on Data 1, the rankings become {2, 3, 8, 4, 7, 6, 10, 5, 9, 1}. Apparently, {Hengyang, Chuzhou}, {Hangzhou, Huaibei} and {Shiyan, Zhengjiang} are three pairs where the relative positions changed. Since the number of evaluation pairs is equal to the combination of n things taken k at a time without repetition, i.e., $C_{10}^2$, then concordance index for variation coefficient method related to Data 1 is $(C_{10}^2-3)/C_{10}^2=0.9333$. Then, for each method, five concordance index were derived from five sampled datasets, and their average is considered as the robustness measurement. The average concordance index for variation coefficient method, entropy weighting method, CRITIC method, Ideal Point method and our proposed method are 0.8978, 0.9111, 0.8889, 0.8533, and 0.9778, respectively. In this manner, it is concluded that our proposed group-decision-framework is more robust in assessing urban flooding resilience

4. Conclusions

Urban flood resilience is a key index for describing the capacity of cities to cope with flooding impacts and recover from the impacts. The assessment of flood resilience is viewed essential to assist decision making in flood risk management, and can be treated as a multi-criteria decision making problem. In view of the fact that different MCDM tools would generate different evaluation results, this study proposes a group-decision-making framework which consists of three stages, namely, data normalization, weight determination and results aggregation. For the MCDM objective, methods are imbedded in the proposed framework. SMAA-2 is applied to achieve a holistic evaluation results by considering all possible preferences among different MCDM methods. A case study on the Yangtze River is presented to demonstrate the effectiveness and robustness of our methodology.

This study uses four MCDM methods, but the framework is able to incorporate more methods that can be either objective, subjective or even integrated. We expect researchers to apply the proposed methodology to other natural disaster management problem. In addition, the criteria value could be uncertain in practice, and such uncertainty of criteria could be considered by future research. Since only 41 alternatives with 17 criteria are evaluated, how to reduce the computational burden when the dimension of alternatives and criteria are both large is another valuable research direction.