Next Article in Journal
The Digital Divide? Analyzing Regional Differences of Tablet PC Use in Korean Middle Schools for Sustainable Development
Next Article in Special Issue
Impact of Land Use/Land Cover Change on Hydrological Components in Chongwe River Catchment
Previous Article in Journal
Crop Mapping Based on Historical Samples and New Training Samples Generation in Heilongjiang Province, China
Previous Article in Special Issue
Optimizing the Grouting Design for Groundwater Inrush Control in Completely Weathered Granite Tunnel: An Experimental and Field Investigation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Social Acceptability of Flood Management Strategies under Climate Change Using Contingent Valuation Method (CVM)

by
Fatemeh Fadia Maghsood
1,2,
Hamidreza Moradi
1,*,
Ronny Berndtsson
2,
Mostafa Panahi
3,
Alireza Daneshi
4,
Hossein Hashemi
2 and
Ali Reza Massah Bavani
5
1
Department of Watershed Management Engineering, College of Natural Resources, Tarbiat Modares University, Noor 46414-356, Iran
2
Centre for Middle Eastern Studies & Department of Water Resources Engineering, Lund University, SE-221 00 Lund, Sweden
3
Department of Environmental Economics, Science and Research Branch, Islamic Azad University, Tehran 1477893855, Iran
4
Department of Watershed Management Sciences and Engineering, Gorgan University of Agricultural Sciences and Natural Resources, Gorgan 4918943464, Iran
5
Department of Irrigation and Drainage Engineering, Aburaihan Campus, University of Tehran, Tehran 3391653755, Iran
*
Author to whom correspondence should be addressed.
Sustainability 2019, 11(18), 5053; https://doi.org/10.3390/su11185053
Submission received: 22 August 2019 / Revised: 6 September 2019 / Accepted: 10 September 2019 / Published: 16 September 2019
(This article belongs to the Special Issue Sustainable Water Resources Management for Disaster Risk Reduction)

Abstract

:
Floods are natural hazards with serious impact on many aspects of human life. The Intergovernmental Panel on Climate Change (IPCC) reported that climate change already has significant impact on magnitude and frequency of flood events worldwide. Thus, it is suggested to adopt strategies to manage damage impacts of climate change. For this, involving the local community in the decision-making process, as well as experts and decision-makers, is essential. We focused on assessing the social acceptability of flood management strategies under climate change through a socio-hydrological approach using the Contingent Valuation Method (CVM). For this purpose as well, hydro-climate modelling and the Analytical Network Process (ANP) were used. Among twelve investigated flood management strategies, “river restoration”, “agricultural management and planning”, and “watershed management” were the publicly most accepted strategies. Assessment of the social acceptability of these three strategies was carried out by use of the CVM and Willingness to Pay (WTP) methodology. Generally, 50%, 38%, and 18% were willing to pay and 44%, 48%, and 52% were willing to contribute flood management strategy in zones 1, 2, and 3, respectively. Overall, peoples’ WTP for flood management strategies decreased with increasing distance from the river. Among different investigated dependent variables, household income had the highest influence on WTP.

1. Introduction

Floods are natural hazards with detrimental effects on the economy, social facets, and environment [1]. According to the Intergovernmental Panel on Climate Change (IPCC), climate change has had significant influence on magnitude and frequency of flood events in regions all over the world [2]. Thus, IPCC has suggested adaption to actual or expected climatic impacts on human life [3]. Adaptation refers to implementation of effective protection measures to moderate potentially destructive impacts [4,5]. Consequently, adaptive strategies are considered a key component in management strategies to present and future impact of climate change [6].
For effective adaptive strategies against climate change, it is important to involve local participation and community opinion in the decision making process as well as experts and decision-makers [7]. However, realizing an integrated approach in practical flood management remains a challenging issue. This study investigates the social acceptability and feasibility of adaptive flood management strategies in view of climate change for an experimental catchment. Social acceptability can be defined as determination of a group of individuals support dealing with a set of regulations and management tools mostly based on the geographic, social, economic, and cultural factors. Generally, assessing the social acceptability is a complex matter due to the lack of clear and straightforward definition of its concept that it relies on [8].
Generally, Multi-Criteria Decision-Making (MCDM) approaches provide effective tools for exploring different alternatives for decision-makers. In this regard, flood management professionals use this approach to help managers and policy makers to make proper decisions [9]. In this study, the Analytical Network Process (ANP) was employed to prioritize among a set of flood management strategies. The highest ranked flood management strategy as evaluated by experts through ANP was used to assess social acceptability by the local community through the Contingent Valuation Method (CVM). The CVM approach is widely used to monetize non-market goods and issues through simulating actual market conditions [10]. Flood management strategies often affect local citizens and a way to quantify social acceptability is to assess the WTP for implementation of flood risk mitigation measures [11].
CVM approaches have to some extent been used in previous flood management studies. For instance, the CVM approach used for evaluating household WTP for a flood control project in Brazil [12]. They investigated that on some features that have significant influence on the outcome of CVM results. The study showed that this approach is not appropriate for flood control investments in low-income areas of developing countries [12]. The CVM approach for flood risk improvement in Żuławy of the Vistula delta showed that the residents agree to pay for protection and restoration of polder areas and possible resettlement [13]. Markantonis and Bithas (2010) [14] used CVM to estimate the Greek national mitigation and adaptation costs in view of climate change. The study indicated that the largest part of the gross domestic product (GDP) for flood mitigation measures depends on efforts to reduce greenhouse gas emissions. [14]. Moreover, Devkota (2014) [5] found that people’s perception and preferences for particular adaptation strategies changed according to the climate change scenarios in the West Rapti River basin of Nepal. In their study, socio-economic factors like age, gender, education, income level, and flood damage cost showed noticeable impact on the level of WTP for flood mitigation.
In view of the above, the objective of this study is to explore the connection between local community, researchers, and decision-makers using ANP as a decision-making approach and CVM as economic valuation approach to quantify the flood mitigation efficiency and to assess social acceptability of flood mitigation strategies in the study area.

2. Materials and Methods

2.1. Study Area

The Talar River Basin is a mountainous area in the Mazandaran Province in northern Iran close to the Caspian Sea. It covers an area of 1727 km2 with a river length approximately 150 km flowing through the neighborhood of Ghaemshahr City with about 321,000 dwellers and discharge to the Caspian Sea at 21 m amsl [15].
The climate of the Talar Basin is semi-humid and cold with an average annual precipitation and temperature of about 610 mm and 11 °C, respectively. The altitude varies between 216 and 3967 m amsl (Figure 1).

2.2. Multi-Criteria Decision-Making

The conceptual framework used in this study is illustrated in Figure 2. The objective of the study was to investigate the relationship between local community, researchers, and decision-makers. For this purpose, ANP as a decision-making approach and CVM as economic evaluation were used to quantify the flood mitigation efficiency and to assess social acceptability of flood mitigation strategies in the study area.
Multi-Criteria Decision-Making (MCDM) is a widely known technique for decision-making by formally structuring multi-aspect problems [9,16]. Using MCDM will not lead forward to a final and optimum decision, rather to provide a set of available alternatives to better inform decision-making. In this regard, MCDM approaches try to integrate interests and possible choices of decision-makers and present a priority of available choices through weighting calculations for each criteria [16]. Among various MCDM methods, ANP is a mathematical-based method developed based on an analytical hierarchy process, which is explained below [17,18].

2.3. Analytical Network Process Questionnaire

In order to follow the ANP methodology, pairwise comparisons between suggested flood management strategies by selected experts (decision-makers) were considered. A pairwise comparison of alternatives was scaled based on 1–9, where 1, 3, 5, 7, and 9 denote equal, moderate, strong, very strong, and extremely dominant, respectively. Note that the ANP method was processed in Super Decisions software. The detailed information of ANP procedures in Super Decisions software is documented in [18,19]. The questionnaires were sent to twenty selected experts by mail. These experts were selected among managers and researchers with at least 5 years of experience in the field of flood management in Iran. In the ANP questionnaire, in total, twelve alternatives for flood management strategies were ranked through the ANP method. These strategies are categorized as structural, non-structural, and a combination of structural and non-structural measures to manage the flood frequency and magnitude impacts in view of climate change. The pairwise comparisons of proposed flood management strategies are described in Table 1.
It should be mentioned that the questionnaires were accompanied by a cover letter giving detailed information on climate change impacts on flood frequency and source areas based on future climate scenarios and results from hydro-climate modelling in the study area conducted by [2]. The following section briefly describes the climate change impact on flood behavior in the study area. Further, physical information for a better understanding of the basin condition, such as physiography, soil characteristics, and land use, was presented in the cover letter.

2.4. Hydro-Climatic Modeling

Assessment of the impact of climate change on flood frequency and flood source area at basin scale considering Coupled Model Intercomparison Project phase 5 General Circulation Models (CMIP5 GCMs) under two Representative Concentration Pathways (RCP) (2.6 and 8.5) for the experimental basin was carried out by Maghsood et al. (2019). In that study, the Soil and Water Assessment Tool (SWAT) was calibrated and validated. Then, the calibrated SWAT model was run under the two RCP scenarios using a combination of twenty GCMs from CMIP5 for the near future (2020–40). Two indices including Flood Frequency Index (FFI) and Sub-basin Flood Source Area Index (SFSAI) were employed to investigate the impact of climate change on flood frequency pattern and to quantify the contribution of each sub-basin on the total discharge [2]. The output of that study was used to help managers and policy makers to better define mitigation and adaptation strategies through the ANP approach.

2.5. Contingent Valuation Method

The Contingent Valuation Method (CVM) is a widespread approach used by economists and policy makers [20] to assess individual’s WTP for valuing environmental non-market goods and services. This is done in order to find the value that stakeholders place on an economic good or environmental service and quantify costs and benefits that the society obtains [21]. The assumption behind the WTP estimation is that individuals tend to accept or refuse a proposed payment for using the good or environmental service to improve their benefit:
υ ( 1 ,   Y A ; s ) + ε 1   υ ( 0 ,   Y ; s ) + ε 0  
where υ is the individual’s indirect benefit, which is expected to equal the value of benefit υ; Y is income, A is an offered bid (payment), and s consists of different socio-economic features influencing individual choice such as age, education level, etc. The identical, independently distributed random variables with zero means are denoted ε0 and ε1. The benefit difference can be expressed as:
Δ υ = υ ( 1 ,   Y A ; s ) υ ( 0 ,   Y ; s ) + ( ε 1 + ε 0   )
In the case that Δυ is larger than zero, the individuals amplify their benefit by answering “Yes” and accepting to pay the suggested payment. The probit and logit models are usually employed as a qualitative choice method and the logit has advantage of simple computation. The probability, Pi, that an individual will accept an offered bid, A, can be described as the following logit model:
P i = F η ( Δ υ ) = 1 1 + exp ( Δ υ )   =   1 1 + exp { ( α     β A   +   γ Y   +   θ S ) } ,
here Fη() is the Cumulative Distribution Function (CDF) for a standard logistic variate and considered socio-economic feature. The β, γ, and, θ are coefficients that are estimated using collected data [22,23].
Generally, three different methods are used in WTP calculations. These methods are mean WTP, overall mean WTP, and truncated mean WTP that calculate the expected value of WTP through numerical integration ranging from 0 to ∞, −∞ to +∞, and 0 to maximum bid, respectively. The truncated method was used in this study due to its straightforward interpretation and statistical efficiency [24].
The logit model was calculated using the maximum likelihood (ML) estimation technique and the expected value of WTP can be calculated by numerical integration, ranging from 0 to maximum offered bid, A, as:
E ( W T P ) = 0 M a x   .     A F π ( Δ υ ) d A = 0 M a x   .     A ( α * + β A ) d A ,
where E(WTP) is the expected value of WTP and α * is the adjusted intercept that is added by the socioeconomic factor to the original intercept term a. The area under the function can be inferred to estimate the truncated mean of WTP [22,25]

2.6. Contingent Valuation Method Questionnaire

The CVM questionnaire to assess the WTP was designed to state the preferences and local community’s WTP for selected flood management adaptive strategies under climate change scenarios. The questionnaire was based on a dichotomous-choice contingent valuation method (DC-CVM) format as it is more cognitively controllable and mimics a real marketing scenario [21,26]. Two well-trained and experienced interviewers were employed specifically for conducting face-to-face interviews with members of the local community for both pre-tests and final data collection. Since the accuracy of the CVM estimation strongly depends on the presented information during the interviewing [27], the respondents were provided illustrative pictures regarding flood and climate change impacts during the interviews.
The CVM survey was performed during two weeks in July 2018 in three different zones according to distance to the main river (Figure 3). The zones considered were 500, 1000, and 1500 m distance to the river (Zone 1, 2, and 3). In total, 100 interviewees including men and women settlers of the basin were randomly chosen in each zone in different villages.
In general, the questionnaire was organized in three sections. The first section of the questionnaire included attitude and perception questions on topics about climate change impact and flood risk and management. The attributed questions are shown in Table 2.
In the second section, respondents were asked about their WTP for flood management adaptive measures (Table 2). In this section, two ways of questioning were applied; firstly, direct determination of WTP thorough paying by cash (C-WTP) in three determined bids 40,000 (~US $1), 80,000 (~US $2), and 160,000 (~US $4) Rials per month in a year. It should be mentioned that historical exchange rates for Iranian Rials (IRR) were obtained from XE online (xe.com). 1 US dollar equaled 0.000028 IRR on 1 January 2018 and 0.000024 IRR on 30 December 2018. The midpoint of this range was taken for the conversion here. Purchasing power parities from the World Bank are only available until 2017 and are expressed in international dollars.
Secondly, indirect determination of WTP as a non-cash through contributing labor in flood projects (Nc-WTP). For the direct way, people were asked according to the below:
Question 1—Would you be willing to pay money to financially assist the government for the implementation of mentioned flood management adaptive measures? (Yes/No)
If a negative answer according to the above, respondents were asked to give a reason [28] in four follow up questions: (1) This is government responsibility; (2) Implementation of these measures is not necessary; (3) We have economic problems; and 4) Other reason. If the answer was positive, this question followed:
Question 2—Would you like to pay 80,000 Rials (~US $1) for flood management adaptive measures? (Yes/No)
If the answer was “yes” respondents were referred to the 3rd question, otherwise they were referred to the 4th as follows:
Question 3—Would you like to pay 160,000 Rials (~US $2) for flood management adaptive measures? (Yes/No)
Question 4—Would you like to pay 40,000 Rials (~US $4) for flood management adaptive measures? (Yes/No)
Finally, they were asked about the maximum amount they were willing to pay (Question 5).
For the indirect way, respondents were asked whether they would like to participate as labor in the flood management projects. If they answered “yes” they were asked about the number of days per month that they were willing to work. The Nc-WTP was calculated according to:
N c W T P i = D a v e , i × I n c o m e a v e , i × N i
where Nc-WTPi stands for the non-cash WTP, Dave is the average of accepted days in zone i (sum of accepted days/total respondents), Incomeave is the average daily income for a labor in zone i, and N is the total number of people living in zone i.
In addition, respondents were asked to rank, by percentage of importance, the flood management strategies. The third section of the CVM questionnaire concerned respondents’ socioeconomic conditions such as age, gender, education level, occupation, and income level according to Table 3 [5].

3. Results and Discussion

3.1. Analytical Network Process Results

This section presents the socioeconomic key characteristics of selected experts and decision-makers (Table 4) and final weight and ranking of the different criteria and sub-criteria (Table 5). Table 4 shows the key statistics of experts and decision-makers involved in this study. According to the table, 75% of the interviewed experts are male and a majority of their ages ranges between 35 and 40. Half of the expert group have 5 to 10 years’ experience in water related studies and management. In terms of education level, half of them have a PhD degree.
Table 5 presents the final weight and rank of criteria and sub-criteria that were obtained from the Super Decisions software. According to this, it can be seen that a combination of structural and non-structural measures (C3) was chosen as a best option. Regarding structural measures, “river restoration” was given rank one by receiving a 0.33 weight. Among non-structural measures, “agricultural management and planning” was given the highest rank and a weight of 0.34. In terms of combined measures, “watershed management practices” was ranked highest and a weight of 0.46.

3.2. Contingent Valuation Method Results

Respondents’ Attitudes

Figure 4a–c shows the percentage of responses regarding their perception on climate change and flooding. In this part, the interviewees in the three different zones were asked three questions. The responses showed that more than half of the respondents (Zone 1 (64%), Zone 2 (54%), and Zone 3 (52%)) feel that temperature has changed in recent years by selecting the “High and Very High” option (Figure 4a). Moreover, respondents were asked about recent rainfall changes and flood events due to extreme rainfall. The answers showed that a majority in all three zones feel that rainfall has changed and has led to more frequent flood events (Figure 4b). Regarding climate change awareness, their responses showed that most feel that they do not have much information, the “don’t know” option was selected by 36%–44% of participants in all three zones (Figure 4c).
Regarding flood damages in recent years (Figure 4d–f), however, a large number of respondents stated that they have not suffered much by flooding by choosing “Very Low and Low” options (Figure 4d), but their concerns about flood risk in their area still remained “High” and “Moderate” (Figure 4e). This might be a result of experiencing flood events in the vicinity of their villages during recent years [29]. The perception on flood risk management measures and their efficiency showed that most respondents are aware of flood risk management, and they stated that the efficiency of flood management strategies is necessary in their area by choosing “High and Very High” options (Figure 4f).
In order to define WTP for implementation of the three flood management strategies (FMS) described in previous section, respondents in the three zones were asked about their WTP (by cash). Totally, 50%, 38%, and 18% of respondents were willing to pay in zones 1, 2, and 3, respectively. The respondents who showed willingness to pay, were asked about WTP in three bids according to 80,000(~US $2), 160,000 (~US $4), and 40,000 (~US $1) as cash (Table 6). It can be seen that the WTP 80,000 (~US $2), 160,000 (~US $4), and 40,000 (~US $1) Rials per month were accepted by 35%, 21%, and 11% of respondents in zones 1, 2, and 3, respectively. Among respondents that said “Yes”, 12, 7, and 3% were WTP 160,000 (~US $4) Rials. Among those who chose “No”, 18%, 17%, and 7% were WTP 40,000 Rials (~US $1) for implementation of selected FMS. The maximum WTP (more than 160,000 Rials) in zones 1, 2, and 3 was 4%, 2%, and 2% of respondents, respectively.
Figure 5 shows the allocation of each FMS in the three different zones in view of respondents’ WTP. According to this, river restoration strategy was the most popular in zone 1 (47%), zone 2 (45%), and zone 3 (60%).
As for non-cash payment, respondents were asked about their willingness to participate in FMS projects in the region. Table 7 shows the percentage of respondents that is willing to contribute labor in flood management projects instead of payment. It can be seen that 44%, 48%, and 52% of respondents were willing to contribute in this way in zones 1, 2, and 3, respectively.
Table 8 shows the willingness by respondents in terms of number of days to work in flood management strategy projects. Among respondents who are willing to contribute, the willingness in terms of number of days to work in flood management strategy projects in five categories is shown in Table 8. It is seen that the majority of respondents were willing to work at least 2 days per month during a year. For example, among 44% of respondents who were willing to contribute in zone 1, 33% (about 14 respondents) were willing to work 2 days per month.
Among respondents that were not willing to pay, less than 2% thought that implementation of flood mitigation measures is not necessary in the study area (Table 9). About 34, 41, and 70% in zones 1, 2, and 3, respectively, considered that the government is responsible for this.
Table 10 shows socioeconomic characteristics of respondents in the three different zones. According to Table 10, out of the 100 respondents in each zone, 80%, 85%, and 70% were male in zones 1, 2, and 3, respectively. In terms of education, the respondents could be divided into seven different groups including: illiterate, elementary degree, middle school degree, high school degree, college degree, bachelor’s degree, master or Ph.D. degree. The dominant category was high school degree for all three zones. Furthermore, the respondents’ occupations were classified in seven different groups including: office work, labor, military, farmer, cattle farmer, private business, and other. More than 40% of respondents in all three zones could be categorized as private business occupation. About 20% of people participating in the study depend on agriculture and cattle for their livelihood. Very few people (<1%) are employed in the military. Besides this, respondents’ income was grouped into four classes with about 40% earning 10–20 million Rials per month. The respondents’ age is well distributed in four groups.

3.3. Inferential Results of WTP

Logit regression model was used to investigate the characteristics affecting the WTP [30]. In this method, the bid is the dependent variable, and other variables such as age, gender, education, and income are independent variables for the three different zones. The model parameters were estimated by using a maximum likelihood approach. Logit regression model estimation results are presented in Table 11. Total WTP through the direct (Cash) and indirect (Non-cash) WTP calculations are presented in Table 12. Percentage of each flood management strategy in the three different zones is presented in Table 13.
According to Table 11, in zone 1, among independent variables, the respondents’ monthly income is significant at the 99% level and amount of bid is significant at 95% level, and other dependent variables such as age, gender, and education had no significant effect on the WTP. The negative coefficient and elasticity estimation for amount of bid show that by increasing bid amount, the probability of accepting the bid decreases by 0.68%. Considering the marginal effect of the monthly income variable, with an increase of 10 million Rials to household income, the probability of accepting the proposed amount will increase by 32% (Table 11).
Further, the expected value of mean C-WTP to implement selected flood management strategies was estimated through numerical integration from the range of zero to the maximum bid (160,000 Rials, ~ US $4) using estimated parameters from logit model:
E ( C W T P ) = 0 160000 ( 1 1 + exp { ( 0.21186492 + 0.000013498 × Bid ) } ) d Bid = 122692   R i a l s U S   $ 3.19  
The C-WTP calculated for zone 1 shows that respondents are on average willing to pay about 122,700 Rials (~US $3.19) per month which is about 1,472,000 Rials per year (~US $38) for the implementation of selected flood management strategies. Thus, as 772 dwellers live in zone 1, the total C-WTP in this zone can be calculated as 1137 million Rials per year (~US $30,000) (Table 12). According to the Nc-WTP calculation in zone 1, a total of 118 labor days were accepted by the respondents in zone 1, and the average daily income for labor is 505,627 Rials (~US $13), so the final Nc-WTP can be estimated at about 5527 million Rials (~US $144,000). Thus, the total WTP in zone 1 was about 6664 million Rials (Table 12). On average during 2018, this is equivalent to US $173,261 per year. Considering that in zone 1, respondents are willing to pay 47% of the payment for the river restoration measure, the annual WTP for that is estimated about 3132 million Rials (US $81,433) per year. This value for agricultural management and planning measures is about 2132 million Rials (US $55,444) per year as well as for watershed management measures is 1399 million Rials (US $36,385) per year (Table 13).
In zone 2, the results show that the monthly income and amount of bid are statistically significant at 99% and 95%, respectively, while the other dependent variables (age, gender, and education) are not significant. The positive coefficient estimated for monthly income and negative for the bid amount shows that if the monthly income increases and amount of bid decreases, the probability of intention in accepting to pay will increase. The marginal effect value shows that in a case of increasing the monthly income variable and considering other variables as constant, the probability of paying for flood management strategies will not change. The percentage of statistically correct predictions in this zone was 78%, which indicated a relatively good fit to the data (Table 11). The expected value of mean C-WTP for selected flood management strategy implementation was estimated as:
E ( C W T P ) = 0 160000 ( 1 1 + exp { ( 0.503478055 + 0.000018929 × Bid ) } ) d Bid = 47506   R i a l s U S   $ 1.25  
Regarding to C-WTP calculated in zone 2, households are averagely willing to pay 47,506 Rials per month which is about 570,072 Rials (~US $15) per year for the flood management strategy implementation in this zone. Considering 1119 inhabitant in zone 2, the final C-WTP can be calculated as 638 million Rials (~US $17,000) per year (Table 12). According to the Nc-WTP calculation in zone 2, 121 labor days were accepted by the respondents in zone 2 and the final Nc-WTP can be estimated at about 8215 million Rials (~US $214,000). The total WTP in zone 2 is approximately 8853 million Rials (Table 12). This is equivalent to US $230,185 per year. Regarding Table 13, respondents in zone 2 are willing to pay about 4161 million Rials of the total payment for river restoration measures (US $108,187). This value for agricultural management and planning measures is about 2883 million Rials (US $73,659) per year as well as for watershed management measures is 1859 million Rials (US $48,339) per year (Table 13).
In zone 3, the monthly income, bid amount, and age are factors that significantly affect the WTP at 99%, 95%, and 90%, respectively. Gender and education did not have a significant effect on WTP. The estimated coefficient for bid amount and age are negative and show that increases of bid and age lead to decreased probability of WTP for flood management strategies. By increasing monthly income, the probability for paying decreases (Table 11). Thus, 1% increase in age, results in reduced probability of accepting the proposed amount by 2% (Table 11).
The expected mean C-WTP in zone 3 was calculated as:
E ( C W T P ) = 0 160000 ( 1 1 + exp { ( 0.627291241 + 0.000026713 × Bid ) } ) d Bid = 38537   R i a l s U S   $ 1  
The C-WTP means that people in zone 3 are willing to pay 38,537 Rials (~US $1) per month for flood management strategy implementation. Since 639 people live in zone 3, the total C-WTP can be calculated as 295 million Rials (~US $8000) per year (Table 12). In total, 127 labor days were accepted by the respondents in zone 3 and the total Nc-WTP was about 4924 million Rials (~US $129,000). The total WTP in zone 3 was about 5219 million Rials which is equivalent to US $ 135,706 per year (Table 12). Respondents in zone 2 are willing to pay about 2453 million Rials of the total payment for the river restoration measure (US $63,782). Moreover, the percentage of payment for agricultural management and planning measure is about 1670 million Rials per year (US $43,426) and for watershed management measure it is about 1096 million Rials per year (US $28,498) (Table 13).

4. Conclusions

This study focused on assessing the social acceptability and feasibility of flood management strategies regarding climate change impacts in the Talar River Basin, northern Iran, through addressing local participation, expert assessment, and integrative decision making. For this purpose, we used Analytical Network Process (ANP) as multiple criteria decision making (MCDM), Contingent Valuation Method (CVM) as an economic valuation approach, and hydro-climate modelling to study the socio-hydrological system in the study area. Among twelve available flood management strategies in three categories, structural, non-structural, and combination of structural and non-structural measures, three highest rank measures were chosen by experts and decision-makers through ANP approach. These measures included “river restoration”, “agricultural management and planning”, and “watershed management” practices that gained the highest weight values of 0.33, 0.34, and 0.46, respectively.
The respondents’ attitude regarding effects of flood damage during recent year shows that a large number of them has not experienced flood problems similar to those of other studies, e.g., [11]. However, due to experiencing flood events in other nearby basins, respondents’ concerns about flood risk are considerable. The perception of flood risk has strong influence on the WTP, which in regions with different level of flood risk and priority of flood leads to highly variable WTP [32]. The social acceptability of the three selected FMS was assessed by WTP calculation through a CVM approach. Among 100 respondents in each zone (determined based on the distance to the river), 50%, 38%, and 18% were willing to pay for implementation of the three selected FMS and 44%, 48%, and 52% were willing to contribute in FMS projects in zones 1, 2, and 3, respectively. The total WTP calculated C-WTP and Nc-WTP were about 6664 (US $173,261), 8853 (US $230,185), and 5219 (US $135,706) million Rials in zones 1, 2, and 3, respectively. These sums represent the protection value of FMS against flood under climate change impact. The CVM results show that among dependent variables, income has the highest impact on the acceptance or rejection rate. Consequently, respondents with higher income in all three zones were willing to pay more for flood mitigation measures. These results are similar to previous studies [30,33,34,35]. Except for age in the third zone, other variables did not have a significant effect on the WTP in the three zones. The results indicate that the larger distance from the river, the larger decrease in peoples’ WTP for flood mitigation measures. This is logical since flood risk decreases with distance from the main river stream [21,36]. However, about half of the population has concern about flood issues in the region.
For integrated flood management measures, it is necessary to consider public participation by local communities in decision-making. In zones 1 and 3, agricultural management and planning measures were ranked second in terms of acceptance by the respondents, but in zone 2, the watershed management measures were ranked second. It should be noted that although two other watershed management measures were considered, respondents tended to have a low WTP for these.
Increasing the social acceptability plays an important role in sustainability and maintenance of FMS and in this regard news and social media as near real-time information channels share opinions regarding concerns on climate change [37] and flood risk issues [38]. Thus, news and social media are important as opinion builders for climate change and flood management and citizens’ willingness to actively participate. The socio-hydrological framework used in this study is expected to be adaptable for other local communities in different river basins of the world [5]. In this regard, the results of this study are important for policy makers to make better decisions in order to accomplish effective flood management strategies in view of climate change.

Author Contributions

Conceptualization, F.F.M., H.M., M.P, R.B, A.D., and A.R.M.B.; methodology, F.F.M., H.M., R.B., M.P., and A.D.; validation, F.F.M., H.H., M.P., and R.B.; formal analysis, F.F.M.; investigation, F.F.M. and A.D.; writing—original draft preparation, F.F.M.; writing—review and editing, F.F.M., R.B., H.M., A.D., H.H., M.P., and A.R.M.B.; visualization, F.F.M.; supervision, H.M., R.B., and H.H.

Funding

This research received no external funding.

Acknowledgments

We would like to thank experts and local community who warmly contributed by filling out the questionnaires. Additionally, we thank the Centre for Middle Eastern Studies at Lund University and the Ministry of Science, Research, and Technology of the Islamic Republic of Iran for financial support.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Quintero, F.; Mantilla, R.; Anderson, C.; Claman, D.; Krajewski, W. Assessment of Changes in Flood Frequency Due to the Effects of Climate Change: Implications for Engineering Design. Hydrology 2018, 5, 19. [Google Scholar] [CrossRef]
  2. Maghsood, F.F.; Moradi, H.; Massah Bavani, A.R.; Panahi, M.; Berndtsson, R.; Hashemi, H. Climate Change Impact on Flood Frequency and Source Area in Northern Iran under CMIP5 Scenarios. Water 2019, 11, 273. [Google Scholar] [CrossRef]
  3. Arnell, N.; Berry, P.; Dodman, D.; Fankhauser, S.; Hope, C.; Kovats, S.; Nicholls, R.; Sattherwaite, D.; Tiffin, R.; Wheeler, T. Assessing the Costs of Adaptation to Climate Change: A Critique of the UNFCCC Estimates; Imperial College: London, UK, 2009; ISBN 9781843697459. [Google Scholar]
  4. Morid, S.; Massah Bavani, A.R. Exploration of potential adaptation strategies to climate change in the Zayandeh Rud irrigation system, Iran. The journal of the International Commission on Irrigation and Drainage 2010, 59, 226–238. [Google Scholar] [CrossRef]
  5. Devkota, R.P. Flood Adaptation Strategies under Climate Change in Nepal: A Socio-Hydrological Analysis; University of Southern Queensland: Toowoomba, QLD, Australia, 2014. [Google Scholar]
  6. Carraro, C.; Sgobbi, A.; Eni Enrico Mattei, F.; Alberini, A.; Bigano, A.; Bosello, F.; Breil, M.; Catenacci, M.; Chiabai, A.; Crimi, J.; et al. Climate Change Impacts and Adaptation Strategies In Italy: An Economic Assessment; Fondazione Eni Enrico Mattei: Milano, Italy, 2008. [Google Scholar]
  7. Chang, C.T. Risk-trading in flood management: An economic model. J. Environ. Manag. 2017, 200, 1–5. [Google Scholar] [CrossRef]
  8. Thomassin, A.; White, C.S.; Stead, S.S.; David, G. Social acceptability of a marine protected area: The case of Reunion Island. Ocean. Coast. Manag. 2010, 53, 169–179. [Google Scholar] [CrossRef] [Green Version]
  9. De Brito, M.M.; Evers, M. Multi-criteria decision-making for flood risk management: A survey of the current state of the art. Nat. Hazards Earth Syst. Sci. 2016, 16, 1019–1033. [Google Scholar] [CrossRef]
  10. Verlynde, N.; Voltaire, L.; Chagnon, P. Exploring the link between flood risk perception and public support for funding on flood mitigation policies. J. Environ. Plan. Manag. 2019, 1–22. [Google Scholar] [CrossRef]
  11. Zhai, G. Public preference and willingness to pay for flood risk reduction. In A Better Integrated Management of Disaster Risks: Toward Resilient Society to Emerging Disaster Risks in Mega-Cities; Ikeda, S., Fukuzono, T., Sato, T., Eds.; TERRAPUB: Tokyo, Japan, 2006; pp. 57–87. [Google Scholar]
  12. Fuks, M.; Chatterjee, L. Estimating the willingness to pay for a flood control project in Brazil using the Contingent Valuation Method. J. Urban Plan. Dev. 2008, 134, 42–52. [Google Scholar] [CrossRef]
  13. Liziński, T.; Wróblewska, A.; Rauba, K. Application of CVM method in the evaluation of flood control and water and sewage management projects. J. Water Land Dev. 2015, 24, 41–49. [Google Scholar] [CrossRef]
  14. Markantonis, V.; Bithas, K. The application of the contingent valuation method in estimating the climate change mitigation and adaptation policies in Greece. An expert-based approach. Environ. Dev. Sustain. 2010, 12, 807–824. [Google Scholar] [CrossRef]
  15. Yousefi, S.; Moradi, H.R.; Pourghasemi, H.R.; Khatami, R. Assessment of floodplain landuse and channel morphology within meandering reach of the Talar River in Iran using GIS and aerial photographs. Geocarto Int. 2018, 33, 1367–1380. [Google Scholar] [CrossRef]
  16. Brito, M.M.D. A Participatory Multi-Criteria Approach for Flood Vulnerability Assessment. Ph.D. Thesis, Universitäts-und Landesbibliothek Bonn, Bonn, Germany, 2018. [Google Scholar]
  17. Saaty, T.L. Fundamentals of the analytic network process. In Proceedings of the 5th International Symposium on the Analytic Hierarchy Process, Kobe, Japan, 12–14 August 1999. [Google Scholar]
  18. Saaty, T.L.; Vargas, L.G. Decision Making with the Analytic Network Process; Springer: New York, NY, USA, 2006. [Google Scholar]
  19. Saaty, R.W. Decision Making in Complex Environments. The Analytic Network Process (ANP) for Dependence and Feedback. Including a Tutorial for the SuperDecisions Software and Portions of the Encyclicon of Applications. 2016. Available online: https://superdecisions.com/sd_resources/v28_man02.pdf (accessed on 20 March 2019).
  20. Tussupova, K.; Berndtsson, R.; Bramryd, T.; Beisenova, R. Investigating willingness to pay to improve water supply services: Application of contingent valuation method. Water 2015, 7, 3024–3039. [Google Scholar] [CrossRef]
  21. Shang, Z.; Che, Y.; Yang, K.; Jiang, Y. Assessing local communities’ willingness to pay for river network protection: A contingent valuation study of Shanghai, China. Int. J. Environ. Res. Public Health 2012, 9, 3866–3882. [Google Scholar] [CrossRef] [PubMed]
  22. Lee, C.K.; Han, S.Y. Estimating the use and preservation values of national parks’ tourism resources using a contingent valuation method. Tour. Manag. 2002, 23, 531–540. [Google Scholar] [CrossRef]
  23. Ji, S.; Choi, Y.; Lee, C.K.; Mjelde, J.W. Comparing willingness-to-pay between residents and non-residents using a contingent valuation method: Case of the Grand Canal in China. Asia Pac. J. Tour. Res. 2018, 23, 79–91. [Google Scholar] [CrossRef]
  24. Duffield, J.W.; Patte, D.A. Inference and optimal design for a welfare measure in dichotomous choice contingent valuation. Land Econ. 1991, 67, 225–239. [Google Scholar] [CrossRef]
  25. Jin, M.; Juan, Y.; Choi, Y.; Lee, C.K. Estimating the preservation value of world heritage site using contingent valuation method: The case of the Li River, China. Sustainability 2019, 11, 1100. [Google Scholar] [CrossRef]
  26. Berrens, R.P.; Bohara, A.K.; Silva, C.L.; Brookshire, D.; Mckee, M. Contingent values for New Mexico instream flows: With tests of scope, group-size reminder and temporal reliability. J. Environ. Manag. 2000, 58, 73–90. [Google Scholar] [CrossRef]
  27. Markantonis, V.; Meyer, V.; Lienhoop, N. Evaluation of the environmental impacts of extreme floods in the Evros River basin using Contingent Valuation Method. Nat. Hazards 2013, 69, 1535–1549. [Google Scholar] [CrossRef]
  28. Amigues, J.P.; Boulatoff, C.; Desaigues, B.; Gauthier, C.; Keith, J.E. The benefits and costs of riparian analysis habitat preservation: A willingness to accept/willingness to pay contingent valuation approach. Ecol. Econ. 2002, 43, 17–31. [Google Scholar] [CrossRef]
  29. Modarres, R.; Sarhadi, A.; Burn, D.H. Changes of extreme drought and flood events in Iran. Glob. Planet. Chang. 2016, 144, 67–81. [Google Scholar] [CrossRef]
  30. Amirnejad, H.; Tonakbar, P. The willingness to pay for organic milk by consumers in Tehran. J. Agric. Sci. Technol. 2015, 17, 1685–1694. [Google Scholar]
  31. Vice Presidency Plan and Budget Organization Statistical Centre of Iran. Population and Housing Censuses; Statistical Center of Iran: Tehran, Iran, 2016; ISBN 9786004092425. (In Persian) [Google Scholar]
  32. Zhai, G.; Sato, T.; Fukuzono, T.; Ikeda, S.; Yoshida, K. Willingness To Pay for Flood Risk Reduction and Its Determinants in Japan1. J. Am. Water Resour. Assoc. 2006, 42, 927–936. [Google Scholar] [CrossRef]
  33. Loureiro, M.L.; McCluskey, J.J.; Mittelhammer, R.C. Will consumers pay a premium for eco-labeled apples. J. Consum. Aff. 2002, 36, 203–219. [Google Scholar] [CrossRef]
  34. Gracia, A.; de Magistris, T. The demand for organic foods in the South of Italy: A discrete choice model. Food Policy 2008, 33, 386–396. [Google Scholar] [CrossRef]
  35. Yahaya, I. Measuring Consumers’ Willingness to Pay for “Safer” Vegetables in Urban and Peri-Urban Ghana. Ph.D. Thesis, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana, 2008. [Google Scholar]
  36. Brouwer, R.; Akter, S.; Brander, L.; Haque, E. Economic valuation of flood risk exposure and flood control in a severely flood prone developing country. J. Environ. Econ. Manag. 2006, 32, 251–270. [Google Scholar]
  37. Ogunjinmi, A.; Ogunjinmi, K.; Sunday, O.; Adekoya, O. Influence of Social Media on Climate Change Knowledge and Concerns. Nig. J. Agric. Food Environ. 2016, 12, 23–30. [Google Scholar]
  38. Aisha, T.S.; Wok, S.; Manaf, A.M.A.; Ismail, R. Exploring the Use of Social Media During the 2014 Flood in Malaysia. Procedia Soc. Behav. Sci. 2015, 211, 931–937. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Location of the study area in northern Iran.
Figure 1. Location of the study area in northern Iran.
Sustainability 11 05053 g001
Figure 2. Schematic framework of the methodology.
Figure 2. Schematic framework of the methodology.
Sustainability 11 05053 g002
Figure 3. Location of villages with interviewees and the three different zones included in the survey.
Figure 3. Location of villages with interviewees and the three different zones included in the survey.
Sustainability 11 05053 g003
Figure 4. Respondents’ perception regarding climate change and flooding. (a): Q 1.1; (b): Q 1.2; (c): Q 1.3; (d): Q 2.1; (e): Q 2.2; (f): Q 2.3.
Figure 4. Respondents’ perception regarding climate change and flooding. (a): Q 1.1; (b): Q 1.2; (c): Q 1.3; (d): Q 2.1; (e): Q 2.2; (f): Q 2.3.
Sustainability 11 05053 g004
Figure 5. Preferred FMS depending on the three investigated zones.
Figure 5. Preferred FMS depending on the three investigated zones.
Sustainability 11 05053 g005
Table 1. List of investigated flood management strategies.
Table 1. List of investigated flood management strategies.
CriteriaDescription of Strategies (Sub-Criteria)
Structural (C1)(S11): Dams and reservoirs
(S12): Construction standards and building codes
(S13): Flood relief channels
(S14): River restoration
(S15): Embankments
Non-structural (C2)(S21): Rising public awareness
(S22): Agricultural management and planning
(S23): Flood forecasting and warning
(S24): Flood insurance or compensation
Combination of structural and non-structural (C3)(S31): Watershed management practices
(S32): Land use policy, planning, zoning, and regulation
(S33): Flood spreading systems
Table 2. Attitude and perception questions included in the questionnaire.
Table 2. Attitude and perception questions included in the questionnaire.
Section 1: Respondents’ Perception of Climate Change and Its Impacts
Q 1.1How much do you feel temperature has changed over the past years?
Q 1.2How much do you feel extreme rainfall has changed over the past years?
Q 1.3How much are you aware of the climate change and its impacts?
Section 2: Respondents’ Perception of Flood Risk and Management
Q 2.1How much has floods affected your family over the past years?
Q 2.2What is your concern about the flood risk in your area?
Q 2.3Do you think flood control plans are effective to reduce flood risks?
Table 3. Summary of socioeconomic conditions.
Table 3. Summary of socioeconomic conditions.
FactorsCategory
GenderMale/ Female
Education LevelIlliterate/Elementary degree/Middle school degree 1/High school degree/College degree 2/Bachelor’s degree/Master or Ph.D. degree.
OccupationOffice work/Labor/Military/Farmer/Cattle farmer/Private business/ Other
Income level (Monthly)<10, 10–20, 20–30, 30 < (Million Rials)
Age<30, 30–40, 40–50, 50 <
1 9th grade degree. 2 Two years’ university studies.
Table 4. Key characteristics of experts and decision-makers.
Table 4. Key characteristics of experts and decision-makers.
Key CharacterSpecificationFrequency (%)
GenderMale75
Female25
Age (Year)<3010
30–3525
35–4040
>4025
Work experience (Year)<520
5 to 1050
10 to 1530
Education level (degree)PhD50
Master40
Bachelor10
Table 5. Final weight and rank of criteria and sub-criteria.
Table 5. Final weight and rank of criteria and sub-criteria.
Criteria and Sub-CriteriaWeightRank
CriteriaStructural (C1)0.253
Non-structural (C2)0.332
Combination of structural and non-structural (C3)0.411
Sub-criteria structural(S11): Dams and reservoirs0.095
(S12): Construction standards and building codes0.242
(S13): Flood relief channels0.114
(S14): River restoration0.331
(S15): Embankments0.203
Sub-criteria non-structural(S21): Rising public awareness0.292
(S22): Agricultural management and planning0.341
(S23): Flood forecasting and warning0.193
(S24): Flood insurance or compensation0.164
Sub-criteria combination of structural and non-structural (S31): Watershed management practices0.461
(S32): Land use policy, planning, zoning and regulation0.342
(S33): Flood spreading systems0.143
Table 6. Mean monthly WTP per household in three determined bids in three different zones.
Table 6. Mean monthly WTP per household in three determined bids in three different zones.
Bid (Rials)ResponseZone 1 (%)Zone 2 (%)Zone 3 (%)
80,000Yes352111
No657989
160,000Yes1273
No889397
40,000Yes18177
No828393
Maximum (Rials)>160,000422
Table 7. Respondents’ willingness to contribute as labor in FMS.
Table 7. Respondents’ willingness to contribute as labor in FMS.
ResponseZone 1 (%)Zone 2 (%)Zone 3 (%)
Yes444852
No565248
Table 8. Respondents’ willingness to spend x number of working days as labor (day/month).
Table 8. Respondents’ willingness to spend x number of working days as labor (day/month).
Days (Day/Month)Zone 1 (%)Zone 2 (%)Zone 3 (%)
One212214
Two334545
Three161227
Four1246
Five19168
Table 9. Reasons given by respondents who are not willing to pay.
Table 9. Reasons given by respondents who are not willing to pay.
ReasonZone 1 (%)Zone 2 (%)Zone 3 (%)
It is a government responsibility344170
I don’t think these measures are necessary120
I have economic problems182326
Other reasons47344
Table 10. Socioeconomic characteristics of the interviewees in the three zones.
Table 10. Socioeconomic characteristics of the interviewees in the three zones.
CharactersCategoriesZone 1 (%)Zone 2 (%)Zone 3 (%)
GenderMale808570
Female201530
Education LevelIlliterate525
Elementary degree10811
Middle school degree16128
High school degree334444
College degree91211
Bachelor degree201919
Master and Ph.D. degree732
OccupationOffice work171816
Labor162110
Military010
Farmer151511
Cattle farmer520
Private business404262
Other711
Income (Million Rials)<10181928
10 to 20444245
20 to 30253122
>301385
Age<30272621
30–40232827
40–50241927
>50262725
Table 11. Logit model results of the demand function.
Table 11. Logit model results of the demand function.
Variables NameZone 1 aZone 2 bZone 3 c
CoefficientT-StatisticElasticity in MeanMarginal EffectCoefficientT-StatisticElasticity in MeanMarginal EffectCoefficientT-StatisticElasticity in MeanMarginal Effect
Bid−0.135 × 10−4−2.31 **−0.683−0.29 × 10−5−0.189 × 10−4−2.616 **−0.96−0.347 × 10−5−0.267 × 10−4−2.064 **−1.31−0.16 × 10−5
Age−0.0162−0.953−0.461−0.00348−0.0113−0.617−0.36−0.00208−0.0516−1.543 *−2.052−0.0031
Gender0.7280.8380.5340.1560.3540.2660.274−0.0649−0.09890.0950.09970.594 × 10−2
Education−0.0607−0.4080.176−0.0130.1420.6660.4620.0261−0.257 × 10−4−0.845−1.009−0.0154
Income0.150 × 10−64.425 ***2.1970.323 × 10−70.165 × 10−64.302 ***2.3430.303 × 10−70.274 × 10−64.224 ***3.920.165 × 10−7
a Log-likelihood function = −73.6. Likelihood Ratio Statistic (LR Statistic) = 28.98. Probability (LR Statistic) = 0.00002. Maddala R-square = 0.19. Percentage of right predictions = 0.72. b Log-likelihood function = −66.52. Likelihood Ratio Statistic (LR Statistic) = 26.75. Probability (LR Statistic) = 0.00006. Maddala R-square = 0.18. Percentage of right predictions = 0.78. c Log-likelihood function = −32.1. Likelihood Ratio Statistic (LR Statistic) = 32.46. Probability (LR Statistic) = 0.00000. Maddala R-square = 0.24. Percentage of right predictions = 0.89. *: Significant at 10% level. **: Significant at 5% level. ***: Significant at 1%level.
Table 12. Cash and Non-cash WTP calculation for the three different zones (approximate values).
Table 12. Cash and Non-cash WTP calculation for the three different zones (approximate values).
C-WTP 1
Zone 1Zone 2Zone 3
Number of households in zone [31]7721119639
WTP (per month)122,70047,50038,500
WTP (per year)1,472,304570,072462,444
Total C-WTP in zone (million Rials) (~)1,137638295
Nc-WTP 2
Zone 1Zone 2Zone 3
Number of respondents100100100
Number of labor days118121127
Average of respondents14.1614.5215.24
Average income (Rials/day)505,627505,627505,627
Total Nc-WTP in zone (million Rials) 552782154924
Total WTP (C-WTP and Nc-WTP) in zone (million Rials)666488535219
1 Cash WTP, 2 Non-cash WTP.
Table 13. Percentage of WTP for each FMS in the three different zones.
Table 13. Percentage of WTP for each FMS in the three different zones.
Flood Management StrategiesZone 1Zone 2Zone 3
%Million Rials%Million Rials%Million Rials
River restoration473,132454,161602,453
Agricultural management and planning measures322,132252,833231,670
Watershed management measures211,399301,859171,096
Total WTP1006,6631008,8531005,219

Share and Cite

MDPI and ACS Style

Maghsood, F.F.; Moradi, H.; Berndtsson, R.; Panahi, M.; Daneshi, A.; Hashemi, H.; Bavani, A.R.M. Social Acceptability of Flood Management Strategies under Climate Change Using Contingent Valuation Method (CVM). Sustainability 2019, 11, 5053. https://doi.org/10.3390/su11185053

AMA Style

Maghsood FF, Moradi H, Berndtsson R, Panahi M, Daneshi A, Hashemi H, Bavani ARM. Social Acceptability of Flood Management Strategies under Climate Change Using Contingent Valuation Method (CVM). Sustainability. 2019; 11(18):5053. https://doi.org/10.3390/su11185053

Chicago/Turabian Style

Maghsood, Fatemeh Fadia, Hamidreza Moradi, Ronny Berndtsson, Mostafa Panahi, Alireza Daneshi, Hossein Hashemi, and Ali Reza Massah Bavani. 2019. "Social Acceptability of Flood Management Strategies under Climate Change Using Contingent Valuation Method (CVM)" Sustainability 11, no. 18: 5053. https://doi.org/10.3390/su11185053

APA Style

Maghsood, F. F., Moradi, H., Berndtsson, R., Panahi, M., Daneshi, A., Hashemi, H., & Bavani, A. R. M. (2019). Social Acceptability of Flood Management Strategies under Climate Change Using Contingent Valuation Method (CVM). Sustainability, 11(18), 5053. https://doi.org/10.3390/su11185053

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop