Methodology for Selecting Best Management Practices Integrating Multiple Stakeholders and Criteria. Part 2: Case Study

Abstract

The selection of stormwater Best Management Practices (BMPs) for mitigating the effects of urbanization on the hydrological cycle could be a complex process due to conflicting stakeholder views, and varying levels of performance of BMPs across a range of criteria (runoff reduction, erosion control, etc.). Part 1 of this article proposed a methodology based on the application of multi-criteria decision aid (MCDA) methods, which was tested here on a residential stormwater network in the Montreal area. The case study considered green roofs, rain gardens, rain barrels and pervious pavement over a range of economic, social, and water quality and quantity criteria by applying 4 MCDA methods under three different stakeholder views. The results indicated Elimination et Choix Traduisant la Réalité (ELECTRE) III to be the most appropriate method for the methodology, presenting flexibility concerning threshold values, criteria weights, and showing shared top choices across stakeholders (rain gardens, and rain gardens in combination with pervious pavement). The methodology shows potential for more formal applications and research opportunities. Future work may lie in the inclusion of multiple objective optimization, better stakeholder engagement, estimation of economic benefits, water quality modeling, long-term hydrological simulations, and estimating real BMP pollutant removal rates.

1. Introduction

This is Part 2 of the paper on the methodology for selecting Best Management Practices (BMPs) based on multi-criteria decision aid (MCDA) methods integrating multiple stakeholder views. To demonstrate the methodology’s potential, Part 2 presents a case study application following the steps described in detailed in Part 1 [1] along with the corresponding results and discussion. Rather than focusing on the results, which could be highly case-specific, this work mainly draws attention to how the proposed methodology could be applied and interpreted.

2. Methodology

The methodology described in Part 1 considered a step of problem definition, during which the site, MCDA methods, criteria and technical guidelines are selected. The next step calls for a preliminary analysis of the site in order to obtain the most relevant BMPs adapted to the case. Afterwards, the alternatives for consideration are analysed, including individual BMPs and possible combinations. Next, the stakeholder analysis is needed, during which relevant stakeholders are identified and data are collected for determining criteria priorities and for calculating the social performance of the different BMPs. The next step consists in the application of the MCDA methods themselves, providing a final ranking of the different BMP alternatives for each stakeholder. The final step consists in an analysis of these results, including the sensitivity analysis, in order to draw conclusions on the most appropriate MCDA method and recommended BMP alternative(s) [1].

3. Application to a Case Study

The methodology was applied to a case study in the Montreal area. The following subsections describe in detail the application of the steps previously mentioned.

3.1. Problem Definition

An urbanized watershed in the suburbs of the Greater Montreal Area was selected presenting complaints from residents due to frequent flooding, most notably during a storm in 2012 where several basements were flooded due to backflow in the foundation drains. The watershed drains an area of around 9.56 ha, located in a mainly residential zone with a park and a school near the top. The average percentage of impervious area of the watershed is 42% (including roads and driveways). The lowest point is at 13.25 m above sea level while the highest is at 15.63 m. The network is exclusive for stormwater and consists of circular concrete pipes. The methodology was applied using a modified version of the network for simplified analysis. The Stormwater Management Guide from Quebec [2] served as the main document for the analysis, complementing when necessary with guides from other cities across Canada and North America.

Following the recommendations stated in the Methodology, four main criteria were considered: technical performance (water quantity control), water quality control, economic cost, and social performance. Additional sub-criteria were added in some cases, as explained below. The criteria evaluating the quantitative control basically compared the outflow hydrographs from the site before and after the implementation of BMPs. For this case, three sub-criteria were selected: peak flow and runoff volume reduction, and delay to reach the outfall peak flow. Other criteria related to quantitative control could have included runoff velocity in order to address erosion control. However, the thresholds associated with this type of control depend also on the receiving water body (which was not studied in this case), and so they were left out.

Concerning the water quality, the Quebec Stormwater Management Guide suggests analysing a range of pollutants, emphasizing in Total Suspended Solids (TSS) and Total Phosphorous (TP), but also including organic matter, lead, Total Nitrogen (TN), among others [2]. Due to the limited information available on the concentrations of pollutants in the watershed, as well as removal performances from BMPs, only TSS, TN and TP were considered.

The social criteria considered were related exclusively to people’s perception of the different BMPs, adding the following four sub-criteria: aesthetic and landscape benefits, acceptability, perceived improvement to quality of life
and contributions to sustainable development. For the economic performance, no sub-criteria were added, analysing the costs in terms of the Net Present Value (NPV).

Due to time limitations, only four MCDA methods were chosen for this case study, two complete aggregation and two partial aggregation methods. Since some of the latter may not provide a complete ranking order, rather a schematic diagram detailing which alternative is preferred over which [3], specific versions of these methods were used that provided complete rankings. The Analytic Hierarchy Process (AHP), Élimination et Choix Traduisant la Réalité (ELECTRE) III, Preference Ranking Organization Method for Enrichment of Evaluation (PROMETHEE) II, and the Modified Technique for Order Preference by Similarity to Ideal Solutions (MTOPSIS) were selected. Detailed explanations of these methods are described in Part 1 of this work [1].

3.2. Preliminary Site Analysis

Seeing as the municipality had little information concerning the relevant hydrologic parameters, with few resources to spare for on-site verification, most information had to be taken from the literature. According to the Institut de Recherche et de Développement en Agroenvironnement (IRDA) [4], clays are the predominant soil group in the watershed, which was confirmed by the municipality’s engineering staff, though more specific soil properties were not provided. The infiltration capacity, using Horton’s infiltration model, was estimated using a soil type D, with little or no vegetation, resulting in a maximum infiltration capacity $f_o$ of 25 mm/h, and a final infiltration capacity $f_c$ of 1 mm/h [5]. The ensemble of BMPs considered is shown in Table 1, along with the technical constraints considered for this case, using NA for “Not Applicable” and X for unspecified information.

Table 1. Best Management Practices (BMPs) technical constraints [2,6,7].

BMP	Topography	Soil Infiltration	Groundwater	Drained Area	% of BMP Area from Total Drained Area
(1) Green roofs	20° roof slope	NA	NA	NA	NA
(2) Rainwater Capture	NA	NA	NA	NA	NA
(3) Downspout Disconnection	NA	NA	NA	NA	NA
(4) Infiltration drains	NA	Min rate 15 mm/h	>1 m below bottom	<0.5 ha	NA
(5) Infiltration trenches	NA	Min 15 mm/h	>1 m below bottom	<2 ha	Variable
(6) Permeable pavement	<5% slope	* Min 12.5 mm/h	>1.2 m below bottom	<4 ha	>50% **
(7) Filter strips	1%–5% slope	*** None	>0.5 m below bottom	<2 ha	>16.7%
(8) Rain gardens	<2% slope	* Min 25 mm/h	>1.2 m below bottom	<2 ha	>5%
(9) Vegetated swales	1%–4% longitudinal	NA	NA	<2 ha	Variable
(10) Perforated pipes	NA	Min 15 mm/h	>1 m below bottom	NA	X
(11) Dry ponds	NA	NA	NA	>5 ha	Retention volume variable
(12) Wet ponds	NA	NA	NA	>5 ha
(13) Wetlands	NA	NA	NA	>5 ha
* Smaller values accepted for partial infiltration		** Proportional to impervious area drained		*** If not conceived for infiltration

Table 1. Best Management Practices (BMPs) technical constraints [2,6,7].

BMP	Topography	Soil Infiltration	Groundwater	Drained Area	% of BMP Area from Total Drained Area
(1) Green roofs	20° roof slope	NA	NA	NA	NA
(2) Rainwater Capture	NA	NA	NA	NA	NA
(3) Downspout Disconnection	NA	NA	NA	NA	NA
(4) Infiltration drains	NA	Min rate 15 mm/h	>1 m below bottom	<0.5 ha	NA
(5) Infiltration trenches	NA	Min 15 mm/h	>1 m below bottom	<2 ha	Variable
(6) Permeable pavement	<5% slope	* Min 12.5 mm/h	>1.2 m below bottom	<4 ha	>50% **
(7) Filter strips	1%–5% slope	*** None	>0.5 m below bottom	<2 ha	>16.7%
(8) Rain gardens	<2% slope	* Min 25 mm/h	>1.2 m below bottom	<2 ha	>5%
(9) Vegetated swales	1%–4% longitudinal	NA	NA	<2 ha	Variable
(10) Perforated pipes	NA	Min 15 mm/h	>1 m below bottom	NA	X
(11) Dry ponds	NA	NA	NA	>5 ha	Retention volume variable
(12) Wet ponds	NA	NA	NA	>5 ha
(13) Wetlands	NA	NA	NA	>5 ha
* Smaller values accepted for partial infiltration		** Proportional to impervious area drained		*** If not conceived for infiltration

Visits carried out on-site showed the predominance of residential houses with sloped roofs, no sidewalks in the majority of roads, and a fairly constant line of driveways along house lots. Considering the space availability and infiltration characteristics presented, four BMPs were identified to be the best suited for implementation. However, future use of the results from this case study would require further verification of all the hydrological parameters shown in Table 1. The selected BMPs and their possible locations are listed below:

• Green roofs (GR): to be installed in the flat roof of the school.
• Rainwater harvesting (Br): for individual households (maximum two per household).
• Permeable pavement (PP) (partial infiltration): pervious asphalt to be installed on driveways.
• Rain gardens (RG) (partial infiltration): for implementation on front lawns.

3.3. Analysis of Alternatives

As mentioned in Part 1, Storm Water Management Model (SWMM) is one of the most popular hydrological models for urban stormwater management in North America [1]. In this case, the Personal Computer Storm Water Management Model (PCSWMM) was used, a modified version of the software developed by Computational Hydraulics International (CHI)^® (Guelph, ON, Canada), using the same computational base as SWMM.

Figure 1 presents the watershed configuration, showing conduits in yellow (with direction of flow), the junctions in blue, subcatchments in green, and the outfall marked by a red triangle. For this modified version of the network, house lot, flat roof, and the transversal street slopes were set to 2%, with open spaces (like parks and sports courts) set at 0.5%. Following recommend values, the conduits (made of concrete) were modeled using a Manning n coefficient of 0.014. For the overland flow, this parameter was taken at 0.11 for impervious areas, and 0.15 for the pervious areas. The depression storage depths used were of 1.21 mm and 2.54 mm, respectively [8]. The percentage of impervious areas without depression storage was kept at the default value of 25% since no special conditions in the area were known to exist. Dynamic wave hydraulic routing was used.

Figure 1. Watershed configuration.

Figure 1. Watershed configuration.

For pollutant build up, Even Mean Concentrations (EMCs) were used for the different land-use covers, which are summarized in Table 2. In order to better model water quality, subcatchments were further sub-divided to have one single land-use cover for each of them. If EMCs are used in SWMM, every drop of water that comes into contact with a specific land-use cover will be given the same pollutant concentration. Pollutants are then conveyed through the network together with the water flow, and are only removed when water leaves the system through infiltration, or when it is treated as the flow passes through BMPs.

Table 2. Pollutant Event Mean Concentrations (EMCs) [2].

Pollutant	Residential	Commercial	Non Developed	Roads
TSS (mg/L)	48	43	51	99
TN (mg/L)	2.00	2.20	1.20	2.30
TP (mg/L)	0.30	0.22	0.25	0.25

Table 2. Pollutant Event Mean Concentrations (EMCs) [2].

Pollutant	Residential	Commercial	Non Developed	Roads
TSS (mg/L)	48	43	51	99
TN (mg/L)	2.00	2.20	1.20	2.30
TP (mg/L)	0.30	0.22	0.25	0.25

Different rain events were considered for the water quantity and quality control performance. The Quebec Stormwater Management Guide states that a network that discharges into another should ideally provide adequate performance (no flooding or surcharge) for storms of a 10-year return period with a duration of 1 h (if the watershed area is less than 50 ha) [2]. A Chicago rain distribution using a ratio coefficient (time to maximum rain intensity over total event duration) of 0.45 and time step of 10 min was used [5]. Concerning the water quality scenario, a 25 mm rain lasting 3–6 h is recommended. For this case, the Soil Conservation Society (SCS) six-hour empirical rain event was used [9]. The hyetographs used are shown in Figure 2, with the current network conditions shown in Table 3 and Table 4.

Figure 2. Rain event for peak flow control (left) and water quality control (right).

Figure 2. Rain event for peak flow control (left) and water quality control (right).

Table 3. Current conditions for peak flow control scenario.

Parameter	Result
Peak flow (m3/s)	1.189
Time to peak (min)	30
Water volume through outfall (m3)	2288

Table 3. Current conditions for peak flow control scenario.

Parameter	Result
Peak flow (m3/s)	1.189
Time to peak (min)	30
Water volume through outfall (m3)	2288

Table 4. Current pollutant wash off (rain event for water quality scenario).

Water Volume through Outfall (m3)	TSS (kg)	TN (kg)	TP (kg)
1581	164.69	5.08	0.681

Table 4. Current pollutant wash off (rain event for water quality scenario).

Water Volume through Outfall (m3)	TSS (kg)	TN (kg)	TP (kg)
1581	164.69	5.08	0.681

The municipality did not have data on real flows. A formal proposition to collect measurements was done, but never accepted. This prevented a more accurate calibration of the model from being done. However, a sensitivity analysis was carried out on the most relevant hydrological parameters: slope, percentage of impervious area, flow length, and infiltration capacity. The response of any watershed is less sensitive to many of the hydrological parameters during strong storms [2]. Therefore, a smaller rain event with a return period of two years was used for the test to bring out the effect of the individual parameters as much as possible. The variables were modified individually within ranges deemed reasonable by the authors (i.e., parking lots remained impervious, etc.). The results for the peak flow sensitivity analysis are shown in Figure 3.

Figure 3. Peak flow sensitivity analysis.

Figure 3. Peak flow sensitivity analysis.

It can be seen that the percentage of imperviousness was the parameter with the greatest impact among those tested, but only after being significantly increased (30% or more), and even then had a relatively small impact.

In light of the sensitivity analysis, the model was deemed reasonably adequate for the purposes of this work. However, any future usage of the results found here would require a full calibration and verification of the input values.

The BMP modeling was done using the integrated Low Impact Development (LID) tool in PCSWMM, requiring a number of different parameters for each practice. All BMPs were implemented as separate sub-catchments in order to be able to specify a pollutant removal rate exclusive to each BMP. Clogging and long-term simulations were not considered. The design configurations for the model are shown in Table 5.

Table 5. BMP parameters used in the Personal Computer Storm Water Management Model (PCSWMM) [2,7,8,10,11,12,13,14].

Layer	Parameter	RG	GR	PP	Br
Surface	Berm Height (mm)	250	5	2.5	-
	Vegetative Cover Fraction	0.1	0.1	0	-
	Surface roughness (Manning)	0	0	0.03	-
	Surface slope (%)	0	0	2	-
Pavement	Thickness (mm)	-	-	100	-
	Void ratio	-	-	0.165	-
	Impervious surface fraction	-	-	0	-
	Permeability (mm/h)	-	-	36000	-
	Clogging Factor	-	-	0	-
Soil	Thickness (mm)	600	150	-	-
	Porosity	0.453	0.453	-	-
	Field Capacity (fraction)	0.19	0.19	-	-
	Wilting Point (fraction)	0.085	0.085	-	-
	Saturated Conductivity (mm/h)	10.92	10.92	-	-
	Conductivity curve slope	7.5	7.5	-	-
	Suction head (mm)	110	110	-	-
Storage	Height (mm)	400	-	700	1300
	Void ratio	0.4	-	0.4	-
	Filtration rate (mm/h)	1	-	1	-
	Clogging Factor	0	-	0	-
Drainage Mat	Thickness (mm)	-	25.4	-	-
	Void ratio	-	0.33	-	-
	Surface roughness (Manning)	-	0.2	-	-
Drain	Drain Coefficient	1.47	-	1.1	3.0
	Drain Exponent	0.5	-	0.5	0.5
	Drain offset height (mm)	0	-	0	0
	Drain Delay (h)	-	-	-	24

Table 5. BMP parameters used in the Personal Computer Storm Water Management Model (PCSWMM) [2,7,8,10,11,12,13,14].

Layer	Parameter	RG	GR	PP	Br
Surface	Berm Height (mm)	250	5	2.5	-
	Vegetative Cover Fraction	0.1	0.1	0	-
	Surface roughness (Manning)	0	0	0.03	-
	Surface slope (%)	0	0	2	-
Pavement	Thickness (mm)	-	-	100	-
	Void ratio	-	-	0.165	-
	Impervious surface fraction	-	-	0	-
	Permeability (mm/h)	-	-	36000	-
	Clogging Factor	-	-	0	-
Soil	Thickness (mm)	600	150	-	-
	Porosity	0.453	0.453	-	-
	Field Capacity (fraction)	0.19	0.19	-	-
	Wilting Point (fraction)	0.085	0.085	-	-
	Saturated Conductivity (mm/h)	10.92	10.92	-	-
	Conductivity curve slope	7.5	7.5	-	-
	Suction head (mm)	110	110	-	-
Storage	Height (mm)	400	-	700	1300
	Void ratio	0.4	-	0.4	-
	Filtration rate (mm/h)	1	-	1	-
	Clogging Factor	0	-	0	-
Drainage Mat	Thickness (mm)	-	25.4	-	-
	Void ratio	-	0.33	-	-
	Surface roughness (Manning)	-	0.2	-	-
Drain	Drain Coefficient	1.47	-	1.1	3.0
	Drain Exponent	0.5	-	0.5	0.5
	Drain offset height (mm)	0	-	0	0
	Drain Delay (h)	-	-	-	24

The pollutant removal rates achieved by the different BMPs are still a subject of debate. Performance can vary over time, and from place to place, with no universally agreed values [15]. In some BMPs, like green roofs, nutrients might leech under certain conditions, and it is therefore best recommended to assume that no removal of these pollutants is achieved [16]. The values used in this case for all BMPs are shown in Table 6.

Table 6. BMP pollutant removal efficiencies (%) [2,10,16,17].

BMP	TSS	TN	TP
RG	80	60	50
GR	85	0	0
PP	65	25	25
Br	0	0	0

Table 6. BMP pollutant removal efficiencies (%) [2,10,16,17].

BMP	TSS	TN	TP
RG	80	60	50
GR	85	0	0
PP	65	25	25
Br	0	0	0

The BMPs were implemented in the locations specified in the preliminary analysis, using the maximum area available. The individual performances of the BMPs are shown in Figure 4. Firstly, it can be seen that none of the alternatives had significant impact on the response time to attain the peak flow at the outfall. This could be explained by the fact that the four BMPs selected are only capable of on-site control, and while they could help mitigate peak flows and volume, none of them provide conveyance control that could significantly slow the flow before reaching the outfall.

Figure 4. Single BMP implementation performances.

Figure 4. Single BMP implementation performances.

The installation of the green roof had very little impact on the overall watershed response since it was installed on a relatively small area only. Considering that rain barrels have a fixed maximum retention volume (regardless of the rain event), the control provided was relatively small for a strong storm like the one used in the model. The implementation of pervious pavement in all driveways had a much larger impact than the previous two, reducing both total volumes and peak flows. This could be due to the great depth of the storage layer, and the extremely high permeability of the pavement layer that allowed the water to easily enter the storage layer. For similar reasons, rain gardens also provided high levels of control. However, they also showed a second peak. This phenomenon could be due to the permeability of the soil layer. As the rain garden’s surface filled rapidly, the overflow was conveyed quickly across the network (resulting in the first peak). Some time later, the flow that remained in the rain garden finally reached the storage layer after passing through the soil layer and was collected by the underdrain and then conveyed across the network (resulting in the second peak).

The addition of a second BMP control provided results (shown in Figure 5) that are coherent with those obtained for single BMPs. The combination of barrels with green roofs provided minimal control. This is to be expected, since their individual contributions were relatively small, and did not interact together in the watershed. However, it is possible to see improved performance by having two BMPs in series (i.e., gardens plus pavements). The combination of pervious pavement and the rain gardens provided the maximum benefits, seeing as they both provided good individual control, and can work in series.

Figure 5. Performances of combinations of two BMPs.

Figure 5. Performances of combinations of two BMPs.

Adding a third or a fourth BMP did not significantly improve the performance of the network when compared to some of the two BMP combinations, as shown in Figure 6. Particularly, having all four BMPs was practically the same as having any of the combinations including rain gardens and pervious pavement together (with all of these curves nearly fully overlapping), which were the two BMPs providing the greatest control. Most importantly, the results seem to indicate that the benefits provided by an additional BMP are not a simple addition of the individual results. The performance of the combination may not only depend on the individual contributions, but also on how well the control measures work together, thus resulting in a non-linear relationship.

Figure 6. Performances of combinations of three and four BMPs.

Figure 6. Performances of combinations of three and four BMPs.

The levels of pollutant removal are shown in Table 7. Concerning individual contribution, it can be seen that the performance of each alternative is a result of both the potential treatment provided, and the reductions in runoff volumes. Green roofs, with low levels of treatments and volume reductions, provided minimal water quality control. Rain barrels showed greater runoff reductions, but without treatment, resulting in modest removal rates. Rain gardens and pervious pavement can potentially achieve both high levels of treatment and runoff reduction, resulting in the greatest removal rates.

Table 7. Pollutant removal rates achieved.

Alternative	TSS Removal (%)	TN Removal (%)	TP Removal (%)
Br	9.42	9.35	9.25
RG	70.00	73.44	76.51
GR	1.17	2.83	2.79
PP	66.82	62.54	58.00
Br + RG	70.06	73.52	76.65
Br + GR	10.59	12.17	11.89
Br + PP	67.36	64.00	60.06
RG + GR	69.75	73.31	76.36
RG + PP	81.83	85.39	87.81
GR + PP	67.97	65.33	60.65
Br + RG + GR	69.81	73.39	76.51
Br + RG + PP	81.91	85.49	87.96
Br+ GR + PP	68.51	66.79	62.70
RG + GR + PP	81.65	85.26	87.67
Br + RG + GR + PP	81.65	85.33	87.67

Table 7. Pollutant removal rates achieved.

Alternative	TSS Removal (%)	TN Removal (%)	TP Removal (%)
Br	9.42	9.35	9.25
RG	70.00	73.44	76.51
GR	1.17	2.83	2.79
PP	66.82	62.54	58.00
Br + RG	70.06	73.52	76.65
Br + GR	10.59	12.17	11.89
Br + PP	67.36	64.00	60.06
RG + GR	69.75	73.31	76.36
RG + PP	81.83	85.39	87.81
GR + PP	67.97	65.33	60.65
Br + RG + GR	69.81	73.39	76.51
Br + RG + PP	81.91	85.49	87.96
Br+ GR + PP	68.51	66.79	62.70
RG + GR + PP	81.65	85.26	87.67
Br + RG + GR + PP	81.65	85.33	87.67

Concerning the combinations of BMPs, it can be seen that any combination having both rain gardens and pervious pavement provided the highest levels of removal, a similar trend to the water quantity control. However, it should be noted that adding another BMP did not always result in higher removal rates (i.e., rain gardens combined with green roofs, versus rain gardens alone), even though the differences were minimal. This could be due to the different concentration of pollutants found in each land-use. Ultimately, the BMPs can only provide treatment for flow passing through them. Implementing a control measure that slows a heavily polluted flow before it reaches the desired BMP for treatment might delay it in excess, reaching the BMP after it has reached its full capacity, therefore overflowing without receiving any treatment whatsoever. This situation then poses the challenge of strategic BMP implementation, ensuring the more polluted flows reach the BMPs faster, resulting in more efficient treatment. However, this kind of analysis is beyond the scope of this project and is left for future research opportunities.

In terms of economic performance, the Sustainable Technologies Program (STEP) tool developed by the Toronto and Region Conservation Authority [18] was used for a timespan of 25 years with a 2% inflation rate and 3% discount rate. Pervious asphalt was not included in the STEP tool, so values were used from the literature. Cahill [10] presented the following costs for porous asphalt:

• Demolition Cost: $3.75 per ft²
• Subbase and excavation: $1.88 per ft²
• Pavement installation: $1.87 per ft²
• Annual Maintenance Cost: $0.04 per ft²

3.4. Stakeholder Analysis

As recommended in the Methodology, three stakeholder groups were sought: land developers and planners, engineers, and ordinary citizens. Surveys were applied sharing the same overall structure, but with small differences between stakeholders. Apart from basic questions on background information of the person in order to see if there were any particular factors influencing people’s response, the core of the survey consisted of a table in which the person specified the level of importance (5 level scale, where 5 is the maximum level of importance) for each criterion and subcriterion. The citizen’s survey included at the end a similar table to obtain the social performance scores for each of the four BMPs. The survey designs are included as supplementary material.

A number of engineering firms and planning bodies from different municipalities of the Greater Montreal Region were contacted and asked to answer the survey. Response levels were poor, with only seven engineers and three planners answering all questions. Concerning the citizens as stakeholders, this group should have ideally been composed of local residents directly impacted by the project. Due to time and resource constraints, a more formal and complete sampling process could not be carried out. Ultimately, the survey was converted to an online format and distributed via email. In total, 15 people answered the citizens’ survey.

In this case, to get a better representation of the majority of each stakeholder group, the mode values were used, which are shown in Table 8 and Table 9. When no single mode existed, the average was then retained. For the alternatives consisting of BMP combinations, the score of the social criteria was taken as a weighted average according to the area occupied by each practice. Due to the extremely limited sample size, no further conclusions were drawn concerning trends in the answers for each stakeholder group.

Table 8. Criteria level of importance.

Criterion	Citizens	Engineers	Planners
Technical Criteria	3	5	4
Peak flow reduction	4	4	4
Volume reduction	4	5	4
Peak flow delay	3	4	4
Water Quality Criteria	5	4	3
TSS Removal	4	4	3
TN Removal	4	3	2
TP Removal	4	3	2
Economic NPV	4	5	3
Social Criteria	3	5	3
Aesthetics	3	4	3
Acceptability	3	4	3
Life Quality Contributions	4	5	3
Sust. Dev. Contributions	4	5	4

Table 8. Criteria level of importance.

Criterion	Citizens	Engineers	Planners
Technical Criteria	3	5	4
Peak flow reduction	4	4	4
Volume reduction	4	5	4
Peak flow delay	3	4	4
Water Quality Criteria	5	4	3
TSS Removal	4	4	3
TN Removal	4	3	2
TP Removal	4	3	2
Economic NPV	4	5	3
Social Criteria	3	5	3
Aesthetics	3	4	3
Acceptability	3	4	3
Life Quality Contributions	4	5	3
Sust. Dev. Contributions	4	5	4

Table 9. BMP social performance scores.

Criterion	Barrels	Rain Gardens	Green Roof	Pervious Pavement
Aesthetics	0	4	3	2
Acceptability	2	3	4	2
Life Quality	2	3	3	2
Sust. Dev.	3	3	3	2

Table 9. BMP social performance scores.

Criterion	Barrels	Rain Gardens	Green Roof	Pervious Pavement
Aesthetics	0	4	3	2
Acceptability	2	3	4	2
Life Quality	2	3	3	2
Sust. Dev.	3	3	3	2

3.5. MCDA Method Application

Some final considerations were made before applying the MCDA methods. First, as seen in the hydrographs presented in Figure 4 to Figure 6, peak flow delay was practically the same for all alternatives, so it was ultimately dropped from the analysis. Additionally, the alternatives presenting flooding or surcharge were removed (rain barrels, green roofs, and the combination of both). The final performance matrix used is shown in Table 10.

Table 10. General performance matrix.

Alternative	Peak Flow Reduction (m3/s)	Volume Reduction (m3)s	TSS Removal (%)	TN Removal (%)	TP Removal (%)	NPV ($)	Aesthetics	Accep.	Life Quality	Sust. Dev.
RG	0.746	491	70.0	73.4	76.5	2,805,967	4.00	3.00	3.00	3.00
PP	0.706	44	66.8	62.5	58.0	866,135	2.00	2.00	2.00	2.00
Br + RG	0.746	607	70.1	73.5	76.7	3,686,044	3.94	3.00	2.99	2.99
Br + PP	0.709	126	67.4	64.0	60.1	1,746,212	1.98	2.01	2.00	2.00
RG + GR	0.746	523	69.7	73.3	76.4	3,172,915	3.74	3.00	3.00	3.26
RG + PP	0.900	515	81.8	85.4	87.8	3,672,103	2.83	2.41	2.41	2.41
GR + PP	0.739	75	68.0	65.3	60.6	1,233,082	2.19	2.19	2.19	2.39
Br + RG + GR	0.746	640	69.8	73.4	76.5	4,052,992	3.70	3.00	2.99	3.24
Br + RG + PP	0.900	624	81.9	85.5	88.0	4,552,180	2.81	2.42	2.41	2.41
Br + GR + PP	0.744	158	68.5	66.8	62.7	2,113,159	2.18	2.20	2.19	2.38
RG + GR + PP	0.900	547	81.6	85.3	87.7	4,039,050	2.85	2.49	2.49	2.61
Br + RG + GR + PP	0.900	657	81.7	85.3	87.7	4,919,127	2.83	2.49	2.48	2.61

Table 10. General performance matrix.

Alternative	Peak Flow Reduction (m3/s)	Volume Reduction (m3)s	TSS Removal (%)	TN Removal (%)	TP Removal (%)	NPV ($)	Aesthetics	Accep.	Life Quality	Sust. Dev.
RG	0.746	491	70.0	73.4	76.5	2,805,967	4.00	3.00	3.00	3.00
PP	0.706	44	66.8	62.5	58.0	866,135	2.00	2.00	2.00	2.00
Br + RG	0.746	607	70.1	73.5	76.7	3,686,044	3.94	3.00	2.99	2.99
Br + PP	0.709	126	67.4	64.0	60.1	1,746,212	1.98	2.01	2.00	2.00
RG + GR	0.746	523	69.7	73.3	76.4	3,172,915	3.74	3.00	3.00	3.26
RG + PP	0.900	515	81.8	85.4	87.8	3,672,103	2.83	2.41	2.41	2.41
GR + PP	0.739	75	68.0	65.3	60.6	1,233,082	2.19	2.19	2.19	2.39
Br + RG + GR	0.746	640	69.8	73.4	76.5	4,052,992	3.70	3.00	2.99	3.24
Br + RG + PP	0.900	624	81.9	85.5	88.0	4,552,180	2.81	2.42	2.41	2.41
Br + GR + PP	0.744	158	68.5	66.8	62.7	2,113,159	2.18	2.20	2.19	2.38
RG + GR + PP	0.900	547	81.6	85.3	87.7	4,039,050	2.85	2.49	2.49	2.61
Br + RG + GR + PP	0.900	657	81.7	85.3	87.7	4,919,127	2.83	2.49	2.48	2.61

Concerning the specific MCDA methods, the following considerations were made:

• AHP: the binary comparisons for criteria and sub criteria were based on levels of importance obtained from the surveys. A difference of 1 level was given a score of 3 in the fundamental scale, a score of 5 for 2 levels of difference, 7 for 3 levels and 9 for 4. The performance of alternatives underwent linear normalization to obtain values in the fundamental scale. The criteria priorities were used as weights for all other methods.
• ELECTRE III: The criteria thresholds were fixed by the authors, considering a preference threshold set at twice the indifference threshold, and the veto threshold set to at least three times the preference threshold [19]. The effects of these values are considered in the sensitivity analysis. The values used are shown in Table 11.
• PROMETHEE II: To keep as much consistency as possible with ELECTRE III, most criteria were treated as Type 5, with a linear increase in preference after the indifference threshold was crossed. For the social criteria, considering the scale used, only one threshold was deemed relevant, and so they were treated as Type 2 criteria. The threshold values themselves were the same as those used in ELECTRE III. The only “cost” criterion was the economic NPV.

Table 11. Criteria threshold values.

Criterion	Indifference (q)	Preference (p)	Veto (v)
Peak flow	0.03 m3/s	0.06 m3/s	0.18 m3/s
Volume	50 m3	100 m3	400 m3
TSS	3% removal	6% removal	18% removal
TN	3% removal	6% removal	18% removal
TP	3% removal	6% removal	18% removal
NPV	$150,000	$300,000	$3,000,000
Aesthetics	0.5	1	3
Acceptability	0.5	1	3
Life Quality	0.5	1	3
Sustainable Dev.	0.5	1	3

Table 11. Criteria threshold values.

Criterion	Indifference (q)	Preference (p)	Veto (v)
Peak flow	0.03 m3/s	0.06 m3/s	0.18 m3/s
Volume	50 m3	100 m3	400 m3
TSS	3% removal	6% removal	18% removal
TN	3% removal	6% removal	18% removal
TP	3% removal	6% removal	18% removal
NPV	$150,000	$300,000	$3,000,000
Aesthetics	0.5	1	3
Acceptability	0.5	1	3
Life Quality	0.5	1	3
Sustainable Dev.	0.5	1	3

4. Results

4.1. MCDA Rankings

The final rankings obtained are summarized in Table 12 for all stakeholders, citizens (C), engineers (E) and planners (P). It can be seen that, according to the citizen’s point of view (which prioritized water quality, followed by economic NPV), all methods except MTOPSIS ranked the combination of rain gardens and pervious pavement in first place. This alternative provided practically the highest pollutant removal rates, while still being much cheaper than other alternatives with similar performance. On the other hand, from the engineers’ perspective (which gave less importance to water quality), all methods pointed towards the solution of rain gardens. Rain gardens had the best social performance, the second best level of quantitative control, and were much cheaper than other alternatives with similar performances. Therefore, it could be said that the answers appear to be logical according to the stakeholder priorities and the performance values used.

Results for the planners were more varied, with no two MCDA methods indicating the same “best” alternative, where the greatest differences between rankings concerned the best-ranked solutions. Again, looking at the priorities shown by the stakeholder provides an idea as to reason of the results. Planners prioritized quantitative control, and gave the same level of importance to the other three main criteria, which ultimately led to a compromise as no alternative showed high performance in those three areas. This situation was dealt with differently in each MCDA since they have different axiomatic bases, and thus the final rankings were not the same.

Table 12. Multi-criteria decision aid (MCDA) rankings for all stakeholders.

Alternative	AHP			ELECTRE III (Descending)			ELECTRE III (Ascending)			PROMETHEE II			MTOPSIS
	C	E	P	C	E	P	C	E	P	C	E	P	C	E	P
RG	5	1	7	3	1	3	2	1	2	5	1	7	5	1	7
PP	11	11	12	5	2	8	3	2	3	9	10	12	1	5	12
Br + RG	7	3	8	8	2	3	10	1	4	7	2	5	8	3	1
Br + PP	12	12	11	9	4	8	7	6	8	11	11	11	3	6	11
RG + GR	6	2	6	4	7	4	5	3	7	6	3	8	6	2	4
RG + PP	1	6	3	1	2	1	1	2	1	1	5	4	7	9	8
GR + PP	9	9	10	6	3	3	6	5	5	10	9	10	2	4	10
Br + RG + GR	8	4	5	11	5	6	11	3	7	8	4	6	10	7	2
Br + RG + PP	3	8	4	7	4	2	7	5	1	3	7	2	11	11	3
Br+ GR + PP	10	10	9	12	8	7	8	7	5	12	12	9	4	8	9
RG + GR + PP	2	5	1	2	3	5	4	4	6	2	6	3	9	10	5
Br + RG + GR + PP	4	7	2	10	6	8	9	6	6	4	8	1	12	12	6

Table 12. Multi-criteria decision aid (MCDA) rankings for all stakeholders.

Alternative	AHP			ELECTRE III (Descending)			ELECTRE III (Ascending)			PROMETHEE II			MTOPSIS
	C	E	P	C	E	P	C	E	P	C	E	P	C	E	P
RG	5	1	7	3	1	3	2	1	2	5	1	7	5	1	7
PP	11	11	12	5	2	8	3	2	3	9	10	12	1	5	12
Br + RG	7	3	8	8	2	3	10	1	4	7	2	5	8	3	1
Br + PP	12	12	11	9	4	8	7	6	8	11	11	11	3	6	11
RG + GR	6	2	6	4	7	4	5	3	7	6	3	8	6	2	4
RG + PP	1	6	3	1	2	1	1	2	1	1	5	4	7	9	8
GR + PP	9	9	10	6	3	3	6	5	5	10	9	10	2	4	10
Br + RG + GR	8	4	5	11	5	6	11	3	7	8	4	6	10	7	2
Br + RG + PP	3	8	4	7	4	2	7	5	1	3	7	2	11	11	3
Br+ GR + PP	10	10	9	12	8	7	8	7	5	12	12	9	4	8	9
RG + GR + PP	2	5	1	2	3	5	4	4	6	2	6	3	9	10	5
Br + RG + GR + PP	4	7	2	10	6	8	9	6	6	4	8	1	12	12	6

All stakeholders used the same thresholds and performance values, and yet yielded very different results. This goes to show how sensitive the methods are to the priorities or criteria weights, confirming the need for a full sensitivity analysis of the input parameters.

4.2. Ranking Analysis and Comparison

The lack of agreement between methods and stakeholders complicated the final recommendations. As mentioned in Part 1, there is no definitive answer on how to integrate various MCDA methods, as they all could present different axiomatic bases, and it is up to the decision maker to decide how to best test the results [1]. In this case, three steps were taken: ranking correlation analysis, comparison of best-ranked alternatives, and a sensitivity analysis on the most important input parameters of the MCDA methods.

Ultimately, the methodology seeks to find a BMP alternative that is acceptable to all stakeholders. Thus, differences in rankings between each group should be relatively small. These comparisons require the calculation of rank correlations. In this case, Spearman’s rank correlation coefficient test (SCCT) was used, as it is relatively simple to calculate and to interpret. The values range from −1 to 1, where the absolute value of the coefficient shows the strength of the relationship between both rankings. It should be noted that when there are many alternatives sharing the same rank, an average of the ranks involved should be used [20]. Spearman’s coefficient $r_s$ is calculated between rankings $X$ and $Y$ using Equation (1).

$$r_s=1-\frac{6{{\displaystyle \sum }}_{i=1}^n{d}_i^2}{n\left(n^2-1\right)} d_i=y_i-x_i$$

(1)

where $n$ is the number of alternatives in each ranking and $d_i$ is the difference in rank of alternative $i$ in rankings $X$ and $Y$. The critical $r_s$ value for $n=12$ with a 0.05 level of significance is equal to 0.503 [20]. The results of the correlations are shown in Table 13.

Table 13. Spearman correlation results.

Stakeholders	AHP	E. Descending	E. Ascending	PROMETHEE II	MTOPSIS
Citizens–Engineers	0.54	0.63	0.39	0.54	0.59
Citizens–Planners	0.90	0.53	0.45	0.84	0.79
Engineers–Planners	0.48	0.44	0.44	0.40	0.07

Table 13. Spearman correlation results.

Stakeholders	AHP	E. Descending	E. Ascending	PROMETHEE II	MTOPSIS
Citizens–Engineers	0.54	0.63	0.39	0.54	0.59
Citizens–Planners	0.90	0.53	0.45	0.84	0.79
Engineers–Planners	0.48	0.44	0.44	0.40	0.07

It can be seen that the correlations between engineers and planners were weak across all methods, which might indicate future conflicts between stakeholders, and the decision makers should keep this in mind. Aside from this disagreement, AHP showed the strongest correlations. However, this alone is not enough to select it as the final MCDA to retain, especially when looking at the top choices the method presented across all stakeholders. From Table 12, it can be seen that there was no single BMP alternative consistently ranked among the top three choices with AHP. The same can be said for both PROMETHEE II and MTOPSIS (which also showed the weakest of all correlations, close to zero). ELECTRE III on the other hand, presented several alternatives that satisfied this condition. In the case of the descending distillations, the rain gardens and the combination of rain gardens and pervious pavement were both ranked among the top three across all stakeholders. The ascending distillations included both of these, plus the pervious pavement alternative as well. Despite the fact that the statistically significant correlations in ELECTRE III were weaker than those of AHP or PROMETHEE, it shows more promise as an overall MCDA method in this case because it provides alternatives that will most likely be accepted by the different stakeholders. The weaker correlations were mostly due to individual alternatives that were awarded ranks across a wider range of values, rather than a disagreement in the general trend of the ranking itself. However, the sensitivity analysis is still needed before issuing final recommendations.

Table 14. Main criteria range of level of importance.

Main Criterion	Max Level of Importance	Min Level of Importance
Technical	5	1
Water quality	5	1
Economic NPV	5	2
Social Performance	5	2

Table 14. Main criteria range of level of importance.

Main Criterion	Max Level of Importance	Min Level of Importance
Technical	5	1
Water quality	5	1
Economic NPV	5	2
Social Performance	5	2

To evaluate weight sensitivity, different scenarios were run, maximizing the importance of one of the four main criteria while minimizing the importance for the others. Table 14 shows the ranges of values found in the surveys for all main criteria (regardless of the stakeholder). For brevity, only the importance of the main criteria was modified, using the engineer’s rankings as a point of reference. The scenarios were analysed by looking at the number of ranks the alternatives moved across scenarios. The results are shown in Table 15.

Table 15. Rank variation results for weight sensitivity analysis.

Alternative Rank Variations	AHP	ELECTRE III (Des)	ELECTRE III (Asc)	PROMETHEE II	MTOPSIS
Minimum	4	2	1	5	5
Maximum	11	9	8	11	11
Average	7.58	5.33	4.92	7.92	8.25

Table 15. Rank variation results for weight sensitivity analysis.

Alternative Rank Variations	AHP	ELECTRE III (Des)	ELECTRE III (Asc)	PROMETHEE II	MTOPSIS
Minimum	4	2	1	5	5
Maximum	11	9	8	11	11
Average	7.58	5.33	4.92	7.92	8.25

For AHP, PROMETHEE II and MTOPSIS, it is clear that at least one alternative moved from top to bottom ranks depending on the scenario. In ELECTRE III, this also happened but was not as evident because of the incomparability that arose in the different scenarios (meaning that the actual bottom rank was not 12, but rather a smaller number). However, ELECTRE III showed the least variations, both by having the alternatives that moved the least, and the smallest variation average. It was considerably more stable when compared to AHP, PROMETHEE and MTOPSIS, which showed average rank variation ranges of 7.58, 7.92, and 8.25, respectively. Nevertheless, the analysis carried out clearly shows the importance of verifying and validating criteria priorities when proceeding in future applications.

ELECTRE III and PROMETHEE II could also be affected by the choice of threshold values. To analyse the methods’ sensitivity to these parameters, the concept of “family of base solutions” was used, as presented by Maystre et al. [3], which refers to all solutions satisfying a given condition. The engineers’ rankings were used as a reference point (the base solution). Having identified the choices that were consistently ranked at the top with ELECTRE III, the family of base solutions were all those in which rain gardens and the combination of rain gardens with pervious pavement remained ranked among the top three. For PROMETHEE II the condition was set simply to keep the top three choices from the engineer’s ranking.

Only the indifference threshold was modified, seeing as the preference and veto thresholds (where applicable) were set proportional to it. Criteria using the same measurement scales (like all pollutant removal rates) were modified together. The $q$ threshold was modified between 0 and the range of performance values of each set of criteria using the same scale. The ranges of threshold values resulting in rankings belonging to the family of base solutions are shown in Table 16.

Table 16. Elimination et Choix Traduisant la Réaité (ELECTRE) III and Preference Ranking Organization Method for Enrichment of Evaluation (PROMETHEE) II threshold sensitivity analysis.

Criteria Parameters	Peak Flow Reduction (m3/s)	Volume Reduction (m3)	Removal Rate (%)	Economic NPV ($)	Social Performance
Criteria weight	0.075	0.225	TSS: 0.06 TN, TP: 0.02	0.3	Aesth, Accep: 0.038 LQ, SD: 0.113
Initial Threshold value	0.03	50	3	150,000	0.5
Range of performance values	0.19	613	30	4,052,993	2.02
ELECTRE III Min acceptable value	0	20	0.03	60,000	0.3
ELECTRE III Max acceptable value	0.092	613	6.71	706,000	2.02
PROMETHEE II Min acceptable value	0	17.8	0	0	0
PROMETHEE II Max acceptable value	0.19	613	30	245,000	0.9

Table 16. Elimination et Choix Traduisant la Réaité (ELECTRE) III and Preference Ranking Organization Method for Enrichment of Evaluation (PROMETHEE) II threshold sensitivity analysis.

Criteria Parameters	Peak Flow Reduction (m3/s)	Volume Reduction (m3)	Removal Rate (%)	Economic NPV ($)	Social Performance
Criteria weight	0.075	0.225	TSS: 0.06 TN, TP: 0.02	0.3	Aesth, Accep: 0.038 LQ, SD: 0.113
Initial Threshold value	0.03	50	3	150,000	0.5
Range of performance values	0.19	613	30	4,052,993	2.02
ELECTRE III Min acceptable value	0	20	0.03	60,000	0.3
ELECTRE III Max acceptable value	0.092	613	6.71	706,000	2.02
PROMETHEE II Min acceptable value	0	17.8	0	0	0
PROMETHEE II Max acceptable value	0.19	613	30	245,000	0.9

Since the family of base solutions are not the same for ELECTRE III and PROMETHEE II, the differences in threshold ranges cannot be directly compared. Nevertheless, several conclusions can be drawn. PROMETHEE II in general showed a large tolerance to fluctuations of the indifference threshold, being completely insensitive to changes in it (meaning that $q$ could take any value between 0 and the range of performance values) for the water quality criteria, and the peak flow reductions. When looking at the levels of importance granted to these criteria, it can be seen that they corresponded to those granted smaller weights (between 0.02 and 0.075). On the other hand, NPV, which was the individual criterion with the highest weight, showed the smallest tolerance to changes in $q$, (the maximum acceptable value of $245,000 was only 6% of the range of performance values found for that criterion). This is to be expected, as small changes in the level of preferences in the criteria with the highest weights will have a greater influence in the net flows, and therefore the rankings.

Rankings can be greatly modified in ELECTRE III if any of the veto thresholds are crossed since the consequence of this action is independent of the criteria weights. Therefore, in general, the tolerance to changes in ELECTRE III was smaller than in PROMETHEE II. However, it should be noted that very high and very low levels of indifference thresholds could potentially increase incomparability. If $q$ is very high, most alternatives (if not all) will be considered indifferent to one another for that criterion. In this case, the impact on the final ranking will depend on the criterion weight. On the other hand, if $q$ is very low (and assuming the proportions between $q$, $p$ and $v$ remain constant), then it is possible that the veto threshold will most likely be crossed more times, decreasing the number of possible outranking relations. As such, acceptable values for $q$ in ELECTRE III tended to show both upper and lower limits.

In any case, it can be seen that both methods did present certain flexibility to the threshold values, with the initial thresholds set well within the limits of the tolerable values. Despite this flexibility, since the authors set the values used, it would be important to verify the stakeholders view on the matter. There is even the possibility that each stakeholder could present different threshold values. However, this is beyond the scope of this case study and is left for future applications of the proposed methodology.

In light of the MCDA results and the sensitivity analysis, a number of final recommendations can be made. ELECTRE III proved to be the most robust of the methods used, showing the least variations when modifying criteria weights. Additionally, it was the only method that presented several alternatives consistently ranked among the top three. As such, it would seem to be the method best adapted to the methodology proposed in this work. The final BMP recommendations are, according to the results from ELECTRE III, rain gardens or the combination of rain gardens and pervious pavement. Though not always the first choice, they appeared to be the alternatives most likely to be accepted by all stakeholders. In this case, social participation workshops could be helpful to bring the stakeholders together, presenting a reduced set of alternatives (including those previously mentioned) to simplify the process.

5. Discussion

It should be noted that the rankings themselves are not a definitive answer. As the name implies, the methods are decision aid tools, and still require judgment and critical interpretation from the decision makers. The results should be used with caution, recognizing that real preferences from the stakeholders may differ. However, the rankings are useful to help reach a final solution between the stakeholders more quickly and effectively by knowing where to focus people’s attention. As mentioned in Part 1, the inclusion of the different stakeholder priorities could be a step towards a more sustainable land development, albeit resulting in a more complex decision process [1]. It is entirely possible that the stakeholders might disagree completely on which alternative to implement, and the decision maker should exert caution if it becomes necessary to bring them together to find common ground, particularly to keep a balance of power between stakeholders.

Concerning the results of the case study, other researchers have reached similar conclusions concerning the robustness of ELECTRE III, like Chitsaz and Banihabib [21] and Maté Marín [22], who found it to be the MCDA method least sensitive to criteria weights among those analysed by each author. Even though ELECTRE III was recommended as the method of choice to retain, the other MCDA methods could also show potential applications.

AHP, though less robust than ELECTRE III, showed higher correlations between stakeholders, and presented several shared top choice alternatives between some of them. However, the complete aggregation component of the method could limit its use. If the stakeholders allow for compensation between the criteria analysed and no specific thresholds need to be met, then it could be a relatively easy method to apply. PROMETHEE II is also simple to use. Despite the fact that it showed more sensitivity to the criteria weights, it showed some flexibility concerning the threshold values used. Therefore, if the decision makers are relatively certain about the criteria weights, it could be a useful choice of MCDA. Additionally, its partial aggregation component could present an advantage over others, like AHP, for instance. Finally, MTOPSIS proved to be sensitive to criteria weights and often disagreed with the other methods. The different axiomatic bases it uses (like the way it normalizes performance values) led to very different rankings not only as compared with other methods, but also between stakeholders, as shown by the correlations calculated and the weight sensitivity analysis. Nevertheless, it is the simplest of the four methods used and requires the least amount of user input, only needing the performance values and the criteria weights. However, if sufficient information were available on the levels of preference, thresholds or other parameters, it would be recommended to use a different MCDA method.

The results and conclusions drawn from this case study should only be used recognizing its different limitations and considerations made. Since the quality of information on alternative performance and stakeholder priorities could impact the final results, it is important to be as transparent as possible concerning the different assumptions made for the application of the methodology when communicating results.

Concerning this case study, limited information was available on the existing network, so a modified version had to be used (which limits direct applicability of the results found). Further verification of on-site data would be needed to have a formal calibration of the model, particularly the infiltration parameters and soil characteristics, as these can vary greatly from one place to another and have a large impact on the suitability of BMPs and their sizing. The modeling itself is another issue that should be addressed. Even though SWMM and PCSWMM are commonly used for stormwater management, they do present limitations, especially concerning water quality. The performance of the different BMPs was also simplified for this study. Real removal efficiencies could vary from place to place and will depend on final design specifications. The model considered that all flow passing through a specific BMP received the same continuous and homogenous treatment, which is fairly unrealistic. For this report, only EMCs where considered, which ignores a number of real processes that affect pollutant build up and wash off. Winter pollutant build up could be particularly important for the case of Quebec and other places presenting cold climate [2]. There is also the issue of continuous modeling for evaluating long-term performance, seeing as single rain events cannot show the effects of the natural ageing process of the network (including the BMPs) and the performance losses this implies. Additionally, a chain of flow was assumed for the BMPs working in series. The feasibility of such flow path would need to be verified. Since only a preliminary design was considered, the performance values of the economic criterion remain a rough estimate, especially since economic benefits were not included.

The survey design is important as well. A compromise might be needed between simplicity and the quality of information derived from them. In this case, the questions were chosen having the MCDA methods in mind already, but the design still presented limits and obstacles. In particular, the evaluation of social performance needs work so that it becomes more objective. For instance, what one person believes to be “strong aesthetic contributions” could very well be different from what someone else thinks.

Future work and research opportunities could address the issues previously mentioned. Additionally, concerning the methodology itself, both sizing and emplacement of BMPs were kept relatively simple, taking only a small number of alternatives and possible combinations, but with no optimization. This presents the possibility of coupling at some point multi-criteria analysis with multi-objective optimization in an improved methodology, as discussed in Part 1 [1].

6. Conclusions

Best Management Practices could help restore the natural hydrological cycle after urbanization. The performance of the different BMPs varies over a large range of criteria, making their selection a complicated process, possibly requiring compromise between different criteria. This work aimed to present a methodology for selecting BMPs based on MCDA methods, considering various stakeholder views and priorities, while seeking to obtain solutions under a sustainable development paradigm.

The proposed methodology comprises six steps, beginning with the problem definition, and continuing with the preliminary site analysis, analysis of alternatives, stakeholder analysis, application of the MCDA methods, and the final analysis and recommendations.

The methodology was tested on a case study of a residential stormwater network in the Greater Montreal Area. The case study evaluated green roofs, rain gardens, rain barrels and pervious pavement over a range of economic, social, water quality and water quantity criteria by applying AHP, ELECTRE III, PROMETHEE II and MTOPSIS under three different stakeholder views.

The results showed differences between MCDA methods and stakeholders, with initially no clear general preference of BMP alternative. Further evaluation of the MCDA method correlations, comparisons of the top choices and the sensitivity analysis provided better bases for recommendations. The analysis suggested ELECTRE III to be the most appropriate method for the proposed methodology, presenting flexibility (to a certain degree) in the threshold values, criteria weights, and the only method to show shared top choices across stakeholders (rain gardens, and rain gardens with pervious pavement). However, other possible application cases were suggested for the other MCDA methods.

This study explored the impact of BMP implementation in urban watersheds. In order to address the possible differences between case studies, it is important to present a methodology stating a clear sequence of actions to be taken, but also permitting great flexibility in order to adapt the procedure to the specific case study, which is the main contribution of this work. The results obtained show potential for more formal application as well as more research opportunities.