Field Evaluation of a Stormwater Treatment Train with Pit Baskets and Filter Media Cartridges in Southeast Queensland

Abstract

Field monitoring of a stormwater treatment train has been underway between November 2013 and May 2015 at a townhouse development located at Ormiston, southeast Queensland. The research was undertaken to evaluate the effectiveness of a 200 micron mesh pit basket in a 900 square format and an 850 mm high media filtration cartridge system for removing total suspended solids and nutrients from stormwater runoff. The monitoring protocol was developed with Queensland University of Technology (QUT), reflecting the Auckland Regional Council Proprietary Device Evaluation Protocol (PDEP) and United States Urban Stormwater BMP Performance Monitoring Manual with some minor improvements reflecting local conditions. During the 18 month period, more than 30 rain events have occurred, of which nine comply with the protocol. The Efficiency Ratio (ER) observed for the treatment devices are 32% total suspended solids (TSS), 37% for total phosphorus (TP) and 38% total nitrogen (TN) for the pit basket, and an Efficiency Ratio of 87% TSS, 55% TP and 42% TN for the cartridge filter. The performance results on nine events have been observed to be significantly different statistically (p < 0.05) for the filters but not the pit baskets. The research has also identified the significant influence of analytical variability on performance results, specifically when influent concentrations are near the limits of detection.

1. Introduction

The release of the Queensland State Planning Policy (SPP) requires local planning schemes to integrate the state’s interest in water quality by applying stormwater management objectives relevant to the climatic region, or demonstrating current best practice environmental management for urban developments. The SPP seeks to facilitate innovative and locally appropriate solutions to achieve the stormwater management design objectives typically 80% total suspended solids (TSS), 60% total phosphorus (TP), and 45% total nitrogen (TN) [1].

Several documents have been released in Australia over the past decade providing guidance on the design, modelling, construction, implementation and maintenance of stormwater quality management measures to achieve these objectives [2,3,4]. These guidelines have typically focussed on the constructed “natural” treatment measures including swales, biofiltration and wetlands.

Few of the guidelines include sections for demonstrating the performance of other types of stormwater treatment solutions. When compared with international evaluation protocols, it is apparent that the necessary detail to demonstrate performance to local conditions is omitted from these local guidelines. This paper presents a protocol developed for local Australian conditions by local universities in conjunction with SPEL Environmental (SPEL), a stormwater technology supplier, and applies it to testing an innovative stormwater treatment train in southeast Queensland, with discussion of the performance results observed.

2. Local Field Testing Site Details

Testing has been under way for more than 18 months at a townhouse complex at Ormiston, Queensland. The site is about 28 km east of the Brisbane Central Business District. Runoff from the site enters the local drainage network via grated inlets and is transported to an underground chamber for further treatment and detention prior to its discharge into the Council network. The site has a total area of 2028 m² with approximately 1140 m² of roof area (56%), 500 m² of impervious driveway (25%) and the balance, 388 m² (19%) of pervious area. The stormwater treatment train includes rainwater tanks for roof water, 900 mm square pit baskets (also known as catch basin inserts) with a 200 micron mesh bag in each of the gully pits (catch basins), and an underground vault with two 850 mm high media cartridge filters. The surface runoff from the site drains through the pit baskets into the pipe network, whereas the roofwater overflow from the rainwater tanks enters the pipe network beneath the pit baskets. This configuration is a typical stormwater treatment train for a medium- to high-density residential development in southeast Queensland. The site is also representative of typical applications for the pit basket and media filter treatment train.

The pit baskets are designed to capture the gross pollutants and coarse sediment leaving the pervious and impervious ground surfaces and are installed in the gully pits (catch basins). Figure 1 shows a plan view of a typical installation. The cartridge filters utilise a perlite, zeolite and activated alumina media to provide physical filtration and adsorption of stormwater pollutants, including nutrients. Overflow from the small rainwater tanks (3 kL per dwelling) enters the pipe drainage network beneath the pit baskets, and hence will provide significant dilution to the stormwater water quality exiting the pit baskets. Figure 2 is a photograph of the monitoring site. Figure 3 is a schematic of the catchment, and Figure 4 is a schematic cross-section of the filter vault and monitoring installation.

Figure 1. Typical pit basket (catch basin insert) installation plan view (a); side view (b).

Figure 1. Typical pit basket (catch basin insert) installation plan view (a); side view (b).

Figure 2. Townhouse development at Ormiston, QLD, showing driveway area, landscaping and filter cartridge for installation.

Figure 2. Townhouse development at Ormiston, QLD, showing driveway area, landscaping and filter cartridge for installation.

Runoff samples are collected by four ISCO GLS auto-samplers at the locations shown in Figure 4. Runoff is sampled as it leaves the driveway surface and enters the pit basket (1). A second sampler to determine the water quality of runoff in the conveyance pipe is installed because the roofwater enters the pipe drainage network beneath the pit baskets and provides dilution to the surface runoff. A third sampler collects water from the outlet pipe of the cartridge media filters, which are located upstream of the 850 mm baffle wall in the detention chamber (3). A fourth sampler collects filtered water from a tray beneath the pit basket (4). A photograph of the StormSack collection point is presented in Figure 5.

Figure 3. Schematic of the runoff pathways for the townhouse development at Ormiston. The red arrows indicate stormwater runoff entering the pit baskets. Excess rainfall (blue lines) enters the central drainage pipe, which then passes into the filter vault.

Figure 3. Schematic of the runoff pathways for the townhouse development at Ormiston. The red arrows indicate stormwater runoff entering the pit baskets. Excess rainfall (blue lines) enters the central drainage pipe, which then passes into the filter vault.

Figure 4. Schematic cross section of the filter vault with the stormwater treatment system.

Figure 4. Schematic cross section of the filter vault with the stormwater treatment system.

Figure 5. StormSack sample collection photograph (inlet circled).

Figure 5. StormSack sample collection photograph (inlet circled).

3. Local Field Testing Methodology

Due to the lack of formalised testing protocols in Australia, the University of the Sunshine Coast (USC), Queensland University of Technology (QUT), Griffith University (GU) and SPEL have formulated testing protocols based on the Auckland Council Proprietary Device Evaluation Protocol (PDEP), Washington Department of Ecology (WDOE) and Stormwater Equipment Manufacturers Association (SWEMA) protocols [5,6,7]. The protocols have been formalised to deliver a robust, scientifically defensible outcome. Even so, the protocols developed at the initiation stage have needed refinement once actual site data was observed, influenced by local hydrological conditions and equipment constraints. The protocol applied at the Ormiston test location, and monitored by QUT is detailed in Table 1.

Much of these protocol criteria appear in the Stormwater Quality Improvement Device Evaluation Protocol (SQIDEP) released as a consultation draft by Stormwater Australia [8].

Table 1. QUT-SPEL Field Testing Protocol Requirements for Ormiston.

Parameter	Ormiston
Minimum Storm Duration	5 min
Catchment type	Medium density townhouse property
Stormwater Treatment Device Type	Full scale—200 micron mesh pit basket and radially-wound media filter combination
Target Number of Storm events	15
Minimum rainfall depth per event	5 mm
Minimum inter-event period	48–72 h, depending on influent concentrations > Limit of Detection (LOD)
Minimum hydrograph sampling	First 60% of hydrograph
Flow rates tested	At least 3 events >75% of the treatable flow rate (TFR) with 1 exceeding the TFR.
Minimum number of water sub-samples collected per event	Minimum 8 influent and 8 effluent subsamples for each event. (Based on advice from the laboratory regarding minimum sample amount)
Sampling method	Auto-sampler, flow-weighted in 1000 L intervals (pipe network) and 0.5 mm rainfall for pit basket samples
Data Management	Campbell Scientific CR800 Data logger with Ethernet Modem
Particle Size Distribution (PSD) analysis via Laser Diffraction	Continuously stirred, without chemical dispersion or sonication
Total Suspended Solids (TSS)	American Public Health Association (APHA) (2005) 2540 D [9]
Total Nitrogen and species (water samples only)	APHA (2005) 4500 N, APHA (2005) 4500 NH3, APHA (2005) 4500 NO3
Total Phosphorus and Orthophosphate (water samples only)	APHA (2005) 4500 P
pH and Electrical Conductivity (EC)	Handheld probe, calibrated to manufacturer’s specifications

Table 1. QUT-SPEL Field Testing Protocol Requirements for Ormiston.

Parameter	Ormiston
Minimum Storm Duration	5 min
Catchment type	Medium density townhouse property
Stormwater Treatment Device Type	Full scale—200 micron mesh pit basket and radially-wound media filter combination
Target Number of Storm events	15
Minimum rainfall depth per event	5 mm
Minimum inter-event period	48–72 h, depending on influent concentrations > Limit of Detection (LOD)
Minimum hydrograph sampling	First 60% of hydrograph
Flow rates tested	At least 3 events >75% of the treatable flow rate (TFR) with 1 exceeding the TFR.
Minimum number of water sub-samples collected per event	Minimum 8 influent and 8 effluent subsamples for each event. (Based on advice from the laboratory regarding minimum sample amount)
Sampling method	Auto-sampler, flow-weighted in 1000 L intervals (pipe network) and 0.5 mm rainfall for pit basket samples
Data Management	Campbell Scientific CR800 Data logger with Ethernet Modem
Particle Size Distribution (PSD) analysis via Laser Diffraction	Continuously stirred, without chemical dispersion or sonication
Total Suspended Solids (TSS)	American Public Health Association (APHA) (2005) 2540 D [9]
Total Nitrogen and species (water samples only)	APHA (2005) 4500 N, APHA (2005) 4500 NH3, APHA (2005) 4500 NO3
Total Phosphorus and Orthophosphate (water samples only)	APHA (2005) 4500 P
pH and Electrical Conductivity (EC)	Handheld probe, calibrated to manufacturer’s specifications

The sampling program listed in Table 1 is triggered by two criteria. Firstly, >2 mm of rainfall over a rolling 30-min window must occur, based on field experience. This was programmed into the datalogger, to ensure sufficient water depth was available in the pipe to collect samples. Rainfall is measured onsite by a 0.2 mm waterlog tipping bucket rain gauge. The second criteria is flow volume, where a sample is initiated after 1000 L of stormwater discharge past each of the two pipe sampling points shown in Figure 4. Flow rate/volume was measured by two Starflow ultrasonic probes installed at the inlet and outlet pipes of the concrete chamber shown in Figure 4. For the pit basket where flow measurement was impracticable, sample intervals were triggered at 0.5 mm rainfall intervals. As the basket effectively has zero residence time, the inlet and outlet samples were triggered simultaneously.

Ultrasonic probes were selected for flow measurement due to a reported accuracy of ±2% for flow and ±0.25% for depth [10]. This accuracy is comparable to flumes and weirs but without the associated interference with water quality, especially TSS, observed with the latter.

A 1000 L volume of water was chosen as the sampling interval as this is 50% of the cartridge vault volume. This volume also corresponded to 0.5 mm of runoff over the site, assuming zero losses. Analysis of a smaller flow volume trigger indicated that it could challenge the physical limitations of the samplers’ purge/collection cycle (about a 90 s cycle). All the subsamples collected during a runoff event were composited within the sampler in a 9 L bottle. Each subsample was 200 mL to ensure sufficient volume was available for the suite of subsequent chemical analyses (listed in Table 1). This flow-weighted sampling protocol provides an Event Mean Concentration (EMC).

As has been noted previously [11], the physical limitations of the equipment and analysis process can subsequently affect the protocol. Therefore, any nominated protocol needs flexibility to respond to these potential constraints. For example, to collect eight subsamples practically restricts the minimum time for a “qualifying” storm to greater than 12 min, even though flow may occur quicker. Hence, for this site, a storm event less than ten minutes duration is unlikely to provide sufficient time to collect eight aliquots even if sufficient volume were present. As the project progressed, the laboratory advised that analyses could be performed on much smaller volumes, thereby permitting as few as three aliquots to be sufficient from short duration events. The intent of the monitoring program, however, is to collect a spread of subsamples across the hydrograph of every event regardless of the duration.

On the other hand, a maximum number of subsamples can be collected before the container is full, and therefore an analysis of the likelihood of rainfall events exceeding the maximum capacity of the containers was undertaken to identify the likely upper event size. As the ISCO sampler can collect a maximum of 9 L of sample, 45 sub samples, each of 200 mL, are possible. For Ormiston, this equates to approximately 22.5 mm total rainfall. Statistical analysis of rainfall events for Brisbane between July 2000 and July 2010 (assuming no runoff losses) indicates that this 9 L capacity would allow capture of >90% of the daily runoff events.

The inter-event period (antecedent duration) was set in the protocol to 48–72 h between rainfall events, as previous QUT research into pollutant build-up and wash-off on urban surfaces indicated that this was the optimal point at which pollutants reach a detectable level in runoff [12,13]. This research has shown that low intensity, low-volume events do not produce detectable concentrations for antecedent periods (ADP) less than 72 h. However, the Ormiston project has shown that high intensity events with less than 72 h ADP can produce detectable concentrations. Therefore, the protocol has been adjusted to include events where ADP might be <72 h, if pollutant concentrations are measurable. The minimum rainfall depth for a qualifying storm will vary between monitoring sites, depending on the catchment characteristics. For the medium density Ormiston townhouse site with a high fraction of impervious area, the minimum daily rainfall for monitoring has been set to 5 mm, as this is the level at which observable runoff can be measured. Other monitored sites with larger, more pervious catchments will require more rainfall to produce sufficient runoff for sampling. Therefore, we caution against setting a rigid minimum rainfall volume for qualifying storms in monitoring protocols, as this is inherently site-specific.

The draft SQIDEP also requires a minimum of three flow events >75% of the maximum treatable flow rate (TFR), with at least 1 event greater than the TFR [8]. It should be noted that a requirement for all events be at the TFR, or >75% of the TFR, may be statistically rare. For example, an evaluation of the hydrology for the Ormiston site across 10 years of historical data, indicates that this may be achieved less than three times annually. Hence to achieve 15 qualifying events at the TFR, would require a minimum of five years of sampling.

The monitoring equipment and sample collection were independently undertaken by staff from QUT, and analysed in NATA registered laboratories. Reports on the findings were prepared by QUT [14]. This maintains independence and integrity of the sampling, collection and analysis process. As there is a range of possible metrics used to assess performance data, this paper presents several of them.

Average Concentration Removal Efficiency (CRE) is calculated from the function:

$$Avg. CRE=\frac{\sum \left[\frac{\left\{EMCin-EMCout\right\}}{EMCin}\right]}{no. of events}$$

(1)

Efficiency Ratio (ER) is calculated from the function:

$$ER=1-\frac{Mean EMCout}{Mean EMCin}$$

(2)

To briefly paraphrase the above, the CRE is the average of the removal ratios (percentages) for every event, whereas the ER is the removal ratio of the average inflow and outflow concentrations for all events. The ER weights EMCs (flow-weighted concentrations) from all storms equally, regardless of the pollutant concentration or runoff volume, and minimises the potential impacts of smaller, cleaner events on performance calculations. The ER can, however, be influenced by a small number of high influent concentration events that skew average concentration results. Therefore, other metrics, including Average CRE, should also be considered [15]. The Average CRE quantifies the percent removal for each event, and calculates an average value of the percentages, allowing the smaller, cleaner events to have greater influence on the average CRE, and hence minimise the influence of the few, large influent concentrations.

4. Results and Discussion

A report on 18 months of monitoring has been released by QUT [14]. Of sixteen (16) captured rainfall events > 5 mm, nine events are qualifying. Where the results have been less than the limits of detection (LOD), they have been shown as 50% of the LOD. All events reported in this paper had flows >75% of the TFR, with one event exceeding the TFR.

Table 2 presents the water quality data observed at the pit basket (catch basin insert) and shows influent concentrations for TSS similar to those reported as typical by guidelines for urban residential catchments, whereas the TN and TP concentrations are mostly below guideline figures. The preliminary results indicate that the relatively simple 200 micron filter bag removes about 32% of the suspended solids and 37% and 38% of the TP and TN concentrations respectively based on the ER metric. The performance indicated by the CRE metric, is strongly influenced by very low inflow concentrations and slightly higher outflow concentrations, generating a negative ratio.

Water quality data from the media filter samples is presented in Table 3. It can be seen that the pollutant concentrations observed in the pipe inflow (inlet to the filters) is significantly lower than the pit basket outflow concentrations shown in Table 2. This is a direct result of stormwater dilution by overflow from the rainwater tanks entering the network at the base of the gully pits (catch basins). Even so, the data indicates that the filters are removing TSS, TN and TP, at very low concentrations. Mean ERs of 87%, 55% and 42% for TSS, TP and TN respectively are observed. Of particular note, the outflow TSS concentrations from the media filter are consistently below detection limits (<5 mg/L), for most events. Similarly the outflow TP concentrations are very close to the limits of detection.

Table 2. Pit Basket Water Quality Results.

Parameter	TSS		TN		TP
LOD (mg/L) 1	5		0.1		0.01
Event	In (mg/L)	Out (mg/L)	In (mg/L)	Out (mg/L)	In (mg/L)	Out (mg/L)
23 June 2014	10	NC 2	0.60	NC	0.04	NC
16 August 2014	122	40	0.90	0.40	0.13	0.10
18 August 2014	12	2.5	0.20	0.20	0.05	0.05
23 August 2014	9	2.5	0.20	0.10	0.05	0.15
26 September 2014	346	253	2.40	1.90	0.58	0.36
9 December 2014	202	186	3.85	2.20	0.40	0.10
18 December 2014	2.5	10	0.30	0.30	0.03	0.07
20 Febuary 2015	34	6	0.70	0.30	0.09	0.02
30 April 2015	90	58	0.90	0.50	0.13	0.07
Average Conc.	102.19	69.75	1.18	0.74	0.18	0.11
Median Conc.	34.00	25.00	0.70	0.35	0.09	0.08
Efficiency Ratio (Avg)	-	32%	-	38%	-	37%
Average CRE	-	9%	-	34%	-	−9%
Efficiency Ratio (Median)	-	26%	-	50%	-	7%
Median CRE	-	51%	-	44%	-	31%

Notes: ¹ LOD = Limits of Detection of the analytical method; ² As Outflow samples were not collected (NC) from this event, it has been excluded from the calculations.

Table 2. Pit Basket Water Quality Results.

Parameter	TSS		TN		TP
LOD (mg/L) 1	5		0.1		0.01
Event	In (mg/L)	Out (mg/L)	In (mg/L)	Out (mg/L)	In (mg/L)	Out (mg/L)
23 June 2014	10	NC 2	0.60	NC	0.04	NC
16 August 2014	122	40	0.90	0.40	0.13	0.10
18 August 2014	12	2.5	0.20	0.20	0.05	0.05
23 August 2014	9	2.5	0.20	0.10	0.05	0.15
26 September 2014	346	253	2.40	1.90	0.58	0.36
9 December 2014	202	186	3.85	2.20	0.40	0.10
18 December 2014	2.5	10	0.30	0.30	0.03	0.07
20 Febuary 2015	34	6	0.70	0.30	0.09	0.02
30 April 2015	90	58	0.90	0.50	0.13	0.07
Average Conc.	102.19	69.75	1.18	0.74	0.18	0.11
Median Conc.	34.00	25.00	0.70	0.35	0.09	0.08
Efficiency Ratio (Avg)	-	32%	-	38%	-	37%
Average CRE	-	9%	-	34%	-	−9%
Efficiency Ratio (Median)	-	26%	-	50%	-	7%
Median CRE	-	51%	-	44%	-	31%

Notes: ¹ LOD = Limits of Detection of the analytical method; ² As Outflow samples were not collected (NC) from this event, it has been excluded from the calculations.

Table 3. Media filter cartridge Water Quality Results.

Parameter	TSS		TN		TP
LOD (mg/L) 1	5		0.1		0.01
Event	In (mg/L)	Out (mg/L)	In (mg/L)	Out (mg/L)	In (mg/L)	Out (mg/L)
22 June 2014	2.72	2.50	0.70	0.60	0.02	0.01
16 August 2014	22.69	2.50	0.40	0.30	0.04	0.02
18 August 2014	24.95	2.50	0.39	0.20	0.06	0.04
23 August 2014	2.50	2.50	0.20	0.10	0.03	0.03
25 September 2014	74.39	2.50	0.75	0.30	0.08	0.02
8 December 2014	31.59	15	0.47	0.20	0.03	0.02
18 December 2014	2.50	2.50	0.44	0.20	0.02	0.01
20 Febuary 2015	20.65	2.50	0.41	0.40	0.01	0.01
30 April 2015	65.65	0.50 *	0.31	0.05	0.03	0.01
Average Conc.	24.7	3.67	0.45	0.26	0.04	0.02
Median Conc.	12.6	2.5	0.41	0.2	0.03	0.02
Efficiency Ratio (Avg)	-	87%	-	42%	-	55%
Average CRE	-	58%	-	44%	-	56%
Efficiency Ratio (Median)	-	89%	-	52%	-	33%
Median CRE	-	88%	-	49%	-	64%

Notes: ¹ LOD = Limits of Detection of the analytical method; * LOD = 1 mg/L for this event.

Table 3. Media filter cartridge Water Quality Results.

Parameter	TSS		TN		TP
LOD (mg/L) 1	5		0.1		0.01
Event	In (mg/L)	Out (mg/L)	In (mg/L)	Out (mg/L)	In (mg/L)	Out (mg/L)
22 June 2014	2.72	2.50	0.70	0.60	0.02	0.01
16 August 2014	22.69	2.50	0.40	0.30	0.04	0.02
18 August 2014	24.95	2.50	0.39	0.20	0.06	0.04
23 August 2014	2.50	2.50	0.20	0.10	0.03	0.03
25 September 2014	74.39	2.50	0.75	0.30	0.08	0.02
8 December 2014	31.59	15	0.47	0.20	0.03	0.02
18 December 2014	2.50	2.50	0.44	0.20	0.02	0.01
20 Febuary 2015	20.65	2.50	0.41	0.40	0.01	0.01
30 April 2015	65.65	0.50 *	0.31	0.05	0.03	0.01
Average Conc.	24.7	3.67	0.45	0.26	0.04	0.02
Median Conc.	12.6	2.5	0.41	0.2	0.03	0.02
Efficiency Ratio (Avg)	-	87%	-	42%	-	55%
Average CRE	-	58%	-	44%	-	56%
Efficiency Ratio (Median)	-	89%	-	52%	-	33%
Median CRE	-	88%	-	49%	-	64%

Notes: ¹ LOD = Limits of Detection of the analytical method; * LOD = 1 mg/L for this event.

As can be seen in Table 2 and Table 3, the ER and CRE metrics vary, though both use the same concentration data. This is the result of the two methods using different mathematical logic. For example, the pit basket result for TP on 23 August 14 indicates a CRE of −200% that subsequently causes the average CRE to be negative, even though all the other events show positive CRE values. Results near the limits of detection, such as that for 23 August 14, can skew the average CRE metric. A recorded inflow concentration of 0.01 mg/L, for example, and an outflow concentration of 0.02 mg/L will provide an individual CRE of −100% and influence the average CRE, yet be as a result of analytical error. Results on duplicate samples from Ormiston have been observed to differ by 0.3 mg/L for TN and 0.02 mg/L for TP, and result in a “theoretical export” of pollutants of ~200% for these very low influent concentrations. This large negative percent removal then has a knock-on effect on the average CRE value, and so we suggest CRE is not an appropriate metric when influent concentrations are close to the LOD.

We suggest that ER is the better metric for evaluation of this dataset. However, in the instances that high concentration influent outliers are recorded (for example, above the Water by Design MUSIC modelling guidelines [4]) as the dataset grows, we suggest that Average CRE, Comparison of Medians, and statistical analyses should all be used to validate performance. In the dataset observed by this research, there are no outliers based on the Water by Design MUSIC Modelling Guidelines for storm concentrations from an urban residential catchment [4]. In fact, the observed concentrations at Ormiston are low in comparison with the guideline values, as shown in Table 4. We therefore maintain that ER is the more suitable metric to be used at this point in time, for this site.

Table 4. Comparison of Ormiston Surface Water Quality Results with Brisbane MUSIC Guidelines for urban residential areas.

Parameter	MUSIC Guideline Values (Lumped Urban Residential Catchment) 1			Ormiston Surface Influent (Pit Basket Inflow)
	−1SD	Mean	+1SD	−1SD	Mean	+1SD
TSS (mg/L)	61.7	151	372	0	102.2	222.52
TP (mg/L)	0.162	0.339	0.708	0	0.182	0.380
TN (mg/L)	1.07	1.82	3.09	0	1.181	2.474

Note: ¹ Reference: [4].

Table 4. Comparison of Ormiston Surface Water Quality Results with Brisbane MUSIC Guidelines for urban residential areas.

Parameter	MUSIC Guideline Values (Lumped Urban Residential Catchment) 1			Ormiston Surface Influent (Pit Basket Inflow)
	−1SD	Mean	+1SD	−1SD	Mean	+1SD
TSS (mg/L)	61.7	151	372	0	102.2	222.52
TP (mg/L)	0.162	0.339	0.708	0	0.182	0.380
TN (mg/L)	1.07	1.82	3.09	0	1.181	2.474

Note: ¹ Reference: [4].

Significant debate continues as to the “best” method to calculate device performance. Statistical validation (Paired t-test) of the dataset is also recommended to confirm significant differences between the influent and effluent sample sets [5]. The Auckland PDEP also indicates that where the median and the mean of the performance metric (e.g., ER) vary by more than 10%, additional sampling events are recommended. The median concentrations presented in Table 2 and Table 3, when used to calculate an Efficiency Ratio, result in a difference of more than 10% for TN and TP when compared with a Mean Efficiency Ratio. In comparison, however, the Median CRE values appear to be generally consistent with the Average ER.

5. Normality and Log-Normality Tests

Environmental monitoring data is typically log-normally distributed, therefore, the data sets recorded for this site were evaluated against a normal distribution, in a log-transformed basis. The Anderson-Darling Normality test identifies normal distributions where the p-value is >0.05 (alpha). As can be seen from Table 5, all of the datasets are log-normally-distributed, except for TSS outflow and TP outflow on the media filters.

Table 5. Anderson-Darling Normality tests on log-transformed dataset, statistically significant results shown in bold.

Treatment Device	p-value
	TSS in	TSS out	TN in	TN out	TP in	TP out
StormSack	0.748	0.497	0.506	0.328	0.518	0.615
SPELFilter	0.054	0.005	0.418	0.413	0.906	0.015

Table 5. Anderson-Darling Normality tests on log-transformed dataset, statistically significant results shown in bold.

Treatment Device	p-value
	TSS in	TSS out	TN in	TN out	TP in	TP out
StormSack	0.748	0.497	0.506	0.328	0.518	0.615
SPELFilter	0.054	0.005	0.418	0.413	0.906	0.015

This is likely a result of the outflow concentrations converging on similar results (i.e., Below or at Detection limits). This information is relevant for confirming that a Student’s paired t test is a valid method to compare the influent and effluent datasets. Figure 6 presents the Q-Q plots for the StormSack pollutant data in log-transformed format. Figure 7 presents the Q-Q plots for the SPELFilter pollutant data in log-transformed format.

Figure 6. Q-Q Plots of log-normal distributions—Pit Basket (a) TSS Inflow; (b) TSS Outflow; (c) TP inflow; (d) TP outflow; (e) TN inflow; (f) TN outflow.

Figure 6. Q-Q Plots of log-normal distributions—Pit Basket (a) TSS Inflow; (b) TSS Outflow; (c) TP inflow; (d) TP outflow; (e) TN inflow; (f) TN outflow.

Figure 7. Q-Q Plots of log-normal distributions—SPELFilter Inflow and Outflow, TSS, TN and TP respectively. (a) TSS Inflow; (b) TSS Outflow; (c) TP Inflow; (d) TP Outflow; (e) TN Inflow; (f) TN Outflow.

Figure 7. Q-Q Plots of log-normal distributions—SPELFilter Inflow and Outflow, TSS, TN and TP respectively. (a) TSS Inflow; (b) TSS Outflow; (c) TP Inflow; (d) TP Outflow; (e) TN Inflow; (f) TN Outflow.

6. Statistical Significance Tests

To evaluate whether the data demonstrate that the treated flow is statistically different from the inflow, statistical tests were performed on the log-transformed datasets. The paired Student’s t test evaluates whether the two datasets have the same mean. Therefore, if the datasets are considered statistically to be significantly different, they are shown in bold below. As can be seen from t tests on the log-transformed data, all pollutants in and out of the media filter are statistically significantly different. The pit basket concentrations data is not statistically significantly different, according to this test. The t test results are presented in Table 6. Given the inherent variability of environmental data, a further statistical test was performed on the raw dataset. The Mann-Whitney (Wilcoxon) Rank-Sum test was performed on the raw datasets for the pit basket and media filter. As can be seen in Table 7, the media filter results are confirmed to be statistically significantly different, however, the pit basket datasets are not.

Table 6. Student’s t tests on log-transformed dataset, statistically significant results shown in bold.

Treatment Device	p-value (Two-Tailed)
	TSS	TN	TP
StormSack	0.117	0.006	0.412
SPELFilter	0.015	0.005	0.002

Table 6. Student’s t tests on log-transformed dataset, statistically significant results shown in bold.

Treatment Device	p-value (Two-Tailed)
	TSS	TN	TP
StormSack	0.117	0.006	0.412
SPELFilter	0.015	0.005	0.002

Table 7. Wilcoxon-Mann-Whitney Rank-Sum test on raw data, statistically significant results shown in bold.

Treatment Device	TSS		TN		TP
	p-value	Sig.	p-value	Sig.	p-value	Sig.
StormSack	0.496	no	0.429	no	0.711	no
SPELFilter	0.008	yes	0.021	yes	0.031	yes

Table 7. Wilcoxon-Mann-Whitney Rank-Sum test on raw data, statistically significant results shown in bold.

Treatment Device	TSS		TN		TP
	p-value	Sig.	p-value	Sig.	p-value	Sig.
StormSack	0.496	no	0.429	no	0.711	no
SPELFilter	0.008	yes	0.021	yes	0.031	yes

These results confirm international observations that environmental data may require large datasets that are economically unviable to demonstrate statistical significance [15]. Further, the concentrations observed on the catchment are beyond the control of the researcher. For example, an estimation of the number of samples required for a paired comparison on the pit basket dataset, as indicated by the equation described by Burton and Pitt [16] shown below, suggests that 160 samples are required for TSS, 103 samples are required for TP and 220 samples are necessary for TN.

$$n=2{\left[\frac{Z_{1-\text{α}}+Z_{1-\text{β}}}{{\text{μ}}_1-{\text{μ}}_2}\right]}^2{\text{σ}}^2$$

(3)

where n = number of sample pairs needed; α = false positive rate (1−α is the degree of confidence. A value of α of 0.05 is usually considered statistically significant, corresponding to a 1−α degree of confidence or 95%); β = false negative rate (1−β is the power. If used, a value of β of 0.2 is common but it is frequently ignored, corresponding to a β of 0.5); Z_1−α = Z score (associated with area under normal curve) corresponding to 1−α; Z_1−β = Z score corresponding to 1−β value; µ₁ = mean of dataset one; µ₂ = mean of dataset two; σ = standard deviation (same for both datasets, assuming normally-distributed).

SPEL and Drapper Environmental Consultants (DEC) are monitoring seven research sites across southeast Queensland, and have observed that for each qualifying event there are three others discarded for non-conformance with the protocol. Continuing a monitoring program to achieve 220 qualifying events required for statistical certainty (>600 events overall) would be financially prohibitive for any research program and delay outcomes for many years.

7. Conclusions

Evaluation of alternate stormwater treatment devices has been under way for decades internationally and, appears to be gaining momentum in Australia. While a number of existing guidelines stipulate that performance of alternate stormwater treatment devices must be demonstrated for local and regional conditions, the guidelines generally do not define how this should be accomplished. USC, QUT, GU, DEC and SPEL have worked together to adapt international protocols to suit local and regional conditions on a variety of sites and treatment measures in southeast Queensland. This paper details the protocol being implemented on one of the monitoring sites at Ormiston, Southeast Queensland. A report published by QUT on the nine complying events at the site indicate Efficiency Ratios of 32% TSS, 37% TP and 38% TN for the 900 square StormSack pit basket, and 87% TSS, 55% TP and 42% TN for the 85mm high, radially-wound, multi-media SPELFilter cartridge. Given the dataset analyses on the field testing of this treatment train indicates that the performance of the SPELFilter is statistically proven, and, when combined in a treatment train, it will comply with the QLD SPP water quality objectives of 80% TSS, 60% TP and 45% TN removal.