Modeling and Forecasting of nanoFeCu Treated Sewage Quality Using Recurrent Neural Network (RNN)

Abstract

Rapid industrialization and population growth cause severe water pollution and increased water demand. The use of FeCu nanoparticles (nanoFeCu) in treating sewage has been proven to be a space-efficient method. The objective of this work is to develop a recurrent neural network (RNN) model to estimate the performance of immobilized nanoFeCu in sewage treatment, thereby easing the monitoring and forecasting of sewage quality. In this work, sewage data was collected from a local sewage treatment plant. pH, nitrate, nitrite, and ammonia were used as the inputs. One-to-one and three-to-three RNN architectures were developed, optimized, and analyzed. The result showed that the one-to-one model predicted all four inputs with good accuracy, where R² was found within a range of 0.87 to 0.98. However, the stability of the one-to-one model was not as good as the three-to-three model, as the inputs were chemically and statistically correlated in the later model. The best three-to-three model was developed by a single layer with 10 neurons and an average R² of 0.91. In conclusion, this research provides data support for designing the neural network prediction model for sewage and provides positive significance for the exploration of smart sewage treatment plants.

1. Introduction

Water is a crucial resource for life. Population growth, expansion of irrigated areas, and industrial development increase the global demand for freshwater supply. The global demand for freshwater supplies has increased by more than 600% since the 1960s [1]. Consequently, freshwater shortages have become a threat to sustainable human development [2]. It is prevalent in developing countries because of the poor enforcement of environmental laws and the low awareness of freshwater protection. Water pollution exacerbates the shortage issue. The major causes of water pollution include chemical spills [3], illegal industrial wastewater discharge [4], rural aquaculture wastewater [5], and domestic sewage pollution [6].

Studies showed that sewage polluted the world’s coastlines. Cascade et al. [7] used a geospatial model to measure and map the nitrogen (N) and pathogen-fecal indicator organisms (FIO) found in human sewage in approximately 135,000 watersheds worldwide. The results show that 63% of the nitrogen in coastal waters, which was equivalent to 3.9 Tg N, comes from sewage systems. It affects the safety of seafood, and human pathogens in seafood can lead to outbreaks of foodborne disease [8]. Eventually, it would be a threat to biodiversity and ecosystem health. Efforts have been made by government bodies and researchers to overcome the water pollution issue, especially for sewage treatment. Generally, the sewage treatment process is classified into three stages (primary, secondary, and tertiary treatment) controlled by a combination of physical, chemical, and biological processes. The primary treatment mostly adopts physical methods to treat larger suspended solids and sand in sewage through grid retention, filtration, and sedimentation. This treatment technology is relatively mature. Secondary treatment typically involves biological treatment to remove dissolved and suspended biological matter, such as the activated sludge process (ASP). It is worth noting that the effluent quality of the secondary treatment was unstable due to the microbe’s sensitivity to changes in pH, dissolved oxygen, pollutant concentration, and temperature [9]. Tertiary treatment processes include physical-chemical treatment such as coagulation, flocculation, sedimentation, and filtration, as well as advanced oxidation processes, for instance, photocatalytic oxidation and electrochemical oxidation. Usually, tertiary treatment processes have the advantage of high treatment efficiency and reliable effluent quality, but the process is expensive and possibly causes secondary pollution, such as the degradation of photocatalysts and electrodes [10].

Thus, it is important to create cross-disciplinary synergies to develop innovative technology for sewage treatment to address space and cost-effectiveness issues [11]. Lately, Chan et al. [12] found that FeCu nanoparticles could be used for ammonia removal via the oxidation process. The results demonstrated that the immobilized FeCu nanoparticles exhibited good ammonia removal performance when handling 10–100 ppm of ammonia solutions. The reusability study revealed that the immobilized FeCu could be reused at least three times without deterioration [13]. Meanwhile, the pilot-scale studies also revealed the full potential of immobilized FeCu for sewage treatment [14]. Recently, with the development and availability of computing power and big data, artificial neural networks (ANN) have been successfully applied in many fields, especially sequential prediction, such as rainfall-runoff [15] and weather and hydrological time sequence prediction [16]. In water treatment plants, sensors are used for measuring water level, flowrate, and water quality [17]. This generated a pool of time series data that can be used to monitor treatment efficiency. It is noteworthy that the acquired data are related in complex and nonlinear ways [18]. It is impractical to manually compute and analyze the data. Hence, it is essential to incorporate machine learning into water research.

Table 1. Recent works on the use of machine learning to model and estimate the water quality.

Applications	Model Description	Variables	Results	Limitations	References
River water quality prediction	Combines auto-regressive integrated moving average (ARIMA) and clustering model	The water quality total phosphorus (TP)	Mean absolute error (MAE) = 0.0082	Inaccurate rainfall data will affect the model’s prediction accuracy.	[19]
Predicting water quality data (obtained from the water quality monitoring platform)	CNN-long short-term memory network (LSTM) combined model	Dissolved oxygen (DO)	RMSE = 0.8909	Multi-layer hidden layer experiments were not explored. Fewer input variables.	[20]
Predicting aquaculture water quality	BPNN, RBFNN. SVM. least squares support vector machine (LSSVM).	DO, pH, NH3-N, NO3-N, NO2-N	SVM obtained the most accurate and stable prediction results.	Hyperparameter tuning experiments have not been performed in more detail.	[21]
Monitoring water quality parameters	LSTM -RNN	pH, DO, chemical oxygen demand (COD), NH3-H	R2 = 0.83 Mean Relative Error (MRE) = 0.18	The number of hidden layers can be further adjusted.	[22]
Predict the water quality of urban sewer networks.	Multiple linear regression (MLR), Multilayer perception (MLP) RNN, LSTM and gated recurrent unit (GRU)	Biological oxygen demand (BOD), (COD), ${\mathrm{NH}}_4^{+}$-N total nitrogen (TN), TP	GRU achieved a 0.82–5.07% higher R2 than RNN and LSTM.	The contribution of each input indicator to the model predictions needs to be explored.	[23]
Predicting water quality data	Multi-task temporal convolution network (MTCN)	DO and Temperature	Temperature (RMSE = 0.59) DO(RMSE = 0.49)	Long training time (9 hours:58 minutes)	[24]
Prediction of DO in river waters	General regression neural network (GRNN), BPNN, RNN	Water flow, temperature, pH and electrical conductivity	RNN > GRNN > BPNN	No adjustment to the structure and parameters of the individual models.	[25]
Lake temperature modeling	physics-guided neural networks (PGNN)	11 meteorological drivers	Compared to SVM, least squares boosted regression trees (LSBoost) and ANN models, PGNN ensures better generalizability as well as scientific consistency of results.	The spatial and temporal nature of the data is not taken into account.	[26]

summarizes recent research on the use of machine learning to model and estimate water quality. Wu [19] combined the auto-regressive integrated moving average (ARIMA) and clustering models to improve the poor predictive performance of the traditional time-series ARIMA model for data with random characteristics. Tan [20] proposes a hybrid model for water quality prediction that combines the respective advantages of the convolutional neural network (CNN) model and the long short-term memory (LSTM) model. After the feature extraction of the CNN layer, the original data will get a new sequence with more vital feature ability than the original sequence. Compared with the conventional LSTM model, its mean absolute error was optimized by 11.63%. Li [21] uses several models, including back propagation neural networks (BPNN), radial basis function neural networks (RBFNN), and support vector machines (SVM), to simulate and predict water quality parameters. The results showed that the SVM achieved the best prediction effect, with an accuracy of 99% for both published and measured data.

The ANNs simulate the functions of the human brain [27], in which a large number of widely interconnected neurons form a network to conduct continuous learning, summarize past experience, and store the acquired knowledge. This information is then used for future forecasting [28]. During the training process, the network continuously adjusts its weights to minimize errors. If the network makes a poor prediction, the connection weights in the network will be adjusted in order to increase accuracy. Thus, the network is less likely to make the same mistake during the next iteration [29,30]. ANN is a general approximator that extracts the nonlinear relationships and interactions from different data features and solves large-scale problems by identifying the relationships in a given pattern [31], such as time series forecasting, pattern recognition, nonlinear modeling, classification, and control [32].

The recurrent neural network (RNN) is a versatile and easily assembled ANN [33]. The conventional ANN architecture is composed of an input layer, multiple hidden layers, and an output layer. Each layer consists of numerous interconnected neurons [34]. A neuron is a non-linear algebraic function [35]. When the signal is applied to the input of the neuron, its weights are modified [36]. Contrary to other neural network architectures, RNN uses state variables to store previous information. The output is then computed based on the current and previous states. Therefore, the RNN is widely used in the applications of time series, such as natural language processing, speech recognition, and machine translation [37]. However, it is noted that conventional RNNs usually do not perform well for long sequences due to gradient disappearance and gradient exploding [38]. These issues can be effectively addressed by optimizing the activation function or changing the network architecture [39].

The analysis based on experiments showed that the changes in the water quality parameters are not entirely random. Not only are there correlations among the water quality parameters, but the current statistical moments of the parameters are also closely related to the past moments, and these changes have certain asymptotic characteristics [40]. This characteristic makes it advantageous to use RNN for modeling water quality parameters. It is noted that the selection of appropriate input variables is an important step in the development of deep learning models. The work aims to develop an RNN model to predict the performance of immobilized nanoFeCu for sewage treatment with a focus on nitrogen compounds, including ammonia, nitrate, and nitrite. In this work, the water quality parameter data is obtained from a local sewage treatment plant. An ANN is designed for predicting the water quality to explore the correlation between the data, improve the prediction accuracy, and finally obtain the prediction results based on different network architectures and parameters. The study’s findings provide an exemplary model for predicting nano-FeCu-treated sewage using an RNN approach, which has not been studied previously. This work would serve as an important reference for the researchers and engineers who are exploring smart sewage treatment and nanotechnology.

2. Methodology

2.1. Data Collection and Processing

The data was collected from a pilot-scale study conducted at a local sewage treatment plant. 100 g of immobilized nanoFeCu was placed in a 50-L reactor and used to treat the sewage for a period of 27 weeks. A total of 84 sets of data were collected for ammonia, nitrate, nitrite, and pH readings at varied flowrates in the range of 210 mL/min to 1200 mL/min, from time t = 0 h to t = 7 h.

The data preprocessing is done by data screening, data cleaning, and data normalization. The purpose of data screening and cleaning is to filter out missing values and address the problem of outliers, which could be due to unpredictable weather during the process of data acquisition. The process of data cleaning to identify outliers in the dataset is performed based on the Pauta criterion [41]. Lastly, the data was normalized using the min-max normalization method, where the data was linearly transformed within the range of 0 to 1 for both inputs and outputs [42]. This is done to ensure that the data is properly fitted to the model. The min-max normalization method is mathematically expressed as Equation (1)

$$y_i=\frac{x_i-\underset{1\le j\le n}{\mathit{min}}\left\{x_j\right\}}{\underset{1\le j\le n}{\mathit{max}}\left\{x_j\right\}-\underset{1\le j\le n}{\mathit{min}}\left\{x_j\right\}}$$

(1)

where $x_i$ denotes the raw data and $y_i$ denotes the normalized value. Xmin and Xmax represent the minimum and maximum data, respectively.

The statistically analyzed data was presented in

Table 2. A statistical analysis of the processed data.

Data Set	Unit	Count	Mean	Min	Max	Std Dev
pH		80	7.600	6.240	9.310	0.816
Nitrate	mg/L	80	5.694	1.100	18.300	3.247
Nitrite	mg/L	80	0.02284	0.006	0.081	0.014
Ammonia	mg/L	80	23.434	1.700	47.400	9.731
Flowrate	mL/min	80	742.00	210.000	1200.000	374.788

and illustrated using a box-and-whisker plot, as illustrated in Figure 1. The processed data were divided into 10 groups according to the collection time and flowrate. 90% of the data was used for training, and the remaining 10% was used for testing purposes.

2.2. Pearson Correlation Coefficient

The Pearson correlation coefficient, as presented in Equation (2), was used to identify the relation between the inputs using SPSS 26 software

$$r=\frac{{\displaystyle \sum }\left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)}{\sqrt{{\displaystyle \sum }{\left(x_i-\overline{x}\right)}^2}{\displaystyle \sum }{\left(y_i-\overline{y}\right)}^2}$$

(2)

$r$ = correlation coefficient

$x_i$ = values of the x-variable in a sample

$\overline{x}$ = mean of the values of the x-variable

$y_i$ = values of the y-variable in a sample

$\overline{y}$ = mean of the values of the y-variable

2.3. Model Setup and Implementation

Table 3. Specification of the RNN architecture.

Inputs at t = 0 h to 7 h	Ammonia, Nitrite, Nitrate, pH
Outputs at t = 0 h to 7 h	Ammonia, Nitrite, Nitrate, pH
Number of neurons	10, 20, 30, 40, 50
Number of hidden layers	1–5
Window size	2
Activation function	ReLU
Number of iterations	1000

shows the RNN architectures adopted in this study, which are ideal to model time series data. pH, ammonia, nitrate, and nitrite at different time intervals were used as the inputs and outputs of the model. In this study, flowrate is the manipulated variable, which affects the performance of immobilized nanoFeCu. The flowrate was not affected by the time; thus, it was not used to develop the model. The number of neurons and hidden layers were manipulated within the range of 10–50 and 1–5, respectively, to increase the model’s performance. The dropout rate was set at 0.09. ReLU was chosen as the activation function as it is faster than Sigmoid and Tanh due to its simple function composition (when x ≤ 0, output = 0, and when x > 0, output = x) [43]. In addition, the non-saturating activation function of ReLU avoids the vanishing gradient. The RNN model is developed using TensorFlow, which is open-source software for deep learning [44]. 1000 iterations were used in this study due to minimum training loss, as illustrated in Figures S1–S5.

Figure 2a,b present the one-to-one and three-to-three RNN architectures. Figure 2c presents the design scheme of the RNN model. In the one-to-one model, one of the inputs, for example pH at t = 1 and 2 h, was used to predict the subsequent output at the following time interval, which was pH at t = 3 h. Meanwhile, three inputs (ammonia, nitrate, and nitrite) at a specific time were used to estimate the subsequent outputs in the three-to-three model. This means that ammonia, nitrate, and nitrite at t = 1 h and 2 h served as the inputs for ammonia, nitrate, and nitrite at t = 3 h.

The accuracy of prediction is measured by the coefficient of determination (R²) as presented in Equation (3)

$${\mathrm{R}}^2=\frac{{\left({{\displaystyle \sum }}_{i=1}^n\left(c_{ti}-{\overline{c}}_t\right)·\left(c_{ti}-{\overline{c}}_p\right)\right)}^2}{{{\displaystyle \sum }}_{i=1}^n{\left(c_{ti}-{\overline{c}}_t\right)}^2·{{\displaystyle \sum }}_{i=1}^n{\left(c_{pi}-{\overline{c}}_p\right)}^2}$$

(3)

n denotes the number of data, $c_{ti}$ represents the true value at the ith sample, $c_{pi}$ represents the predicted value for the ith sample, and ${\overline{c}}_p$ and ${\overline{c}}_t$ represent the average of the predicted results and the real results, respectively.

3. Result and Discussion

3.1. The Performance of One-to-One Model in Ammonia, Nitrate, Nitrite and pH Prediction

Figure 3 shows the prediction results of the one-to-one model for (a) pH, (b) ammonia, (c) nitrate, and (d) nitrite. In Figure 3a, when the number of hidden layers is 1, the R² value is almost not affected by the number of neurons. However, as the number of hidden layers increases, the R² value decreases. It could be clearly observed in the model, which consisted of 3 hidden layers and 40 neurons, where the R² was equal to 0.844, while the model with a single layer and 40 neurons has a R² value of 0.966. This could be due to the fact that the change in pH was not significant during the nanoFeCu-triggered ammonia oxidation process, as shown in Table 2, where the SD value of pH is 0.816. This indicates that a single layer is sufficient to develop a good model to predict the pH. A similar trend was observed in the nitrite model, where a small SD value of 0.014 was observed in Table 2, and Figure 3d showed that a single layer was good enough to predict its performance with a high R² of ~0.8. The best nitrite prediction was found in the model with a single hidden layer, 30 neurons, and an R² value of 0.872.

A similar approach was used to predict ammonia and nitrate, as shown in Figure 3b,c. In Figure 3b, one hidden layer is insufficient to support the model and achieve good results in predicting the value of ammonia. The increase in the number of neurons improves the performance of the prediction (R² from 0.611~0.823), which indicates that the model structure of one hidden layer is too simple to extract the features of complex data. When the hidden layers are increased to 2–5 layers, the performance of the model is basically stable, and the R² of the models is all greater than 0.793. The highest R² value of 0.9770 was achieved from 3 hidden layers and 50 neurons in the RNN. For nitrate prediction, different variations of hidden layers and neurons can all achieve results with R² > 0.8, except when the hidden layer is 2 and the neurons >10, as depicted in Figure 3c. A maximum R² of 0.9456 was found in the RNN model with a single layer and 40 neurons. Comparatively, the R² of pH and nitrite were higher than ammonia and nitrate, which could be due to the higher SD of ammonia and nitrate, which were 3.247 and 9.731, as tabulated in Table 2.

The result of the existing study is better than the work published by Lei [45], where back propagation neural network (BPNN) and radial basis function neural network (RBFNN) were used to predict pH, ammonia, nitrite, and nitrate. The highest R² values of pH (R² = 0.84) and ammonia (R² = 0.88) were obtained by using RBFNN. Meanwhile, the highest R² value of 0.96 for nitrate and 0.87 for nitrite were reported for BPNN. In the current study (one-to-one model), the highest R² values for pH, ammonia, nitrate, and nitrite were 0.985, 0.977, 0.946, and 0.872, respectively. This could be due to the suitability of RNNs to handle time-series data.

3.2. The Performance of Three-to-Three Model in Ammonia, Nitrate and Nitrite Prediction

A three-to-three model to predict the performance of ammonia, nitrate, and nitrite was developed, and the results are presented in Figure 4. This model was made as there was a close relationship between these three reactants in ammonia oxidation processes [12,13], as indicated in Equations (4)–(10). Nitrate and nitrite are the intermediate products used to oxidize ammonia to nitrogen gas with the aid of nanoFeCu. Fe releases the electrons to facilitate the oxidation process, while Cu assists in the electron transfer [46,47].

In Figure 4, for a single hidden layer, the added neurons decrease the R² value of the model, especially in Figure 4a,b. This could indicate that the increasing number of neurons leads to the risk of overfitting the model. It is noteworthy that the change in neurons from 10 to 50 did not cause a dramatic change in R² values when the number of hidden layers was increased from 1 to 5, as shown in Figure 4a,b. This suggests that when the hidden layer is small, even increasing the number of neurons may not lead to better prediction results. The best R² predictions for ammonia, nitrate, and nitrite were 0.961 with 5 layers and 30 neurons, 0.940 with a single layer and 50 neurons, and 0.941 with 5 layers and 20 neurons, respectively. In addition, the three-to-three model predicted ammonia, nitrate, and nitrite well with an R² value of > 0.9, except for the results of the single hidden layer.

$${\mathrm{NH}}_4\mathrm{Cl}+{\mathrm{H}}_2\mathrm{O}\to {\mathrm{H}}_3{\mathrm{O}}^{+}+{\mathrm{NH}}_3+{\mathrm{Cl}}^{-}$$

(4)

$$2{\mathrm{NH}}_3+4{\mathrm{O}}_2\to {\mathrm{NO}}_2^{-}+{\mathrm{NO}}_3^{-}+3{\mathrm{H}}_2\mathrm{O}$$

(5)

$${\mathrm{Fe}}^0\to 2{\mathrm{e}}^{-}+{\mathrm{Fe}}^{2+}$$

(6)

$${\mathrm{NO}}_3^{-}+2{\mathrm{e}}^{-}+{\mathrm{H}}_2\mathrm{O}\to {\mathrm{NO}}_2^{-}+2{\mathrm{OH}}^{-}$$

(7)

$${\mathrm{H}}_2\mathrm{O}+{\mathrm{e}}^{-}\to {\mathrm{OH}}^{-}+{\mathrm{H}}_{\mathrm{ads}}$$

(8)

$$2{\mathrm{NO}}_2^{-}+4{\mathrm{H}}_{\mathrm{ads}}\to {\mathrm{N}}_2+4{\mathrm{OH}}^{-}$$

(9)

$$2{\mathrm{NO}}_2^{-}+12{\mathrm{H}}_{\mathrm{ads}}\to 2{\mathrm{NH}}_4^{+}+4{\mathrm{OH}}^{-}$$

(10)

3.3. The Comparison of One-to-One and Three-to-Three Models in Ammonia, Nitrate and Nitrite Estimation

The best RNN architectures of one-to-one and three-to-three models for predicting ammonia, nitrate, and nitrite were identified from Figure 3 and Figure 4. The selection criteria were made based on the R² values and simplicity of RNN architectures. Based on the average R² values of three-to-three models, a single hidden layer with 10 neurons was identified as the best RNN architecture, where the R² average was equal to 0.9132. The R² values for ammonia, nitrate, and nitrite were 0.8736, 0.9295, and 0.9366, respectively, as presented in

Table 4. The best RNN architectures for one-to-one and three-to-three models.

Model	Hidden Layers	One-to-One	Three-to-Three
	Hidden Layers	R2	R2
Ammonia	Single: 10 neurons	0.6110	0.8736
Nitrate	Single: 10 neurons	0.8201	0.9295
Nitrite	Single: 10 neurons	0.7943	0.9366
Average R2	-	-	0.9132

The best RNN architectures with the highest R² in the one-to-one model are extracted from Figure 3.

. Compared to the one-to-one model, the R² values of ammonia, nitrate, and nitrite were much lower, whereas the values were 0.6110, 0.8201, and 0.7943 for the same design of RNN architecture. The high R² value found in the three-to-three model could be due to the chemical reaction between the three nitrogen compounds, as shown in Equations (4)–(10). This is also proved by the Person correlation coefficient chart, as illustrated in Figure 5, where ammonia, nitrate, and nitrite were significantly correlated at the 0.01 level. The details of the comparison between the one-to-one and three-to-three models are available in the supplementary data, Tables S1 and S2.

The Pearson correlation coefficient chart in Figure 5 showed that the correlation between pH and flowrate is lowest at r = 0.092. This indicates that pH and flowrate are independent parameters. Hence, these two parameters were not considered when developing the three-to-three model. It is notable that there is a weak correlation among pH, ammonia, and nitrate. This is because pH affects the ${\mathrm{NH}}_4^{+}$ and NH₃ equilibrium [48], which eventually affects the formation of nitrate, as shown in Equation (10).

It was observed that the stability of the prediction model is higher in the three-to-three model compared to the one-to-one model, as depicted in Figure 3 and Figure 4. When a one-to-one model was used to predict nitrate, the R² fluctuated from 0.740 (2 hidden layers and 30 neurons) to 0.946 (1 hidden layer and 40 neurons), as shown in Figure 3c. In Figure 4b, the three-to-three model becomes relatively stable, and R² ranges from 0.805 (4 hidden layers and 10 neurons) to 0.940 (1 hidden layer and 30 neurons). This demonstrates the improved predictive stability of the model.

The coefficient matrix and bias values for the three-to-three model (single hidden layer and 10 neurons) are shown in Figure 6.

4. Conclusions

In this work, one-to-one and three-to-three RNN models were developed, optimized, and compared to identify the best model to predict and forecast the performance of nanoFeCu in sewage treatment. The R² values of the one-to-one model for predicting pH, ammonia, nitrate, and nitrate fell within the range of 0.87 to 0.98. However, the overall performance of the model decreases and the results fluctuate when the number of hidden layers and neurons is increased. Comparatively, the stability of the three-to-three model was better as the nitrogen compounds are chemically related in the oxidation process. It is also proved by the Person correlation coefficient, where the nitrogen compounds were significantly correlated at the 0.01 level. Besides, the three-to-three model also improved the overall prediction performance. The best RNN architecture for predicting the nanoFeCu-treated sewage was identified at a single layer with 10 neurons, where an average R² of 0.91 was recorded. The findings of this study showed that the RNN model developed in this study is suitable for a contact time of up to 7 h. The accuracy of the model could be reduced if the data is collected for >7 h due to the exploding and vanishing gradients problem in time series [49,50]. It is recommended to extend the research by developing the model using other machine learning approaches for a comparative study.