Development and Deployment of a Virtual Water Gauge System Utilizing the ResNet-50 Convolutional Neural Network for Real-Time River Water Level Monitoring: A Case Study of the Keelung River in Taiwan

Abstract

Climate change has exacerbated severe rainfall events, leading to rapid and unpredictable fluctuations in river water levels. This environment necessitates the development of real-time, automated systems for water level detection. Due to degradation, traditional methods relying on physical river gauges are becoming progressively unreliable. This paper presents an innovative methodology that leverages ResNet-50, a Convolutional Neural Network (CNN) model, to identify distinct water level features in Closed-Circuit Television (CCTV) river imagery of the Chengmei Bridge on the Keelung River in Neihu District, Taiwan, under various weather conditions. This methodology creates a virtual water gauge system for the precise and timely detection of water levels, thereby eliminating the need for dependable physical gauges. Our study utilized image data from 1 March 2022 to 28 February 2023. This river, crucial to the ecosystems and economies of numerous cities, could instigate a range of consequences due to rapid increases in water levels. The proposed system integrates grid-based methods with infrastructure like CCTV cameras and Raspberry Pi devices for data processing. This integration facilitates real-time water level monitoring, even without physical gauges, thus reducing deployment costs. Preliminary results indicate an accuracy range of 83.6% to 96%, with clear days providing the highest accuracy and heavy rainfall the lowest. Future work will refine the model to boost accuracy during rainy conditions. This research introduces a promising real-time river water level monitoring solution, significantly contributing to flood control and disaster management strategies.

1. Introduction

The escalating impact of climate change, marked by a worldwide increase in severe weather events, particularly unpredictable and rapid fluctuations in river water levels, necessitates the development of reliable, real-time water level detection systems [1,2,3]. In Taiwan, a country experiencing a heightened frequency of typhoons and heavy rainfall [4,5], the increased risk of rising river water levels underscores the need for remote hydrological monitoring, especially during typhoons or significant precipitation [6]. This situation highlights the vital importance of automated water level measurement systems [7,8,9,10].

Traditional methods, which heavily rely on physical river gauges, are becoming increasingly unreliable due to environmental degradation [11,12,13,14,15]. An alternative approach using CCTV cameras to monitor water gauges installed in significant rivers and flood-prone areas has been explored. Yu [16] proposed a differencing image technique that detects minor changes in water levels by analyzing the Region of Interest (ROI) between previous and current frames and applying the Otsu threshold method. However, the robustness of this method under different illuminations and locations still needs to be tested. Kim [17] developed a cloud-based system, the River Eye Image Water Level Gauge, which integrates video surveillance for river flow and water level measurements. This system is currently undergoing testing at four sites. Hiroi [18] presented a water-level sensor system that uses infrared image processing for real-time river-level monitoring and accurate flood prediction in urban areas. Pan [19] developed a low-cost unmanned surveillance system that uses a map-based web service, video cameras, water level analyzers, and wireless communication routers for real-time water level measurements. The deep learning-based method demonstrated superior performance in terms of accuracy and stability. Sabbatini [20] proposed a computer vision solution for automatic river water-level monitoring, showing excellent performance in discerning frame quality, especially during nighttime. Narayanan [21] introduced a method that uses participatory sensing and computer vision to estimate flood levels.

However, the absence of gauges in some rivers and inaccurate readings due to inadequate maintenance hinder precise water level detection [22,23]. As illustrated in Figure 1, poorly maintained gauges often present unclear numerical readings, preventing image recognition technology from accurately determining the current water level and leading to potential misjudgments. This highlights an urgent need for further research to mitigate these issues.

To address this challenge, this paper introduces an innovative approach that utilizes existing CCTV footage to establish a virtual water gauge. This method subsequently applies image processing techniques to determine the current water level of the river. The proposed model, which employs ResNet-50 [24,25,26], a Convolutional Neural Network (CNN) model [27,28,29,30,31], is trained using data derived from the CCTV river imagery of the Chengmei Bridge on the Keelung River in Neihu District, Taiwan. This data was collected from 1 March 2022 to 28 February 2023.

The Keelung River, an essential water system in Northern Taiwan, has an approximate length of 96 km and a catchment area of around 512 square kilometers. It traverses prominent cities in Northern Taiwan, including Taipei, New Taipei, and Keelung, significantly influencing these regions’ geographical and economic landscapes.

The Keelung River has been the site of numerous severe floods throughout the years, marked by notably devastating events brought on by Typhoon Lynn in October 1987, Typhoon Winnie in August 1997, Typhoon Xangsane in October 2000, and Typhoon Nari in September 2001. These typhoons unleashed torrential rains, leading to extensive flooding within the Keelung River basin. For example, Typhoon Xangsane in 2000 resulted in an inundation of approximately 465 hectares spanning various districts in Taipei City, New Taipei City, and Keelung City, causing 59 fatalities and flooding around 10,000 households. Moreover, Typhoon Nari in 2001 led to severe flooding in the Keelung River basin, submerging numerous areas in Taipei City, New Taipei City, and Keelung City, resulting in 104 deaths and approximately 20,000 flooded households.

The most recent incident occurred on 16 October 2022, when Typhoon Nisha swept across the Keelung River basin. This event led to a dramatic surge in the river’s water level, peaking at 5.04 m, exceeding the alert level by two meters. This occurrence highlighted the Keelung River basin’s vulnerability to flood risks under extreme weather conditions. Such disasters can significantly impact residents in low-lying areas and the surrounding communities, resulting in property damage, road traffic disruptions, and casualties. Therefore, implementing effective flood prevention measures and disaster management strategies in the Keelung River basin is critically important to mitigate the impacts of future extreme weather events.

This study is particularly significant for the Keelung River, especially near the Chengmei Bridge. Its geographical location, impact on the surrounding cities, and environmental challenges make it an ideal site for testing our innovative water level detection methods. Influenced by heavy rainfall, the river’s water level fluctuations allow us to refine and test our system to enhance the region’s flood prevention and disaster management strategies.

A unique feature of this study is the utilization of the existing CCTV infrastructure installed across numerous rivers. By integrating a cost-effective hardware device, Raspberry Pi [32,33,34], the pretrained grid-based virtual water gauge model can be executed to determine the current river water level. This approach significantly reduces the cost of establishing river water level monitoring facilities and can be readily implemented in various locations.

The paper is structured as follows: The subsequent section will detail the methodology, including the model’s training and operation. The following section will present the study’s results, including its effectiveness during heavy rainfall. The concluding section will discuss the implications of the study, its limitations, and future research directions. This innovative approach represents a significant advancement in flood control and disaster management, offering considerable potential for enhancing cities’ resilience to flooding and other water-related disasters.

2. Materials and Methods

In this section, we elaborate on developing and implementing three primary models designed to enhance the accuracy and reliability of river water level monitoring. Firstly, the Grid Selection Model uses CCTV footage and image processing techniques to identify the optimal grid for establishing a virtual water gauge. Secondly, the Grid State Recognition Model accurately determines the state of the selected grid, categorizing it as devoid of water, partially filled with water, or filled with water. This categorization is crucial for determining the river’s current water level. Lastly, the Water Level Calculation Model calculates the water level height of the virtual water gauge, converting the grid’s state into a numerical value that represents the river’s current water level. These models work collectively to provide accurate, real-time water level detection. Their methodologies, execution, and collaborative functioning are detailed in the following sections and depicted in Figure 2.

2.1. Grid Selection Model

The Grid Selection Model, a critical system component, is designed to select representative grids from CCTV footage. These grids effectively reflect water level variations, forming the foundation for a virtual water gauge. The model operates in five stages. River imagery and weather data are initially collected to guide the subsequent processes. Following this, the recognition area within the imagery is identified, focusing on relevant areas to enhance efficiency and precision. The images are then divided into smaller, manageable grids for individual examination. These images undergo dynamic binarization, transforming into a binary format for a streamlined analysis. The final stage involves selecting grids that accurately reflect water level changes. These grids provide data for precise water level monitoring. Each stage is meticulously fine-tuned to maximize the model’s effectiveness, with an in-depth exploration of each stage offering a comprehensive understanding of the Grid Selection Model’s operation.

2.1.1. Collection of River Imagery and Weather Data

The acquisition of river imagery is a critical phase in our methodology. For a meaningful analysis, gathering images representing various water level changes is essential. Notably, images captured after rainfall events, which cause water levels to surge, are particularly valuable, as they distinctly record fluctuations in water levels.

Simultaneously, we collect weather information specific to the river’s location, encompassing the current weather conditions and sunrise and sunset times. This data aid in refining the threshold for binarization during the dynamic image binarization stage, considering the environmental factors in the imagery. These weather data are typically acquired from meteorological observation stations or similar entities.

For this study, river imagery was sourced from the CCTV at the Chengmei Bridge on the Keelung River in Taiwan. This open data was gathered from 1 March 2022 to 28 February 2023 via the Water Resources Agency’s Water Situation Image Monitoring Station’s cloud service platform, a Ministry of Economic Affairs subsidiary. The CCTV river images, with a resolution of 1920 × 1080, are updated every minute, as illustrated in Figure 3.

Additionally, weather data, including hourly weather conditions and sunrise and sunset timings from 1 March 2022 to 28 February 2023, was compiled for the Neihu District of Taipei City, where the Chengmei Bridge is situated. These data were sourced from the Central Weather Bureau’s website, Taiwan’s Ministry of Transportation and Communications Division.

This comprehensive collection of river imagery and weather data forms a solid foundation for the subsequent stages of our Grid Selection Model, significantly enhancing the accuracy of our water level monitoring system.

2.1.2. Determination of the Recognition Area

The initial step in establishing a virtual water gauge within the CCTV footage of a river involves selecting an appropriate region. This region is the basis for filtering suitable unit grids to construct the virtual water gauge. The selection of this region is critical and should align with the actual water level fluctuations in the river.

This designated region assists in selecting the ideal grid units that constitute the virtual water gauge. The lower limit of this region corresponds with the lowest point of the actual water level, represented as zero on the water gauge. Conversely, the upper limit of this region aligns with the highest point of the water level. This process and the corresponding water levels are depicted in Figure 4.

In locations equipped with water gauges, the proposed grid-based virtual water level can be directly inferred from the height of the water gauge, as demonstrated in the image. However, for areas lacking water gauges, it becomes necessary to calculate the conversion formula and parameters that link the virtual water level to the actual water level. This process requires on-site measurements. Using this information, the current water level in the river can be accurately determined, even in locations where water gauges are absent or the readings are obscured.

In this case, when identifying the recognition area from the collected CCTV river imagery at Chengmei Bridge, the lowest point within the recognition area corresponds with the part of the river where the water level is 0 m on the water gauge. Conversely, the highest point aligns with the highest point on the river water gauge, situated at 7.5 m. This is illustrated in Figure 5.

2.1.3. Image Gridification

The next step in creating a virtual water gauge from the river imagery involves segmenting the identified area within the image into a grid format. The dimensions of these grids should closely correspond to the pixel count in the image that represents the unit height of the actual water level. With appropriate segmentation, we can select the most suitable grids to form the virtual water gauge.

For the ensuing grid selection, we employ the ResNet50 model, which is implemented using the TensorFlow package in Python, to identify the features of the grid images. This methodology will assist us in selecting grids suitable for constructing the virtual water gauge. It is imperative to note that the image input size for the ResNet50 model is 224 × 224. Therefore, we must ensure that the grid dimensions do not introduce any distortions or alterations to the inherent features of the grid during the scaling process to match the required input size.

To this end, the number of pixels corresponding to the grid height, denoted as Grid_Pixel, is determined using Equation (1). Assuming the unit height of the actual water level corresponds to P pixels in the image, the formula is as follows:

$${\mathrm{Grid}}_{\mathrm{Pixel}}=\{7\times 2^{\mathrm{N}}|\mathrm{N}=⎣{\mathrm{l}\mathrm{o}\mathrm{g}}_2^{\mathrm{P}/7}⎦\}$$

(1)

N is an exponential term in this equation that adjusts the grid size to closely approximate P. This formula aims to align the grid height (Grid_Pixel) with the image height (P) corresponding to the unit height of the actual water level. Simultaneously, it ensures that, when increased by a power of 2 (to the Nth power), the final size is 224 × 224. The choice of 7 as the base in this formula is informed by the fact that, when 224 is continuously halved, the smallest value achievable is 7.

In this study, involving the river imagery from Chengmei Bridge, an actual 1 m corresponds to 65 pixels in the image. According to Equation (1), where P is given as 65, the value of Grid_Pixel can be calculated by using Equation (1): Grid_Pixel = {7$\times 2^{\mathrm{N}}$|N = $⎣{\mathrm{l}\mathrm{o}\mathrm{g}}_2^{65/7}⎦$ = 3} = 7 $\times$ 2³ = 56, which simplifies as the Grid_Pixel is 56.

Therefore, when performing image gridification, a grid height unit of 56 pixels should be used for processing. Figure 6 shows the recognition region partitioned into grids of uniform size, each approximating the unit height of the real-world water level.

2.1.4. Dynamic Image Binarization

The binarization process is crucial in image recognition, particularly when recognizing river water levels [35,36]. However, employing dynamic image binarization is necessary due to several potential complications. Traditional static thresholding methods often need to perform better when applied to images captured under various environmental conditions. A single threshold value may not be universally suitable for different lighting and weather conditions or times of the day. This can lead to suboptimal binarization and, consequently, decreased recognition accuracy. Additionally, a static approach needs more flexibility to adapt in real time to environmental changes, which can further compromise the reliability of the image recognition process.

To illustrate this, refer to Figure 7, which showcases images taken at 8:00 a.m. at the Nanhu Bridge over the Keelung River under different weather conditions. These images were binarized using a fixed threshold of 150. Figure 7a depicts a sunny day, and Figure 7b represents an overcast day. The results demonstrate that a fixed threshold can discern the water gauge’s numbers in Figure 7b. However, in Figure 7a, some numbers need to be more identifiable, indicating the limitations of using a fixed threshold under varying weather conditions.

We have proposed a polynomial regression model [37] to address these shortcomings and establish a dynamic binarization threshold prediction model. This model is designed to predict the optimal binarization threshold in real time, enabling it to adapt to changing environmental factors. In this study, we implemented this dynamic binarization threshold prediction model to binarize image data within a specified timeframe. For instance, Figure 8 demonstrates the dynamically predicted threshold values for each hour from 0:00 to 23:00 between 15 March and 18 March 2022.

Using the threshold values predicted by our model, we successfully improved the image recognition accuracy of CCTV images for Chengmei Bridge on the Keelung River spanning 2022 to 2023, as shown in Figure 9.

The binarization process of images requires distinct threshold values depending on varying weather conditions and times. For instance, Figure 9a illustrates that a threshold of 70 is optimal for binarizing images taken on a clear, sunny day at 5:00 p.m. Conversely, for images captured on a rainy day at noon, a higher threshold value of 110 is recommended, as demonstrated in Figure 9b.

This dynamic binarization strategy allows for acquiring binarized images ideally suited for recognition tasks, effectively circumventing the constraints of traditional static image binarization. This adaptability to environmental variations ensures reliable and accurate image recognition, regardless of the prevailing conditions during image capture.

2.1.5. Selection of Grids for Virtual Water Gauge

This study introduces a grid selection method for creating a virtual water gauge. As illustrated in Figure 10, following the binarization of the image, the process selects an appropriate grid from each row to construct the virtual water gauge. The crucial characteristic of the grids chosen as candidates for the virtual water gauge within the same column is their ability to distinctly differentiate between water-free, partially water-filled, and fully water-filled states. This method enhances the precision in recognizing water levels under varying environmental conditions, thus contributing to advancements in image recognition.

Figure 11 exemplifies this concept with two images from the same row in their original and binarized forms. Figure 11a depicts a clear day without water, while Figure 11b represents a rainy day with a water-filled grid. It is evident from these images that the blue grid exhibits significant differences in the binarized images between the water-free and fully water-filled states, while the yellow grid does not. Therefore, the blue grid is more suitable than the yellow grid for constructing the virtual water gauge. This innovative method enhances the precision of water level recognition, contributing significantly to advancements in image recognition techniques.

The grid selection method proposed in this study encompasses the following four primary steps:

• Grid Clustering
  Each identical grid from the collected river images undergoes a binarization process, with all historical data of the same grid categorized into three groups: “no water”, “partial water”, and “full water”, as displayed in Figure 12.
  To facilitate this process, we employ a semi-supervised auto-labeling technique using ResNet50, and we start with a small set of manually labeled images under the categories of “no water”, “partial water”, and “full water”. Using this labeled set, we train an initial model. This trained model is then used to predict labels for the unlabeled images.
  Images where the model’s predictions are highly confident are identified and added to the labeled set with their predicted labels, known as pseudo-labeling. We then retrain the model on this newly augmented labeled set.
  This iterative process performs pseudo-labeling and retraining the model until a specified stopping condition is met, such as no significant improvement in the model performance. This semi-supervised auto-labeling approach allows the model to progressively learn from a more extensive dataset, improving its performance, even if the initial labeled dataset is small.
• Grid Preprocessing
  Before extracting features with the ResNet50 model, ensuring that the grid size meets the model’s input requirements, i.e., 224 × 224, is imperative. If the grid size does not meet the ResNet50 model’s input requirements, the original grid is expanded to 224 × 224 using bilinear interpolation [38,39,40,41]. This preserves the details and features of the original grid, ensuring visual consistency and preventing shape or pixel distortion, thereby enabling the extraction of accurate feature vectors.
• Grid Feature Extraction
  This step primarily involves extracting features of each grid’s “no water”, “partial water”, and “full water” states using ResNet50. Preprocessed grid historical images are input into the ResNet50 model. Assuming the number of images is n, the input shape is (n, 224, 224). After convolution calculations, the feature vector is extracted from the layer before the fully connected layer, with a shape of (n, 7, 7, 2048), which is then transformed into a one-dimensional array for the subsequent similarity calculations. Figure 13 illustrates that we ultimately obtained the feature vectors for the three categories.
• Virtual Water Gauge Grid Selection
  This step aims to select the most suitable grid from each row to serve as a virtual water gauge. Here is the process:
  • Feature Vector Similarity Calculation for Each Grid
    For each grid, compute its feature vectors under three different water level states: “no water” (denoted as N), “partial water” (denoted as P), and “full water” (denoted as F). These feature vectors, which encapsulate the image characteristics of each grid under different water level states, are obtained through the previously mentioned Grid Feature Extraction model.
  • Cosine Similarity Calculations
    The cosine similarity formula calculates the similarity between feature vectors of different water level states [42,43,44,45], denoted as S_A×B. This formula is given by

    $${\mathrm{S}}_{\mathrm{A}}{\times }_{\mathrm{B}}=\frac{\mathrm{A}\mathrm{B}}{\left|\right|\mathrm{A}\left|\right| \left|\right|\mathrm{B}\left|\right|}$$

    (2)

    where A and B are the feature vectors, • denotes the dot product, and ||A|| and ||B|| are the magnitudes of vectors A and B, respectively.
    For this analysis, we substitute (A, B) in the formula with (N, P), (N, F), and (P, F) to calculate the cosine similarities for these combinations.
    A value closer to 1 for a cosine similarity indicates a higher similarity between the feature vectors of two water level states. Therefore, if a grid’s feature vectors under different water levels have a high cosine similarity, it implies that the grid’s feature vectors are ineffective in distinguishing between water level states, making it less suitable as a virtual water level gauge.
  • Similarity Average Calculations
    For each grid, we calculate the average of the cosine similarities S_N×P, S_N×F, and S_P×F to get Similarity[r][k], which represents the similarity of the kth grid in the rth row:

    $$\mathrm{S}\mathrm{i}\mathrm{m}\mathrm{i}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{t}\mathrm{y}\left[r\right]\left[k\right]=\frac{({\mathrm{S}}_{\mathrm{N}\times \mathrm{P}}+{\mathrm{S}}_{\mathrm{N}\times \mathrm{F}}+{\mathrm{S}}_{\mathrm{P}\times \mathrm{F}})}{3}$$

    (3)
    Figure 14 provides a box plot illustrating the similarity values for each row of grids. These values were derived from images captured at the Chengmei Bridge on the Keelung River, collected from 1 March 2022 to 28 February 2023.
  • Select the grid with minimum similarity
    Finally, we select the grid with the smallest average similarity for each row to serve as the virtual water level gauge. Grid[i]_{virtual water gauge} represents the index of the grid with the smallest similarity in the ith row, and argmin_k [46] is a function that delivers the index k of the minimum value in the sequence Similarity[i]:

    Grid [i]_{virtual water gauge} = argmin_k(Similarity[i][k])

    (4)

The steps outlined earlier form the proposed grid selection method in this research. This method aids in identifying the most fitting grid to serve as a virtual water gauge for detecting water levels. As depicted in Figure 15, the grid with the lowest similarity value within its row, marked by a red box, is selected as the virtual water gauge.

2.2. Grid State Recognition Model

In this study, we design individual Grid State Recognition Models for each grid identified by the Grid Selection Models to compose the virtual water gauge. The primary purpose of these Grid State Recognition Models is to discern the current water level status of the grid from real-time CCTV footage, categorizing it as “no water”, “partial water”, or “full water.”

To build the Grid State Recognition Model, we employ the architecture of the ResNet50 model. The model input for each grid of the virtual water gauge is the binary grid image of all the historical image data after being clustered into the “no water”, “partial water”, and “full water” categories. We resized the binary grid image to 224 × 224 using bilinear interpolation. After multiple training iterations, we successfully trained a model capable of identifying the grid status as “no water”, “partial water”, or “full water” based on the input binary grid image, as depicted in Figure 16.

2.3. Water Level Calculation Model

Our Water Level Calculation Model uses the “no water”, “partial water”, and “full water” states of a virtual water gauge grid, as identified by the Grid State Recognition Model, as the input. These states are utilized to calculate the water level of the virtual water gauge. We assume that the height of a grid corresponds to the actual water level, referred to as Grid_Height, where Grid_Height is equal to 0.86 m based on a grid size of 56 pixels and a correspondence of 1 m to 65 pixels in the river image from the Chengmei Bridge CCTV.

For the “no water” state, the corresponding river water level is 0, while, for the “full water” state, it is 1 $\times$ Grid_Height. For the “partial water” state, we first calculate the grid water level height ΔH within the grid, which is then converted into the corresponding river water level height ΔH $\times$ Grid_Height.

The calculation of the water level height ΔH within the “partial water” grid is carried out in four steps, as depicted in Figure 2:

• Preprocessing
  The binary “partial water” state is converted by dividing the image values by 255. Pixels with water are converted to 0 and those without water to 1, producing a 56 × 56 matrix.
• Horizontal Water Ripple Filtering
  We employ two convolution operations to filter out horizontal water ripples [47,48,49]. The first convolution operation uses a 1 × 7 filter matrix with all values set to 1 and a stride of 7. Each row undergoes separate convolution operations, resulting in a 56 × 8 matrix (see Figure 17).
  To calculate the vertical height of the water level in the grid, the 56 × 8 matrix obtained from the first convolution operation undergoes a second convolution operation to produce a 56 × 1 matrix. This operation uses a 3 × 8 filter matrix with all values set to 1 and a stride of 1, with convolution performed from bottom to top. To prevent the original grid height from being affected by the convolution operation, a padding operation [50,51] is performed on the 56 × 8 matrix before the second convolution. The padding matrix is 2 × 8 with all values set to 1, as illustrated in Figure 18.
• Vertical Water Ripple Filtering
  For a more precise determination of the water level height, we further process the matrix obtained from the lateral ripple filtering for vertical ripple filtering. A single convolution operation is used to eliminate vertical water ripples gradually. Before this operation, the 56 × 1 matrix undergoes padding, with a 2 × 1 padding matrix where all values are 0. Then, a convolution operation is performed using a 3 × 1 filter matrix with all values set to 1 and a stride of 1. This operation yields a 56 × 1 matrix of water probabilities for each row (see Figure 19).
• Grid Water Level Height Calculation
  Lastly we transform the matrix of water probabilities into a matrix of 1s and 0s by setting a threshold value θ. Values below θ are converted to 0 and those above θ to 1, resulting in the grid water level height matrix H. The grid water level height ΔH is then calculated using Equation (5); in this case, θ is 0.7, and the Grid_Pixel is 56.

  $$\mathsf{\Delta }\mathrm{H}=\frac{{\sum }_1^{{\mathrm{G}\mathrm{r}\mathrm{i}\mathrm{d}}_{\mathrm{P}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}}}\mathrm{H}\left[\mathrm{i}\right]}{{\mathrm{G}\mathrm{r}\mathrm{i}\mathrm{d}}_{\mathrm{P}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}}}$$

  (5)

Subsequently, the river water level is calculated using Equation (6), where N_{Full_Water} represents the total number of “full water” grids.

$${\mathrm{Level}}_{\mathrm{Height}}=(1\times {\mathrm{N}}_{\mathrm{Full}\_\mathrm{Water}}+\mathsf{\Delta }\mathrm{H})\times {\mathrm{Gird}}_{\mathrm{Height}},$$

(6)

Our Water Level Calculation Model provides a comprehensive approach for estimating water levels from CCTV images. By integrating the Grid State Recognition model, we can interpret the state of the grid and calculate the corresponding water level. This model is a highly effective tool for real-time monitoring and predicting river water levels, significantly contributing to disaster prevention and management.

3. Results

This section delves into assessing the model’s accuracy, cost-effectiveness, and overall performance. A crucial aspect of the validation procedure entails determining the feasibility of implementing real-time river water level monitoring on a Raspberry Pi platform. The platform from which the study’s data are sourced solely provides CCTV river images for the Keelung River Chengmei Bridge, devoid of any physical measurement data. Consequently, our precision assessment relies exclusively on the model’s results. Nevertheless, during the preliminary evaluation stages, we manually interpreted the water levels from the images, marking them to establish a baseline for the model assessment. Consequently, we expanded the grid matrix of the virtual water gauge from its original 9 grids, which correspond to the actual water gauge, to 15 grids. This augmentation ensures that the coverage spans the lowermost section of the image.

3.1. Model Accuracy

The process of authenticating the model’s accuracy principally involved the following essential steps:

• Image Clustering
  Leveraging the weather data from the Central Weather Bureau, the river images under investigation were categorized into three distinct classes: sunny, cloudy, and rainy.
• Initial Water Level Determination
  The water level for the first frame in the test video was manually annotated to establish an initial reference point.
• Evaluation of the Virtual Water Gauge Accuracy
  This process acknowledges that the river water level exhibits sudden surges or drops when an anomaly is observed in the virtual water gauge level, as demonstrated in Figure 20. A rapid decline promptly follows an increase, which we classify as a spike, signifying an error in the water level assessment. Figure 20a,b illustrate the outcomes of the Water Level Calculation Model under rainy and heavy rain conditions, respectively. The results of the Water Level Calculation Model under clear weather conditions are presented in Figure A1. If the current water level diverges from the previous one by more than θ, it is marked as a spike, indicating a potential error in the water level assessment. In this study, θ was assigned to correspond to Grid_Pixel. The formula utilized to compute accuracy is provided by Equation (7):

  $$\mathrm{Accuracy}=\frac{(\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{s}-\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{s}\mathrm{p}\mathrm{i}\mathrm{k}\mathrm{e}\mathrm{s})}{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{n}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{s}}$$

  (7)

During the initial evaluation, the accuracy of the virtual water gauge level was found to be 93% on sunny days, 88.4% on cloudy days, and a mere 50.1% on rainy days. A thorough analysis of the inaccurate image data indicated that the primary source of these errors was wrong predictions of the dynamic binarization values, which significantly distorted the water level estimates.

Crucial adjustments were implemented to rectify these issues, primarily focusing on refining the dynamic binarization threshold prediction model. However, the original model incorporated weather factors, and the influence of the rainy day factor was reduced due to the limited volume of image data from rainy days.

The dynamic binarization threshold prediction model was restructured into three distinct models, each tailored for sunny, cloudy, and rainy conditions. This modification aimed to bolster the model’s adaptability to various weather conditions and, importantly, to equip each model with a dedicated dataset for enhanced learning and prediction accuracy.

The results of these adjustments were highly encouraging. The accuracy of the virtual water gauge level increased to 96.3% on sunny days, 90.9% on cloudy days, and, remarkably, 83.6% on rainy days, as illustrated in Figure 21. These improvements underscore the effectiveness of the refined approach in minimizing errors associated with inaccurate dynamic binarization predictions, particularly under adverse weather conditions.

Upon rectifying the dynamic binarization values, the model demonstrated enhanced accuracy across all weather conditions, with a substantial improvement observed under rainy conditions. This underscores its effectiveness and robustness in accurately determining water levels.

The model’s accuracy ranged from 83.6% to 96%. The highest accuracy was recorded during clear weather conditions, while the lowest was observed during heavy rainfall. This fluctuation can be attributed to the complexity and clarity of river imagery under different weather conditions.

3.2. Virtual Water Gauge System

This research combines the proposed grid-based virtual water gauge with the Raspberry Pi and corresponding river water level sensor components to design a low-cost system for real-time river water level monitoring.

The system encompasses the following characteristics:

• Main Functions: The CCTV monitoring function can capture images with a resolution of 1920 $\times$ 1080. It also detects the image grid water level and calculates the water level height utilizing a virtual water gauge.
• Specifications: The system has sensors for the temperature, humidity, light intensity, and rain detection. It also incorporates a waterproof infrared camera, an Internet of Things communication module, Arduino Mega2560, Raspberry Pi 4B, solar panels, a solar power manager, a rechargeable lithium battery, a waterproof box, and support rods.
• Power Supply: The system can accommodate a maximum power supply of 5 V, 3 A and utilizes a 20 W monocrystalline silicon solar panel and a 10 Ah lithium battery.
• Cost: USD 750.

The developed physical system is depicted in Figure A2. Figure A2a presents an exterior view of the virtual water gauge system, while Figure A2b illustrates the configuration of the components within the waterproof box.

3.3. Grid-Based Methods and Infrastructure Integration

To verify if the grid-based virtual water gauge system could fulfill the requirements of real-time river water level monitoring using a Raspberry Pi, we implemented the system on both a standard PC and a Raspberry Pi 4 to analyze an image of Chengmei Bridge’s water level. The time required for a single frame was recorded as 0.1259 s and 0.5428 s, respectively.

Table 1. Hardware specifications of the PC and Raspberry Pi 4B used for performance testing.

Model Type	Specification	Computing Time
PC	CPU: i5-8300H GPU: GTX: GTX-1650 RAM: 20 GB	0.1259 s
Raspberry Pi 4 B	CPU: ARM Cortex A72 RAM: 8 GB	0.5418 s

shows the PC and Raspberry Pi 4B hardware specifications used in this validation process.

We amalgamated grid-based methods with infrastructure elements like CCTV cameras and Raspberry Pi devices for data processing. This fusion of technologies enabled us to devise an efficient system capable of real-time monitoring, thereby enhancing the reliability of our model.

Integrating the virtual water gauge system significantly enhanced the real-time monitoring capabilities of our model. This system, which operates independently of the reliability of physical gauges, demonstrated its ability to provide accurate water level readings while effectively reducing deployment costs.

When applied to real-time river monitoring, this system presents several benefits:

• Speed and Efficiency: As indicated in the tests, the system can process images swiftly, even on a low-powered device like a Raspberry Pi. This facilitates near-real-time monitoring, essential when immediate responses to fluctuating water levels are needed, such as during floods or heavy rainfall.
• Accessibility and Cost-Effectiveness: Using a Raspberry Pi makes the system highly accessible and cost-effective. A Raspberry Pi is affordable and widely available, allowing system deployment in multiple locations without substantial financial expenditure.
• Automation and Accuracy: The system automates the process of water level monitoring, eliminating the need for time-consuming and error-prone manual measurements. Incorporating image recognition and the virtual water gauge also enhances the precision of water level readings.
• Flexibility: The system can be adapted to monitor various rivers or bodies of water by simply changing the image source, making it a flexible solution tailored to diverse monitoring needs.
• Environmentally Friendly: The ability of the system to operate on a low-powered device like a Raspberry Pi means it can be powered by renewable energy sources, such as solar panels, marking it as an environmentally friendly solution.

4. Discussion

This section examines the model’s accuracy and performance validation, particularly emphasizing its performance under various weather conditions. Furthermore, we investigate how weather conditions affect the precision of the virtual water gauge to enhance the model’s decision-making in future scenarios. Despite this, we acknowledge certain constraints of our study, including the necessity to gather data in differing climates and the challenge of establishing the virtual water gauge scale without a physical water gauge. We propose the following solutions to mitigate these challenges and increase the system’s practicality.

• Clarifying Model Accuracy
  The initial data accuracy for rainy days was 50.1%, mainly due to threshold prediction errors within the dynamic binarization prediction model. This resulted from the reduced sample size during heavy rainfall periods, which diluted the binarization threshold characteristics during the modeling process.
  While the adjusted accuracy increased to 83.6%, we could further augment the model accuracy by applying techniques such as Resampling [52], Cost-Sensitive Learning [53], or other methods to manage unbalanced datasets and enhance the model’s ability to predict binarization thresholds.
  Alternatively, installing a photometric sensor at the monitoring site to set the binarization threshold directly based on lumen values could minimize errors in the judgment of the virtual water gauge water levels due to the binarization threshold.
• The Relationship between Weather and Virtual Water Gauge Accuracy
  We categorized the images into sunny, cloudy, and rainy conditions to analyze the influence of different weather scenarios on the model.
  • During sunny days, spikes were primarily attributed to misinterpretations caused by sun reflections on the grid, fallen leaves or other debris, and large waves created by strong winds on the water’s surface.
  • On cloudy days, the reflection and shadow on the water surface, influenced by cloud variations, could cause the intensity of the sunlight on the river surface to change rapidly as the cloud layer moves. This may induce fluctuations in the light intensity during the image processing stage, potentially leading to inaccuracies in water level detection. Consequently, future measures may require algorithm adjustments or the utilization of data from other sensors to mitigate the impact of these light fluctuations on water level measurements.
  • On rainy days, especially during intense rainfall, the selected grid was filled with rainwater, leading to misjudgments of a full water level, or the rain hitting the water surface caused large waves, leading to misjudgments of a waterless state.
  In the future, we can deploy the Boyer–Moore majority algorithm [54,55] or other data mining techniques to identify the primary categories or samples to circumvent or rectify situations where the grid is misjudged.
• Merits of the Grid-Based Approach
  The grid-based method used in this study offers two key benefits:
  • Efficiency in Image Processing: With the image size of 1920 × 1080 pixels and the grid size of 56 × 56 pixels, this approach substantially reduces the computational complexity and workload by up to 98.6%. This efficiency enables real-time river water level monitoring, enhancing the system’s overall performance.
  • Precision in Height Calculations: Unlike traditional water gauges that use meter units, the grid height calculations allow for a more detailed height scale, capturing exact height data, such as 1.23 m. This granularity provides comprehensive height information, enhancing the accuracy and precision of water level measurements and predictions.
• Limitations and Future Directions
  Compared to physical water gauges, a fundamental limitation of this research is the necessity to gather river image data under various climatic conditions for training before deployment unless preexisting historical river image data are available.
  We plan to introduce an automated process in future research to overcome this limitation. This approach will involve establishing a computerized data collection and processing workflow to save time and ensure data consistency.
  In instances where rivers are equipped with actual water gauges, the scale of the virtual water gauge can be defined based on the water gauge in the image. However, another challenge arises for rivers without real water gauges. In these cases, a temporary ruler must be established on-site during the initial setup. This ruler is then screen-captured, and the scale of the virtual water gauge is defined through it.
  In response to these challenges, our lab is harnessing Augmented Reality (AR) technology to measure object heights using CCTV or mobile camera devices such as mobile phones. This approach aims to alleviate the need for physically establishing a ruler on-site.
  Furthermore, we plan to apply geometric and trigonometric functions to correct height deviations caused by camera angles. These advancements can significantly enhance the accuracy and applicability of virtual water gauges. By implementing these strategies, we aim to address the current limitations and expand the application of virtual water gauges, enabling swift deployment in most rivers.

This study illuminates the complexities and considerations of utilizing AI technology for intelligent water resource management and environmental sciences. Despite the identified challenges and limitations, this research has demonstrated promising potential for applying AI in this domain. We anticipate that the insights garnered will not only enhance the accuracy and reliability of our existing model but also lay the groundwork for future innovations. As we refine our methodology and explore new techniques, we are committed to advancing our understanding and application of AI technology in hydro-informatics systems toward a more sustainable future.

5. Conclusions

The research presented in this paper demonstrates the significant potential of AI technology in environmental sciences and water resource management. Our study focused on developing and validating a grid-based virtual water gauge model, which utilized a Raspberry Pi platform, for real-time river water level monitoring. The model’s accuracy varied across different weather conditions, with the best results achieved on sunny days at 96.3%, cloudy days at 90.9%, and rainy days at 83.6%.

This study also culminated in the design of a cost-effective and efficient real-time river water level monitoring system. The system combines the grid-based virtual water gauge with a Raspberry Pi and other corresponding components, providing an accessible and automated solution for water level monitoring. The system demonstrated a strong performance, even on a low-powered device, and offered flexibility in monitoring various rivers or bodies of water.

However, the research also identified certain limitations and challenges. These included data collection under varied climatic conditions and the challenge of establishing a virtual water gauge scale without a physical water gauge. Future work should address these limitations and improve the model’s accuracy and robustness.

Potential future improvements include applying unbalanced dataset management techniques to enhance binarization threshold predictions, deploying data mining techniques to rectify grid misjudgments, and automating model training. Additionally, future research could explore applying geometric and trigonometric functions to correct height deviations caused by camera angles.

Despite the challenges, this study provides valuable insights into the application of AI technology in hydro-informatics systems. As we continue to refine our methodology and explore new techniques, we remain committed to enhancing our model’s accuracy, reliability, and practicality, laying the groundwork for future innovations in this critical area. Ultimately, we aim to advance the understanding and application of AI technology in environmental sciences, contributing to a more sustainable future.