An Automated System to Monitor River Ice Conditions Using Visible Infrared Imaging Radiometer Suite Imagery

Abstract

This study presents an innovative, automated deep learning-based technique for near real-time satellite monitoring of river ice conditions in northern watersheds of the United States and Canada. The method leverages high-resolution imagery from the VIIRS bands onboard the NOAA-20 and NPP satellites and employs the U-Net deep learning algorithm for the semantic segmentation of images under varying cloud and land surface conditions. The system autonomously generates detailed maps delineating classes such as water, land, vegetation, snow, river ice, cloud, and cloud shadow. The verification of system outputs was performed quantitatively by comparing with existing ice extent maps in the northeastern US and New Brunswick, Canada, yielding a Probability of Detection of 0.77 and a False Alarm rate of 0.12, suggesting commendable accuracy. Qualitative assessments were also conducted, corroborating the reliability of the system and underscoring its utility in monitoring hydraulic and hydrological processes across northern watersheds. The system’s proficiency in accurately capturing the phenology of river ice, particularly during onset and breakup times, testifies to its potential as a valuable tool in the realm of river ice monitoring.

1. Introduction

The complex significance of river ice, with its varied roles in hydrological processes and its impact on water resources management operations, holds an acute level of importance. Its influence extends over a wide range of areas, including flood dynamics, water resources management, water supply, and hydropower generation, underscoring its essentiality in maintaining and managing our water-dependent ecosystems and infrastructures [1,2]. River ice phenomena, which encompass ice formation, growth, and subsequent breakup can instigate flooding and inflict considerable damage to infrastructure, notably at the higher latitudes of the Northern Hemisphere. Here, river ice influences the hydrological patterns of nearly 60% of rivers for a significant portion of the winter [1]. In higher latitudes, it asserts substantial control over a majority of fluvial processes [3]. The interruptions caused by these ice-related events can lead to disruptions in water supply and hydropower generation, triggering substantial socioeconomic ramifications [4].

Despite the indispensable role river ice plays in the hydrological cycle and water resources management, its systematic monitoring poses formidable challenges. While in situ observations hold substantial worth, their utility is curtailed by issues of accessibility, particularly during severe winter conditions, rendering them inadequate for comprehensive and continuous data acquisition [5]. There exists a notable deficiency in the systematic mapping of river ice at a continental scale, primarily attributed to logistical constraints and safety concerns inherent in ground-based measurements. In the United States, river ice data can be gleaned from reports published by the US Geological Survey (USGS) Water Resources Division and from USGS stream-gauging station data. Additionally, river freeze up and breakup and ice jam information is cataloged by the US Army Cold Regions Research and Engineering Laboratory (CRREL) and is available in the Ice Jam Database [6]. However, the incorporation of satellite data appears as a promising adjunct to in situ data and aerial surveys, potentially enhancing the scope and reliability of river ice monitoring [3].

For instance, in the realm of river ice research, pioneering studies have utilized remote sensing data for river ice mapping. Chaouch et al. (2014) developed an automated algorithm using the Moderate Resolution Imaging Spectroradiometer (MODIS) data, offering reliable information for accurate hydrological forecasts on a regional scale [5]. Yang et al. (2020) employed a trove of clear-sky Landsat images to evidence the global decline in river ice extent, providing a validated model for future predictions [3]. Further, Altena and Kääb (2021) showcased the feasibility of a multi-satellite monitoring system, demonstrating the potential of leveraging low- and medium-resolution satellite sensors to monitor river ice movement [7]. These seminal works provide a robust foundation for further innovation in river ice monitoring.

Satellite-based remote sensing has emerged with the advent of several new and enhanced sensors in recent years as a robust avenue to circumvent these limitations. Specifically, the Visible Infrared Imaging Radiometer Suite (VIIRS), embarked on the Suomi-NPP (SNPP), NOAA-20, and NOAA-21 satellites, offers repetitive and wide-ranging coverage, thereby establishing itself as an exceptional candidate for river ice condition monitoring. Nevertheless, most extant studies employing VIIRS imagery for hydrological applications have predominantly focused on flood mapping [8,9,10] and snow cover detection [11]. Despite the incorporation of numerical models and machine learning methodologies, most of these mapping algorithms continue to be predominantly threshold-based, largely dependent on terrestrial observations. This factor considerably constrains their adaptability and scalability, particularly in regions where data is scarce. Furthermore, to the authors’ knowledge, the developed system is unique in its capability to automatically detect and map river ice on a continental scale, estimate its concentration and thickness regularly, and allow the user to infer its motion leveraging the coincident multi-satellite observations.

This study introduces a novel satellite-based system for river ice mapping using VIIRS imagery that addresses the existing operational gap and provides forecasters and decision makers with daily updates on river ice conditions on a continental scale, covering northern watersheds in the US and the entire Canadian territory. The system leverages the capabilities of deep learning techniques, which have demonstrated extraordinary efficacy in the realm of remote sensing. The use of deep learning-based techniques with their advanced capabilities for the segmentation of VIIRS images is expected to address the detection and mapping of river ice as well as the generation of additional classes such as snow, water, and cloud, which are also useful for the forecasters to conduct a holistic analysis of the hydrological processes. More precisely, the presented study that leverages the utilization of deep learning not only aspires to amplify the accuracy of river ice detection but also assists in the extraction of more intricate data, an endeavor that conventional methodologies generally struggle to achieve [12]. Another novelty that the proposed system holds is the fact that all the generated images are disseminated in a Google Earth Engine (GEE) interface that allows the users to display, compare, and query the different products.

2. Materials and Methods

2.1. Study Domain

The geographic domain covered by the proposed system lies between 30 degrees north and 80 degrees north, which stretches between the southern US and the northern limit of Alaska. In terms of longitude, the system’s domain stretches between −180 degrees and −60 degrees, which encompasses all rivers and waterbodies from western Alaska to the east coast of the US and Canada (Figure 1d). Given the large extent of the covered domain, the use of moderate spatial resolution satellite images is more straightforward from a feasibility perspective. The use of high-resolution imagery has certainly more potential in terms of the level of information to generate. However, the existing sensors to date lack the exhaustive spatial coverage to encompass a continental domain such as the ones covered by VIIRS images.

The geographic extent of the system covers all seven northern River Forecast Centers (RFCs), namely, Northwest RFC (NWRFC), Ohio RFC (OHRFC), Northcentral RFC (NCRFC), Mid-Atlantic RFC (MARFC), Missouri Basin RFC (MBRFC), Alaska-Pacific RFC (APRFC), and northeast RFC (NERFC) (Figure 1a,b). All the covered RFCs are under the National Weather Service. Their domains are also part of the National Water Model (NWM), a US-wide hydrological model operated by the NOAA National Water Center (NWC). It is worth noting that the Colorado Basin RFC might be partially impacted by river ice during winter, but we do not have a record of active ice breakup seasons in CBRFC.

The seven northern RFCs are impacted by river ice every year. However, the frequency and the risk of ice-induced flooding vary among these RFCs. According to CRREL, the Missouri Basin and Alaska-Pacific RFCs have the highest number of ice jams on record due to the significant extent of the territory and the fact that all rivers in the state freeze in winter and are impacted subsequently by the breakup season (Figure 1c). In 2012, for instance, the village of Galena along the Yukon River was impacted by a record ice jam, which caused significant flooding that led to the evacuation of the village (Figure 1e).

Numerous other instances highlight the detrimental effects of ice jams, further emphasizing the urgency of this issue. For example, NOAA (https://www.nesdis.noaa.gov/river-ice-flooding-initiative accessed on 19 August 2023) and CRREL [6] have documented several significant ice jams that have induced substantial disruptions and damage. Despite the insights these records provide, it is important to consider the inherent limitations of these sources of data. Notably, the CRREL database operates on a point-based system, which does not incorporate remote sensing data into the published records [6]. This constraint essentially narrows the scope of the information and potentially overlooks broader trends in ice jam occurrences and their impacts. Cases reported by CRREL underscore the adverse impacts that ice jams can have on communities and the environment and the need for more comprehensive monitoring systems. While CRREL’s database provides valuable information about these instances, the integration of remote sensing data could further enhance the depth and reach of this information, allowing for more informed, accurate, and timely responses to such environmental crises.

2.2. Datasets

2.2.1. VIIRS Images

The Joint Polar Satellite System (JPSS) provides essential observations for severe weather prediction and environmental monitoring, facilitated by three currently operational satellites: the Suomi National Polar-Orbiting Partnership (S-NPP, launched October 2011), NOAA-20 (JPSS-1, launched November 2017), and NOAA-21 (launched November 2022). These satellites, orbiting at an altitude of 824 km, offer global coverage and cross the equator at 1:30 am and 1:30 pm local time, effectively making two observations per day at lower latitudes and more frequent observations towards the poles. Each satellite completes approximately 14 orbits per day, taking around 100 min to complete a single orbit [14,15,16].

The Visible Infrared Imaging Radiometer Suite (VIIRS) is the next-generation scanning radiometer integrated into the JPSS, responsible for measuring the properties of the Earth’s atmosphere, ocean, and land. It builds upon the legacy of the Moderate Resolution Imaging Spectroradiometer (MODIS) on the Terra and Aqua satellites and the Advanced Very High-Resolution Radiometer (AVHRR) on the NOAA and Metop series of satellites [14,15,16].

VIIRS operates with 22 spectral bands that range from visible to longwave infrared. These include 16 moderate resolution bands (M-bands) that provide a resolution of 750 m at the nadir, five imaging resolution bands (I-bands) offering 375 m resolution at the nadir, and one day-night band (DNB) ensuring 750 m resolution across the scan. It is noteworthy that the VIIRS’s swath width is broader (~3040 km) compared to that of MODIS (~2330 km), providing daily full global coverage in both day and night sides of the Earth, with minimal degradation in resolution at swath edges. Notably, VIIRS ensures no coverage gaps in the tropics. The distribution of VIIRS data takes the form of granules, each comprising 48 scans that encompass approximately 85 s of data, covering roughly 570 km in the along-track direction [14,15,16].

This study employs I-band images derived from the NOAA-20 and S-NPP satellites’ ascending daytime passes. Specifically, the following VIIRS I-bands are used: I01, I02, I03, and I05. In addition, the zenith, the satellite, and the solar angle layers are used. The angles are important to define the geometry of the observation and capture its spatial and temporal context. This is particularly important in the case of such a large domain.

This study uses VIIRS images from two platforms, namely NPP and NOAA-20. The identical sensors flown on these platforms have similar orbits, but they are 50 min apart. The acquisition of images from similar sensors with a reasonable lag is useful to generate composites that should minimize the cloud coverage, assuming that surface conditions do not change significantly within the 50 min separating the subsequent acquisitions.

2.2.2. Auxiliary Datasets

Other ancillary datasets are used in this study to support the use of deep learning and enhance the segmentation with additional inputs and explanatory variables. Land cover is a key input that should impact the segmentation of VIIRS images. The algorithm will use the Terra and Aqua combined Moderate Resolution Imaging Spectroradiometer (MODIS) Land Cover Type (MCD12Q1) product as one of the input variables for river ice remote sensing [17]. In addition, the climatology of Land Surface Temperature (LST) was proven to be useful in the identification of clouds via the calculation of the vertical temperature gradient between LST climatology, originally calculated under cloud-free conditions and the I05 thermal band of VIIRS. A narrow gap between the climatology of LST and I05 indicates that the scene is cloud free, whereas the presence of clouds may lead to a significant difference that may vary depending on the type of clouds, their thickness, and their altitude. The LST climatology was obtained from the NASA MODIS product MOD11C3 Version 6, which provides monthly Land Surface Temperature and Emissivity (LST&E) [18]. Another dataset that was necessary to conduct the segmentation of VIIRS images and delineate waterbodies was the high-resolution land-water mask, which was generated using Landsat images collected over around 20 years, a period that is long enough to capture extreme flooding conditions to ensure that floodplains are included in the water mask [19].

2.2.3. Validation Datasets

The evaluation of the system performance was conducted using qualitative and quantitative methods. The qualitative methods make use of collected on-the-ground observations from cameras deployed and maintained by USGS at several stations around the country. Other photographs are also obtained from surveys conducted by NOAA NWS, whether during field surveys along the rivers that are ice-prone or airborne surveys such as the RiverWatch program that takes place in Alaska to monitor river ice conditions across the state. In addition, the qualitative evaluation relies on the comparison with other collocated satellite observations from high-resolution sensors such as Sentinel 1 and 2 or moderate-resolution sensors such as Sentinel 3. Moreover, when reporting streamflow observations at river stations, USGS adds a flag that is indicative of ice presence. USGS usually filter out streamflow observations that are collected in the presence of river ice. The flag is often added to a posteriori during the quality control of the collected data. It assumed that the absence of streamflow observation and the addition of the quality flag is useful in the context of river ice as the flag confirms the presence of ice at the location of the streamflow station.

In the absence of operational and official river ice products, the quantitative evaluation of the proposed system is challenging. Nonetheless, over the years, the New Brunswick Department of Environment and the local Government in Canada generated river ice charts showing the extent of ice in the main rivers in the province and those in shared watersheds with the state of Maine in the US. This is a unique product that uses the River Ice Observation and Reporting System tool internally to help inform the flood forecasting process and emergency response coordination. Hence, the charts issued by the authorities in New Brunswick, Canada, were used for the quantitative assessment of the proposed VIIRS product.

It is worth noting that despite the fact that the system generates various classes that cover vegetation, snow, water, ice, and clouds, the focus of the verification here is on the performance of mapping river ice. Although the variability of the extent of the other classes such as snow and water are indicative of ice dynamics as their change could often be a precursor for ice breakup, their quantitative verification is beyond the scope of the paper.

2.3. Use of U-Net for VIIRS Images Classification

The employed methodology leverages the sophisticated U-Net network architecture. This deep learning framework is deployed to seamlessly integrate all the VIIRS channels with supplementary surface layers, thereby facilitating an efficient image segmentation process. This strategy paves the way for generating several distinct classes, which encompass clouds, river ice, snow, water, bare land, and vegetation. This diversified classification capability stands as a testament to the efficacy and versatility of the U-Net network and its applicability in the context of VIIRS data. The overall flow chart of the system is presented in Figure 2.

Cloud detection is one of the challenging tasks when using optical images for the monitoring of surface conditions. Cloud spectral signature can exhibit a significant variability depending on cloud type, height, thickness, latitude, and observation geometry, i.e., solar and satellite angles. Here, we propose a two-stage approach that consists of using the first cloud-specific net to map clouds. Then, the cloud free part of the satellite image is segmented separately with the second net that was trained using scenes with dominant cloud free conditions.

The first step addresses cloud detection using a trained net over scenes with prevailing cloudy conditions. The scenes used in the training that are specific to cloudy conditions were carefully examined visually to select those with various cloud types. In addition, cloud-free scenes, which were identified after careful visual examination were used to train another net that is specific for clear conditions. The selected scenes that were used for the training of the UNet were labeled, and the following classes were identified: water, ice, snow, bare land, vegetation, and clouds. The defined classes during the training step will be generated using the trained UNet in the validation and testing steps.

Cloud detection using the trained net leverages the sensitivity of the vertical gradient of temperature to cloud presence. The vertical gradient of temperature corresponds to the difference between LST climatology and the observation in the thermal I5 channel. This gradient that is calculated for each pixel is one of the main input variables that guides the detection of clouds. The training of the first net included cloud and cloud shadow pixels despite their distinct spectral signatures. The likelihood of finding cloud shadows is higher when using cloudy scenes. Therefore, the cloud shadow class was generated as an output of the first clustering along with the cloud class. So, the output of the first net is two classes, namely, cloud and cloud shadow.

The second net, which is focused on cloud-free conditions is used to generate the following classes: water, land, vegetation, snow, and ice. The first class includes inland waterbodies (rivers and lakes) and potentially overbank flooding beyond this work’s scope. The land class corresponds to bare soil and grasslands. The vegetation class encompasses different vegetation types. The snow and ice classes exhibit similar spectral signatures in the VIIRS I-bands when compared to the other classes. In this study, snow is identified over land pixels only, and ice is only detected over lakes and rivers. It is possible for ice to be covered with snow. In such a case, the system will classify the pixel as ice. The training and validation of both nets follow the same process. The same hyper parameters are used. Each scene used for the training of each net is divided into training, test, and validation fractions that cover 80%, 10%, and 10% of the scene, respectively.

Segmentation tasks commonly employ encoder–decoder architectures [20,21,22,23] due to their widespread usage. In this framework, input images undergo a process of encoding at various stages, followed by upsampling by the decoder to restore the original size. The fundamental idea behind this architecture originated from the Fully Convolutional Network (FCN) [22]. FCN replaces the fully connected layers of the network with convolutional layers and employs deconvolutional layers to perform upsampling, thereby restoring the segmentation results. Building upon the FCN paradigm, U-Net introduces a more sophisticated decoder that incorporates compensatory elements at corresponding levels to account for local information [20].

The U-Net architecture, a type of Convolutional Neural Network (CNN) approach, was initially proposed to enhance segmentation performance on biomedical images [20]. This architecture proved particularly effective for applications that require output images of similar size to the input images. To achieve this, the U-Net architecture employs a two-step process of down sampling (encoder) using convolutional layers, followed by upsampling (decoder) to restore the image while performing segmentation [20].

The encoder path captures the contextual information of the image, generating feature maps. It constitutes the initial component, typically employing a pre-trained classification network such as VGG or ResNet. The encoder’s purpose is to encode the input image into feature representations at various levels by employing convolution blocks, followed by downsampling via maxpooling operations [20]. However, the decoder path uses transposed convolutions to enable precise localization. The decoder constitutes the latter part of the architecture, serving the purpose of semantically projecting the discriminative features learned by the encoder at a lower resolution onto the pixel space at a higher resolution. This process aims to achieve dense classification. The decoder is composed of up sampling and concatenation operations, which are subsequently followed by regular convolution operations. Consequently, the U-Net architecture produces an output of the same size as the input image with a specified class depth. Leveraging the U-Net architecture, each pixel in the input image can be estimated individually, making it highly suitable for tasks such as building extraction [20].

In addition to the determination of ice extent as an output of the applied segmentation, the system calculates ice concentration for each pixel within the scene that falls in a waterbody (river or lake). The ice concentration is calculated using a linear model that uses two tie points of the I02 channel, one for open water reflectance and another for pure ice pixel reflectance. Given the significant variability of reflectance spatially and seasonally, the tie points used in the calculation of ice concentration are updated regularly as new scenes are being processed using the 10% and 80% quantiles of the reflectance values for the water and ice tie points, respectively.

2.4. Evaluation Metrics

In the present study, a quantitative verification of the U-Net deep learning model was executed via a raster-to-raster comparison. This approach involved a juxtaposition of the river status data derived from the model output and the rasterized river status data extracted from the ice charts published by the authorities in New Brunswick, Canada. The evaluation metrics for the comparison were encapsulated in a 2 × 2 confusion matrix, representing the status of a pixel as either ice or water ice.

The components of the confusion matrix comprised True Positives (TP), False Positives (FP), False Negatives (FN), and True Negatives (TN). Here, TP represented the accurate predictions of river ice presence, and a high TP count signified commendable predictive accuracy for positive instances. Conversely, FP denoted erroneous predictions of river ice presence, with a high FP count highlighting a tendency to incorrectly identify river ice presence. Similarly, FN stood for incorrect predictions of river ice absence, with a high FN count indicating missed instances of river ice presence. Lastly, TN pertained to correct predictions of river ice absence, with a high count of TNs underscoring predictive accuracy for negative cases. To measure the model’s overall accuracy, the Proportion Correct (PC) was utilized. PC is the ratio of correct predictions (TP and TN) to the total number of instances, calculated as follows:

$$\mathrm{P}\mathrm{C}=\frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}+\mathrm{T}\mathrm{N}},$$

(1)

The Bias ratio (B), defined in Equation (2), was employed to ascertain the model’s bias. The metric indicates whether the predicted occurrences (in this case, river ice conditions: (TP + FP) are overpredicted or underpredicted compared to the observed occurrences (total actual occurrences: TP + FN). A bias ratio of 1 implies perfect prediction, whereas a value greater than 1 indicates over-prediction, and a value less than 1 signaled under-prediction.

$$\mathrm{B}=\frac{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}},$$

(2)

The model’s efficacy in correctly detecting positive instances (here, the river ice condition) was assessed via the Probability of Detection (POD). As shown in Equation (3), POD represents the fraction of the actual occurrences that were correctly predicted. A POD value close to 1 suggests high detection capabilities, while a value close to 0 indicates poor detection. Concurrently, the False Alarm Ratio (FAR), or the ratio of false positive instances to the total predicted positive cases, was used to quantify the number of incorrectly predicted positive cases (Equation (4)).

$$\mathrm{P}\mathrm{O}\mathrm{D}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}},$$

(3)

$$\mathrm{F}\mathrm{A}\mathrm{R}=\frac{\mathrm{F}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}},$$

(4)

Finally, the Critical Success Index (CSI) was employed as a comprehensive accuracy metric that considered TP, FP, and FN. The CSI ranged from 0 to 1, with 1 being the ideal score, computed as follows:

$$\mathrm{C}\mathrm{S}\mathrm{I}=\frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}},$$

(5)

These measures collectively offer an extensive understanding of the model’s performance, particularly useful in binary classification tasks such as this study’s prediction of the presence or absence of river ice. They provided insights into the model’s accuracy in predicting river ice presence (POD), its bias towards over- or under-prediction (Bias ratio), its general accuracy (PC), the frequency of incorrect predictions (FAR), and an all-encompassing measure of the model’s performance (CSI).

3. Results

3.1. Remote Sensing-Based Validation of the VIIRS River Ice Product

Figure 3 shows an example of the three different products that the river ice system generates, namely ice extent in Figure 3b, ice concentration in Figure 3d, and a map that integrates all classes in Figure 3e. In this example, VIIRS observation over the Lake Saint-Pierre area in the Saint Lawrence River, downstream of Montreal, Canada, on 14 January 2022 is shown. The generated maps in this example are compared to a scene from Sentinel 3 captured on the same day over the same region. Ice extent in Figure 3b agrees reasonably well with the extent shown in the Sentinel 3 image, as the VIIRS product clearly captures ice presence along the southern and northern banks of the Saint Lawrence River and Lake Saint Pierre. The central part of the river and lake’s cross section is maintained ice free in part due to ice breaking activities that are regularly conducted by the Canadian Coast Guard to maintain the Saint Lawrence seaway ice-free and prevent ice jams and flooding in its tributaries, especially close to spring breakup season [24]. This section of the Saint Lawrence River tends to freeze between January and March, with varying onset and breakup dates depending on the year. For instance, in 2023, Saint Lawrence had one of the lowest ice extents on record [25].

The calculated ice concentration in Figure 3d agrees well with the ice extent in the VIIRS product and the RGB Sentinel 3 image. Ice concentration values are the highest along the northern and southern riverbanks and in Saint Pierre Lake. The concentration of ice gradually decreases towards the river’s seaway in the center of the cross-section where ice concentration values are around zero, which matches the appearance of ice-free and water pixels in the sentinel 3 RGB image and VIIRS ice extent product in Figure 3b. Ice concentration values were also high in the Saint Lawrence tributaries on the southern side of the river, indicating prevailing freezing conditions in these shallow and narrow rivers, which corroborated by the freezing conditions reported in the ice extent map and Sentinel 3 RGB image. It is worth noting that ice extent and ice concentration are calculated independently in the system. While ice concentration values are determined using adaptive tie points adjusted depending on the scene, ice extent is solely a result of the U-Net segmentation. The agreement between ice concentration distribution and ice extent is therefore an indication of their reliability. Figure 3e displays all the determined classes, including ice extent that is reported in Figure 3b. Even if the verification of snow extent is beyond the scope of this study, one can notice that the spatial distribution of snow on the surface is in agreement with the one seen in Sentinel 3 RGB. The extent of snow north of the Saint Lawrence River well matches its distribution in the VIIRS product. Despite the focus on mapping river ice in this study, the extent of snow cover and its change, especially during the melt period is highly informative and indicative of river sections where the breakup is more likely.

The assessment of the river ice product was conducted continuously for three winter seasons via regular communications with end-users from NOAA River Forecast Centers (RFCs) under the National Weather Service (NWS). The assessment was qualitative and relied on collected observations from ground-based cameras mostly operated by USGS, field observations from government authorities and/or citizen science programs, and flyovers such as the RiverWatch flight surveys that were conducted in Alaska during the breakup period. Overall, the assessment of the product according to the received feedback shows its capabilities to accurately map river ice extent and concentration, especially in large and moderately wide rivers. Figure 4 shows one example of the assessment of the product using ground-based observation of the Missouri River using a camera that is part of a USGS station [26]. Two observations are presented in Figure 4, one on 6 April 2023 (Figure 4c) and another on 9 April 2023 (Figure 4f). The camera frames showed a significant change in ice conditions as the second photograph taken on the 9th of April showed ice-free conditions, indicating a rapid transition from an ice-covered river cross section to an ice-free one. The corresponding river ice map displaying ice concentration values at the same location showed a similar behavior as the concentration dropped from around 90% (Figure 4b) to values closer to 20% (Figure 4e). The persistent ice concentration value (i.e., 20%) that is determined by the system despite the fact that the camera frame shows open water conditions could be attributed to the existence of snow on the ground as it is shown around the bottom edge of the photographs (Figure 4f) and all classes layer of the segmented image (Figure 4d). Overall, ice concentration values in the Missouri River showed a swift drop that is in line with the local observation obtained from the USGS camera. The rapid transition and change in river ice conditions are common in mid-latitude regions in the US. The quick dynamic of river ice, as illustrated in this example highlights the importance of developing an automated system for the monitoring of quick changes, potential ice jam formation, and the occurrence of ice-induced flooding.

3.2. Quantitative Evaluation and Performance Insights of the VIIRS River Ice Product

The quantitative assessment of the VIIRS river ice product was conducted using river ice charts reported by the New Brunswick Department of Environment and the local Government. Figure 5 displays an example of the charts issued by the authorities in New Brunswick reporting ice conditions as of 31 March 2023. To our knowledge, the generated river ice charts are the only available river ice-focused maps georeferenced that show the spatial distribution of ice in rivers along with its concentration and thickness. The VIIRS river ice product determines ice extent and concentration in all waterbodies, which include rivers and lakes. The river ice charts issued by the authorities in New Brunswick comprise rivers only and do not include lakes. A quantitative assessment was conducted where both products overlap.

Reference data for model evaluation were obtained on specific dates in 2023: 7 February, 9 February, 3 March, 10 March, 21 March, 11 April, 14 April, and 15 April. During these periods, in situ observations were available, and the generated VIIRS river ice images had no or limited cloud cover. The selected dates corresponded to observed river statuses of open water and ice cover on the Saint John River and the Aroostook River. Such conditions allowed for a rigorous assessment of the model’s performance in accurately distinguishing between ice cover and open water.

The Proportion Correct, indicating the model’s overall accuracy, was 0.747. This signifies that approximately 75% of the model’s predictions regarding the presence or absence of river ice were accurate, demonstrating a solid level of predictive accuracy. However, it also implies that there is room for further refinement to improve the model’s performance. The Bias ratio stood at 0.870. Being less than 1, this indicates a slight tendency for the model to under-predict the presence of river ice. This suggests that improvements could be made in the model’s sensitivity towards identifying instances of river ice (Figure 6).

The Probability of Detection was measured at 0.768, indicating that the model accurately detected around 77% of the actual positive cases. This demonstrates a strong capability in identifying the presence of river ice, which is a promising result. However, the False Alarm Ratio was calculated as 0.117, suggesting that about 12% of the model’s predicted positive cases were incorrect. This is a relatively low proportion, indicating that the model has a low tendency to falsely predict the presence of river ice where there is none. Lastly, the Critical Success Index was calculated as 0.697. This value, close to the ideal score of 1, suggests that the model has a good overall performance considering both over-predictions and missed predictions. However, as with the PC, this also indicates potential for further refinement of the model (Figure 6).

Overall, these metrics provide a comprehensive understanding of the model’s performance, highlighting its strengths in accurately detecting river ice presence and areas that could be refined to further enhance its predictive accuracy. To this end, a more granular analysis was carried out to decode the model’s bias by examining the rate of false positives and false negatives in relation to open water status. Specifically, the periods when the river was in open water status but was incorrectly predicted by the model as ice-covered (FPW: False Positive Water), and vice versa when the river was ice-covered yet the model predicted open water status (FNW: False Negative Water).

The investigation revealed that the FPW rate was a remarkable 0%. This outcome suggests that the model exhibited exceptional accuracy in detecting open-water conditions. However, a stark contrast was observed in the rate of FNW, which stood at 25%. This result translates to instances where the model incorrectly predicted the river’s status as open water while it was, in fact, ice-covered (Figure 6).

This discrepancy partially elucidates why the model is underestimating the river ice cover, as indicated by the Bias ratio of less than 1. It is also worth noting that the model predicted snow cover over some river stretches, which introduced an additional source of bias. Nonetheless, the primary focus of our analysis remained anchored to the binary open water and ice-covered states that the river typically assumed during the observation periods.

The obtained findings underline the necessity for refining the model’s sensitivity towards ice cover detection whilst acknowledging its commendable performance in mapping open water conditions. The results highlight the nuanced performance of the model and offer valuable insights into potential areas of improvement, thereby guiding further model refinement.

3.3. Systematic Tracking of Snowmelt and Ice Dynamics Using Automated Surface Mapping

Figure 7 shows the progress of snowmelt across Alaska as a representative example that demonstrates the system’s capability to track the change in surface conditions, especially during season transitions. The statewide scenes shown in Figure 7 are the result of mosaicking of all received VIIRS scenes on a specific day. Given the northern location of the state, several VIIRS swaths can be collected in one day due to the numerous overpasses of the sensor. The mosaic is built by maintaining the most recent cloud-free observations, which led to the minimization of cloud presence. On 19 April 2022, VIIRS product shows the state of Alaska entirely covered by snow, indicating that the breakup season has not started yet. Usually, river ice breakup is preceded by snowmelt, which generates the runoff that increases streamflow, which triggers a mechanical river ice breakup. The following statewide scene of 1 May 2022 shows that the start of the snowmelt occurred in the central region of Alaska around the confluence of the Tanana and Yukon Rivers as well as the downstream region of the Kuskokwim River. Then, snowmelt and breakup rapidly propagated to the entire central region of the state and the south-central region where most of the state population lives. It is only towards the end of May that snow and ice in the north slope region of Alaska (north of the Yukon River) started to melt, according to the generated scene of 25 May 2022. The breakup in the north slope region continued for another three weeks, ending in the second week of June 2022, according to the scene generated on 12 June 2022, which shows a snow- and ice-free region in the north of Alaska.

The interplay between snowmelt and river ice breakup forms a pivotal aspect of the hydrological exploration in this study, and this is particularly evident when observing the Tanana River watershed during the 2021 breakup season. As demonstrated in Figure 8, which outlines the temporal shifts in freshwater coverage within the watershed, there is a clear correlation between the progression of snowmelt and subsequent changes in ice coverage. This relationship, although intricate, underscores the complex role that snowmelt plays in governing the dynamics of river ice processes.

Snowmelt flooding, a prevalent hydrological phenomenon in cold regions, often coincides with the onset of ice breakup, suggesting a complex mutual interaction. As the spring season ushers in warmer temperatures, snowmelt accelerates, thereby increasing the volume of water feeding into river systems. This surge in water levels applies additional pressure on the overlying ice, which in turn induces stress and triggers cracks parallel to the riverbanks. It is worth noting, however, that a time delay inherently exists in the response of ice coverage to the triggers of snowmelt, introducing a nuanced interplay between snowmelt and ice breakup dynamics.

The observations obtained during the 2021 breakup season on the Tanana River (Figure 8) provided a compelling case study for this interaction. Over a brief span of five days, from 17 April to 21 April, we observed a ~41% reduction in snow-covered areas, instigated by a rapid snowmelt. Notably, this substantial reduction in snow coverage was followed by only a ~16% decrease in ice-covered areas within the following four days, solidifying the theory of a lagged response of ice breakup to snowmelt. This four-day delay, although seemingly minor, has profound implications for our understanding of river ice phenomena and their potential impacts on water resources management.

The obtained findings reinforce the concept that snowmelt acts as a precursor to river ice mechanical breakup, with a temporal offset between these two processes playing a crucial role in hydrological forecasting. Accurate prediction of these events can inform proactive strategies, minimizing risks associated with flooding and potential infrastructure damage. These results contribute to the expanding understanding of these interlinked processes, offering a foundation for further research aimed at refining and improving predictive models. As we continue to grapple with the implications of changing climate conditions, such insights into river ice dynamics will prove indispensable for sustainable and efficient water resource management. It is worth noting that the reported lag highlights the lag between snowmelt and ice breakup specific to the watershed. Several factors related to the watershed properties such as topography and morphology, and snow cover such as its depth and density, control the phase lag between snowmelt and the start of breakup. This aspect can be investigated further in future work, leveraging the daily mapping of snow and ice with the system.

The proposed system’s capacity for automated and consistent mapping of ice and snow presents addresses a current operational gap in the field of hydrology. By facilitating the near real-time monitoring and spatial representation of different surface classes, the system helps users in the monitoring and understanding of evolving environmental conditions. Significantly, this continuous and automated approach enables the deduction of trends and changes in surface areas, including the distribution of ice and snow cover. This information is invaluable in predicting crucial phenomena, such as the timing and extent of ice breakup events, with greater accuracy and foresight than was previously possible. By identifying shifts in surface conditions and linking these with pertinent hydrological data, the system allows the user to anticipate events with a degree of certainty that has broad implications for water management and hazard prediction.

The proposed automated system transcends the limits of traditional, intermittent observational methods, providing a dynamic and richly informative tool for understanding and predicting the complexities of ice and snow dynamics. Consequently, it fosters a more robust, nuanced, and proactive approach to the study and management of our water resources and climatic conditions. The current system is automated, and all generated maps are hosted and disseminated via a Google Earth Engine-based interface hosted by the iSMART laboratory at the Stevens Institute of Technology [13]. Eventually, a full integration of the system in the decision-making process of NOAA NWS forecasters requires the transition of the system and its integration into the Advanced Weather Interactive Processing System (AWIPS), which is a platform used to gather several datasets from various sources for NWS experts to analyze and generate their forecast reports. The generated river ice product was disseminated using a Google Earth engine interface and made publicly available online [13]. The system is operational between October 1st and June 15th of the following year, a period that is assumed to cover all freeze-up and breakup events within the continental geographic domain of the product. In addition to VIIRS maps, the user can access other satellite images that are readily accessible via the Google Earth engine catalog. Hence, the user can overlap the generated river ice product with RGB images from different satellites such as Sentinel 2 and 3 that were acquired on the same day. This is particularly important for cross validation and product verification using independent verifications. It is worth noting that this study focuses solely on the segmentation of VIIRS images and that future work will address the classification of scenes from other sensors such as SLSTR on 3. Nevertheless, the display of the RGB images along with the VIIRS product has proven to be useful beyond the validation purposes. Figure 9, for instance, compares the VIIRS river ice product, precisely the ice concentration values, over the Hudson River north of New York City on 26 February 2022 to a coincident Sentinel 2 RGB scene over the same area and same day. Both cloud-free scenes were visually inspected, and ice floes were manually delineated. One should note that the overpass times of VIIRS NOAA-20 used here to calculate ice concentration and Sentinel 2 are 2:20 p.m. and 10:30 a.m., respectively. The comparison of both scenes shown in Figure 9 shows a shift of the delineated ice floes over an estimated distance of about 3 km. Knowing that both scenes were acquired around 4 h apart, this implies that the surface velocity was about 0.2 m/s. This value of ice motion velocity that was inferred in this example from the analysis of two satellite scenes is close to other values reported in the literature such as those in the study by Kääb et al. (2019) that reported an ice motion velocity of 0.8 m/s in the Yukon River [26]. Even if ice motion velocity is site-specific and depends on several other factors, obtaining values of similar magnitude in two large rivers of the US is indicative of the potential of using multi-satellite observations. Furthermore, the comparison of the inferred velocity to the simulated one using the operational hydrodynamic model of the Stevens Flood Advisory System (SFAS) [27] indicates that the simulated surface velocity around the ice displacement was about 1 knot, which is equivalent to approximately 0.51 m/s that is in the order of the inferred values from space. The SFAS system accounts for the effect of river ice and its attenuation of the tidal effect in the Hudson. Nonetheless, the system lacks the assimilation of the actual ice information to accurately calculate the effect on the circulation of the river. This can be addressed in future studies. This example demonstrates the possible inference of river ice motion and its velocity using multi-satellite observations. The future development of a multi-satellite product can lead to the automated calculation of ice motions, which is critical for the prediction of ice jam formations.

4. Discussion

The qualitative and quantitative assessments of the developed river ice mapping system demonstrated its reliability in capturing river ice dynamics across a large continental scale. Thus, as such, the developed system is a candidate for operational deployment, so it could be used and hosted by NOAA NWS and used by forecasters at the River Forecast Centers and National Water Centers. Nonetheless, the conducted assessment of the product by NOAA end-users during the past winters has revealed that there is a need for further improvements that could be addressed in future work.

The current version of the river ice mapping system relies solely on VIIRS images, which are of moderate 375 m spatial resolution. This suggests that in its current version, the system is more suitable for large rivers and those of a moderately wide cross section, wider than 375 m. Future work will cover narrow rivers, which requires downscaling the collected observations over mixed pixels that includes both land and ice or water reflectance. The challenge is to unmix the coarse pixel observation, which could be the result of different combinations of surface conditions that may lead to the same overall pixel-level observations. For instance, a high reflectance from a smaller land fraction due to snow presence mixed with an ice-free river fraction within the pixel can lead to similar satellite observations in the case of a larger land snow-free land fraction mixed with ice-covered river fraction within the same pixel. An accurate estimate of land and water fractions within the observed pixels and the conditions of each of these fractions is necessary for the downscaling of the moderate-resolution product to cover narrow rivers.

Overall, the system has shown an acceptable performance in detecting clouds of different types over the entire geographic range. It was noted that the system is capable of detecting cirrocumulus clouds that are commonly known as ‘’popcorn clouds” reasonably well despite their local extent. The limitation faced in the case of low and thin clouds is common when optical images are used. The proposed approach for cloud detection that relies on the vertical difference between I05 temperature and the LST climatology (DLST) works well with high altitude and thick clouds but requires further training in the case of low and thin clouds. In addition, a lower performance was noticed in the detection of cloud shadow, which was merged with cloud class in the automated system. This limitation can be attributed to the relatively limited geographic extent of cloud shadow compared to other classes, which implies a lower number of cloud shadow pixels used in the training of the U-Net compared to the other geographically prevailing classes. Future work will focus on enhancing cloud shadow detection, which can be achieved by collecting more scenes and enlarging the training data sample that corresponds to the cloud shadow class. Another approach that can be tested may consist of introducing new explanatory variables such as cloud base height and temperature profile, which are at the basis of geometry-based methods of cloud shadow detection.

Some noticed challenges were region specific. The false classification of relief shade issue was reported mostly in Alaska, where the sun angle can be relatively low and therefore maximizes the extent of relief shade, especially in central and northern regions of the state where mountains obstruct sunlight to create darker spots even in the presence of snow on the ground, which leads to their classification as water since their reflectance drops to the order of water pixels. One attempt to avoid this type of false alarm is to allow water to be detected only within a waterbody’s boundary and within a certain buffer area around it, which corresponds to the floodplain.

Despite the overall acceptable performance of the system. Some of the reported limitations are inherent to the physics of the optical satellite data used for the detection of river ice. The system makes use of the spectral difference between the different classes, especially ice and water in the different optical bands. However, under specific circumstances, ice and water may behave similarly. Figure 10 illustrates a challenging situation in Peoria Lake and Upper Peoria Lake in the Illinois River on 10 January 2022, where NOAA NWS reported the presence of river rice conditions that were not clearly visible in the high-resolution Sentinel-2 in Figure 10a since NWS reported the presence of thin ice. However, the system showed an increase in ice concentration around the section of the river, Figure 10b, indicating a better sensitivity of the concentration to ice presence, especially in the case of thin ice. The ice extent map did not include the thin ice part on the western riverbank and only showed the brighter and thicker ice on the easter riverbank (Figure 10c). This demonstrates the importance of analyzing both ice concentration and extent to develop a better understanding of ice conditions in rivers.

Future work will include the processing of more recent sensors such as NOAA-21 and future similar sensors such as those from the European Sentinel constellation and future Joint Polar Satellite System (JPSS) missions. The use of moderate-resolution images from multiple sensors will generate seamless gap-free mosaics that could be equivalent in terms of coverage to the geostationary satellites, with a noticeable improvement in terms of spatial resolution. In addition, the use of multiple images, especially at high latitude regions where the revisit cycles are frequent, offers the advantage of mitigating cloud coverage [28]. This is in addition to other potential byproducts such as the inference of ice motion, as was demonstrated in this study.

The proposed river ice system is automated and capable of generating daily maps that inform end-users of ice conditions. The compositing of multiple images daily and weekly and the use of multiple acquisitions can mitigate the effect of cloud obstruction. The information derived from the system regarding ice extent and ice concentration is certainly very useful in predicting ice transport in rivers and the possible occurrence of ice jams. This is of primary importance for large-scale models such as the National Water Model operated by NOAA National Water Center, an operational hydrologic and hydraulic model that predicts streamflow across the entire continental US. Delving further into the broader implications of this study, it is pertinent to consider the significant effect that ice conditions have on water supply during the colder seasons. Considering a recent study, it is apparent that the current version of the NWM necessitates improvements in its streamflow estimation, specifically during periods of low temperatures and river ice conditions [29]. The developed river ice system can provide the needed information on ice extent and concentration. This information can be complemented by an estimate of ice thickness that can be retrieved from the cumulative degree day of freezing that is commonly used as a proxy for ice thickness. When combining ice extent and concentration with ice thickness, one can estimate ice volume, which is essential to simulate ice transport and its impact on the hydraulics in the river. The National Water Model, in its model agnostic NextGen configuration, could allow the introduction of such components in northern watersheds to account for the effect of river ice on streamflow prediction.

5. Conclusions

This study was primarily aimed at addressing the gap in systematic, large-scale river ice monitoring via the development of an automated, high-resolution mapping system to provide real-time, reliable data for water resources management. The novelty of the research lies in the application of deep learning techniques to VIIRS data, reinforced by extensive validation against ground-based and remote sensing observations, thereby culminating in the creation of the first continental-scale river ice monitoring system. This innovative approach has the potential to significantly enhance our understanding of river ice dynamics and their implications for water resources, thereby providing crucial insights for improved risk management and mitigation strategies. The maps generated by the system have proven to successfully capture the phenology of river ice, particularly during the onset and breakup periods. The consistent reflection of changes in ice, water, and snow conditions, as inferred from the VIIRS product, corroborates its reliability, underlining its usefulness in monitoring hydraulic and hydrological processes across northern watersheds in the US.

The evaluation of the river ice product was carried out continuously over three winter seasons, involving regular interaction with end-users from NOAA River Forecast Centers under the National Weather Service. Ground-based observations from USGS-operated cameras, field observations, citizen science programs, and aerial surveys such as the RiverWatch flight surveys in Alaska formed the basis of the assessment. This constant monitoring and feedback from users demonstrated the system’s capability to accurately map river ice extent and concentration, particularly in large and moderately wide rivers. Analysis of data from the Missouri River, for example, indicated a swift transition from an ice-covered river to an ice-free one, as evidenced by ground-based camera observations. The corresponding river ice map showed a similar trend, validating the system’s ability to accurately track rapid changes in river ice conditions. Such rapid shifts are common in mid-latitude regions of the US and underline the necessity for an automated monitoring system that can quickly respond to potential ice jam formation and ice-induced flooding.

A quantitative assessment of the model revealed an overall accuracy (Proportion Correct) of 0.747, indicating that approximately 75% of the model’s predictions regarding the presence or absence of river ice were accurate. This not only demonstrates an admirable level of predictive accuracy but also suggests room for further model refinement. With a Bias ratio of 0.870, the model exhibited a slight under-prediction of river ice presence, hinting at a potential area for improving the model’s sensitivity. The study also established a relationship between changes in the extent of snow, ice, and water within specific watersheds. For instance, in the Tanana River watershed, Alaska, a rapid snow melt typically occurs around 4 days before the mechanical breakup in the river, as detected by an increase in water extent and a decline in ice surface area in the VIIRS river ice product.

In conclusion, this study represents a significant stride towards the establishment of an automated, reliable system for large-scale river ice monitoring. The accuracy of the developed model, its validation via rigorous qualitative and quantitative assessments, and its ability to track swift changes in river ice conditions together underscore its potential as a valuable tool for future water resources management. Further refinement and enhancements building upon the insights garnered from this study can contribute to improved risk management and water resource sustainability under changing climate conditions.