Methodology for Severe Convective Cloud Identification Using Lightweight Neural Network Model Ensembling

Abstract

This study introduces an advanced ensemble methodology employing lightweight neural network models for identifying severe convective clouds from FY-4B geostationary meteorological satellite imagery. We have constructed a FY-4B based severe convective cloud dataset by a combination of algorithms and expert judgment. Through the ablation study of a model ensembling combination of multiple specialized lightweight architectures—ENet, ESPNet, Fast-SCNN, ICNet, and MobileNetV2—the optimal EFNet (ENet- and Fast-SCNN-based network) not only achieves real-time processing capabilities but also ensures high accuracy in severe weather detection. EFNet consistently outperformed traditional, heavier models across several key performance indicators: achieving an accuracy of 0.9941, precision of 0.9391, recall of 0.9201, F1 score of 0.9295, and computing time of 18.65 s over the test dataset of 300 images (~0.06 s per 512 × 512 pic). ENet shows high precision but misses subtle clouds, while Fast-SCNN has high sensitivity but lower precision, leading to misclassifications. EFNet’s ensemble approach balances these traits, enhancing overall predictive accuracy. The ensemble method of lightweight models effectively aggregates the diverse strengths of the individual models, optimizing both speed and predictive performance.

1. Introduction

Severe convective clouds, typically associated with thunderstorms, high winds, hail, and other extreme weather phenomena, are characterized by robust convective clusters formed under specific humidity conditions in an unstable atmospheric layer [1]. These clouds, featuring a relatively low cloud base and a significantly higher cloud top, contain intense internal vertical airflows [2,3,4,5] and are crucial for meteorological forecasting and disaster warning systems, potentially mitigating the impacts of natural disasters [6,7,8]. With advances in remote sensing technology [2,9,10,11,12,13], the automatic detection and classification of these clouds using satellite images has become feasible, enhancing the timeliness and accuracy of meteorological services. Simultaneously, the use of deep learning techniques, particularly convolutional neural networks [14,15,16] (CNNs), has greatly improved the detection and classification of severe convective clouds in satellite imagery. Cloud segmentation based on deep learning is a long-standing and ongoing area of research in the field of remote sensing. This includes general cloud segmentation without differentiating cloud types [5,11,14,17,18,19,20,21,22], as well as segmentation of severe convective clouds [2,4,6,8,15,23,24,25,26,27], which continue to be actively studied, underscoring the enduring significance of this problem. In practical applications, particularly where real-time processing of vast datasets is required, lightweight neural networks have garnered attention due to their lower computational demands and rapid processing capabilities. Lightweight models such as ENet [28], ESPNet [29], Fast-SCNN [30], ICNet [31], and MobileNetV2 [32], though smaller, demonstrate performance comparable to larger networks like U-Net [33] and DeepLabV3 [34] in semantic segmentation tasks. These networks maintain high accuracy while significantly enhancing computational speed, making them suitable for use in resource-constrained environments [5,16,35].

Photogrammetry [36,37,38] and GNSS radio occultation [39,40] are exemplary in providing precise object details and valuable profile data, respectively, but satellite remote sensing images offer distinct advantages for large-scale environmental monitoring. Satellite imagery excels in providing extensive geographical coverage and frequent updates [41], capabilities that are indispensable for tracking dynamically changing severe convective clouds. This comprehensive and continuous coverage allows for the real-time observation of weather patterns on a global scale, which is not feasible with photogrammetry’s relatively limited area coverage or the atmospheric profiles provided by GNSS radio occultation. Additionally, the integration of multispectral imaging in satellites enhances the detection and analysis of various cloud properties and atmospheric conditions, leading to more accurate weather forecasting and climate studies. Therefore, while other methods provide valuable insights at different scales and dimensions, satellite remote sensing remains crucial for holistic and continuous monitoring.

To further improve recognition accuracy and the generalizability of models, ensemble techniques have been incorporated into deep learning. By aggregating predictions from multiple models, ensemble methods can effectively reduce the error rates that might occur in individual models, thereby achieving higher accuracy [3,42,43]. Additionally, ensemble methods enhance the predictive capability of models on unseen samples, which is particularly vital under the fluctuating conditions of meteorological phenomena [24,25,44]. As lightweight neural network technologies [45] continue to advance, their integration into ensemble frameworks for recognizing severe convective clouds offers significant research value and practical prospects. Integrating multiple lightweight models not only balances efficiency and accuracy but also enhances the robustness of the system [35].

For light, medium, or large neural networks, researchers have made many meaningful and significant improvements to make them more adaptable to cloud recognition tasks [20,24,42,46,47,48,49,50] (including the identification of severe convective clouds). However, beyond these improvements, we can also consider how to use model ensemble techniques to better integrate and utilize existing lightweight neural networks, enabling them to exhibit capabilities and computational efficiency far beyond a single large neural network.

The model ensemble techniques have been widely used in many fields [3,7,11,19,51,52,53], but they have not yet been applied to the task of remote sensing image recognition of severe convective clouds. In the field of meteorology, model integration techniques have been extensively utilized for analyzing synoptic scale weather phenomena [54] and enhancing meteorological forecasts [55,56]. This approach has also been widely adopted in the domain of cloud segmentation within satellite remote sensing imagery [3,11,19]. However, these efforts predominantly employ integrated models for RGB channel segmentation in visible light satellite images, without differentiation of cloud types, and do not engage the infrared and water vapor channels necessary for segmenting severe convective clouds. Currently, the singular application of model integration techniques to severe convective weather focuses on extreme weather events, including the use of radar reflectivity data for thunderstorm warnings [57] and high-resolution rapid refresh (HRRR) 1–24 h forecasts to predict catastrophic weather conditions over the United States [58]. Additionally, existing cloud segmentation generally involves the integration of lightweight and non-lightweight networks to achieve higher accuracy, albeit at a marginally slower speed than non-lightweight networks [3], or solely through non-lightweight network integrations [11,19], yet these methods typically underemphasize computational speed. Therefore, we innovatively propose a multi-lightweight model integration technique for severe convective cloud segmentation, which not only surpasses the accuracy of single non-lightweight networks but also improves speed. The novelty of this research lies in the successful first-time application of this multi-lightweight model integration approach specifically for severe convective cloud segmentation.

Given the ample excellent research in the field [2,4,6,7,8,15,23,24,25,26,27,59,60], we do not propose a specific neural network. While developing new networks has its merits, optimizing the use of existing networks is equally valuable [61]. Our work achieves efficient, high-precision cloud detection by enhancing the utilization of existing lightweight networks. We demonstrate the efficacy of integrating multiple lightweight neural network models, a more impactful approach likely to see broader adoption among researchers than introducing a single network. This study employs five lightweight neural networks—ENet, ESPNet, Fast-SCNN, ICNet, MobileNetV2—as submodules in an ensemble to achieve efficient and accurate detection of severe convective clouds. The FY-4B satellite [62,63] is the first operational satellite of China’s new generation of geostationary meteorological satellites, the Fengyun-4 series, with significant improvements compared to FY-4A [64,65]. Therefore, this article uses data from FY-4B and takes the severe convective weather in the southeast of the Eurasian continent and nearby sea areas as the target of research. Comparative experiments demonstrate that this ensemble method not only maintains high recognition accuracy but also significantly improves processing speed, showcasing the potential and advantages of lightweight deep learning models in practical applications.

2. Study Area and Data

2.1. Study Area

The study area is located between 0–45°N and 100–160°E (Figure 1), primarily encompassing the southeastern part of the Eurasian continent and adjacent maritime regions. This region exhibits significant seasonal and regional characteristics in severe convective weather activities, influenced by both topography and regional atmospheric circulation patterns. The area includes a wide range of terrestrial and marine zones, such as China, Japan, the Korean Peninsula, the South China Sea, and the East China Sea. Each area’s unique weather patterns are driven by a combination of geographical, oceanographic, and atmospheric factors [66,67], ranging from intense thunderstorms in China’s southeast, influenced by moist and cool air masses, to Japan’s typhoon-induced convective systems during summer. The Korean Peninsula shows sharp seasonal contrasts in convective activity, while the South and East China Seas are hotspots for cyclonic developments and associated severe weather, respectively, fueled by warm sea temperatures and monsoonal interactions.

2.2. FY-4B Data

Fengyun-4B [62,63] (FY-4B) is a new-generation geostationary meteorological satellite in the Fengyun series from China, an important member following FY-4A. It is specifically designed for meteorological observation, climate environment monitoring, and natural disaster warning. Compared to FY-4A, FY-4B has improved performance [62,68,69], providing more accurate and comprehensive meteorological data to meet the growing demand for meteorological services. As of 1 April 2024, the satellite’s subsatellite point is located at 105°E.

The Advanced Geosynchronous Radiation Imager (AGRI) is one of the core payloads of the FY-4B satellite. It is a high-performance multi-channel imager primarily used to obtain high-quality atmospheric, cloud, and surface images. The AGRI sensor inherits the technical advantages of similar sensors on FY-4A, while optimizing and improving some performance indicators. The successful deployment of FY-4B and its AGRI sensor has further enhanced the capabilities in the field of geostationary meteorological observation, which is of great significance for improving the accuracy of weather forecasts, deepening the understanding of the Earth’s climate system, and effectively responding to natural disasters. Through coordination with other meteorological satellite systems, FY-4B can provide more robust and comprehensive support for global meteorological services. The main features and capabilities of the FY-4B AGRI sensor include:

• Multi-channel imaging. AGRI is equipped with multiple observation channels, including visible light, near-infrared, mid-infrared, and far-infrared bands, supporting wide-band atmospheric, cloud, and surface observations. This multi-channel observation capability allows AGRI to capture detailed features under different meteorological and environmental conditions.
• High spatial resolution. AGRI provides a spatial resolution of up to 500 m in the visible light and near-infrared channels, and even higher resolution in other channels, enabling it to capture more detailed meteorological and surface information.
• Fast scanning capability. AGRI provides rapid observation updates, including global scans every 15 min, regional scans every 5 min, and rapid scans of key areas every minute, greatly improving the monitoring and response speed to extreme weather events.

The observational parameters for each channel are presented in

Table 1. FY-4B AGRI parameters by channel *.

Channel Type	Band	Center Wavelength (μm)	Bandwidth (μm)	Spatial Resolution (km)	Main Applications
VIS/NIR	1	0.47	0.45–0.49	1	Small-particle aerosols, true color synthesis
	2	0.65	0.55–0.75	0.5	Vegetation, image navigation registration, stellar observations
	3	0.825	0.75–0.90	1	Vegetation, aerosols over water surfaces
Shortwave IR	4	1.379	1.371–1.386	2	Cirrus clouds
	5	1.61	1.58–1.64	2	Low cloud/snow identification, water/ice cloud discrimination
	6	2.25	2.10–2.35	2	Cirrus, aerosols, particle size
Midwave IR	7	3.75	3.50–4.0	2	Clouds and high albedo targets, fire spots
	8	3.75	3.50–4.0	4	Low albedo targets, surface
Water vapor	9	6.25	5.80–6.70	4	Upper-level water vapor
	10	6.95	6.75–7.15	4	Mid-level water vapor
	11	7.42	7.24–7.60	4	Low-level water vapor
Longwave IR	12	8.55	8.3–8.8	4	Clouds
	13	10.80	10.30–11.30	4	Clouds, surface temperature, etc.
	14	12.00	11.50–12.50	4	Clouds, total water vapor, surface temperature
	15	13.3	13.00–13.60	4	Clouds, water vapor

* Available online: https://www.nsmc.org.cn/nsmc/cn/instrument/AGRI.html (accessed on 20 April 2024).

.

2.3. Data Preprocessing

The radiometric calibration and geographical repositioning of FY-4B data are crucial steps before FY-4B data labeling and severe convective cloud dataset construction. Radiometric calibration ensures that the satellite’s sensor outputs accurately reflect the electromagnetic energy it captures. This process corrects any system biases or errors, translating raw data into meaningful measurements that can be compared over time and with other sensors. Geographical repositioning, on the other hand, aligns the satellite data with geographic coordinates on Earth’s surface. Here, the geographic repositioning includes converting nominal projection to a 0.04° lat-lon grid projection, and converting the 0.04° lat-lon grid projection back to nominal projection. This is essential for accurate mapping and analysis, as it corrects any positional discrepancies due to the satellite’s orbit, sensor geometry, or Earth’s rotation. Together, these processes ensure that the data from FY-4B is reliable and precise for practical applications.

Radiometric calibration [70] mainly includes two steps: Firstly, Level 1 data for the solar reflective bands requires the conversion of DN values to reflectance, radiance, or apparent reflectance. Secondly, for the infrared bands, Level 1 data requires the conversion of DN values to radiance ($W/(m^2·sr·um)$) or brightness temperature (K) according to the lookup table.

In geographical repositioning [8,71], we need to convert nominal projection to a 0.04° lat-lon grid projection. The FY-4 satellite employs the geostationary orbit nominal projection defined by the CGMS LRIT/HRIT global standard, with geographic coordinates calculated based on the WGS84 reference ellipsoid. Projection transformation involves mapping the pre-projection data points according to the projection transformation formula into the coordinates of the post-projection image. We first determine the geographic extent of the projection transformation and calculate the number of rows and columns in the post-projection image. Then, we map each pixel from the original cloud map through the lat-lon projection transformation formula to the post-projection image. When converting the FY-4B data from 0.04° lat-lon grid projection to nominal projection, the opposite operation can be performed.

2.4. FY-4B Data Labeling: Severe Convective Cloud Dataset Construction

The geographical regions from which the dataset samples were collected are shown in Figure 1, spanning the period from 1 June 2022, to 30 June 2023. All samples were selected from areas with distinct convective cloud clusters, ensuring that each sample contained convective clouds. Regarding the number of samples used for the cloud segmentation neural network, Tian et al. [3] employed 22,432 samples of 256 × 256 pixels, while Ma et al. used 7200 samples of the same size [19]. Zhang et al. [72] utilized 784 samples of 125 × 125 pixels and 2543 samples of 256 × 256 pixels. Li et al. [73] used 3000 samples of 512 × 512 pixels for single cloud category classification in lightweight network model. As this study focuses on ensemble training based on lightweight models, the demand of lightweight models for sample size is relatively small. Therefore, a compromise of 3000 samples of 512 × 512 pixels was adopted.

We have constructed a severe convective cloud dataset using 3000 multi-channel satellite data samples from the AGRI onboard the FY-4B satellite, each with dimensions of 512 × 512 pixels. This dataset was developed through a combination of algorithms and expert judgment. The labeling process focuses primarily on images from the FY-4B satellite, based on actual needs. In the labeling process, severe convective cloud formations such as convective cells, thunderstorm clusters, and squall lines are identified. The process eliminates interference from cirrus clouds in the longwave infrared data.

Before labeling, data from reflective and infrared bands are preprocessed—reflective band data are calibrated as reflectance, and infrared band data as brightness temperature. Data from channels at 0.47, 0.65, and 0.825 μm are combined to create true-color images. For example, the water vapor channel data (WV at 6.25 μm) and longwave infrared channel data (LWIR at 10.8 μm) are separated to create a water vapor-infrared brightness temperature difference ($BTD$), calculated as $BTD={TBB}_{WV}-{TBB}_{LWIR}$.

After calculating the $BTD$, each remote sensing image is subjected to dynamic thresholding to coarsely extract mesoscale convective cloud clusters, identifying their approximate regions and applying morphological operations to fill gaps within the convective clusters. The dynamic $BTD$ threshold of convective clouds in each period is specifically set based on the expert visual interpretation of the images during that time frame. Then, combined with the synthesized true-color cloud map, the expert manual visual interpretation corrects misidentifications of convective areas (typically caused by high-altitude clouds like cirrus), further enhancing the labeling accuracy.

The severe convective cloud dataset example is illustrated in Figure 2, with convective cloud clusters identified and edges detected using the Canny operator [74]. The Canny operator, a multi-stage edge detection algorithm, is widely employed in computer vision tasks for its optimal performance in identifying object boundaries within digital images [74,75]. This operator applies a sequence of Gaussian filters to smooth the image, reducing noise while preserving the integrity of edge structures [76]. The algorithm then computes the intensity gradients, followed by non-maximum suppression to eliminate spurious edge pixels [77]. Finally, hysteresis thresholding is applied to track and connect the remaining edge pixels, resulting in a binary edge map. The Canny operator’s adaptability to various image conditions and its ability to produce clear, continuous edges make it a preferred choice for edge detection in numerous applications like delineation of convective cloud clusters in meteorological image analysis [78].

3. Method

The framework for our method of severe convective cloud identification using lightweight neural network model ensembling is illustrated in Figure 3. Initially, from the aforementioned dataset of severe convective clouds, we extract three-channel information from 3000 samples of 512 × 512 resolution, specifically at 6.25 μm, $BTD\left(6.25\text{–}10.8\mathsf{\mu }\mathrm{m}\right)$, and $BTD\left(10.8\text{–}12\mathsf{\mu }\mathrm{m}\right)$. This forms a four-dimensional array (3000 × 3 × 512 × 512). Concurrently, the corresponding binary labels are extracted, resulting in another four-dimensional array (3000 × 1 × 512 × 512). Subsequently, these 3000 data samples are randomly divided into training, validation, and test sets with a ratio of 8:1:1. Following this partition, we employ model ensembling and ablation study techniques to test the performance of 31 combinations (${\sum }_{k=1}^{5}C\left(5,k\right)$) of five different lightweight neural networks (ENet, ESPNet, Fast-SCNN, ICNet, and MobineNetV2) in identifying severe convective clouds to determine the optimal combination. Upon identifying the most effective ensemble, we build a framework for severe convective cloud identification based on this optimal lightweight neural network ensemble, and compare its efficacy with traditional, non-lightweight neural networks such as U-Net and DeepLabV3, thereby validating the effectiveness of our approach.

In the context of satellite remote sensing, infrared and water vapor channels are indispensable for the segmentation of severe convective clouds due to their capability to detect essential atmospheric conditions [4,8,25,26,27]. These channels excel in capturing thermal variations and monitoring moisture dynamics—key factors in identifying and predicting severe weather events. Infrared imagery, effective in delineating cold cloud tops indicative of severe conditions, and water vapor channels, crucial for assessing moisture content essential for storm intensity and development, operate continuously, thus facilitating round-the-clock monitoring and significantly enhancing forecasting accuracy. Conversely, visible light and shortwave infrared (SWIR) channels present limitations: they are bound to daylight operations and lack comprehensive monitoring of atmospheric water vapor, reducing their effectiveness in continuous severe weather prediction. Due to these constraints and the unavailability of nighttime data from FY4 series satellites in visible and SWIR channels [8,64,70], we have opted to utilize the water vapor and longwave infrared band data from the FY4B satellite. The selected 6.25 μm, $BTD\left(6.25\text{–}10.8\mathsf{\mu }\mathrm{m}\right)$, and $BTD\left(10.8\text{–}12\mathsf{\mu }\mathrm{m}\right)$ have previously been employed in the FY4A series for the task of severe convective cloud segmentation, as demonstrated by Chen et al. [8]. Additionally, the selection of these three channels is justified as they sufficiently enable high-precision segmentation of severe convective clouds. The innovation of this paper is reflected in surpassing traditional networks in both speed and accuracy. Given that the three channels are adequate, introducing additional channels would result in marginal improvements and could potentially reduce the computational efficiency of the model.

3.1. Parameter Configuration

This experiment was conducted using the PyTorch2.2.0 + cu121 (Python 3.10.14) deep learning framework [79] on an NVIDIA RTX A5000 GPU equipped with 24 GB of memory. The adaptive moment estimation (Adam) optimizer [80] was employed, configured with the exponential decay rate for the first moment estimates ${\beta }_1=0.9$, the exponential decay rate for the second-moment estimates ${\beta }_2=0.999$, and a very small constant $ϵ={10}^{-8}$ to prevent any division by zero in the implementation.

3.2. Loss Function

In the task of binary classification for severe convective cloud detection, the binary cross-entropy loss [81] (BCE) is employed to effectively measure model accuracy. Cross-entropy serves as a crucial metric in evaluating classification models by quantifying the divergence between predicted and true probability distributions. A cross-entropy score of zero represents an ideal where predictions perfectly match actual labels, whereas scores approaching infinity indicate severe mispredictions, with log loss amplifying errors where predicted probabilities of the correct class near zero. This measure not only prioritizes accuracy but also considers the probabilistic confidence of predictions, enhancing both the robustness and reliability of model evaluations. It is advantageous for gradient-friendly properties that facilitate effective model training via backpropagation. BCE quantifies the discrepancy between predicted probabilities and actual binary labels, defined mathematically as:

$$\mathrm{BCE}=-\frac{1}{N}\sum _{i=1}^N\left[y_i\log \left(\widehat{y_i}\right)+\left(1-y_i\right)\log \left(1-\widehat{y_i}\right)\right]$$

(1)

where $N$ is the sample count, $y_i$ represents the true label, and $\widehat{y_i}$ denotes the predicted probability of severe convective cloud presence.

3.3. Model Evaluation Index

To objectively evaluate the performance of our models in the binary classification of severe convective clouds, we employ five standard metrics. Each metric provides insights into different aspects of model accuracy and robustness, thus allowing a comprehensive assessment across various dimensions of prediction performance.

• Accuracy: Defined as the ratio of correctly predicted results to the total results, accuracy offers a straightforward measure of overall model effectiveness. Specifically, the accuracy measures the proportion of total predictions that are correct. It ranges from 0 to 1, where 1 indicates perfect accuracy, and 0 indicates complete inaccuracy.

$$\mathrm{Accuracy}=\frac{\mathrm{Number}\mathrm{of}\mathrm{Correct}\mathrm{Predictions}}{\mathrm{Total}\mathrm{Number}\mathrm{of}\mathrm{Predictions}}$$

(2)

2.

Precision: Precision assesses the accuracy of positive predictions and evaluates the incidence of false positives. It is particularly useful in situations where the cost of a false positive is high. Like accuracy, precision ranges from 0 to 1, with 1 being perfect (no false positives) and 0 indicating all positive predictions are incorrect.

$$\mathrm{Precision}=\frac{\mathrm{True}\mathrm{Positives}}{\mathrm{True}\mathrm{Positives}+\mathrm{False}\mathrm{Positives}}$$

(3)

3.

Recall: Also known as sensitivity, recall measures the model’s ability to detect all actual positives. Its range is also between 0 and 1, where 1 means all true positives are correctly identified, and 0 signifies no true positives are detected.

$$\mathrm{Recall}=\frac{\mathrm{True}\mathrm{Positives}}{\mathrm{True}\mathrm{Positives}+\mathrm{False}\mathrm{Negatives}}$$

(4)

4.

F1 Score: The F1 score is the harmonic mean of precision and recall, providing a balance between the two when an equal importance is assumed. It is particularly useful when the cost of false positives and false negatives is high. The F1 score also ranges from 0 to 1, where 1 is the best possible score, indicating perfect precision and recall.

$$\mathrm{F}1\mathrm{Score}=2\times \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}$$

(5)

5.

Intersection over Union (IoU): Also known as the Jaccard index, IoU is the ratio of the intersection to the union of the predicted and true labels. An IoU of 1 indicates a perfect prediction where the predicted labels or boundaries completely coincide with the ground truth, while an IoU of 0 signifies no overlap at all between the predicted and actual labels.

$$\mathrm{IoU}=\frac{\mathrm{Area}\mathrm{of}\mathrm{Overlap}}{\mathrm{Area}\mathrm{of}\mathrm{Union}}$$

(6)

6.

Overall Performance (OP): To synthesize the insights provided by individual metrics into a single performance indicator, we introduce the OP index. The higher the OP score, the better, as it suggests a model that performs well across all key aspects of binary classification. OP is computed as the sum of the standardized scores of the five aforementioned metrics:

$$\mathrm{OP}=\mathrm{Accuracy}+\mathrm{Precision}+\mathrm{Recall}+\mathrm{F}1\mathrm{Score}+\mathrm{IoU}$$

(7)

This composite metric provides a holistic view of the model performance, encompassing accuracy, precision, sensitivity, and the balance between precision and recall, as well as the degree of overlap between predicted and true classes. OP is particularly useful when selecting the optimal combination from a large set of models. Here, “optimal” refers to the best overall performance, rather than necessarily being the top performer in every individual metric.

3.4. Training

The models were trained for a total of 30 epochs, the loss records and detailed parameters setting of which are shown in Figure 4 and

Table 2. Training details.

Model	Avg Train Time (s/Epoch)	Avg Val Time (s/Epoch)	Number of Parameters	Model Size (MB)	Batch Size	Initial Learning Rate
ENet	75.21	13.63	214,465	0.95	8	0.001
ESPNet	26.80	12.99	103,784	0.42	8	0.001
Fast-SCNN	118.45	13.52	1,112,350	4.12	8	0.0005
ICNet	21.11	13.43	3741	0.02	4	0.0005
MobileNetV2	221.24	15.65	3,363,602	11.39	16	0.0005
DeepLabV3	536.48	28.91	41,999,209	160.56	4	0.0001
U-Net	483.16	18.57	7,383,234	28.22	4	0.001

, respectively.

For most networks, the training loss tends to stabilize after 20 epochs, and it generally remains stable after 30 epochs, with the exception of MobileNetV2, which exhibits slight fluctuations in the later stages (Figure 4). The fluctuations in validation loss are more pronounced, with ESPNet experiencing the largest variations in validation loss. Notably, MobileNetV2 shows significant fluctuations during the 5–10 epoch range, but eventually stabilizes. Importantly, the lightweight network ENet demonstrates the fastest decline in validation loss at the onset of training and achieves the lowest stable loss subsequently. The performance of non-lightweight networks, such as U-Net and DeepLabV3, in both training and validation is commendable as well.

However, when focusing on training duration and model size (as shown in Table 2), the advantages of lightweight models become apparent. Even when aggregating the training times of the five lightweight networks, the required training time, the number of model parameters, and the model size are all less than those of U-Net and DeepLabV3. Thus, the methodology proposed in this paper for severe convective cloud identification using an ensemble of lightweight neural network models aims to leverage this compactness of lightweight network architectures.

Our findings also indicate that networks with excessively small model parameters do not yield desirable results. As evidenced in Table 2, ICNet’s parameters are at least two orders of magnitude smaller than those of other networks. When a model’s parameter count is particularly low, its training loss tends to be significantly greater than that of more complex models with larger parameter counts. This discrepancy stems from the constrained expressive, learning, and optimization capabilities of models with minimal parameters, which tend to underfit, as demonstrated by the high loss observed in Figure 4.

3.5. Building the EF Network via Model Ensembling and Ablation Studies

In this section, we have achieved computational efficiency and accuracy that surpass those of single non-lightweight networks by ensembling various lightweight networks with the optimal combination. To ascertain the most effective model ensembling combination for severe convective cloud identification, we utilize ablation study techniques to evaluate the efficacy of 31 different combinations (${\sum }_{k=1}^{5}C\left(5,k\right)$) involving five distinct lightweight neural networks (ENet, ESPNet, Fast-SCNN, ICNet, and MobileNetV2).

Figure 5 and Figure 6 illustrate the results of these 31 model combinations. During the evaluation of the 31 model combinations, we conducted separate computations and recorded the resulting parameters for each combination, executing a total of 31 runs. In the model ensembling process, we fuse the outputs of the last linear layer of each network within the combinations, which are the raw score logits not yet normalized by probabilities. Utilizing logits directly in the loss function enhances numerical stability as transforming logits to probabilities via softmax before computing the cross-entropy loss involves logarithmic and exponential operations, which can lead to numerical instability. This approach also improves computational efficiency by avoiding dual transformations (first softmax, then log). For each specific combination, all modules within the ensemble were simultaneously executed, and their logits were summed to obtain the final output probability. Based on this output, the accuracy and runtime of each combination were obtained. The process of incrementally adding or removing lightweight neural network modules to identify the critical components and optimal combination is referred to as an ablation study in this paper. In this context, the ablation study serves not only to assess performance but also as a method for determining the most favorable ensemble configuration.

Initially, we hypothesized that the full model combination of ENet + ESPNet + Fast-SCNN + ICNet + MobileNetV2 (E + ES + F + I + M) would yield the highest-quality results, albeit with slightly lower computational efficiency. However, upon comparing the performance parameters of these 31 model combinations (as shown in

Table 3. Performance parameters of 31 model combinations of ENet (E), ESPNet (ES), Fast-SCNN (F), ICNet (I), and MobileNetV2 (M). Results are arranged in descending order according to the OP metric.

Model Combination	Test Time (s)	Test Loss	Accuracy	Precision	Recall	F1	IoU	OP
E + F	18.65	0.0156	0.9941	0.9391	0.9201	0.9295	0.8684	4.6512
E + F + M	25.27	0.0202	0.994	0.9442	0.9122	0.9279	0.8655	4.6438
E + ES + F + M	24.95	0.0258	0.9939	0.9606	0.8911	0.9246	0.8597	4.6299
E + ES + F	19.63	0.0215	0.9938	0.959	0.89	0.9232	0.8574	4.6234
ES + F + M	21.24	0.0215	0.9935	0.9538	0.8891	0.9203	0.8524	4.6105
E + F + I + M	25.05	0.0258	0.9936	0.9628	0.8814	0.9203	0.8524	4.6091
E + M	19.51	0.0184	0.9935	0.9553	0.8869	0.9199	0.8516	4.6072
ES + F	17.23	0.0176	0.9933	0.9511	0.8871	0.918	0.8485	4.5983
E + F + I	20.76	0.0215	0.9934	0.9623	0.877	0.9177	0.8479	4.598
E + ES + F + I + M	25.76	0.0337	0.9932	0.9704	0.8659	0.9151	0.8436	4.5882
F + I + M	21.56	0.0228	0.9928	0.9538	0.8726	0.9114	0.8372	4.5678
F + M	20.73	0.0203	0.9925	0.9083	0.9139	0.9111	0.8367	4.5666
E + ES + M	22.44	0.0265	0.9929	0.9673	0.8601	0.9105	0.8358	4.5646
E	15.29	0.0173	0.9927	0.9437	0.8794	0.9105	0.8356	4.5625
E + ES + F + I	20.67	0.03	0.9928	0.9698	0.8571	0.91	0.8349	4.5619
ES + F + I + M	23.18	0.029	0.9928	0.9694	0.856	0.9092	0.8335	4.5609
F + I	16.98	0.0199	0.9922	0.9503	0.8591	0.9024	0.8222	4.5281
E + I + M	19.28	0.0271	0.9923	0.9712	0.8416	0.9018	0.8212	4.5262
ES + F + I	18.09	0.0259	0.9921	0.9694	0.8394	0.8997	0.8177	4.5183
ES + M	17.15	0.0225	0.992	0.9656	0.8409	0.899	0.8165	4.514
F	18.37	0.0208	0.9912	0.8686	0.9313	0.8989	0.8164	4.5064
M	16.72	0.0199	0.9915	0.9194	0.8762	0.8973	0.8137	4.4988
E + ES	16.31	0.0252	0.9917	0.9612	0.8383	0.8956	0.8109	4.4981
E + ES + I + M	20.59	0.037	0.9918	0.9752	0.8269	0.895	0.8099	4.4977
E + I	15.86	0.0271	0.9905	0.9662	0.803	0.8771	0.7811	4.4179
ES + I + M	17.93	0.0338	0.9905	0.9771	0.7937	0.8759	0.7792	4.4164
I + M	17.21	0.0257	0.9904	0.9679	0.7982	0.8749	0.7777	4.4091
E + ES + I	17.87	0.0373	0.9903	0.9709	0.793	0.873	0.7746	4.4018
ES	16.39	0.0272	0.9884	0.9502	0.7659	0.8481	0.7363	4.2889
ES + I	14.51	0.0402	0.9863	0.9684	0.6979	0.8112	0.6824	4.1462
I	11.95	0.0464	0.9786	0.9156	0.5442	0.6826	0.5182	3.6392

), we discovered that the dual-model combination of ENet + Fast-SCNN (E + F) actually provided the highest overall quality, ranking first in five metrics (test loss, accuracy, F1, IoU, and OP). Additionally, due to its simpler dual model structure, the E + F combination also offers advantages in computation speed. Consequently, we have chosen to construct EFNet (ENet and Fast-SCNN based network) based on the dual-model combination of E + F.

3.6. Parallel Computing

Given our approach uses an ensemble of multiple lightweight submodels, the availability of multiple GPU resources would enable the utilization of multi-GPU hardware acceleration to parallelize the inference process of these submodels. Consequently, we propose a model parallelization architecture that allows multiple submodels to be executed concurrently on different GPUs.

Specifically, we have defined a new model class named CombinedModel, which accepts an arbitrary number of submodels as input parameters. During the forward propagation process, each submodel independently processes input data on its respective GPU, generating intermediate results. Subsequently, through efficient cross-GPU communication primitives, the outputs from all submodels are aggregated and summed to produce the final prediction result. This fine-grained model parallelization method allows us to handle models without being constrained by the memory limits of a single GPU.

During the implementation phase, we leveraged high-level APIs to facilitate more efficient and streamlined training and evaluation of model combinations. The program organizes multiple submodels into a list, iterates over them, and automatically distributes input data across multiple GPUs for parallel computation while aggregating gradients from each GPU, thereby accelerating the training process. Simultaneously, during the inference stage, we employ a context manager to disable gradient calculations, as the inference process does not require backpropagation. This approach reduces unnecessary memory consumption and computational overhead, thereby enhancing inference efficiency. In summary, by utilizing the high-level APIs provided by PyTorch, we achieve more efficient and readable implementation of model combination training and evaluation while optimizing inference performance, laying the foundation for realizing high accuracy and low latency objectives.

4. Results

4.1. Qualitative Analysis

By comparing the results of severe convective cloud identification using EFNet, U-Net, DeepLabV3, ENet, ESPNet, Fast-SCNN, ICNet, and MobileNetV2 (as shown in Figure 7), it is evident that the identification quality of single lightweight neural networks is generally inferior to that of non-lightweight networks (U-Net, DeepLabV3).

We found significant misclassification issues with ICNet (Figure 7k), MobileNetV2 (Figure 7l), ENet (Figure 7h), and Fast-SCNN (Figure 7j). The common issue of ICNet, MobileNetV2, and ENet is the extensive underreporting of clouds (an excess of green areas), with ICNet and Fast-SCNN additionally suffering from a substantial number of false positives (an excess of red areas). However, the combined EFNet (Figure 7e) does not exhibit both issues and achieves significantly better performance compared to U-Net (Figure 7f), and shows mixed results when compared to DeepLabV3 (Figure 7g). When the quality of results is comparable and slightly superior, EFNet is preferred due to its significantly higher computational efficiency.

It is noteworthy that in this study, ENet (Figure 7h) exhibits a considerable number of false negatives (evidenced by the extensive green areas), yet it has fewer false positives. Conversely, Fast-SCNN (Figure 7j) shows a higher number of false positives (indicated by the extensive red areas), but fewer false negatives. The integration of these two models into EFNet (Figure 7e) results in a reduction in both false positives and false negatives. This outcome serves as the most intuitive demonstration of the significance of model integration in improving predictive accuracy and balance.

4.2. Performance Metrics Analysis

Although qualitative analysis suggests that DeepLabV3 appears to perform better than U-Net, in reality, the performance metrics of DeepLabV3 do not show any advantage over U-Net (as indicated in

Table 4. Performance parameters of EFNet, U-Net, DeepLabV3, ENet (E), ESPNet (ES), Fast-SCNN (F), ICNet (I), and MobileNetV2 (M). Results are arranged in descending order according to the OP metric.

Model Combination	Test Time (s)	Test Loss	Accuracy	Precision	Recall	F1	IoU	OP
E + F	18.65	0.0156	0.9941	0.9391	0.9201	0.9295	0.8684	4.6512
E	15.29	0.0173	0.9927	0.9437	0.8794	0.9105	0.8356	4.5625
U-Net	20.41	0.0173	0.9926	0.955	0.854	0.9017	0.8209	4.5242
F	18.37	0.0208	0.9912	0.8686	0.9313	0.8989	0.8164	4.5064
M	16.72	0.0199	0.9915	0.9194	0.8762	0.8973	0.8137	4.4988
DeepLabV3	43.15	0.0256	0.9912	0.9412	0.8323	0.8834	0.7911	4.4392
ES	16.39	0.0272	0.9884	0.9502	0.7659	0.8481	0.7363	4.2889
I	11.95	0.0464	0.9786	0.9156	0.5442	0.6826	0.5182	3.6392

). Furthermore, compared to the non-lightweight networks U-Net and DeepLabV3, EFNet maintains an advantage in five key metrics (Test Loss, Accuracy, F1, IoU, OP) and also demonstrates greater computational speed.

5. Discussion

Severe convective cloud systems are a critical element in weather forecasting, involving complex physical and dynamical processes that make it challenging for a single model to accurately capture all relevant features. Therefore, the use of ensembles of multiple models allows for a more comprehensive analysis and identification of these complex weather systems.

Our analysis suggests that severe convective cloud identification is well suited to ensembles of multiple lightweight neural networks, though such studies are currently scarce. Firstly, the problem of severe convective cloud identification covers a broader feature space, with characteristics that may vary across different temporal and spatial ranges. Lightweight networks can be optimized for specific features or sections of an image, and integrating various networks can cover a wider array of characteristics comprehensively. For instance, ENet (Figure 7h) exhibits numerous false negatives, evidenced by extensive green areas, indicating a conservative predictive approach with high specificity but reduced sensitivity. This results in fewer false positives, demonstrating ENet’s tendency to miss certain convective cloud formations that are subtle or complex. In contrast, Fast-SCNN (Figure 7j) shows a liberal predictive behavior with a higher number of false positives (extensive red areas), indicating high sensitivity but lower specificity. This model identifies a broader range of features as convective clouds, increasing the likelihood of misclassification. The ensembling of these models into EFNet (Figure 7e) effectively balances these characteristics, reducing both false positives and false negatives. This demonstrates the power of model ensembling to enhance predictive accuracy and achieve a more balanced performance by leveraging the complementary strengths of Enet’s precision and Fast-SCNN’s recall. This approach illustrates the broader utility of model ensembles in machine learning to provide a comprehensive view of complex phenomena.

Secondly, ensembling multiple lightweight neural networks can enhance computational efficiency while maintaining model accuracy. Lightweight neural networks typically require fewer computational resources and storage space, making deployment feasible in resource-constrained environments such as direct satellite processing or real-time monitoring systems. Additionally, the parallel processing capabilities of multiple lightweight networks further enhance efficiency. The ensembling also reduces the risk of overfitting, as lightweight models, with fewer parameters compared to large deep networks, are less prone to overfitting. Ensembling multiple lightweight models can preserve the generalization capabilities of the model while improving prediction stability. Lastly, it enhances the robustness of the model; in the face of varying meteorological conditions and environmental changes, a single model may struggle to adapt to all scenarios. Model ensembling, by combining the predictions of multiple models, can provide more robust predictions when faced with unknown or anomalous data.

This paper introduces the EFNet model based on model ensembling methods and an ablation study. This model achieves the aforementioned advantages, ensuring that the quality of severe convective cloud identification surpasses that of U-Net and DeepLabV3 while attaining greater computational efficiency with a smaller model and fewer parameters.

6. Conclusions

The implementation of an ensemble of lightweight neural networks for the detection of severe convective clouds has demonstrated significant advancements in meteorological imaging applications, with the computational time, test loss, accuracy, recall, F1, IoU, and OP surpassing U-Net and DeepLabV3. Based on FY-4B datasets, we constructed a severe convective cloud dataset that supports the deep learning of severe convective cloud segmentation. Our methodology effectively combines the rapid processing abilities of lightweight models with the robustness required for accurate weather forecasting. The ensemble model consistently outperformed traditional, heavier models across several key performance indicators, achieving an accuracy of 0.9941, precision of 0.9391, recall of 0.9201, F1 score of 0.9295, IoU of 0.8684, and computing time of 18.65 s over the test dataset of 300 images. These metrics underscore the efficacy of our approach in maintaining high accuracy while reducing computational demands, making it potential for real-time monitoring and analysis.

This research moves us closer to more dynamic and precise meteorological service systems capable of better predicting and mitigating the impacts of severe weather conditions. The dynamic mechanisms are difficult for a single neural network to fully grasp; only a multi-model ensemble can more easily extract them. However, the multi-model ensemble of non-lightweight networks always faces the problem of low computational efficiency. Here, we demonstrated that the ensembles of several lightweight models can achieve slightly better results using less computing time and power, which means the ensemble of lightweight networks could resolve this issue to some extent. Future work will focus on further refining the ensemble techniques and exploring their applicability to other types of satellite data and environmental monitoring tasks, potentially enhancing the predictive capabilities and operational efficiency of weather services globally.