METEO-DLNet: Quantitative Precipitation Nowcasting Net Based on Meteorological Features and Deep Learning

Abstract

Precipitation prediction plays a crucial role in people’s daily lives, work, and social development. Especially in the context of global climate variability, where extreme precipitation causes significant losses to the property of people worldwide, it is urgently necessary to use deep learning algorithms based on radar echo extrapolation for short-term precipitation forecasting. However, there are inadequately addressed issues with radar echo extrapolation methods based on deep learning, particularly when considering the inherent meteorological characteristics of precipitation on spatial and temporal scales. Additionally, traditional forecasting methods face challenges in handling local images that deviate from the overall trend. To address these problems, we propose the METEO-DLNet short-term precipitation prediction network based on meteorological features and deep learning. Experimental results demonstrate that the Meteo-LSTM of METEO-DLNet, utilizing spatial attention and differential attention, adequately learns the influence of meteorological features on spatial and temporal scales. The fusion mechanism, combining self-attention and gating mechanisms, resolves the divergence between local images and the overall trend. Quantitative and qualitative experiments show that METEO-DLNet outperforms current mainstream deep learning precipitation prediction models in natural spatiotemporal sequence problems.

1. Introduction

Precipitation has been intricately linked to human life since ancient times [1]. With global climate change, predicting precipitation worldwide has become more challenging, and the various disasters caused by extreme precipitation have had a significant impact on people around the world [2]. Therefore, accurately forecasting precipitation is a matter of great concern for countries worldwide [3].

Precipitation can be categorized into short-term, mid-term [4], and long-term precipitation [5] on a time scale [6]. Among these, short-term precipitation (0–2 h) has the most severe impact on residents [7]. Firstly, it can trigger natural disasters such as floods, debris flows, and landslides; secondly, short-term precipitation may lead to flooding, causing urban waterlogging; thirdly, it can damage transportation infrastructure, resulting in traffic accidents [8,9]; and fourthly, it disrupts residents’ daily routines, including outdoor activities and flight plans [10,11]. Therefore, accurate short-term precipitation prediction is of the utmost importance [12,13].

Conventional methods for short-term precipitation forecasting heavily depend on numerical weather prediction (NWP) [14] and extrapolation of radar echo reflectivity. NWP [14], based on physics-based prior knowledge and extensive meteorological data, calculates precipitation forecasts using computers [15]. However, this method has limitations. Firstly, the use of prior physical knowledge in the form of physics models may lead to convergence issues. Secondly, dealing with extensive meteorological data computationally demands significant resources, resulting in low computational efficiency and poor real-time performance. Lastly, physical models have a limited utilization rate of historical meteorological data, making it challenging to integrate existing meteorological data for precipitation prediction [16,17]. Currently, mainstream methods based on radar echo reflectivity extrapolation include the cross-correlation method (TREC) [18], the optical flow method [19], the monomer centroid method [18], and deep learning-based radar echo extrapolation methods [20]. TREC [18] calculates the cross-correlation coefficient between pre- and post-movement images, retains the maximum value during movement, and considers the displacement obtained when the maximum cross-correlation coefficient is achieved as the movement speed of the echo image. It then extrapolates radar echo images for the next moment based on this speed. The main drawback is the difficulty in capturing the growth and decay of weather systems as well as the uncertainty in displacement. The optical flow method [19], originating from target tracking methods in computer vision, calculates motion information based on the correspondence between reflectivity at corresponding positions of adjacent echoes over time. However, it is constrained by two conditions: the total intensity remains constant and the motion does not include rapid nonlinear changes. The monomer centroid method [18] treats radar echoes as a point, conducting recognition and tracking. However, it is sensitive to noise and prone to interference, leading to a decrease in prediction accuracy.

With the development and application of deep learning, radar echo extrapolation methods based on deep learning have been employed in precipitation forecasting [21,22,23,24]. Compared to traditional radar echo extrapolation methods, deep learning-based radar echo extrapolation methods are less sensitive to noise, have fewer constraints, and offer faster computational speed. The general process involves predicting future radar echo image sequences based on input historical radar echo images and then applying the obtained echo distribution to estimate precipitation using the Z-R relationship [25]. The Z-R relationship [25] refers to the Marshall–Palmer relationship, where Z represents radar reflectivity and R represents precipitation intensity. Currently, deep learning-based radar echo extrapolation methods are divided into two technical approaches: convolutional neural networks (CNN) and recurrent neural networks (RNN) [26]. Due to their ability to extract temporal correlations, RNNs are currently more widely used and have developed various variants to better handle spatiotemporal correlations. However, such deep learning models suffer from deficiencies in extracting spatiotemporal features and long-tail blur issues. Moreover, these models, which are general-purpose models for video prediction, still exhibit significant disparities between predicted results and actual values. Real-time precipitation forecasting poses greater challenges compared to typical video prediction problems. Consequently, leveraging radar echo data to improve precipitation forecasting accuracy remains a significant challenge.

This paper proposes a novel meteorology feature-based and deep learning-based quantitative precipitation nowcasting network (METEO-DLNet), which divides radar echo images into fine-scale and coarse-scale levels, predicts and merges radar echo forecast images, and subsequently uses the Z-R relationship for precipitation forecasting. The model overcomes previous spatiotemporal feature extraction deficiencies and the deviation of local image trends from global trends in deep learning-based precipitation forecasting. It enhances the accuracy of long-term forecast images.

In the proposed model, we employ downsampling to divide radar echo images into fine-scale and coarse-scale levels, which are separately input into the recurrent parts of the encoding-decoding structure of RNNs. Within the RNNs, meteorological factors such as terrain and airflow changes during precipitation processes are incorporated. Spatial attention and differential attention are used to learn the spatial and temporal features of radar echo images. Finally, a fusion mechanism combining self-attention and gating mechanisms is utilized to merge the images from the two scales, thereby obtaining the predicted radar echo images.

To fully utilize meteorological features and address the deviation between global and local trends, we designed a short-term precipitation prediction network based on meteorological features and deep learning. Our contributions are as follows:

• We devised a radar echo image prediction framework based on Meteo-LSTM and multiscale fusion, achieving short-term precipitation forecasting;
• A novel Meteo-LSTM was crafted, incorporating spatial attention [27] and differential attention to learn the spatial and temporal features of radar echo images. It maximally leverages meteorological characteristics, enhancing precipitation prediction accuracy from both temporal and spatial perspectives;
• A fusion mechanism, combining self-attention and gate mechanisms, was introduced. This mechanism guided local image along with the overall trend, elevating the accuracy of radar echo image predictions.

The remaining sections of this paper are organized as follows: Section 2 reviews related work. Section 3 presents the short-term precipitation forecasting method based on meteorological features and deep learning. Section 4 displays detailed experimental results. Section 5 is the discussion. Section 6 concludes the paper.

2. Related Work

Currently, deep learning-based precipitation forecasting can be categorized into four types: utilizing recurrent neural networks (RNN) as iterative prediction models, employing convolutional neural networks (CNN) as feedforward prediction models, using multi-scale methods as prediction models, and integrating attention mechanisms as prediction models. In the feedforward models based on CNN, SimVP [28] consists of an encoder, translator, and decoder [29], all constructed entirely with CNN components without additional modules. STRPM [30] proposes a spatiotemporal encoding-decoding scheme to retain more high-resolution spatiotemporal information. It focuses on the spatiotemporal residual features between the previous and subsequent model time steps and introduces a new loss function called learning perceptual loss (LP-loss). Other CNN-based prediction models primarily use U-net [31] as a foundation, such as RainNet [32], SmaAt-Unet [33], and FURENet [34]. However, CNN-based prediction models struggle to capture long-term dependencies in radar echo image sequences, hindering the effective handling of spatiotemporal relationships in predictions.

In the realm of iterative prediction models based on recurrent neural networks (RNN), ConvLSTM [35], proposed by Shi et al., is a classic example. It replaces matrix multiplication in FC-LSTM [36] with convolution operations, achieving the extraction of spatiotemporal relationships. Subsequently, Shi et al. introduced TrajGRU [10], which transforms the previous fixed connections into dynamic connections. Building upon this, Wang proposed PredRNN [37], PredRNN++ [38], PredRNN-V2 [39], MIM [40], and E3D-LSTM [41], further improving precipitation prediction accuracy. PredRNN [37] adds long-term memory on top of ConvLSTM [35] and establishes vertical connections between layers. PredRNN++ [38] introduces a causal LSTM and a gradient highway unit (GHU), addressing the gradient vanishing issue. PredRNN-V2 [39] utilizes memory decoupling loss to let memory cells focus on short-term and long-term memories separately. MIM [40] introduces stationary and non-stationary modules to address stability and non-stationarity during the prediction process. E3D-LSTM [41] replaces the forget gate in PredRNN [37] with an attention mechanism. Wu Haixu proposed MotionRNN [42] and designed MotionGRU to model overall motion trends and instantaneous changes cohesively. Additionally, there are other prediction models like CMS-LSTM [43], MS-LSTM [44], PrecipLSTM [45], and MM-RNN [46], all belonging to iterative prediction models based on recurrent neural networks. Currently, RNN-based precipitation prediction models lack consideration for meteorological features and do not make sufficient use of meteorological characteristics within precipitation.

In the field of attention-based prediction models, MFCA [47] introduced a method for multimodal data fusion, merging data from different sources while utilizing cross-attention mechanisms to facilitate information interaction and feature integration across different data modalities. Tekin, S.F. [48] incorporated attention mechanisms to help the model focus more attentively on important information when dealing with spatiotemporal data. By dynamically adjusting attention weights at each time step, the model can better capture correlations between spatiotemporal data, thereby enhancing prediction performance. Li, Y. [49] introduced an integrated model of spatial-temporal attention network and multi-layer perceptron. By combining these two different types of neural networks, the model effectively leverages their advantages in capturing spatiotemporal information and nonlinear features, thus improving the accuracy and robustness of meteorological forecasts. Self-attention ConvLSTM [35] introduced a self-attention memory module into ConvLSTM to enhance the model’s ability to model spatiotemporal data, more effectively capturing long-range dependencies and important features in spatiotemporal data.

In the realm of multi-scale prediction models, Broad-UNet [50] employs an innovative multi-scale feature learning approach, adaptively learning and utilizing information from different scales. By learning and integrating multi-scale feature representations at different levels, the model can comprehensively understand the spatial and temporal structures of input data, thereby improving its ability to predict complex weather phenomena. TRU-NET [51], combining deep learning techniques with high-resolution rainfall data, introduces spatial and temporal attention mechanisms and multi-level feature learning methods to effectively capture complex spatial and temporal relationships.

3. Methods

3.1. Problem Definition

Due to the significant correlation between radar reflectivity and precipitation intensity obtained through meteorological radar, the precipitation forecasting problem can be transformed as follows. Firstly, to predict the future radar echo sequence based on the observed radar echo sequence. Then, using the Z-R relationship, calculate precipitation rate values through radar reflectivity. Here, Z represents radar reflectivity, and R represents precipitation intensity. In practical terms, we can represent P measurements at a certain local area (H × W grid points) and time t as a tensor ${\mathrm{X}}_{\mathrm{t}}\in {\mathrm{R}}^{\mathrm{P}\times \mathrm{H}\times \mathrm{W}}$. Hence, this issue can be characterized as:

$$\widehat{Y}=\mathsf{\Gamma }(X,Y),$$

(1)

where $X=\{X_{t-n},X_{t-n+1},X_t\}$ is the collection of radar echo images from time t − n to t, $Y=\left\{Y_{t-n},Y_{t-n+1},Y_t\right\}$ is the real radar echo images from time, $\widehat{Y}=\left\{{\widehat{Y}}_{t+1},{\widehat{Y}}_{t+2}\dots {\widehat{Y}}_{t+m}\right\}$ is the predicted radar echo images from time t + 1 to t + m, and Γ is the precipitation prediction model. The goal of the problem is to get $\widehat{Y}$ as close as possible to $Y$.

3.2. Base Model: ST-LSTM

The spatiotemporal long short-term memory unit (ST-LSTM [37]) is a classical predictive unit in deep learning for spatiotemporal forecasting. It has shown good performance on some video prediction datasets. Internally, ST-LSTM [37] consists of two ConvLSTM [35] units, dividing the memory flow into short-term and long-term memories. Short-term memories flow within the same layer, while long-term memories flow between layers. The formula for ST-LSTM [37] is as follows:

$$\begin{array}{l}i=\sigma \left(W_{ix}\ast X_t+W_{ih}\ast H_{t-1}^k\right)\\ f=\sigma \left(W_{fx}\ast X_t+W_{fh}\ast H_{t-1}^k\right)\\ g=\tanh \left(W_{gx}\ast X_t+W_{gh}\ast H_{t-1}^k\right)\\ C_t^l=f\circ C_{t-1}^k+i\circ g\\ i^{\prime }=\sigma \left(W_{i^{\prime }x}\ast X_t+W_{i^{\prime }m}\ast M_t^{k-1}\right)\\ f^{\prime }=\sigma \left(W_{f^{\prime }x}\ast X_t+W_{f^{\prime }m}\ast M_t^{k-1}\right)\\ g^{\prime }=\tanh \left(W_{g^{\prime }x}\ast X_t+W_{g^{\prime }m}\ast M_t^{k-1}\right)\\ M_t^k=f^{\prime }\circ M_t^{k-1}+i^{\prime }\circ g^{\prime }\\ o=\sigma \left(W_{ox}\ast X_t+W_{oh}\ast H_{t-1}^k\right.\left.+W_{oc}\ast C_t^k+W_{om}\ast M_t^k\right)\\ H_t^k=o\circ \tanh \left(W_{1\times 1}\ast \left[C_t^k,M_t^k\right]\right),\end{array}$$

(2)

Here, $W_{\ast \ast }$ represents the convolutional kernel parameters, “$\ast$” denotes the convolution operation, “$\circ$” is the Hadamard product, $\sigma$ represents the sigmoid activation function, $X_t$ is the input, ${\mathrm{C}}_t^k$ denotes the short-term dependent memory at time t, $M_t^k$ represents the long-term dependent memory at time t, and $H_t^k$ is the hidden state at time t. Input gates i or $i^{\prime }$, forget gates f or $f^{\prime }$, input modulation gates $g$ or $g^{\prime }$, and output gates o, respectively, control the information flow. The structure diagram of ST-LSTM [37] is shown in Figure 1.

Although ST-LSTM [37] has made significant progress in handling spatiotemporal correlations, it is a general video prediction model and has not been specifically designed for precipitation forecasting, making it challenging to learn complex precipitation patterns.

3.3. Network Structure

3.3.1. Whole Network

The model takes in a sequence of 10 consecutive frames of radar echo images, and outputs predictions for the next 10 frames of radar echo images. The resolution of each frame is 128 × 128. The model’s comprehensive structure is depicted in Figure 2, comprising two main components. The model consists of two parts. First, there’s the multiscale RNN model composed of Meteorology-LSTM (Meteo-LSTM), based on the differential self-attention mechanism and spatial attention mechanism. Due to the consideration that low-resolution images contain more global information while high-resolution images contain more local information, The model utilizes max-pooling with a kernel size of 2 × 2 to downsample the original images, resulting in low-resolution images of size 64 × 64. High-resolution images are obtained by duplicating the original images, resulting in images of size 128 × 128. Subsequently, the number of channels is increased from 1 to 64 using an embed layer. Then, both types of images are fed into an encoding-decoding structure composed of three layers of Meteo-LSTM. Each Meteo-LSTM layer has 64 input and output channels, with a convolutional kernel size of 3 × 3, and both stride and padding set to 1. The encoding-decoding structure, composed of three layers of Meteo-LSTM, extracts global spatiotemporal features and local spatiotemporal features from the radar echo images, resulting in radar echo prediction images at two different resolutions. Meteo-LSTM incorporates a spatial differential self-attention mechanism for effective extraction of temporal information and a spatial attention mechanism for spatial information extraction.

The second part is the multiscale fusion module, based on the self-attention mechanism and gate mechanism. Firstly, the low-resolution images are upsampled from 64 × 64 to 128 × 128 using bilinear interpolation to facilitate the subsequent fusion process. In the attention module, the convolutional layer has 1 input channel and 100 output channels, with a kernel size of 1 × 1. In the gating mechanism module, the convolutional kernel size is 3 × 3. The final output is radar echo prediction images with a resolution of 128 × 128. This module effectively merges feature maps from different scales, retaining valuable information while discarding less important details. Next, we will introduce the structure and function of each part.

The RNN recurrent part uses an encode-decode architecture, as shown in Figure 3. To efficiently learn meteorological features across spatiotemporal scales, we employ a three-layer Meteo-LSTM as both an encoder and a decoder. Here, $D{L}_t^l$, $D{S}_t^l$ represent meteorological feature memory on the spatial scale and time scale at layer l at time t. These memories are propagated horizontally. $H_t$, $C_t$, and $M_t$ correspond to the representations in ST-LSTM [37].

3.3.2. Meteo-LSTM

To address the specific issue of precipitation evolution, this paper builds upon ST-LSTM [37] and introduces Meteo-LSTM, designed to capture both temporal and spatial information. Meteo-LSTM incorporates two modules, the Spatial Attention Module (SA), which focuses on learning geographic spatial features, and the Differential Attention Module (DA), which concentrates on understanding the impact of historical precipitation on the present. The structure diagram of Meteo-LSTM is shown in Figure 4.

Spatial features in radar echo images exhibit anisotropy, organized into bands rather than isotropic features, and the statistical structure of rainfall is influenced by topography such as mountains. Therefore, the Spatial Attention Module (SA) is designed to learn geographical features in radar echo images. Experimental results show that differential information is crucial in predicting radar echo images. Hence, the Differential Attention Module (DA) first processes adjacent time frames differentially, then applies an attention mechanism to learn the impact of historical precipitation on the present, and finally employs LSTM to enhance both short-term and long-term memory, effectively capturing temporal changes in radar echo images.

We recognize that precipitation is influenced by topography on a spatial scale. Topography can affect airflow and wind direction in the atmosphere, causing airflow to be blocked or forced to rise. Additionally, topography can induce changes in wind speed, leading to the formation of topographic vortices that result in precipitation. Topographic features can trigger dynamic and thermodynamic effects in the atmosphere, resulting in different precipitation patterns in different regions. Therefore, we introduced the spatial attention mechanism to learn topographic features. The configuration of the Spatial Attention Module (SA) is depicted in Figure 5. The formula for the Spatial Attention Module (SA) is as follows:

$$\begin{array}{l}{\tilde{X}}_t=W_{ax}\ast \left(\sigma \left(W_{am}\ast \left[AvgPool\left(X_t\right),MaxPool\left(X_t\right)\right]\right)\cdot X_t\right)\\ i=\sigma \left(W_{ix}\ast {\tilde{X}}_t+W_{ih}\ast D{L}_{t-1}^k\right)\\ f=\sigma \left(W_{fx}\ast {\tilde{X}}_t+W_{fh}\ast D{L}_{t-1}^k\right)\\ g=\tanh \left(W_{gx}\ast {\tilde{X}}_t+W_{gh}\ast D{L}_{t-1}^k\right)\\ D{L}_t^k=f\circ D{L}_{t-1}^k+i\circ g\end{array}$$

(3)

$X_t$ represents the input, ${\tilde{X}}_t$ is the result after spatial attention processing, “$\ast$” denotes convolution operation, $W_{\ast \ast }$ represents convolution kernel parameters, the convolution kernel size is 3 × 3, “$\circ$” is the Hadamard product, $\sigma$ represents the sigmoid activation function, and $D{L}_t^k$ represents the spatial feature storage of the k-th layer at time t.

We also note that, at the temporal scale, convective activity and changes in airflow during the early stages of precipitation may continue to influence the later stages. Intense convective activity may form in the early stages, leading to localized heavy precipitation and potentially affecting the precipitation pattern in the later stages. Meanwhile, moist soil and high surface humidity in the early stages may increase evaporation and evaporative cooling, affecting the formation and distribution of precipitation in the later stages. Therefore, we propose a module based on the differential attention mechanism, which captures the differences between adjacent time frames and uses an attention mechanism to learn the impact of early-stage precipitation on later-stage precipitation. The configuration of the Differential Attention Module (DA) is depicted in Figure 6. The formula for the Differential Attention Module (DA) is provided.

$$\begin{array}{l} x=X_t^k-X_{t-1}^k\\ Z=x+softmax\left(x\cdot {\left[D{S}_{0:t-1}^k,x\right]}^{\top }\right)\cdot \left[D{S}_{0:t-1}^k,x\right]\\ i=\sigma \left(W_{ix}\ast Z+W_{ih}\ast D{S}_{t-1}^k\right)\\ f=\sigma \left(W_{fx}\ast Z+W_{fh}\ast D{S}_{t-1}^k\right)\\ g=\tanh \left(W_{gx}\ast Z+W_{gh}\ast D{S}_{t-1}^k\right)\\ D{S}_t^k=f\circ D{S}_{t-1}^k+i\circ g\end{array}$$

(4)

$X_t$ and $X_{t-1}^k$ represent the input, $x$ represents the new matrix obtained after differentiation, Z represents the matrix of the impact of historical precipitation on the present obtained through an attention mechanism in feature space, “$\ast$” denotes convolution operation, $W_{\ast \ast }$ represents convolution kernel parameters, The convolution kernel size is 3 × 3, “$\circ$” is the Hadamard product, “$\cdot$” denotes matrix multiplication, σ represents the sigmoid activation function, and $D{S}_t^k$ represents the temporal feature storage of the k-th layer at time t.

To effectively fuse the features obtained by the Spatial Attention Module (SA) and Differential Attention Module (DA), we use Formula (5) for feature fusion. ${\mathrm{H}}_t^k$ represents the hidden state of the k-th layer at time t. $W_{1\times 1}$ represents the convolution kernel parameter, with a kernel size of 1 × 1.

$$\begin{gathered} o=\sigma \left(W_{ox}\ast X_t+W_{oh}\ast H_{t-1}^k+W_{oc}\ast C_t^k+W_{om}\ast M_t^k+W_{ods}\ast D{S}_t^k+W_{odl}\ast D{L}_t^k\right) \\ {H}_t^k=o\circ \tanh \left(W_{1\times 1}\ast \left[C_t^k,M_t^k,D{S}_t^k,D{L}_t^k\right]\right) \end{gathered}$$

(5)

3.3.3. The Fusion Mechanism Based on Self-Attention and Gating Mechanisms

Meteo-LSTM yields predictions at both high and low resolutions, capturing two distinct scales of forecast images. The low-resolution prediction image displays global information, capturing overall features of spatiotemporal movements. In contrast, the high-resolution prediction image reveals local details, capturing specific features of spatiotemporal movements. However, local features might deviate partially from the overall trend during the prediction process. Therefore, using the low-resolution image as a guide, the overall features of the high-resolution prediction image are corrected.

Figure 7 depicts the fusion module based on self-attention and gating mechanisms. Initially, the predictions from both scales are separately input into the self-attention module. Subsequently, using the gating mechanism, the module retains parts of the detailed prediction image that align with the overall trend and discards those that deviate from it. The formula for the fusion module incorporating self-attention and gating mechanisms is as follows:

$$\begin{array}{l}{Z}_l=softmax\left(\left(W_{lq}\ast X_l\right)\cdot {\left(W_{lk}\ast X_l\right)}^{\top }\right)\cdot \left(W_{lv}\ast X_l\right)\\ Z_h=softmax\left(\left(W_{lq}\ast X_l\right)\cdot {\left(W_{hk}\ast X_h\right)}^{\top }\right)\cdot \left(W_{hv}\ast X_h\right)\\ Z_m=\left(W_m\ast \left[Z_l,Z_h\right]\right)\\ i=\sigma \left(W_{i,z}\ast Z_m+W_{i,xl}\ast X_l\right)\\ g=\tanh \left(W_{g,z}\ast Z_m+W_{g,xl}\ast X_l\right)\\ H=\left(1-i\right)\circ X_h+i\circ g\end{array}$$

(6)

The model takes $X_l$ and $X_h$ as inputs, where $X_l$ represents the low-resolution prediction image and $X_h$ represents the high-resolution image. “$\ast$” denotes convolution operation, $W_{\ast \ast }$ represents convolution kernel parameters, The convolution kernel size is 1 × 1, “$\cdot$” denotes matrix multiplication, σ represents the sigmoid activation function, After convolution, corresponding Q, K, and V are obtained. We use $Q_l$ obtained from the low-resolution image as the query for the high-resolution image to focus on regions where local features align with global features. Through attention, we obtain two feature vectors, $Z_l$ and $Z_h$. After concatenation and convolution, they are input into the gating mechanism. The gating mechanism adjusts the weights of the input data, allowing for better information processing in different contexts.

4. Experiment

4.1. Dataset

The MeteoNet dataset [52] was collected and organized by the French Meteorological Office. It consists of two regions, each covering an area of 550 km by 550 km. One region is in the northwest of France, and the other is in the southeast. The spatial resolution of the radar is 0.01°, and the projection system used is EPSG:4326. We chose the data from the northwest region of France, spanning three years from 2016 to 2018. The collected dataset includes subsets of rainfall, rainfall radar data quality, and radar reflectivity. An example diagram is shown in Figure 8. The values in the radar reflectivity dataset range from 0 to 70 dBZ, with each observation having a data grid size of 784 × 565 pixels. Data is recorded every five minutes. To achieve precipitation forecasting for the next two hours, one radar echo image is selected every 15 min. A sample consists of 20 radar echo images. For computational convenience and due to limitations in computing power, we downsampled the original resolution data to 128 × 128 pixels. The rainfall radar data quality dataset associates a quality code with each pixel value of the rainfall product, ranging from 0 (very poor) to 100 (perfect), measured in percentages. If the value is missing, it is set to 255.

We used radar reflectivity as the training and testing dataset for the model. During precipitation forecast evaluation, we first calculate the reflectivity corresponding to R = 0.5, 2, 5, and 10 using the E-R relationship. Then, we use these values as thresholds to compute the CSI and HSS indicators for the predicted radar reflectivity image compared to the true radar reflectivity image. Based on rainfall radar data quality, data with poor quality in the cumulative rainfall dataset was excluded. After preprocessing, the initial dataset is sorted chronologically, and samples are taken at intervals of 15 min. Hence, there is no temporal overlap between samples. The training/evaluation/testing datasets are organized in a ratio of 8:1:1 and split randomly. The training set consists of 4000 data samples, and both the testing and validation sets consist of 500 data samples each. Each data sample includes 20 consecutive radar echo images, with the first 10 frames used as input and the remaining 10 frames for output. The model’s evaluation results are based on the validation set, and to minimize errors, all models were experimented with three times. The final results were averaged over the three validation sets. The early part of the validation set was not involved in model training and testing but served as the ultimate evaluation standard.

4.2. Loss Function

This model utilizes an L1 + L2 loss function for optimization. Additionally, a reverse planning sampling scheme is employed to accelerate the iterative convergence of the model. The loss function is defined as:

$$loss=\frac{1}{n}{{\displaystyle \sum }}_{t=1}^n\left(\left|E_t-{\widehat{E}}_t\right|+{\left|E_t-{\widehat{E}}_t\right|}^2\right)$$

(7)

Here, n represents the total number of output images obtained through the RNN loop in a sample set, set to 19 in this case. $E_t$ denotes the real image, and ${\widehat{E}}_t$ represents the predicted image.

4.3. Implementation Details

This study used Python as the programming language and PyTorch [53] as the deep learning framework, implemented on an NVIDIA RTX A100 GPU. All models were trained using the Adam optimizer [54], with a batch size of four, an initial learning rate of 10⁻⁴, and a training duration of 30 epochs for each model. Early stopping strategy was employed during training, monitoring the validation loss, and halting training when the validation loss ceased to decrease or began to increase, thereby preventing overfitting. To prevent overfitting, this paper employs 5-fold cross-validation. The total of 4500 samples from the training and testing sets are divided into five groups. Validation is conducted on the validation set only when the cross-validation differences are within 1%. The validation set is then used as the final result. To ensure a fair comparison, all models shared the same hyperparameters. The input to the models consisted of 10 consecutive radar echo images with a resolution of 128 × 128, and the output comprised 10 radar echo images. Since the comparative model in this study adopts an encoding-decoding structure, and the first part of the proposed METEO-DLNet also utilizes an encoding-decoding structure, for the sake of fairness and reliability in comparing experiments and ablation experiments, the encoding-decoding structures are equipped with the same number of stacked RNN layers. Moreover, considering that many models compared in the literature often consist of three layers of stacked RNNs in the encoding-decoding structure, this study also employs three layers of Meteo-LSTM stacked to form the encoding-decoding structure. The reverse planning sampling method [39] was also employed to expedite model training. Reverse planning sampling [39] forces the model to learn more about long-term dynamics during the encoding process by reducing the probability of randomly hiding true observations. As the training phase progresses, the probability will decrease.

4.4. Performance Metrics

Due to the broad range of precipitation amounts, evaluating predictions based solely on whether it is raining in a region is not sufficient. This study categorized precipitation amounts into five thresholds (0.5, 2, 5, and 10 mm/h) to assess the model’s precipitation forecasting capabilities at different levels. To evaluate the accuracy of precipitation predictions, the Heidke skill score (HSS) [10] and the critical success index (CSI) [55] were used.

Firstly, the pixel values (P: 0–255) were converted to radar echo reflectivity intensity (dBZ) using the transformation:

$$P=255\times \frac{\mathrm{dBZ}}{60}$$

(8)

Based on the Z-R relationship [25], the conversion between radar echo reflectivity intensity (dBZ) and precipitation intensity values (R: mm/h) for the MeteoNet dataset is given by:

$$\mathrm{dBZ}=10\times \lg \left(200\times R^{1.6}\right)$$

(9)

After obtaining the precipitation intensity prediction matrix, values below the threshold were set to 0 (indicating no precipitation), and values above the threshold were set to 1 (indicating precipitation). The same procedure was applied to the rainfall dataset to obtain the true precipitation intensity matrix. True positive (TP), false positive (FP), true negative (TN), and false negative (FN) predictions were then calculated, and the formulas for calculating the CSI and HSS are as follows:

$$\mathrm{CSI}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}+\mathrm{FP}}\mathrm{HSS}=\frac{\mathrm{TP}\times \mathrm{TN}-\mathrm{FN}\times \mathrm{FP}}{\left(\mathrm{TP}+\mathrm{FN}\right)\left(\mathrm{FN}+\mathrm{TN}\right)+\left(\mathrm{TP}+\mathrm{FP}\right)\left(\mathrm{FP}+\mathrm{TN}\right)}$$

(10)

In addition to the mentioned metrics, mean squared error (MSE), mean absolute error (MAE), and the structural similarity index (SSIM) [56] were also used. Due to the small scale of MSE and MAE values, making it inconvenient to display and compare results, we utilize MSE×N² and MAE×N² instead of MSE and MAE for evaluating the model, where N represents the size of the radar echo image, and N=128. A higher SSIM value indicates greater similarity between the two images. The SSIM formula is as follows:

$$SSIM\left(x,y\right)=\frac{\left(2{\mu }_x{\mu }_y+C_1\right)\left(2{\sigma }_{xy}+C_2\right)}{\left({\mu }_x^2+{\mu }_y^2+C_1\right)\left({\sigma }_x^2+{\sigma }_y^2+C_2\right)}$$

(11)

4.5. Experimental Results and Comparison with SOTA

In this section, we compare our precipitation forecasting network based on meteorological features and deep learning with current outstanding spatiotemporal prediction models in short-term precipitation forecasting. The models for comparison include ConvLSTM [35], TrajGRU [10], PredRNN [37], MIM [40], and MotionRNN [42]. For a fair comparison, all models adopted an encoding-decoding structure. Since MotionRNN [42] needs to be added to an RNN recurrent unit and the original paper suggested that the best combination for prediction was PredRNN [37] + MotionRNN [42], we added MotionRNN [42] to the model based on PredRNN [38]. The average evaluation results for short-term precipitation forecasting are shown in

Table 1. Comparison of radar echo image prediction using different models.

Models	Params	SSIM	MSE × N2	MAE × N2	Time
ConvLSTM [35]	0.94 m	0.793	20.596	187.010	0.8 h
TrajGRU [10]	0.97 m	0.795	20.040	184.981	3.9 h
PredRNN [37]	1.94 m	0.799	20.385	184.374	1.9 h
MIM [40]	3.91 m	0.805	19.961	181.537	4.9 h
MotionRNN [42]	4.02 m	0.807	19.397	180.778	6.4 h
METEO-DLNet	4.07 m	0.814	19.354	176.844	5.5 h

.

From Table 1, it can be observed that, compared to ConvLSTM (2015), METEO-DLNet’s MAE×N² has decreased from 187.010 to 176.844, marking a 5.4% reduction. In comparison to MotionRNN (2021) [42], METEO-DLNet’s MAE × N² has decreased from 180.778 to 176.844, showing a 2.2% reduction. This indicates that METEO-DLNet, as proposed in this paper, effectively reduces prediction errors and enhances the accuracy of image predictions compared to previous models.

Figure 9 illustrates the variation of prediction performance over time steps. It can be observed that MSE shows an increasing trend, while SSIM shows a decreasing trend. This is because as the prediction time lengthens, the error accumulates for each frame, leading to a continuous increase in prediction loss and a decrease in prediction accuracy.

Furthermore, various common precipitation forecast indicators were used to evaluate the accuracy of these methods. Precipitation was categorized into four ranges based on the previously mentioned thresholds: R ≥ 0.5, R ≥ 2, R ≥ 5, and R ≥ 10. The results are shown in

Table 2. CSI and HSS score for R > 0.5 dBZ.

Models	CSI	HSS
ConvLSTM [35]	0.606	0.722
TrajGRU [10]	0.617	0.733
PredRNN [37]	0.621	0.735
MIM [40]	0.631	0.740
MotionRNN [42]	0.635	0.745
METEO-DLNet	0.636	0.746

,

Table 3. CSI and HSS score for R > 2 dBZ.

Models	CSI	HSS
ConvLSTM [35]	0.263	0.393
TrajGRU [10]	0.417	0.557
PredRNN [37]	0.420	0.557
MIM [40]	0.452	0.593
MotionRNN [42]	0.457	0.591
METEO-DLNet	0.466	0.599

,

Table 4. CSI and HSS score for R > 5 dBZ.

Models	CSI	HSS
ConvLSTM [35]	0.083	0.140
TrajGRU [10]	0.243	0.356
PredRNN [37]	0.238	0.342
MIM [40]	0.248	0.364
MotionRNN [42]	0.298	0.415
METEO-DLNet	0.310	0.432

and

Table 5. CSI and HSS score for R > 10 dBZ.

Models	CSI	HSS
ConvLSTM [35]	0.009	0.018
TrajGRU [10]	0.156	0.242
PredRNN [37]	0.108	0.177
MIM [40]	0.091	0.152
MotionRNN [42]	0.198	0.292
METEO-DLNet	0.234	0.336

.

In these tables, higher CSI and HSS values indicate better performance. It can be observed that due to the long-tail distribution of precipitation (few heavy rains but many light rains) [57], the performance of all models deteriorates as the rainfall threshold increases. Since heavy precipitation has the greatest impact on daily life, the focus here is primarily on the R ≥ 10 scenario. As shown in the table, compared to ConvLSTM (2015) [35], METEO-DLNet’s CSI (R ≥ 10) has increased from 0.009 to 0.234, a 25 times increase, and HSS (R ≥ 10) has increased from 0.018 to 0.336, a 17.7 times increase. In comparison to MotionRNN (2021) [42], METEO-DLNet’s CSI (R ≥ 10) has increased from 0.198 to 0.234, a 18.2% increase, and HSS (R ≥ 10) has increased from 0.292 to 0.336, a 15.1% increase. METEO-DLNet consistently outperforms both the early ConvLSTM model (2015) [35] and the more recent MotionRNN model (2021) [42] in nearly all metrics, underscoring its superior performance. This suggests that METEO-DLNet, introduced in this study, excels by proficiently capturing the spatial and temporal characteristics of radar echo images through spatial attention, differential attention, and effective utilization of meteorological features. As a result, it significantly enhances precipitation prediction accuracy across both temporal and spatial dimensions. Additionally, the fusion mechanism based on self-attention and gating has successfully guided overall trends and improved the prediction accuracy of radar echo images. Especially in the prediction of heavy precipitation, METEO-DLNet demonstrates a significant improvement compared to previous models.

To further visually demonstrate METEO-DLNet’s ability to predict high-intensity rainfall, Figure 10 shows randomly selected visualization examples.

As the prediction time increases, ConvLSTM [35] and TrajGRU [10] perform poorly in predicting high-intensity precipitation, primarily manifested in the widespread absence of rainfall. PredRNN [37] seems to exhibit better integrity in predicting the area, but similar to MIM [40] and MotionRNN [42], it experiences blurriness in localized rainfall prediction images. Specifically, in the lower right part of the predicted images, METEO-DLNet is closest to the ground truth, while other models show larger discrepancies. METEO-DLNet, due to the introduction of meteorological features at both temporal and spatial scales and the use of a multi-scale fusion mechanism, excels in handling rainfall details, resulting in clearer precipitation prediction images. In summary, the observations indicate that METEO-DLNet’s advantages become more pronounced with increasing rainfall intensity.

4.6. Ablation Experiment and Analysis

We discuss the effectiveness of Meteo-LSTM and the fusion mechanism in this section.

Effectiveness of Meteo-LSTM: In this comparison between ST-LSTM and Meteo-LSTM, both are evaluated separately to ensure a fair comparison. Both models utilize a three-layer encoding-decoding structure. Additionally, experiments were conducted with Meteo-LSTM configurations lacking the SA module and the DA module. The results are shown in

Table 6. Comparison results between ST-LSTM and Meteo-LSTM on the radar echo test set.

Models	CSI				HSS				SSIM	MSE × N2	MAE × N2
	R ≥ 0.5	R ≥ 2	R ≥ 5	R ≥ 10	R ≥ 0.5	R ≥ 2	R ≥ 5	R ≥ 10
ST-LSTM [37]	0.621	0.420	0.238	0.108	0.735	0.557	0.342	0.177	0.799	20.385	184.374
Meteo-LSTM Without SA	0.628	0.434	0.452	0.136	0.739	0.579	0.371	0.249	0.804	20.012	182.347
Meteo-LSTM Without DA	0.624	0.426	0.440	0.114	0.736	0.568	0.352	0.187	0.800	20.312	183.471
Meteo-LSTM	0.633	0.455	0.269	0.157	0.742	0.594	0.393	0.249	0.806	19.924	180.491

.

Table 6 indicates significant improvements in precipitation forecasting with Meteo-LSTM compared to ST-LSTM. The Critical Success Index (CSI) (R > 10) increased from 0.108 to 0.157, marking a 45.3% improvement, while the Heidke Skill Score (HSS) (R > 10) rose from 0.177 to 0.249, indicating a 40.7% increase. Moreover, the table illustrates that the predictive capability of Meteo-LSTM decreases when either the DA or SA module is absent. These results demonstrate that spatial attention and differential attention effectively leverage meteorological features to learn spatial and temporal characteristics of radar echo images. Meteo-LSTM indeed enhances precipitation forecasting accuracy from both temporal and spatial perspectives.

Effectiveness of the fusion mechanism: a comparison was made between the model without the fusion mechanism (DLNet) and the model with the fusion mechanism (METEO-DLNet). In DLNet, the fusion involves directly adding models of two different scales and equally dividing the result. The results are shown in

Table 7. Comparison results between DLNet and METEO-DLNet on the radar echo test set.

Models	CSI				HSS				SSIM	MSE × N2	MAE × N2
	R ≥ 0.5	R ≥ 2	R ≥ 5	R ≥ 10	R ≥ 0.5	R ≥ 2	R ≥ 5	R ≥ 10
DLNet	0.635	0.459	0.283	0.182	0.743	0.596	0.410	0.273	0.811	19.986	179.346
METEO-DLNet	0.636	0.466	0.310	0.234	0.746	0.599	0.432	0.336	0.814	19.354	176.844

.

After replacement, all indicators of the model show a decline, with a 18.8% decrease in HSS (R ≥ 10) and a 22.2% decline in CSI. The results suggest that the fusion mechanism, based on the combination of self-attention and gating mechanisms, achieves the functionality of guiding overall trends in local images and enhances the prediction accuracy of radar echo images.

5. Discussion

Through comparison with SOTA and ablation experiments, the observations can be summarized as follows. Firstly, the model using Meteo-LSTM performs better in precipitation prediction compared to traditional ST-LSTM [37], indicating that the method of incorporating meteorological features into deep learning models is effective. This method utilizes spatial attention to learn terrain features during precipitation and differential attention to learn differences between adjacent time frames, fully utilizing meteorological features to improve accuracy in precipitation prediction.

Secondly, the fusion mechanism proposed in this paper, based on the combination of self-attention and gating mechanisms, enhances the accuracy of precipitation prediction. This indicates that it retains parts of the fine prediction image consistent with overall trends and discards parts that deviate from overall trends.

Finally, combining Meteo-LSTM with the fusion mechanism based on self-attention and gating mechanisms achieves more accurate precipitation forecasts.

While this study has indeed further enhanced the accuracy of precipitation forecasting, it is important to note that the Z-R formula is an empirical formula derived from precipitation statistics, and thus, there exists error between the predicted and observed precipitation values. In our next step, we aim to explore the utilization of deep learning methods to establish end-to-end precipitation forecasting models to replace traditional Z-R relationships. Additionally, we plan to consider incorporating other meteorological parameters such as wind into the precipitation forecasting model to enhance the accuracy of radar echo variation predictions.

6. Conclusions

Currently, there is still room for significant improvement in radar echo extrapolation methods based on deep learning, with inherent meteorological characteristics in precipitation often not receiving sufficient attention. Precipitation exhibits intrinsic meteorological features. Spatially, terrain has a significant impact, triggering dynamic and thermodynamic effects in the atmosphere and leading to diverse precipitation patterns in different regions. Temporally, convective activity and airflow variations preceding precipitation may persistently influence later precipitation periods. In addition to considering meteorological features, traditional predictions face challenges in handling local images that deviate from the overall motion trend. Therefore, to fully leverage meteorological features and address the deviation between local and global trends, we designed METEO-DLNet, a short-term precipitation forecasting network based on meteorological features and deep learning. Through experiments, the following conclusions were drawn.

Firstly, our designed Meteo-LSTM effectively learns the spatial and temporal features of radar echo images using spatial and differential attention, fully utilizing meteorological features, and enhancing precipitation prediction accuracy at both temporal and spatial scales.

Secondly, the fusion mechanism based on self-attention and gating mechanisms, designed in this paper, guides overall trends in local images and enhances the prediction accuracy of radar echo images.

Thirdly, both quantitative and qualitative experiments indicate that our designed short-term precipitation forecasting network (METEO-DLNet) outperforms current mainstream deep learning-based precipitation forecasting models in this challenging natural spatiotemporal sequence problem.