Enhancing Rainfall Nowcasting Using Generative Deep Learning Model with Multi-Temporal Optical Flow

Abstract

Precipitation nowcasting is critical for preventing damage to human life and the economy. Radar echo tracking methods such as optical flow algorithms have been widely employed for precipitation nowcasting because they can track precipitation motions well. Thus, this method, including the McGill algorithm for precipitation nowcasting by Lagrangian extrapolation (MAPLE), was implemented for operational precipitation nowcasting. However, advection-based methods struggle to predict the nonlinear motions of precipitation fields and dynamic processes, such as the growth and decay of precipitation. This study proposes an enhanced optical flow model using a multi-temporal optical flow field and a conditional generative adversarial network (cGAN). We trained the proposed model using a 3-year radar dataset provided by the Korean Meteorological Administration and performed forecast skill evaluations using both qualitative and quantitative methods. In particular, the model featuring multi-temporal optical flow enhances prediction accuracy for the nonlinear motion of precipitation fields, and the model’s accuracy can be further improved through the use of the cGAN structure. We have verified that these improvements hold for 0–3 h lead times. Based on this performance enhancement, we conclude that the multi-temporal optical flow model with cGAN has a potential role in operational precipitation nowcasting.

1. Introduction

Precipitation nowcasting is crucial for preventing damage to human life and financial losses; thus, nowcasting models have been extensively proposed in recent decades. Using weather radar datasets, radar echo tracking based on the optical flow method has been adopted for precipitation nowcasting [1,2,3,4,5] and has also been implemented in operational precipitation nowcasting such as the McGill algorithm for precipitation nowcasting by Lagrangian extrapolation (MAPLE) [6]. Precipitation nowcasting based on such models has the benefit of maintaining the spatial resolution of predicted images. However, the available prediction timescale is limited, typically around a 1 h lead time [1,2], due to their poor performance when predicting newly developed and/or dissipated precipitation. Additionally, optical flow methods often struggle to capture the temporal evolution of motion, which makes tracking the nonlinear motion commonly observed during precipitation events challenging.

To train and predict the nonlinear evolution of precipitation fields, precipitation nowcasting models based on a deep learning approach have been extensively proposed using a weather radar dataset. For instance, convolutional long short-term memory models (Conv-LSTM) have been employed to predict sequential output from sequential inputs [7,8,9]. In addition, U-Net convolutional neural networks have also been used (e.g., [10,11,12,13,14]). Models based on convolutional neural networks outperform strong baselines such as optical flow methods and numerical weather prediction [9]. However, Fourier analysis reveals that these models produce blurred prediction maps; thus, they are unlikely to be adopted for operational precipitation nowcasting [11].

Compared with the convolutional models listed above, generative models have attracted attention as a technology for precipitation nowcasting [15,16,17]. The generative model produces images or videos based on the distribution of the training image/video dataset. Generative adversarial networks (GAN) [18] have been extensively implemented in various models for efficient training. The GAN simultaneously trains the generator and discriminator. The generator produces images from random input noise, and the images produced by a well-trained generator show distributions similar to real images. The discriminator learns to classify the fake (i.e., images produced by the generator) and real images. Through competitive training of the generator and discriminator, the generator can make the output images indistinguishable from real images. Generative models, such as conditional GAN (cGAN), use additional conditions (e.g., input data of the generator) for training and can generate targeted outputs that suit specific conditions [19]. Generative models based on cGAN, such as deep generative model of radar (DGMR), have been proposed, and these models have the benefit of reducing the blurry effect compared with convolutional neural networks [15,17]. However, the prediction accuracy based on the critical success index (CSI) is poor, particularly when predicting strong rainfall events ($\ge 5\mathrm{m}\mathrm{m}/\mathrm{h}$), and the predicting time is limited so far up to 1.5 h lead time, which is much shorter than the timescale required for operational precipitation nowcasting. Indeed, according to surveys across Europe, for instance, the preferred lead time to start with preparatory measures is 3–6 h lead times [20,21]. Hence, it is unlikely to adopt the currently proposed model for operational precipitation nowcasting.

Recently, deep learning-based methods using optical flow technique have been reported to capture nonlinear motions, both in computer vision [22,23,24,25,26] and atmospheric science using satellite imagery [27]. These methods extract time-sequence input features using deep learning networks to track nonlinear evolution, which cannot be captured by the optical flow algorithm alone. In this study, we adopted these approaches to advance the precipitation nowcasting model based on advection through the optical flow method. Specifically, the model proposed in this study is divided into two parts: (1) input image generation by linear extrapolation using multi-temporal optical flow fields, and (2) a conditional GAN for capturing the features of nonlinear evolution that are not included in the linear extrapolated input image. The forecasting skills of the model were evaluated using both quantitative and qualitative methods.

This paper is organized as follows. In Section 2, we describe the radar dataset and propose the model architecture, including the optical flow method and conditional GAN structure. The baselines for the model comparison are also summarized. Section 3 reports the results of qualitative and quantitative analyses. Finally, Section 4 provides a brief summary.

2. Method

2.1. Radar Dataset

Weather radars measure instantaneous rain rates, and radar data have been used to develop precipitation nowcasting models. The Korean Meteorological Administration (KMA) has provided radar data from ten S-band Doppler radars covering the latitudes of $31.0°$ to $40.5°$N and longitudes of $121.5°$ to $132.5°$E. The Lambert conformal conic method with a central point at $38.0°$N and $126.0°$E was employed for the map projection. The radar reflectivity dataset provided by the KMA was generated using the HSR method. This method involves synthesizing reflectivity from the hybrid surface that is unaffected by non-meteorological echoes, ground clutter, beam blockage, and the bright band [28]. The HSR data have spatial and temporal resolutions of 0.5 km and 10 min, respectively. The precipitation ($R$) in the unit of mm/h can be obtained by the radar reflectivity factor (Z) in the unit of ${\mathrm{m}\mathrm{m}}^6/\mathrm{m}$ through the ZR-relation ($Z=148R^{1.59}$), which is currently employed in the weather radar center of the KMA. Radar data from 2018, 2019, and 2021 were used for training, whereas data from 2020 and 2022 were used for validation and testing, respectively.

2.2. Motion of Precipitation Field

The optical flow method was employed to obtain the motion fields of the precipitation events. The optical flow method extracts the velocity fields, ${\mathit{V}}_{\mathit{f}}$ between the two images. Here, we used the TV-L1 algorithm (see [29] for details of the TV-L1 algorithm), which minimizes the following regularization force,$J$:

$$J\equiv \int \left\{\left|\nabla {\mathit{V}}_{\mathit{f}}\right|+\lambda \left|R_0\left(\mathit{x}\right)-R_1\left(\mathit{x}+{\mathit{V}}_{\mathit{f}}\left(\mathit{x}\right)\right)\right|\right\}dx.$$

(1)

Here, $R_0$ and $R_1$ correspond to precipitation images measured at two different time steps. The first and second terms represent the smoothness of the motion fields and the optical flow constraint, assuming brightness constancy during motion, respectively. The relative importance between these terms is determined by the free parameter $\lambda$. The TV-L1 algorithm is included in the OpenCV library (https://opencv.org, accessed on 17 September 2023).

To analyze the flow motion in detail, we decomposed the optical flow field, ${\mathit{V}}_{\mathit{f}}$ into its divergence component (${\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}$) and curl component (${\mathit{V}}_{\mathit{c}\mathit{u}\mathit{r}\mathit{l}}$) based on Helmholtz’s theorem [30], which are defined as follows:

$${\mathit{V}}_{\mathit{f}}={\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}+{\mathit{V}}_{\mathit{c}\mathit{u}\mathit{r}\mathit{l}},$$

(2)

$${\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}=\nabla \chi , {\mathit{V}}_{\mathit{c}\mathit{u}\mathit{r}\mathit{l}}=\mathbf{k}\times \nabla \psi ,$$

(3)

$${\nabla }^2\chi =\nabla ·{\mathit{V}}_{\mathit{f}}, {\nabla }^2\psi =\mathbf{k}· \nabla \times {\mathit{V}}_{\mathit{f}}.$$

(4)

Note that all flow vectors are comprised of two velocity components (i.e., ${\mathit{V}}_{\mathit{f}}=u_f\mathit{i}+v_f\mathit{j}$; ${\mathit{V}}_{\mathit{c}\mathit{u}\mathit{r}\mathit{l}}=u_{curl}\mathit{i}+v_{curl}\mathit{j}$; ${\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}=u_{div}\mathit{i}+v_{div}\mathit{j}$), where component-$\mathit{i}$ and $\mathit{j}$ represent unit vectors in the in-plane direction, and component $\mathit{k}$ is directed along the out-of-plane direction. $\psi$ and $\chi$ denote the streamfunction and velocity potential, respectively. To calculate ${\mathit{V}}_{\mathit{c}\mathit{u}\mathit{r}\mathit{l}}$ and ${\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}$, we first computed $\chi$ for ${\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}$, using an iterative successive overrelaxation as described in [2]. We then subtracted ${\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}$ from the original optical flow field, ${\mathit{V}}_{\mathit{f}}$. The component ${\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}$ is responsible for growth and decay of the precipitation field, whereas the component ${\mathit{V}}_{\mathit{c}\mathit{u}\mathit{r}\mathit{l}}$ describes the circulation motion of an incompressible field.

To examine the motion characteristics of the precipitation events, we conducted Fourier analysis on the decomposed field components ${\mathit{V}}_{\mathit{c}\mathit{u}\mathit{r}\mathit{l}}$ and ${\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}$. Using the power spectra of the curl component ($F_{curl}(k_x,k_y)$) and the divergence component ($F_{div}(k_x,k_y)$) obtained through Fourier transform, we computed the radially averaged power spectra (see also [31] and the references therein), $P_{curl}\left(k\right)$ and $P_{div}\left(k\right)$, as follows:

$$P_{curl}\left(k\right)=\frac{1}{N_r\left(k\right)}\sum _{i=1}^{N_r\left(k\right)}{F}_{curl}\left(k_i\right),$$

(5)

$$P_{div}\left(k\right)=\frac{1}{N_r\left(k\right)}\sum _{i=1}^{N_r\left(k\right)}{F}_{div}\left(k_i\right),$$

(6)

Here, $k\equiv \sqrt{k_x^2+k_y^2}$ represents the radial wavenumber and $N_r\left(k\right)$ is the number of wavenumber samples that satisfy $\left|k-k_i\right|=dk/2$ within the radial wavenumber bin, $dk$. The ratio $\varphi \left(k\right)\equiv P_{curl}\left(k\right)/P_{div}\left(k\right)$ was also estimated for examining the relative importance of the turbulent flow motions as a function of the spatial scale.

Panels (a) and (b) of Figure 1 display the power spectra $P_{curl}\left(k\right)$ and $P_{div}\left(k\right)$ for 40 arbitrarily selected precipitation events of $30 \mathrm{m}\mathrm{m}/\mathrm{h}$ or more from June to September 2020. The peaks of $P_{curl}\left(k\right)$ and $P_{div}\left(k\right)$ are approximately $~O\left({10}^{-1}\right) {\mathrm{k}\mathrm{m}}^{-1}$ and $~O\left({10}^{-2}\right) {\mathrm{k}\mathrm{m}}^{-1}$, respectively, indicating that the nature of precipitation motions depends strongly on the spatial scales. Such dependence on the motion scales can be prominently characterized by the ratio $\varphi \left(k\right)\equiv P_{curl}\left(k\right)/P_{div}\left(k\right)$ shown in panel (c) of Figure 1. In the large scale corresponding to the synoptic scale, $k/2\pi <{10}^{-2} {\mathrm{k}\mathrm{m}}^{-1}$, the translation motion presented in the ${\mathit{V}}_{\mathit{d}\mathit{i}\mathit{v}}$-component is dominant as expected ($\varphi \left(k\right)<1$). In contrast, in the smaller scale, ${10}^{-2} {\mathrm{k}\mathrm{m}}^{-1}<k/2\pi$, the turbulent motion presented in the ${\mathit{V}}_{\mathit{c}\mathit{u}\mathit{r}\mathit{l}}$-component becomes dominant ($\varphi \left(k\right)>1$). We interpret that the spatial scale, $L=2\pi /k\approx {10}^2 \mathrm{k}\mathrm{m}$, is crucial for operational nowcasting. For instance, considering the averaged flow speed, $V_f \approx 5-10 \mathrm{m}/\mathrm{s}$, the dynamical timescale is roughly $~L/V_f\approx$ 3–6 h, which is comparable to the preferred timescale for nowcasting [20,21]. Hence, for nowcasting at 0–6 h lead times, the nonlinear motion of the precipitation fields significantly affected the accuracy of the nowcasting model.

The nonlinear evolution of the velocity field can be observed using the multi-temporal features of the velocity field evolution. Figure 2 displays the optical flow fields ${\mathit{V}}_{\mathit{f}}(t,t-∆t)$ obtained by three different time intervals, $∆t=10,20$, and $30$ min, for the two examples shown in Figure 1. As shown in the two examples, the characteristics of precipitation motions contain multi-temporal features. In the upper panels, for instance, the direction of movement of the heavy rainfall region depends on the time intervals. With the interval $∆t=10$ min, the heavy rainfall region with $>10 \mathrm{m}\mathrm{m}/\mathrm{h}$ is mainly moving toward the eastern side. With the interval $∆t=30$ min, on the other hand, the vector field associated to the heavy rainfall region is mainly aligned toward the southern area. Such nonlinear evolution of the vector field is also observed in the example displayed in the bottom panels. Hence, considering multi-temporal motions could enhance the accuracy of precipitation nowcasting by capturing nonlinear motions at multi-spatial scales.

2.3. Model Architecture

In this subsection, we propose a model architecture that includes an optical flow method and conditional GAN. As shown in Figure 3, the model consists of two parts: (1) linear extrapolation using multi-temporal optical flow fields, and (2) a conditional GAN (cGAN) to capture the nonlinear evolution of precipitation events.

To resolve the multi-temporal features of precipitation motions, we calculated the temporally-averaged flow field by summing over the optical flow field obtained at different timesteps, as shown in Figure 3a:

$$\widetilde{V_f}\left(t,t-∆t\right)\approx \frac{1}{N}\sum _{i=1}^N\frac{1}{i}{V}_f\left(t, t-i\ast ∆t\right).$$

(7)

Using a parameter $N=3$ and $∆t=$10 min, the field $\widetilde{V_f}$ is the mean field averaged over 30 min. The magnitude of the vector field with a time interval longer than 30 min was smaller than that obtained with a time interval shorter than 30 min. Therefore, we considered the optical flow fields up to $N=3$. The future frame extrapolated by $\widetilde{V_f}\left(t,t-∆t\right)$ can be estimated as follows:

$$\widetilde{I_f}=g\left(I_t, \widetilde{V_f}\left(t,t-∆t\right)\right),$$

(8)

where $g(.)$ represents a backward warping function.

We then employed the conditional GAN (cGAN) method to capture the temporal variation in rainfall intensity and the nonlinear motion of the precipitation fields. The cGAN consists of a generator G and discriminator D. While the generator of the original GAN produces the distribution of data from random noise, the image ($y^{\prime }=G\left(\widetilde{I_f}\right)$) produced through the generator of the cGAN is based on the input image ($x=\widetilde{I_f}$) [19]. Here, the future image obtained by linear advection through a multi-temporal optical flow, $\widetilde{I_f}$, is used as the input image. The discriminator D learns to classify the image from the generator ($y^{\prime }=G\left(\widetilde{I_f}\right)$) or ground truth ($y=I_{t+10}$).

We employed Pix2Pix [32] for the cGAN implementation, which contains a generator based on the U-Net [33] architecture to create $y^{\prime }$ and a discriminator based on patch-GAN to compare $y^{\prime }$ and $y$. Figure 4 depicts the structure of the generator (G) and discriminator (D). In the Pix2Pix model, G minimizes the loss function $L_{cGAN}(G,D)$ when G is trained to produce the image, $\mathrm{y}\approx y^{\prime }$ from input ($\widetilde{I_f}$). By contrast, D maximizes $L_{cGAN}(G,D)$. $L_{cGAN}(G,D)$ can be expressed as follows:

$$L_{cGAN}\left(G,D\right)=\left(\log D\left(y,y^{\prime }\right)\right)+E\left(1-\log D\left(y,y^{\prime }\right)\right).$$

(9)

To improve the quality of the output images, the L1 loss function $L_{L1}$ is additionally considered:

$$L_{L1}\left(G\right)=E\left({‖y-y^{\prime }‖}_1\right).$$

(10)

Combining the adversarial loss function $L_{cGAN}\left(G,D\right)$ and the L1 loss function $L_{L1}$, the total loss function with a free parameter $\lambda$ is described as follows:

$$L_{tot}=\mathrm{a}\mathrm{r}\mathrm{g} \underset{G}{\min }\underset{D}{\max }{L}_{cGAN}\left(G,D\right)+\lambda L_{L1}\left(G\right).$$

(11)

Here, we summarize the details of model training. Owing to the computational cost, we resized the image from $2048\times 2048$ pixels to $512\times 512$ pixels. The resized image contained an area $1024 \mathrm{k}\mathrm{m}\times 1024 \mathrm{k}\mathrm{m}$ with a 2-km spatial interval around the Korean Peninsula. By optimizing the model, the hyperparameters were tuned, which are summarized as follows: The binary cross entropy and Adam optimizers were employed as the loss function and optimizer, respectively. A learning rate of 0.0001 and momentum parameters ${\beta }_1=0.5$, ${\beta }_2=0.999$ were chosen. In addition, after conducting performance tests with various values of 𝜆 within the range of 10–200, we determined that the optimal free parameter 𝜆 for the L1 loss function is 100. Our model was trained using 100 epochs, with a batch size of eight, and the parameters of both the generator and discriminator were updated simultaneously during one epoch.

3. Results

3.1. Models for Performance Comparison

In this Section, we report on the forecasting skill evaluation using qualitative and quantitative methods. Specifically, we examined four types of models to evaluate the performance improvements resulting from the use of multi-temporal optical flow and cGAN. The models are summarized as follows:

●

Single-temporal model without cGAN: A model that includes a single-temporal optical flow field, ${\mathit{V}}_{\mathit{f}}(\mathit{t},\mathit{t}-∆\mathit{t})$, but excludes cGAN structure;

●

Multi-temporal model without cGAN: A model that includes a multi-temporal optical flow field, $\widetilde{{\mathit{V}}_{\mathit{f}}}\left(\mathit{t},\mathit{t}-∆\mathit{t}\right)$, but excludes cGAN;

●

Single-temporal model with cGAN: A model that includes a single-temporal optical flow field, ${\mathit{V}}_{\mathit{f}}(\mathit{t},\mathit{t}-∆\mathit{t})$, and cGAN structure;

●

Multi-temporal model with cGAN: A model that includes a multi-temporal optical flow field, $\widetilde{{\mathit{V}}_{\mathit{f}}}\left(\mathit{t},\mathit{t}-∆\mathit{t}\right)$, and cGAN structure.

As a reference model, we also employed Eulerian persistence (Persistence, hereafter), which assumes that for any lead time $N$ precipitation at $t+N$ is the same as at forecast time $t$. Because this simple model is powerful, particularly for short lead times, it has been widely employed as a baseline model to quantify and compare the performance of developed AI-models [2,13]. In addition, an indirect comparison between the model proposed in this study and the models proposed in previous studies based on the deep learning approach can be performed using Persistence.

3.2. Qualitative Evaluation

We compared the nowcasting outputs obtained by three different models. An example of precipitation nowcasting at 1–3 h lead time is displayed in Figure 5. In the case of the models without multi-temporal optical flow field, the precipitation motions mainly occurred from west to east, which is inconsistent with the ground truth results. By considering the multi-temporal optical flow field, the motion toward the southern area was captured, as shown in the middle panels, up to 3 h lead time. We point out that, although the inclusion of cGAN can capture the temporal changes in rainfall intensity, tracking the multi-temporal optical flow field is of greater importance in predicting the nonlinear evolution of the precipitation field.

To quantify the properties of the motion vectors of the sample shown in Figure 5, we measured their statistics. The frequency distribution of motion fields at the 1–3 h lead times are shown in Figure 6. Here, the frequency distribution of motion fields, $dN/dlog{V}_f$, represents the number of pixels within the logarithmic bin $dlog{V}_f$. The velocity distribution of the models with the multi-temporal optical flow field showed good agreement with that of the ground truth. The models without the multi-temporal optical flow field (blue and magenta lines), on the other hand, slightly overestimated the motion speeds of precipitation compared with the models with the multi-temporal optical flow field and the ground truth. Owing to such overestimations, forecasts using the model without the multi-temporal optical flow field could contain more missing events and false alarm events.

To test the operational feasibility of the model with the cGAN, we examined the blurry effects of the nowcasting model by calculating the radially averaged power spectra of the precipitation fields. Figure 7 shows the power spectra of the same sample at 1–3 h lead times. We noted that the models with cGAN show slightly blurry forecasts in the scale of $k/2\pi >~ 1/20 {\mathrm{k}\mathrm{m}}^{-1}$ due to the adversarial loss function, $L_{cGAN}(G,D)$. Despite the blurry effects of the model with cGAN, we interpret that the models with cGAN could be applicable for precipitation nowcasting, because such blurry effects are independent of the lead time whereas the blurry effects of the model based on the U-Net convolution neural network are more prominent as the lead time increases.

3.3. Quantitative Evaluation

We evaluated forecast skills by estimating pixel-wise scores such as the mean absolute error (MAE), critical success index (CSI), probability of detection (POD), and false alarm rate (FAR).

MAE is the score to capture errors in rainfall rate prediction:

$$\mathrm{M}\mathrm{A}\mathrm{E}=\sum _{j=1}^N\frac{\left|y_j-y_j^{\prime }\right|}{N},$$

(12)

where $y_j$ and $y_j^{\prime }$ denote the rainfall intensities obtained from the ground truth and prediction in the $j$th pixel of the corresponding radar image, respectively, and $N$ is the total number of pixels.

The CSI is the fraction of the forecast event that was correctly forecasted, and it is typically recognized as a score for model accuracy. POD denotes the fraction of correctly forecasted rainfall events to observed events, whereas FAR is the ratio of incorrectly forecasted rainfall events to all forecasted rainfall events. The scores are given as follows:

$$\mathrm{C}\mathrm{S}\mathrm{I}=\frac{\mathrm{h}\mathrm{i}\mathrm{t}\mathrm{s}}{\mathrm{h}\mathrm{i}\mathrm{t}\mathrm{s}+\mathrm{f}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e} \mathrm{a}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{m}\mathrm{s}+\mathrm{m}\mathrm{i}\mathrm{s}\mathrm{s}\mathrm{e}\mathrm{s}},$$

(13)

$$\mathrm{P}\mathrm{O}\mathrm{D}=\frac{\mathrm{h}\mathrm{i}\mathrm{t}\mathrm{s}}{\mathrm{h}\mathrm{i}\mathrm{t}\mathrm{s}+\mathrm{m}\mathrm{i}\mathrm{s}\mathrm{s}\mathrm{e}\mathrm{s}},$$

(14)

$$\mathrm{F}\mathrm{A}\mathrm{R}=\frac{\mathrm{f}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e} \mathrm{a}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{m}\mathrm{s}}{\mathrm{h}\mathrm{i}\mathrm{t}\mathrm{s}+\mathrm{f}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{e} \mathrm{a}\mathrm{l}\mathrm{a}\mathrm{r}\mathrm{m}\mathrm{s}}.$$

(15)

Perfect scores for CSI and POD are 1, and these values range from 0 to 1. FAR ranges from 0 to 1, with a perfect score of 0. Although the accuracy of nowcasting can be tested using the CSI value, it can be higher when nowcasting overestimates the area and intensity of the precipitation fields. We conducted separate evaluations of the CSI, POD, and FAR values at the thresholds of $1$ and $10 \mathrm{m}\mathrm{m}/\mathrm{h}$ [14].

The MAE scores at 0–3 h lead times are shown in Figure 8. While the evaluation results for very short lead times within 20 min could be affected by the optical flow model’s overestimation of motion fields for heavy precipitation in the Korean Peninsula, as shown in Figure 6, we confirmed that the impact of motion fields becomes more significant in terms of accuracy and the evolution of the precipitation field with longer lead times of 0.5 to 3 h. Hence, the models including the optical flow method generally outperformed the Persistence model. As illustrated by the qualitative analysis presented in Figure 5, it is evident that the inclusion of the multi-temporal optical flow field significantly contributes to the accuracy of predicting precipitation field locations. The MAE values obtained from models incorporating the multi-temporal optical flow fields are notably lower than those obtained from models lacking such fields. Furthermore, the multi-temporal model with cGAN exhibits lower MAE values compared to the multi-temporal model without cGAN, demonstrating that the model with cGAN provides more accurate predictions of rainfall intensities.

Figure 9 presents the CSI, POD, and FAR scores at the thresholds of $1$ and $10 \mathrm{m}\mathrm{m}/\mathrm{h}$. Based on the CSI and POD evaluations, the accuracy is influenced by the presence of both the multi-temporal optical flow field and cGAN. When compared to the single-temporal model without cGAN, the CSI values for the single-temporal model with cGAN and the multi-temporal model without cGAN show improvement within 0–3 h lead times. Furthermore, when both the multi-temporal optical flow and cGAN are considered together, the performance enhancement becomes more significant, as indicated by the red solid lines in Figure 9. For example, when compared to the single-temporal optical flow model, the multi-temporal model with cGAN enhances accuracy by approximately $\approx 13–16\%$ in predicting heavy rainfall events with $\ge 10 \mathrm{m}\mathrm{m}/\mathrm{h}$ at 1–3 h lead times. In contrast, the multi-temporal model without cGAN shows an improvement of approximately $\approx 4–6\%$. In this context, it would be possible to further enhance currently operational nowcasting models at KMA based on MAPLE by incorporating the cGAN. Note that the prediction accuracy in terms of the CSI can be improved when the predicted rainfall intensity is overestimated. However, based on the FAR scores, we interpreted the enhancement of CSI scores as independent of such effects. Because the FAR of the multi-temporal model with cGAN is generally lower than that of the other models, we interpret that the multi-temporal model with the cGAN does not overestimate the area or intensity of rainfall.

To assess the generalizability of our findings, we conducted a long-term forecast skill evaluation. Figure 10 displays CSI scores at thresholds of $1$ and $10 \mathrm{m}\mathrm{m}/\mathrm{h}$ for the models over the summer season (June to September) of 2020. Our analysis confirms that both the multi-temporal optical flow method and the refinement using cGAN contribute to improved prediction performance.

4. Summary and Discussion

The optical flow technique has been widely used in operational precipitation nowcasting models, and its performance is comparable to that of other state-of-the-art models. However, because the optical flow model cannot predict the nonlinear evolution, growth, and decay of precipitation fields, its prediction timescale has been limited to a roughly 1 h lead time. This study reported a possible method to improve current optical flow-based precipitation nowcasting by tracking the nonlinear evolution of precipitation fields. To follow the nonlinear motion more precisely, we first analyzed the characteristics of motions typically exhibited in precipitation events, and calculated the multi-temporal optical flow field. We then implemented a cGAN structure to capture dynamic processes such as the growth and decay of precipitation fields. Based on forecast skill evaluations employing both qualitative and quantitative methods, we have verified that the inclusion of the multi-temporal optical flow field significantly improves the accuracy of predicting the nonlinear motion of precipitation fields. Additionally, the refinement using cGAN further enhances the accuracy of nowcasting. Therefore, we interpret that the proposed model can be applied to enhance the currently available model for precipitation nowcasting based on the advection of precipitation fields such as MAPLE.

While this study mainly focuses on predicting heavy rainfall during the summer season, it is also essential to predict ice-phase precipitation, including snow, using the proposed model. Both microphysical processes within the cloud and the conditions related to the ambient dynamics and thermodynamics of the system affect the intensity and amount of snow within the cloud and at the surface, making accurate prediction of snow precipitation challenging (see [34] for more details). In the context of data-driven precipitation nowcasting models based on deep learning architectures, it is possible to consider a multimodal dataset, including radar, satellite observations, and surface observations, to improve the accuracy of predicting different types of precipitation. This can be achieved by training the model with information about cloud types and phases from satellite observations and incorporating thermodynamic effects from surface observations. We leave this as a topic for future work.