Operational Application of Optical Flow Techniques to Radar-Based Rainfall Nowcasting

Abstract

Hong Kong Observatory has been operating an in-house developed rainfall nowcasting system called “Short-range Warning of Intense Rainstorms in Localized Systems (SWIRLS)” to support rainstorm warning and rainfall nowcasting services. A crucial step in rainfall nowcasting is the tracking of radar echoes to generate motion fields for extrapolation of rainfall areas in the following few hours. SWIRLS adopted a correlation-based method in its first operational version in 1999, which was subsequently replaced by optical flow algorithm in 2010 and further enhanced in 2013. The latest optical flow algorithm employs a transformation function to enhance a selected range of reflectivity for feature tracking. It also adopts variational optical flow computation that takes advantage of the Horn–Schunck approach and the Lucas–Kanade method. This paper details the three radar echo tracking algorithms, examines their performances in several significant rainstorm cases and summaries verification results of multi-year performances. The limitations of the current approach are discussed. Developments underway along with future research areas are also presented.

1. Introduction

1.1. Background

Located at the southern coast of China (Figure 1), Hong Kong is affected by heavy rain and severe weather every year. In the early flood season from March to May, surface frontal system and monsoonal trough, in tandem with active mid-tropospheric westerly jets, induce squall lines that lead to heavy rain, severe convective gusts and hail to Hong Kong and the Pearl River Delta (PRD) region. In summertime, heavy rain is typically associated with southwest monsoon, tropical cyclones, as well as the wind convergence zone between a low pressure system near Beibu Wan and a subtropical ridge extending from the western Pacific. Tropical cyclone activities may extend well into September and October, while land-sea breeze convergence and frontal system may also bring occasional excessive rain to the area. A more detailed and quantitative account of the climatological rainfall can be found in [1,2].

Over seven million people reside in this metropolitan with a landmass of just over 1100 km², of which 40% has been designated as country parks or natural reserves. The urban areas and satellite towns where most people reside are very densely populated. With such a setting, even highly localized downpour could easily affect a significant portion of the population. The potential impact of rainstorm is complicated owing to the fact that over 70% of the landmass in Hong Kong is of hilly terrain. Numerous buildings that are built on or next to man-made or natural slopes run into higher risk brought by landslides. There is no dearth of events in which heavy rain and the associated weather have ignited a chain reaction of flooding, landslides, infrastructure failures, fallen trees, traffic congestions, accidents and the like that gravely affected livelihood of the general public. In 1972, in torrential rain a slope in Hong Kong failed, claiming 67 lives, injuring 20 people and destroying three buildings nearby [3].

1.2. Rainfall Nowcast

To mitigate the potential impacts of rainstorms, the Hong Kong Observatory (HKO) introduced a color-coded rainstorm warning system in 1992, and subsequently revised it to the present three-tier form (amber, red and black signals) in 1998 [4]. The amber signal gives alert on potential heavy rain (30 mm in an hour generally over HK) that may develop into a red signal (50 mm in an hour) or a black signal (70 mm in an hour) situation. Key Government departments and major utility companies are put on alert. The red and black signals warn the public of heavy rain which is likely to bring about serious road flooding and traffic congestion. They will trigger response actions by Government departments and major transport operators and utility companies. Details of this rainstorm warning system can be found in [5].

To support the operation of the rainstorm warning system, a radar-based nowcasting system named “Short-range Warnings of Intense Rainstorms in Localized Systems” (SWIRLS) was developed in 1999 and has since been in operational use. A brief history of and an introduction to SWIRLS are given in [6].

Other meteorological services and research institutes around the world have also developed similar observationally based nowcasting systems, some of which with outputs from numerical weather prediction (NWP) models blended into nowcast. Notable operational systems in active service include the followings:

a)

Integrated Nowcasting through Comprehensive Analysis (INCA) by Zentralanstalt für Meteorologie und Geodynamik (ZAMG);

b)

Nimrod by Met Office, United Kingdom (UKMO);

c)

Spectral Prognosis (S-PROG) by (Australian) Bureau of Meteorology (BoM);

d)

Short-Term Ensemble Prediction System (STEPS) by UKMO/BoM;

e)

Auto Nowcasting System (ANC) by National Center for Atmospheric Research (NCAR);

f)

McGill Algorithm for Precipitation Nowcasting Using Semi-Lagrangian Extrapolation (MAPLE) by (Canadian) McGill University etc.

A comprehensive review of nowcasting systems was conducted by Royal Meteorological Institute of Belgium back in 2008 [7], which includes verification results for those available.

As meteorological services normally run only one nowcasting system in operation, cross system comparison is relatively rare. Under the auspices of World Meteorological Organization (WMO) World Weather Research Programme (WWRP), Forecast Demonstration Projects (FDP) were held as parts of the Olympic games in Sydney 2000 (S2000) and in Beijing 2008 (B08FDP), which offered unique opportunities to demonstrate the benefits and quantify the benefits of a real-time nowcast service. Keenan et al. [8,9] documents the experience gained in S2000, with verification results given in [10]. Wang et al. [11] reports on B08FDP with verification included.

SWIRLS is one of the first operational nowcasting systems with urban-scale applications. It continues to be updated with innovative approaches, such as optical flow, from time to time. There are other nowcasting systems that have experimented with or adopted optical flow: Peura and Hohti [12] demonstrates the potential use of optical flow on nowcasting, while Bowler et al. [13] documents a nowcasting algorithm based primarily on Horn and Schunck algorithm. SWIRLS currently adopts “ROVER”, an algorithm to be elaborated in this paper that combines the global Horn and Schunck approach and the Lucas–Kanade method. To the best of the knowledge of the authors, it is the first if not the only nowcasting system doing so. Due to sheer amount of data collected over the years, SWIRLS is also amongst the few nowcasting systems that have undergone long-term verification with publishable results.

This paper elaborates on the correlation-based algorithm and two optical-flow-based algorithms adopted for radar echo tracking in Section 2, and examines their performances through case studies as well as seasonal and multi-year systematic verifications in Section 3. Section 4 discusses the limitations of these tracking algorithms and highlights certain development work being made, while Section 5 concludes the paper.

2. Operational Experiment Setup

2.1. Nowcasting System

The main function of SWIRLS is to provide quantitative precipitation estimation (QPE) and quantitative precipitation forecast (QPF) to support rainstorm warning services. QPE is computed based on radar and rain gauge data analyzed through a variety of techniques including Barnes analysis, co-kriging analysis and a blending of both. The application of co-kriging analysis with performance assessment is given at [14]. QPF is computed using an algorithm based on backward semi-Lagrangian advection scheme. For both QPE and QPF, SWIRLS employs the Z = aR^b (Z − R) relationship with the parameters a and b dynamically calibrated by comparing radar reflectivity with rain gauge measurements in near real-time. The detailed implementation is elaborated in [15]. Quality control on QPE is implemented as described in [16].

SWIRLS uses data primarily from the weather radar in Tai Mo Shan (TMS), which is sited on a hilltop at 968 m above mean sea level with unobstructed view in almost all azimuths. The radar is an S-band radar operating at a frequency of 2.82 GHz, pulse widths of 1.0 and 2.0 µs, and antenna beam width of 0.9°. Another radar, at about 583 m above mean sea level on top of another hilltip named Tate’s Cairn, serves as a backup. The Tate’s Cairn radar has a transmitter frequency of 2.92 GHz, pulse widths of 1.0 and 2.0 µs, and antenna beam width of slightly less than 1.0°. Basic radar data processing, including removal of ground clutters, are handled by the IRIS software [17]. Both radars complete a volume scan every 6 min. Radar data and imagery are generated based on a composite of the two radars with Tai Mo Shan radar being the primary source. SWIRLS is configured to process radar data immediately upon reception of radar data, and thus also runs once every 6 min.

The typical operational products of SWIRLS are based on 2 km CAPPI (Constant Altitude Plan Projection Indicator) radar data covering a domain of 256 km in radius, embedded in a rectangular grid of 480 × 480 pixels. Each pixel represents an area of approximately 1.1 km × 1.1 km. The output QPF products maintain a grid of 480 × 480 pixels but the spacing between grids is halved, so the output domain represents a rectangular area of 256 km × 256 km. The configuration is illustrated in Figure 2.

SWIRLS’s products are generated to aid weather forecasters in the preparation of weather forecasts and the operation of the rainstorm warning system. These products include hourly forecast rainfall map up to 6 h, a time series showing an ensemble of estimated rainfall in the next 6 h, a plot of estimated number of rain gauges reaching user-selected thresholds, etc. (Figure 3).

QPF from SWIRLS is also used to support the operation of the landslip warning in Hong Kong [18]. In addition, SWIRLS is also capable of generating forecasts for thunderstorms [19], squalls [20] and hail. Areas with potential threats are identified through a combination of statistical means and physical modeling. These weather phenomena are extrapolated along the motion field for up to half an hour to generate forecasts.

In recent years, SWIRLS was extended to provide enhanced nowcasting services to the public. For instance, an Internet web page named “Two-hour Rainfall Nowcast for Hong Kong and Pearl River Delta” [21] that provides animation of the evolution of radar echoes for the next two hours has been in operational use since 2007. In late 2012, a location-specific rainfall nowcast service delivered through mobile platform was also launched, as elaborated in [22,23,24].

Since 2008, SWIRLS had been deployed to several international events, including the Beijing Olympic Games in 2008 [25], the Shanghai Expo 2010 [26], the India Commonwealth Games in 2010 [27], and the Shenzhen Universiade 2013. As part of regional and international co-operations, SWIRLS has been provided to meteorological services in Mainland China, India, Macao, the Philippines, Vietnam and Thailand for research and development in radar nowcasting techniques. In 2015, a community version of SWIRLS was developed and made available to interested National Meteorological and Hydrological Services. Malaysian Meteorological Department and South Africa Weather Service have successfully adopted SWIRLS for nationwide operational use and for aviation nowcast research, respectively. Zhuhai Meteorological Bureau is also using SWIRLS operationally.

2.2. Radar Echo Tracking Algorithms

2.2.1. Tracking of Radar Echoes by Correlation (TREC)

In the first operational version of SWIRLS, the motion field of radar echoes was retrieved by a correlation-based technique named as “Tracking Radar Echoes by Correlation (TREC)” that is described in [15,28]. This method essentially compares two successive radar reflectivity images and from which identifies a motion vector for each divided block of pixels through maximizing the correlation coefficient R, defined as follows:

$$R=\frac{{\displaystyle \sum _k{Z}_1(k)\times Z_2(k)}-\frac{1}{N}{\displaystyle \sum _k{Z}_1(k)}{\displaystyle \sum _k{Z}_2(k)}}{\sqrt{{\displaystyle \sum _k{Z}_1^2(k)}-N{\overline{Z}}_1^2}\times \sqrt{{\displaystyle \sum _k{Z}_2^2(k)}-N{\overline{Z}}_2^2}}$$

(1)

where Z1 and Z2 are the array of pixels of reflectivity in the first field and in the second field respectively, and N denotes the number of data points within an array.

In the implementation of SWIRLS, 93 × 93 overlapping blocks, each of size 19 × 19 pixels (i.e., N = 19 × 19) with the centers of two successive blocks separated by five pixels, are extracted from the 480 × 480 pixel reflectivity field, with each pixel representing an area of approximately 1.1 km × 1.1 km. For each block at time T, the correlation coefficients between itself and every other block within a searching radius of 19 pixels recorded 6 min earlier (i.e., T − 6 min) are computed. The block at time (T − 6 min) that yields the highest correlation coefficient with the block at time T is chosen, and the distance between the centers of these two blocks is then used to calculate the motion vector that represent the motion of the block at (T − 6 min). Repeating the procedure for each block, a motion field is obtained for all blocks with echoes. Cressman analysis [29] is subsequently performed to assign motion vectors to those blocks without echoes.

While TREC is successful in tracking motions of individual radar echoes, in practice it generally captures the direction of individual rain cell instead of larger scale motion of weather system, e.g., a squall line containing the rain cell in it. As a result, it occasionally produces a faster movement of convective system such as squall lines in southern China in spring, leading to weaker intensity in QPF. In fact, in typical implementation of TREC, the block size and resolution are pre-determined, making it impractical to track motion of variable scales. To address the issues, there emerged several alternative approaches, some are also correlation-based such as “COTREC” in [30] and “MTREC” in [31] while others adopt variational optical flow technique, to be described in the following paragraphs.

2.2.2. Multi-Scale Optical-Flow by Variational Analysis (MOVA)

To overcome the limitation of TREC, another radar echo tracking method utilizing variational optical flow technique, named as “Multi-scale Optical-flow by Variational Analysis (MOVA)”, was developed in 2009 [32], then replacing TREC for operational use in SWIRLS in 2010.

Defined as the apparent motion of brightness patterns in an image, optical flow would ideally equalize motion field barring lighting changes. Assuming that the brightness of a point of an object would remain unchanged along its path (i.e., “brightness constancy assumption”), the optical flow can be derived by solving the optical flow constraint (OFC) below:

$$\frac{\partial I}{\partial t}+u\frac{\partial I}{\partial x}+v\frac{\partial I}{\partial y}=0$$

(2)

where I represents the brightness of a point projected on a two-dimensional plane.

It can be seen that with two unknowns but just one equation, at least one more constraint is required to arrive at a unique solution. There are several approaches to tackle the issue. One of them is proposed by [33], which applies variational method to solving OFC by minimizing the following cost function:

$$J=J_{\mathrm{o}}+\alpha \cdot J_{\mathrm{HS}}$$

(3)

where

$$J_{\mathrm{o}}={\displaystyle \iint {\left[\frac{\partial I}{\partial t}+u\frac{\partial I}{\partial x}+v\frac{\partial I}{\partial y}\right]}^{2}dxdy}$$

(4)

and J_HS is a global constraint on the smoothness of the gradient of the optical flow field:

$$J_{\mathrm{HS}}={\displaystyle \iint \left({\left|\nabla u\right|}^2+{\left|\nabla v\right|}^2\right)dxdy}$$

(5)

By assuming Lagrangian persistence for echoes over their paths, Germann and Zawadzki [34] adopts a proposal by Wahba and Wendelberger [35] and developed an algorithm named as the variational echo tracking (VET) [36], using a smoothness penalty function:

$$\begin{array}{cc}{J}_{\mathrm{WW}}& ={\displaystyle \iint \left[{\left(\frac{{\partial }^{2}u}{\partial x^2}\right)}^2+{\left(\frac{{\partial }^{2}u}{\partial y^2}\right)}^2+2{\left(\frac{{\partial }^{2}u}{\partial x\partial y}\right)}^2\right]dxdy}\hfill \\ & +{\displaystyle \iint \left[{\left(\frac{{\partial }^{2}v}{\partial x^2}\right)}^2+{\left(\frac{{\partial }^{2}v}{\partial y^2}\right)}^2+2{\left(\frac{{\partial }^{2}v}{\partial x\partial y}\right)}^2\right]dxdy}\hfill \end{array}$$

(6)

In MOVA, seven cascading levels of various box sizes that range from 256 km down to 1.6 km are used together with the smoothness constraint. The smoothness constants for all level is controlled by seven parameters tuned for optimal performance, as given in

Table 1. Parameters adopted in MOVA

No. of Boxes	1	5	10	20	40	80	160
Box Size (km)	256	51.2	25.6	12.8	6.4	3.2	1.6
Smoothing constant γ	0.0005	0.01	0.1	0.1	1	50	100

:

When calculating the cost function, three consecutive radar reflectivity images are used to improve continuity of the retrieved motion field, i.e.,

$$J_{\mathrm{o}}=J_{\mathrm{o}}(T_{-12\min }\mathrm{to}{T}_{-6\min })+J_{\mathrm{o}}(T_{-6\min }\mathrm{to}{T}_0)$$

(7)

2.2.3. Real-Time Optical Flow by Variational Methods for Echoes of Radar (ROVER)

MOVA was put into operational use in 2010. As it will be shown, MOVA achieved improved overall performance over TREC, though it has also been observed in operation that MOVA has a tendency to underestimate the speed of echo motion vectors, in particular at large scale, due to sub-optimal retrieval of echo motion in the minimization procedure. As a result, excessive rainfall was forecast at times. With a view to addressing the issue, another variational optical flow scheme, named as “Real-time Optical flow by Variational methods for Echoes of Radar (ROVER)”, was used instead. In brief, ROVER is similar to MOVA but with two major enhancements: (1) having a pre-processing step to spatially smooth the radar reflectivity images; and (2) adoption of a variant type of optical flow technique.

In convective systems, which dominate precipitation types in summer monsoon in Hong Kong, rain echoes could be so jumpy in reflectivity that their partial derivatives can hardly be accurately calculated. Stable estimates of the derivatives are only possible after the radar rain-rate fields are highlighted. In ROVER, the reflectivity fields are transformed with the following function:

$$G(Z)={\tan }^{-1}\left(\frac{Z-Z_c}{\xi }\right)$$

(8)

where Z_c and ξ control the point of inflection and its sharpness.

In SWIRLS, Z_c and ξ are empirically determined to be 33 dBZ and 4 dBZ, respectively [37] based on a number of rainstorm cases in spring and summer. The transformation effectively enhances the contrast for reflectivity values in the range of 20–40 dBZ through suppressing the most and the least intense parts of rain bands.

The other major enhancement is the adoption of the algorithm proposed in [38,39] in lieu of VET. This algorithm combines the global Horn and Schunck approach and the Lucas–Kanade method [40], thus taking the advantages of both the dense flow fields of the former and the noise robustness of the latter. Through adopting advanced numerical methods, the algorithm further speeds up the computation such that it can be employed for real-time operational use. An open source project named “VarFlow” provides the source codes used to implement the algorithm [41].

The algorithm requires users to specify six parameters to control the Gaussian convolutions that spatially smooth radar images and motion fields to ensure tracking of the most critical features and to reduce jumpiness, to regularize smoothness constraints, to define the finest and coarsest levels, and to set the time intervals. The meanings of the parameters and the empirically determined parameters in ROVER are listed in

Table 2. Optical flow parameters adopted in ROVER.

Parameter	Significance	Value in ROVER
σ	Gaussian convolution for image smoothing	9
ρ	Gaussian convolution for local vector field smoothing	1.5
α	Regularization parameters in the energy function	2000
Lf	The finest spatial scale	1 pixel
Lc	The coarsest spatial scale	7 pixels
Tr	The time interval for tracking radar echoes	6 min

.

As the choice of parameters would affect the quality of the motion field, the performance of the variational optical flow algorithm naturally depends on the parameters adopted. While it might be possible to derive a set of optimal parameters for a particular geographical setting, such a work would require a rich data archive and tremendous computational power to experiment with and explore all the dimensional space. It renders optimisation of parameters difficult if not impractical in practice. In our four years of operational experience, this set of parameters worked reasonably well in most of the time. Readers interested in implementing ROVER may consider adopting this set of parameters for initial trial, and fine tune the parameters as and when situation warrants.

3. Results

3.1. Case Analyses

The performance of the three radar echo tracking algorithms are compared using four rainstorms in Hong Kong during 2013 and 2014. The latter two involved torrential rain with rainfall intensity exceeding 70 mm·h⁻¹ generally over Hong Kong.

In the following discussion, the motion fields computed with the three radar echo motion tracking algorithms together with forecast reflectivity images are shown in order. The actual radar reflectivity images are provided at the bottom row for comparison.

3.1.1. Rainstorm on 5 April 2013

The first case is illustrated in Figure 4. In the morning of 5 April 2013, a surface trough lingered over the southern coast of China. Coupled with a moist and active westerly jet in the lower troposphere, a squall line located to the west of Hong Kong steadily approached the territory at around 07:00 HKT (Hong Kong Time; HKT = UTC + 8 h). While the individual echo tracked largely northeastward along the squall line, the squall line itself traversed primarily eastward towards Hong Kong. The correlation-based algorithm TREC captured the motion of individual echo well but not that of the squall line, the latter is what is required to project accurately the future rainfall condition. As the motion vectors computed by TREC pointed to the northeast, after two hours of extrapolation the squall line was predicted to move to Shenzhen (Figure 4) to the north of Hong Kong. There was hardly any echo predicted within the territory of Hong Kong, and the corresponding QPF was rather inaccurate as a result.

Suggested from its name, MOVA is capable of capturing larger scale motion. The direction of the motion vectors was correct in terms of predicting eastward movement, but the speed was a bit slow. Consequently, it predicted that the squall line would be way to the west of Hong Kong.

In comparison, ROVER managed to capture both the direction and speed of motion of the squall line relatively well, and was successful in predicting that the squall line would have reached the territory in two hours, though the squall line still lagged behind the actual slightly. It demonstrated a significant improvement of ROVER over both TREC and MOVA.

3.1.2. Rainstorm on 26 July 2013

On 26 July 2013, active southerlies in a broad trough of low pressure affected the south China coast. At 07:30 HKT, a line of nearly north–south oriented intense echoes moved across Hong Kong from the south.

As shown in Figure 5, unlike the previous case the motion of individual echoes and that of the convective cell were largely aligned towards the predominately northward direction. Both TREC and ROVER predicted that the line of intense echoes would move to the north of Hong Kong, which was largely correct in direction but slightly slower in terms of speed. MOVA exhibited even slower speed of movement and as a result incorrectly forecast that the line of echoes would still affect Hong Kong.

As it turned out, two other patches (marked A and B) of intense echoes came from the south, one of which traceable to a relatively small patch of echoes about 120 km away (marked A), bringing rain to the Hong Kong Islands and the eastern part of the New Territories by 09:30 HKT. This underlines a major limitation of nowcasting system that works by extrapolating radar echoes, in dealing with growth and decay of storms. No radar echo tracking algorithm could have produced a perfect forecast unless the storm cell remains unchanged in shape, size and intensity, which is usually not the case.

3.1.3. Rainstorm on 22 May 2013

Starting from the small hours of 22 May 2013, a band of heavy rain spread from west to east across the Pearl River Estuary. Weather in Hong Kong deteriorated with torrential rain and intense thunderstorms. More than 150 mm of rain were recorded in many places over the territory. A few districts even recorded over 200 mm. The heavy downpour caused a number of landslides and flooding in Hong Kong.

In this heavy rainfall case, we take a look at one hour forecast. As shown in Figure 6, the motion field and reflectivity field from TREC and ROVER are quite similar, and both correctly predicted heavy rain over the whole territory. MOVA again exhibited a slow bias and as a result forecast intense rainfall covering the western part of the territory only.

3.1.4. Rainstorm on 30 March 2014

An episode of heavy rain came on 30 March 2014. Over the course of 3 to 4 h that evening, more than 100 mm of rainfall were recorded generally in the territory and in certain districts precipitation even exceeded 150 mm. The rainstorm was accompanied by intense thunderstorms, widespread hail, severe squalls and floods, resulting in injuries and inflicting critical damages to a crucial cross-border railway link under construction.

Figure 7 shows a highly organized northeast-southwest oriented line of echoes edging towards the territory from the west at 1900 HKT. In this case, TREC tracked the smaller scale motion of echoes and gave a predominately northeastward motion field. The intense echoes were predicted to move to Shenzhen to the north of Hong Kong, hence grossly underestimating the precipitation in Hong Kong. MOVA generated a motion field that was relatively more accurate in direction but was again slow in speed, resulting in a rain band lying over the western part of the territory and still failed to capture the heaviest rainfall areas. ROVER was, amongst the three, the most accurate algorithm in this case.

3.2. Comparison of Tracking Algorithms

The performance of ROVER for springtime squall lines in Hong Kong is described in [42]. A system to compare the performances of the three algorithms has been developed for continuous verification. In this system, the QPF at each grid point is verified against radar QPE.

The ROVER algorithm started operational trial in mid-May 2012. In this verification, data over a period of two full years, i.e., from 1 June 2012 to 31 May 2014 inclusive, are considered. A total of 2.1 billion data pairs are available for each radar echo tracking algorithm. Each forecast is paired with the corresponding actual value and the outcome is determined with reference to

Table 3. Contingency matrix for systematic verification.

Forecast	Observation
	Yes	No
Yes	Hit	False alarm
No	Miss	Correct negative

.

The performance is then benchmarked in terms of critical success index (CSI), also known as Threat Score (TS), Probability of Detection (POD) and False Alarm Ratio (FAR). CSI(TS), POD and FAR are defined as follows:

$$\mathrm{CSI}(\mathrm{TS})=\frac{\mathrm{hit}}{\mathrm{hit}+\mathrm{miss}+\mathrm{false}\mathrm{alarm}}$$

(9)

$$\mathrm{POD}=\frac{\mathrm{hit}}{\mathrm{hit}+\mathrm{miss}}$$

(10)

$$\mathrm{FAR}=\frac{\mathrm{false}\mathrm{alarm}}{\mathrm{hit}+\mathrm{false}\mathrm{alarm}}$$

(11)

The intensity thresholds of 0.5 mm·h⁻¹, 5 mm·h⁻¹ and 30 mm·h⁻¹, generally representing light, medium and heavy rainfall, respectively, within the forecast range of 1 to 6 h are examined in the following discussion. Forecast rainfall is accumulated hourly in the following results. High intensity rainfall is usually associated with tropical cyclones or convective development in highly unstable atmosphere, which is typically on smaller scale and thus less predictable than low intensity rainfall.

The performances of the various algorithms at 0.5 mm·h⁻¹ threshold, a threshold that is adopted to distinguish between rain and no rain situations, are compared in Figure 8a. ROVER showed minor improvement over TREC for the first two hours of forecast. The improvement becomes more prominent beyond the third hour. For forecast hours of 4–6 h, the performance of ROVER and MOVA is comparable, both outperforming TREC by a significant margin.

At higher rainfall threshold, say, 5 mm·h⁻¹, the performance advantage of ROVER is more obvious even for the first couple of forecast hours, as shown from Figure 8b. Both ROVER and MOVA performed considerably better than TREC for the forecast range of 4–6 h, for which MOVA performs slightly better than ROVER.

Figure 8c further compares the performance of the algorithms in predicting rainstorm as benchmarked against 30 mm·h⁻¹ rainfall threshold, chosen for it being the threshold for the Amber rainstorm warning in Hong Kong. ROVER performs better than MOVA, which in turns outperforms TREC for the first three hour of forecast. Beyond the fourth hour, MOVA performed better than ROVER and both MOVA and ROVER outperformed TREC, though their respective CSIs are rather low, suggesting limiting skill of extrapolation-based QPF in predicting heavy rain at these forecast hours.

A seasonal comparison is also conducted. Figure 9a shows the seasonal results for sixth hour nowcast (i.e., from T + 5 h to T + 6 h) with 0.5 mm·h⁻¹ threshold. In this case, MOVA outperforms ROVER and both perform substantially better than TREC for all seasons.

Figure 9b reveals the performance at 5 mm·h⁻¹ threshold. Here, the second hour forecasts (i.e., from T + 1 h to T + 2 h) are compared due to lower skill at higher threshold that renders the sixth hour forecast at 5 mm·h⁻¹ not having much utility in operation. As shown, the performance advantage of ROVER over the other two algorithms is the most significant in springtime, a season in which precipitation is dominated by persistent squall lines induced by mid-tropospheric westerly waves. In summer, showers are often brought by monsoon troughs, active southwest monsoon and occasionally tropical cyclones. The first two types of precipitation are intensely convective in nature, often undergoing rapid growth and decay and even back-building occasionally, which goes beyond the limit of radar-based nowcasting system that works by extrapolating radar echoes. In autumn, precipitation from tropical cyclones, northeast monsoon and land sea breeze fronts are more predictable and the skills recovers, though still lower than that in spring. In winter, the occurrence of 5 mm·h⁻¹ is relatively rare and so is the number of hits, while at the same time there are still occasional false alarms due to radar noise caused by land clutter, sea clutter, anomalous propagation, etc., which explains the lower CSI compared with other seasons.

Figure 9c further compares the schemes for first hour nowcast with 30 mm·h⁻¹ threshold, which is critical for operating the rainstorm warning in Hong Kong. The result shows that ROVER is more skillful than MOVA, which in turn outperforms TREC. All schemes achieve the highest CSI in spring with the advantage of ROVER over the others the most obvious, as explained earlier.

3.3. Operational Verification of ROVER

In view of the superior performance of ROVER, SWIRLS adopted it as the operational echo tracking algorithm from June 2013. The performance of SWIRLS based on ROVER continues to be verified operationally, as given below.

The same methodology as described in Section 3.2 is adopted for verification. For the year 2016, data are up to 30 September. Performance in terms of CSI is shown in Figure 10.

The verification shows that, while performance vary across years, possibly as a result of different rainfall types and climatological anomalies, the general performances remains largely unchanged. The CSI for the first hour forecast ranges from 49% to 57% at 0.5 mm·h⁻¹ threshold, and remain around 40% at 5 mm·h⁻¹ threshold. This illustrates the robustness of ROVER in the first hour of forecast.

Forecast skills are also measured in terms of POD and FAR. The POD and FAR of ROVER are examined based on four full years of data from 1 October 2012 to 30 September 2016, both dates inclusive and are given in Figure 11.

Generally speaking, a forecast is considered operationally useful when POD exceeds FAR. This can be said for up to 2 h at 0.5 mm·h⁻¹ threshold, and slightly longer than 1 h for 5 mm·h⁻¹ threshold.

Besides, the Heidke Skill Score (HSS) is also used to gauge the measure of skill in forecasts. A positive HSS indicates that a forecast is better than a random based forecast. The formula for calculating HSS is given in [43], copied below:

$$\mathrm{HSS}=\frac{2(\mathrm{hit}\cdot \mathrm{correct}\mathrm{negative}-\mathrm{miss}\cdot \mathrm{false}\mathrm{alarm})}{{\mathrm{miss}}^2+\mathrm{false}{.\mathrm{alarm}}^2+2\cdot \mathrm{hit}\cdot \mathrm{correct}\mathrm{negative}+(\mathrm{miss}+\mathrm{false}\mathrm{alarm})(\mathrm{hit}+\mathrm{correct}\mathrm{negative})}$$

(12)

As shown in Figure 12, verification with multi-year data shows that ROVER provides skillful forecasts for thresholds at 0.5 mm·h⁻¹ and 5 mm·h⁻¹. In both cases, the skill decreases with forecast hours.

4. Discussion

4.1. Limitations

In the current implementation of SWIRLS, the motion field is generated from the two latest radar images and is taken to remain the same throughout the six-hour forecast period. In practice, the motion field could vary according to changes in the synoptic situation, such as passage of westerly wave or passage of tropical cyclones. Hence, errors occur due to the assumed invariability of the motion field. Woo et al. [44] describes a method to separate the motion of a tropical cyclone (TC) in the computation of motion field, and found it capable of better preserving TC rain bands and achieving improvement in skills. The same concept may also apply to other weather systems, such as mid-tropospheric troughs, that traverse Hong Kong at known and constant speed. A suitable blending of numerical weather prediction models may also help mitigate the problem. The STEPS nowcasting system by UKMO demonstrates the benefits of blending radar-based forecasts with NWP forecasts [45].

The parameters adopted in ROVER are currently empirically determined. It may be possible to further improve its performance through optimizing these parameters based on climatological data, ideally specific to the synoptic pattern of the precipitation, or alternatively dynamically determined based on the latest parameter fit. An experimental project to generate probabilistic rainfall nowcast through perturbing optical flow parameters is being pursued.

Despite improvements in the radar echo tracking algorithm, the SWIRLS nowcasting system still bears the common limitation of radar-based nowcasting system in failing to predict growth and decay of radar echoes. Nor does SWIRLS take into account the effect orographic enhancements of precipitation. Mesoscale analyses, application of statistical methods and advanced techniques in blending with NWP models may prove useful in this development area, as assessed in [46,47]. The detection of convective initiation, a precursor to the intensification of precipitation, using the next generation satellite Himawari-8 data, is covered in [48]. The potential to predict convective initiation and growth based on real-time reanalysis data is demonstrated in [49].

4.2. Developments in Progress

Due to uncertainties inherent in observation, radar-based advection and the limitations discussed above, the latest development of SWIRLS is put on moving from deterministic to probabilistic nowcast. A SWIRLS Ensemble Rainfall Nowcast (SERN) based on 36 members with slightly perturbed optical flow parameters was setup for experimental trial. Preliminary results given in [50,51] show that such probabilistic rainfall nowcast products are generally reliable but tend to be over-confident for high intensity thresholds.

Rainfall nowcast really boils down to the prediction of the evolution of radar images given a sequence of recent past images. A novel approach to tackle the problem with convolutional Long-Short-Term-Memory (LSTM) network, a machine learning method, was developed and verified to provide better performance at 0.5 mm·h⁻¹ threshold [52]. Experiments are being conducted to assess its performance at higher intensity thresholds and its suitability for operational use.

Forecasts are typically verified against gridded QPE or rain gauge records. It is equally, if not more, important that the final products and guidance provided to forecasters are rigorously verified as well. This is, however, a challenging task given the much smaller sample and the fact that rainstorm warnings are subject to random errors. A progressive verification scheme to verify rainstorm warnings had since been developed. It was employed to verify automatic warning guidance given by SWIRLS. Based on five years of back-tested data, about 70% of amber rainstorm guidance warnings automatically generated by SWIRLS were considered effective [53].

5. Conclusions

ROVER, a variational optical flow technique to track radar echo motion, has been employed in the SWIRLS nowcasting system of HKO. Case analysis and systematic verification with two-year data demonstrate its advantages over the correlation-based algorithm TREC and the previous optical flow algorithm MOVA in general. Its limitations are discussed and potential future development opportunities are presented.