Evaluation of ICESat-2 Significant Wave Height Data with Buoy Observations in the Great Lakes and Application in Examination of Wave Model Predictions

Abstract

High waves and surges associated with storms pose threats to the coastal communities around the Great Lakes. Numerical wave models, such as WAVEWATCHIII, are commonly used to predict the wave height and direction for the Great Lakes. These predictions help determine risks and threats associated with storm events. To verify the reliability and accuracy of the wave model outputs, it is essential to compare them with observed wave conditions (e.g., significant wave height), many of which come from buoys. However, in the Great Lakes, most of the buoys are retrieved before those lakes are frozen; therefore, winter wave measurements remain a gap in the Great Lakes’ data. To fill the data gap, we utilize data from the Inland Water Surface Height product of the Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) as complements. In this study, the data quality of ICESat-2 is evaluated by comparing with wave conditions from buoy observations in the Great Lakes. Then, we evaluate the model quality of NOAA’s Great Lakes Waves-Unstructured Forecast System version 2.0 (GLWUv2) by comparing its retrospective forecast simulations for significant wave height with the significant wave height data from ICESat-2, as well as data from a drifting Spotter buoy that was experimentally deployed in the Great Lakes. The study indicates that the wave measurements obtained from ICESat-2 align closely with the in situ buoy observations, displaying a root-mean-square error (RMSE) of 0.191 m, a scatter index (SI) of 0.46, and a correlation coefficient of 0.890. Further evaluation suggests that the GLWUv2 tends to overestimate the wave conditions in high wave events during winter. The statistics show that the RMSE in 0–0.8 m waves is 0.257 m, while the RMSE in waves higher than 1.5 m is 0.899 m.

1. Introduction

Coastal communities face potential dangers due to the formidable waves and powerful surges that accompany storms. These natural phenomena present significant risks, endangering the safety and well-being of those residing in coastal areas of the North American Great Lakes (hereafter referred to as the Great Lakes). For example, with winds hitting 20 m/s in Muskegon, Michigan, Lake Michigan’s eastern shore, waves surpassed 3 m in height on 5 December 2017 [1]. This severe storm followed another record of extreme weather in Lake Superior in October, when a buoy in Lake Superior north of Marquette, Michigan, recorded a highest wave of 8.8 m during a high wave event on 24 and 25 October 2017, according to the National Data Buoy Center (NDBC) database. Two people were swept into the lake at Black Rocks, a lookout point in Presque Isle Park north of Marquette [2]. Given these circumstances, the quality of numerical wave models such as WAVEWATCHIII^® (hereafter WW3) [3] holds great significance, as it plays a pivotal role as the foundation of NOAA’s operational Great Lakes Waves-Unstructured Forecast System version 2.0 (GLWUv2) [4,5] in providing wave forecast guidance and warnings for the Great Lakes region. Accurate forecasts enable coastal communities to take precautionary measures to safeguard their properties against potential damages.

In the Great Lakes region, most of the in situ observations of significant wave height (SWH) used to evaluate GLWUv2 are provided by NDBC buoys. However, the buoys are regularly retrieved from the lakes during winter to avoid damage from the ice cover. Therefore, the lack of buoy data in winter leaves the accuracy of the wave model unverified for winter periods. To fill this gap in wave model evaluation, wave data detected by satellites are utilized as an alternative data source.

The ATL13 Along Track Inland Surface Water Data provided by the Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2 ATL13) contains SWH measurements over inland water bodies, including the Great Lakes region [6]. The data offer a potential source for evaluating the accuracy and effectiveness of GLWUv2 in SWH predictions within the Great Lakes for the following three reasons: First, the ICESat-2 ATL13 data products provide the SWH data over unfrozen areas of the Great Lakes during the winter, which makes it competent in complementing the winter data gaps. Second, the ICESat-2 produces a high density of SWH data along its operational track. According to the data over the Great Lakes region in this study, the density of the SWH is approximately 11.8 data points per kilometer. The inland water heights are processed in segments that contain a minimum of approximately 100 signal photons to ensure the segment accurately characterizes the water surface. As such, the segments vary in length from approximately 30 m to 100 m [6]. Therefore, the ICESat-2 ATL13 data products provide the information about water surface conditions at an unprecedentedly high resolution compared with other satellite measurements, such as those produced by the China–France Oceanography Satellite (CFOSAT), whose spatial resolution (nadir beam) is around 70 to 80 km [7], and the Global Navigation Satellite System-Reflectometry (GNSS-R) data, which provide SWH estimates at a resolution of 27.8 km × 27.8 km (0.25° × 0.25°) [8]. Third, the data products from ICESat-2 demonstrate high precision. As presented in the Algorithm Theoretical Basis Document for ATL13 [9], the overall ensemble error per 100 inland water photons is estimated as 6.1 cm, derived from the root-mean-square of five error sources: radial orbit error, tropospheric delay error, forward scattering error, geolocation knowledge uncertainty, and ATLAS (Advanced Topographic Laser Altimeter System) ranging precision per photon. Therefore, it is worth exploring the potential of the ATL13 data products of ICESat-2 in physical wave model verification for the Great Lakes.

Although it has been over 5 years since ICESat-2’s launch (15 September 2018), the SWH data in its ATL13 product have not been broadly explored and applied for examining wave conditions over large water bodies. Most studies regarding the ICESat-2 ATL13 product focus on its inland water orthometric height data, while its SWH data are only treated as auxiliary information. For example, Luo et al. [10] estimated lake water level on the Tibetan Plateau by subtracting the SWH from the orthometric height of the ATL13 dataset. They evaluated how taking SWH into consideration would affect estimating the change rates of water level. Similarly, Liu et al. [11] used the ICESat-2 ATL13 product to monitor the lake water levels in the Yellow River Basin by also subtracting the SWH from the orthometric water level to obtain the true water level, but found this greatly increased the standard deviation of the water level measurements and thus increased the uncertainty of the water volume change calculation, compared to disregarding the effect of SWH. An et al. [12] evaluated the performance of seven ongoing satellite (ICESat-2 included) altimetry missions for measuring inland water levels of the Great Lakes, and concluded that the stability of the average deviation (bias) of the ICESat-2 is the best in inland water level measurements, but the mean root-mean-square error (RMSE) of ICESat-2 is slightly higher than satellites Jason-3 and Sentinel-6. They also pointed out that ICESat-2 has an exceptionally high spatial resolution among the seven satellites. However, their study did not explore the use of SWH data of the ICESat-2 ATL13 product for studying wave conditions in the Great Lakes.

Despite their promising potential, the SWH data of the ICESat-2 ATL13 product remain unexamined. It is necessary to assess their data quality and accuracy before being applied in evaluating the numerical wave model prediction results. Some studies have conducted similar work and investigated the data quality of SWH data provided by other satellites. According to Yang and Zhang’s study [13], the RMSE is about 0.2–0.3 m for the comparison of satellite Sentinel-3A SWH with NDBC buoy data, and that of satellite Sentinel-3B is about 0.18–0.3 m, across the global ocean. Li et al. [14] assessed the performance of the SWH data of the China–France Oceanography Satellite (CFOSAT) in the South China Sea, a unique sea area characterized by a semi-enclosed basin and multi-reef terrain. Compared against observations from mooring or buoy sites, they found that the average correlation coefficient is as high as 0.87, and the average RMSE is 0.47 m in the relatively open and deep areas of the South China Sea. However, the SWH data would be affected by coastlines, topography, and monsoons in certain cases. Moreover, Durrant et al. [15] validated SWH data from Jason-1 and Envisat satellite altimetry against NDBC buoy data across the globe. The Jason-1 SWH was reported to have a bias of −0.010 m, an RMSE of 0.227 m, and a correlation coefficient of 0.983, and the Envisat SWH was reported to have a bias of 0.036 m, an RMSE of 0.219 m, and a correlation coefficient of 0.986. In addition, Peng and Jin [8] estimated the global ocean SWH using space-borne CYGNSS GNSS-R data through the relationship between the square root of the signal-to-noise ratio data of the CYGNSS delayed Doppler map and the SWH, then compared it with buoy-measured data from NDBC. The correlation coefficient between the estimated value and the SWH observation of a buoy is 0.9539, the bias is −0.0496 m, and the RMSE is 0.2761 m. Although all of the above four studies focused on ocean waves, none of them evaluated satellite SWH measurements, particularly in the Great Lakes. Actually, such research remains fairly rare based on our search among academic articles.

Based on the reasons above, this study evaluated the quality of the ICESat-2 ATL13 significant wave height data in the Great Lakes region using the regular buoy measurements sourced from the National Data Buoy Center (NDBC) and the Seagull platform of the Great Lakes Observing System (Seagull GLOS), within two time intervals: April 2021 to December 2021, and May 2022 to October 2022. Then, this study assessed the model quality of GLWUv2 by comparing its retrospective forecast simulations with the ICESat-2 ATL13 significant wave height data in the Great Lakes, and also with emerging low-cost Spotter buoys that have enabled rare winter wave measurements in recent years. This study will fill a critical data gap of wave conditions for the Great Lakes during winter by verifying the accuracy of the ICESat-2 ATL13 significant wave height data. The work will ultimately contribute by providing better wave predictions that improve the preparedness of the people and industries of the Great Lakes communities for dangerous high wave conditions.

2. Materials and Methods

2.1. Study Area

Our study focused on the Great Lakes, which are the largest freshwater system on Earth by area and the second largest by total volume, containing 21% of the world’s surface freshwater and more miles of coastline than the combined Atlantic and Pacific Ocean coastlines in the contiguous United States. The Great Lakes consist of Lake Superior, Michigan, Huron, Erie, Ontario, and the connecting channels (Figure 1). The Great Lakes megalopolis is the most populated and largest megalopolis in North America, containing an estimated population of 53,768,125 as of 2000 and is projected to reach a population of about 63 million by 2025 [16], which indicates that the safeguarding of the Great Lakes coastal communities is imperative.

2.2. Data

2.2.1. ICESat-2 ATL13 Data

The Inland Water Surface Height product (ATL13) of the Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) was used in this study. The ICESat-2 was launched on 15 September 2018, carrying a photon-counting laser altimeter that allows scientists to measure the elevation of ice sheets, glaciers, sea ice and more. The ATL13 contains along-track surface water products for inland water bodies, including lakes, reservoirs, rivers, bays, estuaries, and a 7 km near-shore buffer [6].

The Advanced Topographic Laser Altimeter System (ATLAS) on the satellite ICESat-2 is measuring the surface with 6 laser beams (3 pairs). Each pair consists of one strong beam and one weak beam, different in their transmit energy. The ground tracks corresponding to the 3 pairs are noted as pair track 1, pair track 2, and pair track 3. The two beams within each pair are approximately 90 m apart in the across-track direction and 2.5 km in the along-track direction. The ICESat-2 observatory changes its orientation approximately twice per year, thereby affecting the naming of the 6 beams. Along the direction of travel, depending on their relative position, the three beams on the left of each pair are noted as GTXL, the other three on the right as GTXR, which is a consistent principle (GTX means ground track number). Accordingly, we use GT1R when the ICESat-2 observatory is in forward orientation, and GT3L when it is in backward orientation, as listed in

Table 1. Beams we chose during different periods and their corresponding ICESat-2 orientation.

Period	Orientation	Beam Used
1 April 2021–1 October 2021	Forward	GT1R
2 October 2021–7 June 2022	Backward	GT3L
9 June 2022–11 October 2022	Forward	GT1R

.

2.2.2. In Situ Observations

The SWH data from the ICESat-2 ATL13 product are first verified using the in situ buoy observations. Most of the regular real-time buoy datasets are available from the National Data Buoy Center and the Seagull platform of the Great Lakes Observing System. These regular buoy measurements are limited to non-winter seasons because buoys are retrieved in the late fall to avoid damage from ice cover. Recently, drifting wave buoys were experimentally deployed in the Great Lakes over the winter. These drifting wave buoys, called Spotter buoys, are low-cost, solar powered metocean buoys developed by Sofar Ocean Technologies, Inc., San Francisco, CA, USA [17]. Spotter buoys have been used in studies understanding wave conditions in the marginal ice zone of the Arctic Ocean [18] and other coastal oceans [19]. For the verification work, we utilized the regular buoy data for late summer–fall when high wind conditions are secondly frequent following winter [20] and therefore SWH tends to be higher. The distribution of the regular buoys is shown in Figure 1. Their basic information including buoy ID, coordinates, sampling time interval, water depth, number of observed samples used in the study, lake, and data source (NDBC or Seagull) is listed in

Table A1. Basic information of regular buoys used in this study.

Buoy ID	Latitude	Longitude	Time Interval/min	Water Depth/m	Observation #	Lake	Source
45027	46.860	−91.930	10	52	6	Superior	NDBC
45028	46.814	−91.829	10	49	2	Superior	NDBC
45006	47.335	−89.793	60	194.5	15	Superior	NDBC
45023	47.270	−88.607	5	25	7	Superior	NDBC
45025	46.969	−88.398	5	28	8	Superior	NDBC
45001	48.061	−87.793	60	247	13	Superior	NDBC
45180	48.034	−87.730	30	239	10	Superior	NDBC
45004	47.585	−86.585	60	237.5	15	Superior	NDBC
C45136	48.535	−86.953	60	–	7	Superior	NDBC
C45154	46.050	−82.640	60	–	7	Huron	NDBC
C45137	45.545	−81.015	60	–	3	Huron	NDBC
C45143	44.940	−80.627	60	–	7	Huron	NDBC
45003	45.351	−82.840	60	135	12	Huron	NDBC
45162	44.988	−83.270	20	20	6	Huron	NDBC
45008	44.283	−82.416	60	54.3	9	Huron	NDBC
45163	43.985	−83.596	20	12.5	12	Huron	NDBC
C45149	43.542	−82.075	60	–	7	Huron	NDBC
45209	43.129	−82.391	10	14	2	Huron	NDBC
45175	45.825	−84.772	5	–	12	Michigan	NDBC
45194	45.804	−84.792	30	19.8	12	Michigan	NDBC
45014	44.795	−87.759	30	13	5	Michigan	NDBC
45013	43.100	−87.850	30	20	8	Michigan	NDBC
45199	42.703	−87.649	60	0.5	5	Michigan	NDBC
45187	42.491	−87.779	10	–	8	Michigan	NDBC
45186	42.368	−87.795	10	–	9	Michigan	NDBC
45174	42.135	−87.655	10	–	9	Michigan	NDBC
45198	41.892	−87.563	10	9	8	Michigan	NDBC
45170	41.755	−86.968	10	19	10	Michigan	NDBC
45026	41.982	−86.619	10	20.7	10	Michigan	NDBC
45168	42.397	−86.331	10	20.4	10	Michigan	NDBC
45029	42.900	−86.272	10	27	10	Michigan	NDBC
45161	43.182	−86.360	20	22.5	6	Michigan	NDBC
45024	43.981	−86.556	10	24	6	Michigan	NDBC
45183	44.982	−85.831	30	–	7	Michigan	NDBC
45022	45.405	−85.087	10	36	6	Michigan	NDBC
45002	45.344	−86.411	60	181.4	11	Michigan	NDBC
45007	42.674	−87.026	60	159.1	15	Michigan	NDBC
C45132	42.463	−81.215	60	–	9	Erie	NDBC
45165	41.702	−83.261	10	8	10	Erie	NDBC
45202	41.532	−82.941	10	4.9	2	Erie	NDBC
45201	41.601	−82.781	10	7.6	1	Erie	NDBC
45203	41.393	−82.512	10	–	3	Erie	NDBC
45005	41.677	−82.398	60	9.8	12	Erie	NDBC
45204	41.508	−82.115	10	–	2	Erie	NDBC
45196	41.521	−81.880	10	–	9	Erie	NDBC
45176	41.550	−81.765	60	16.6	11	Erie	NDBC
45205	41.501	−81.748	10	–	3	Erie	NDBC
45206	41.585	−81.583	10	–	3	Erie	NDBC
45197	41.619	−81.617	10	–	10	Erie	NDBC
45164	41.748	−81.698	60	22.6	16	Erie	NDBC
45207	41.762	−81.331	10	–	3	Erie	NDBC
45208	41.934	−80.747	10	–	4	Erie	NDBC
45167	42.185	−80.135	20	–	2	Erie	NDBC
C45142	42.740	−79.290	60	–	6	Erie	NDBC
C45139	43.250	−79.530	60	–	6	Ontario	NDBC
C45159-NWLakeOntario	43.770	−78.980	60	–	8	Ontario	NDBC
45012	43.621	−77.401	60	143.3	14	Ontario	NDBC
C45135-PrinceEdwardPoint	43.785	−76.868	60	–	11	Ontario	NDBC
SPOT-1810	47.914	−89.339	30	–	2	Superior	Seagull
SPOT-0592	47.207	−88.162	30	–	6	Superior	Seagull
SPOT-0700	47.477	−87.870	30	–	7	Superior	Seagull
SPOT-1814	47.192	−87.226	30	–	4	Superior	Seagull
SPOT-1816	46.642	−87.453	30	–	5	Superior	Seagull
SPOT-1360	46.598	−87.372	30	–	5	Superior	Seagull
SPOT-1362	46.570	−86.568	30	–	5	Superior	Seagull
SPOT-1179	46.560	−86.466	30	–	7	Superior	Seagull
SPOT-1361	46.696	−86.004	30	–	5	Superior	Seagull
SPOT-1415	44.346	−87.446	30	–	2	Michigan	Seagull
SPOT-1412	43.392	−87.804	30	–	5	Michigan	Seagull
SPOT-1275	44.758	−86.261	30	–	3	Michigan	Seagull
SPOT-1981	45.045	−86.015	30	–	2	Michigan	Seagull
SPOT-1407	44.788	−85.624	30	–	2	Michigan	Seagull
SPOT-1408	44.769	−85.534	30	–	2	Michigan	Seagull
SPOT-1080	45.921	−84.337	30	–	6	Huron	Seagull
UWRAEON1-22	44.175	−81.653	20	–	5	Huron	Seagull
SPOT-1413	41.741	−83.136	30	–	5	Erie	Seagull
RBS-TOL	41.680	−83.250	60	–	3	Erie	Seagull
UWSS-RAEON2-21	41.913	−82.736	20	–	4	Erie	Seagull
WALNUT	42.132	−80.270	20	–	4	Erie	Seagull
BSC1	42.560	−79.430	10	–	5	Erie	Seagull

. For the wintertime verification, we utilized the Spotter buoy data from the experimental deployment during 10–18 February 2022 in Lake Michigan. The trajectory of the drifting Spotter buoy is shown in the corner of Figure 1.

Finally, wind data from the Coastal Marine Automated Network (C-MAN) at Ludington MI (station ID: LDTM4), Holland MI (station ID: HLNM4), Muskegon MI (station ID: MKGM4), South Haven MI (station ID: SVNM4), and Milwaukee WI (station ID: MLWW3) were used for comparison with the GLWUv2 wind forcing data. The wind speed data from the stations were height-adjusted to 10 m based on the log wind profile assumption [21].

2.2.3. Wave Model Outputs

The NOAA’s Great Lakes Waves-Unstructured Forecast System version 2.0 (GLWUv2) is based on the WAVEWATCHIII^® (WW3) model [3,4,5], which is a physics-based, third-generation wave model where four-wave nonlinear interaction and various other source terms are explicitly modeled in frequency directional space. The current version of WW3 is 6.07. GLWUv2 is implemented on an unstructured grid with resolution ranging from 250 m near the coast to 2.5 km in deep waters. The model is forced by wind fields from the National Digital Forecast Database (NDFD). Considering that the GLWUv2 became operational in May 2023, and the ICESat-2 data are still unavailable for the winter from December 2023 to February 2024, in this study, the retrospective forecast simulation results for January and February in 2022 are used as a proxy for hindcast to evaluate the model quality. GLWUv2 runs on hourly forecast cycles daily, with twenty short cycles out to 48 h and four long cycles out to 150 h. Because the main purpose of this comparison is to examine the best available model estimates for the SWH, in this study, we only use the first-6-h prediction results starting from 1:00 UTC, 7:00 UTC, 13:00 UTC, and 19:00 UTC of each day, thereby constituting the continuous hourly data for each day.

2.3. Data Comparison Methods

Due to the limited distribution of buoys across the Great Lakes region and the ability of buoy observations to reflect the wave conditions only within certain spatial scale, the ideal comparison should be made between buoy data points and ICESat-2 data points that coincide in time and space. The following method is employed to accomplish this matching process:

Each trajectory of ICESat-2 ATL13 over the Great Lakes contains 10²–10³ data points spanning within several seconds due to its high spatiotemporal resolution. The ICESat-2 data point of the closest distance (less than 25 km) to the buoys and the temporally linear-interpolated buoy data point are paired together. As a result, a total of 554 data pairs were obtained during April 2021–October 2022.

Subsequently, the paired data were visually plotted on maps with colors denoting the magnitude of the significant wave height. They were also compared visually by a scatter plot. Then, the ICESat-2 SWH data were evaluated by the statistical analysis of the SWH difference, including the bias, root-mean-square error (RMSE), correlation coefficient (r), and scattering index (SI), defined as follows:

$$\mathrm{Bias}=\frac{1}{n}\sum _{i=1}^n(y_i-x_i),$$

(1)

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}\sum _{i=1}^n{(y_i-x_i)}^2},$$

(2)

$$\mathrm{SI}=\frac{\mathrm{RMSE}}{\overline{x}},$$

(3)

$$r=\frac{{\sum }_{i=1}^n(x_i-\overline{x})(y_i-\overline{y})}{\sqrt{{\sum }_{i=1}^n{(x_i-\overline{x})}^2·{\sum }_{i=1}^n{(y_i-\overline{y})}^2}},$$

(4)

where n represents total number of matched pairs; $y_i$ represents ICESat-2 SWH measurements; $\overline{y}$ represents ICESat-2 SWH average; $x_i$ represents the buoy SWH observations; and $\overline{x}$ represents buoy SWH average.

Similarly, the same comparative approach was extended to examine GLWUv2’s SWH retrospective forecast results. This assessment involved comparisons of GLWUv2 SWH outputs with both ICESat-2 SWH measurements and drifting Spotter buoy observations. In each matched data pair, the GLWUv2 data point was also rigorously selected as the closest counterpart. Consequently, a total of 53,472 matched data pairs were obtained from the matching with ICESat-2 SWH measurements during the period of 1 January to 28 February 2022, and the matching with the Spotter buoy observations yielded 376 matched data pairs during the period of 10–18 February 2022.

3. Results and Discussion

3.1. Data Quality Evaluation of ICESat-2 ATL13 SWH Data

Figure 2 shows comparisons of SWH between ICESat-2 measurements and regular buoy observations on different example days. Illustrated by the color gradient, the data points indicate that the SWH measured by ICESat-2 agrees with regular buoy observations.

The SWH comparisons between ICESat-2 measurements and buoy observations from April 2021 to October 2022 were collected and plotted on an x-y coordinate system. As shown in Figure 3a, the straight line y = x is used for reference. We note that most of the points are located surrounding the y = x line, indicating that the ICESat-2 can achieve a reasonable measurement of the SWH in the Great Lakes.

As shown in

Table 2. Accuracy analysis of ICESat-2 SWH measurements versus buoy SWH observations using all matched data pairs.

Group	Bias	RMSE	SI	r	n
All data	0.074 m	0.191 m	0.46	0.890	554
0–9.2 km	0.077 m	0.188 m	0.47	0.888	186
9.2–16.7 km	0.063 m	0.198 m	0.48	0.898	184
16.7–25.0 km	0.082 m	0.186 m	0.44	0.888	184

, the RMSE for ICESat-2 SWH in all the data pairs collectively is 0.191 m, and the correlation coefficient is 0.890. We also split the data pairs into three groups based on the distance between the ICESat-2 measurement and its corresponding buoy observation. The three groups are noted as 0–9.2 km, 9.2–16.7 km, and 16.7–25.0 km. In this way, the three groups are equal in sample size. The RMSE and correlation coefficient of each of the three groups are also listed in Table 2, which indicates that the ICESat-2 SWH measurements presented reasonable agreement with buoy SWH observations. In addition, the RMSE does not vary significantly as the distance increases within the 25 km range, which shows that distance within 25 km will not affect the capacity of ICESat-2 SWH measurement to be used as a proxy of buoy SWH observation.

In order to focus particularly on high waves, we consider waves with SWH higher than 0.8 m as high waves. Based on this criterion, the matched data pairs with buoy SWH values larger than 0.8 m will be selected to do a further accuracy analysis, thereby evaluating the ICESat-2 SWH data quality particularly in the high wave events. The results are shown in the Figure 3b scatter plot. The statistical results for all the data pairs in high wave events as well as each group are listed in

Table 3. Accuracy analysis of ICESat-2 SWH measurements versus buoy SWH observations using matched data pairs in high wave events (HWE).

Group	Bias	RMSE	SI	r	n
All data (HWE)	−0.077 m	0.340 m	0.26	0.716	61
0–8.5 km (HWE)	−0.104 m	0.299 m	0.23	0.677	20
8.5–18 km (HWE)	−0.150 m	0.395 m	0.28	0.781	21
18–25 km (HWE)	0.028 m	0.315 m	0.26	0.654	20

. From Table 2 and Table 3, it is evident that the ICESat-2 SWH measurements have a larger RMSE and a lower correlation coefficient for high waves, compared to those for all measurements.

3.2. Evaluation of the GLWUv2 SWH Outputs Using ICESat-2 Measurements

The retrospective SWH prediction results of GLWUv2 in winter (January and February 2022) were compared with the ICESat-2 SWH measurements by spatially plotting the data points on maps. Figure 4 illustrates such comparisons in two low wave scenarios and two high wave scenarios. The color gradient pervasive in the lakes displays the magnitude of the SWH predicted by GLWUv2, round dots in the same color pattern represent ICESat-2 SWH data points, while the gray area in each image represents lake surface ice coverage. In the low wave scenarios, depicted in Figure 4a,b, the GLWUv2 SWH predictions align well with ICESat-2 measurements, as indicated by the similar background colors to the round dots in the red box area. However, in the high wave scenarios, depicted in Figure 4c,d, GLWUv2 tends to overestimate SWH.

We also use a scatter plot to better visualize the comparison of GLWUv2 SWH predictions with ICESat-2 SWH measurements during January and February 2022, as shown in Figure 5. The straight line y = x is still used as reference for the comparison. We note that most of the scattered dots follow the trend of this line. The RMSE is 0.312 m, and the correlation coefficient is 0.865. However, GLWUv2’s noticeable overestimation of the SWH occurs where SWH is higher than 1.5 m. In order to analyze the retrospective prediction accuracy of GLWUv2 SWH in different wave height ranges, the data pairs are divided into three groups based on the ICESat-2 SWH values, which are $0<SWH\le 0.8$ m, $0.8<SWH\le 1.5$ m, and $SWH>1.5$ m. The bias, RMSE, SI, and correlation coefficient of the GLWUv2 SWH predictions for the three groups are listed in

Table 4. Accuracy analysis of GLWUv2 SWH retrospective predictions at different ranges of ICESat-2 SWH.

Group	Bias	RMSE	SI	r	n
All data	0.058 m	0.312 m	0.80	0.865	53,472
0–0.8 m	0.033 m	0.257 m	1.06	0.536	46,446
0.8–1.5 m	0.107 m	0.329 m	0.30	0.604	5045
>1.5 m	0.511 m	0.899 m	0.45	0.451	1981

. The bias and RMSE of GLWUv2 SWH both increase with increasing ICESat-2 SWH values. In addition, the correlation coefficient becomes lower than 0.5 for the scenario of SWH larger than 1.5 m, indicating that the GLWUv2 cannot produce a satisfying prediction for high waves. Though such bias in high waves should be addressed in the future modeling work, in actual forecasting operations, overestimation is more acceptable than underestimation for the objective of not underestimating the risks and threats associated with high wave events to the coastal communities around the Great Lakes.

3.3. Evaluation of the GLWUv2 SWH Prediction Using Drifting Spotter Buoy Observations

Having served as the exclusive buoy data source for winter wave conditions, the solitary Spotter buoy, deployed experimentally in Lake Michigan from 10 to 18 February 2022, offers a unique opportunity to directly scrutinize GLWUv2’s significant wave height (SWH) predictions. The Spotter buoy’s trajectory during this period is shown in red in Figure 6. A total of 376 valid Spotter buoy records were matched with the GLWUv2 predictions for this period, forming the visual comparison via scatter plots. As indicated in Figure 7a, it remains evident that GLWUv2 leans towards overestimating SWH in instances of higher waves, while aligning well with Spotter buoy observations in cases of lower waves. The calculated RMSE is 0.486 m, and the bias is 0.263 m.

This performance discrepancy prompted further investigation into whether the model’s overestimation for SWH can be reconfirmed when compared against ICESat-2 measurements within the same region in Lake Michigan. Due to no ICESat-2 data being available for this region during the Spotter buoy deployment (10–18 February 2022), we opted to compare GLWUv2 results to ICESat-2 measurements for the next available ICESat-2 track, which was on 20 February 2022. The locations of ICESat-2 measurements are illustrated in dark blue in Figure 6. The results, as depicted in Figure 7b, reinforce the observation: GLWUv2 consistently yields higher SWH estimates than those measured by ICESat-2, particularly for higher waves. This consistent trend across both comparisons—GLWUv2 versus Spotter buoy and GLWUv2 versus ICESat-2—underscores the model’s inclination to overestimate SWH.

Since the SWH simulations of GLWUv2 are highly impacted by wind speed in the forcing data (NDFD), a bias in wind speed inevitably leads to an overestimation in the projected SWH values. Therefore, we further evaluated the NDFD wind speed data against the Spotter buoy wind speed data. As depicted in Figure 8a, the majority of the wind speed data surpass the wind speed records collected from the Spotter buoy in Lake Michigan between 10 and 18 February 2022, and its bias in wind speed is 2.602 m/s.

However, when comparing the NDFD wind speed data with the wind speed records obtained from five nearby meteorological observation stations at the lake shore (locations shown in Figure 6) during the same period, the high bias in wind speed is not as evident, except for station LDTM4, where the bias is 3.897 m/s, while at stations HLNM4, MKGM4, SVNM4, and MLWW3, the biases are 1.553 m/s, 0.382 m/s, 0.212 m/s, and 0.413 m/s, respectively. The results are shown in Figure 8b–f. The large bias that occurred at the station LDTM4 could have resulted from its location, the back of the inland small lake (Pere Marquette Lake), surrounded by buildings. This could disturb the wind field around the station, resulting in a lowering of the wind speed, which was not represented by NDFD.

Given the above analysis, it is possible that GLWUv2’s overestimation of SWH during 10–18 February 2022 is due to a local high bias in wind speed in the NDFD wind speed data.

4. Conclusions

Based on a thorough assessment of data quality spanning from April 2021 to October 2022, it is evident that ICESat-2’s measurements of significant wave heights align closely with regular buoy observations in the Great Lakes, exhibiting a root-mean-square error (RMSE) of 0.191 m. This robust consistency substantiates the utility of ICESat-2 data for validating significant wave height predictions generated by GLWUv2 during winter in the Great Lakes region. The findings reveal that GLWUv2 demonstrates proficiency in predicting significant wave heights below 1.5 m, while it sometimes tends to overestimate waves higher than that. The overall RMSE for GLWUv2’s significant wave height predictions is 0.312 m. Furthermore, an in-depth comparison of GLWUv2’s SWH outputs with data from a drifting Spotter buoy in Lake Michigan during 10–18 February 2022 also corroborates the model’s overprediction for significant wave height during high wave events. Incorporating the analysis from the perspective of wind speed, such overestimation for significant wave height can be attributed, in part, to a local high bias in the NDFD wind speed data since wind speed serves as a crucial input for significant wave height estimation. The verified utility of the ICESat-2 ATL13 significant wave height product in the Great Lakes contributes to filling wintertime data gaps in the region, which are critical for advancing our understanding of the Great Lakes’ wintertime limnology.