Evaluation of Two Satellite Surface Solar Radiation Products in the Urban Region in Beijing, China

Abstract

Surface solar radiation, as a primary energy source, plays a pivotal role in governing land–atmosphere interactions, thereby influencing radiative, hydrological, and land surface dynamics. Ground-based instrumentation and satellite-based observations represent two fundamental methodologies for acquiring solar radiation information. While ground-based measurements are often limited in availability, high-temporal- and spatial-resolution, gridded satellite-retrieved solar radiation products have been extensively utilized in solar radiation-related studies, despite their inherent uncertainties in accuracy. In this study, we conducted an evaluation of the accuracy of two high-resolution satellite products, namely Himawari-8 (H8) and Moderate Resolution Imaging Spectroradiometer (MODIS), utilizing data from a newly established solar radiation observation system at the Beijing Normal University (BNU) station in Beijing since 2017. The newly acquired measurements facilitated the generation of a firsthand solar radiation dataset comprising three components: Global Horizontal Irradiance (GHI), Direct Normal Irradiance (DNI), and Diffuse Horizontal Irradiance (DHI). Rigorous quality control procedures were applied to the raw minute-level observation data, including tests for missing data, the determination of possible physical limits, the identification of solar tracker malfunctions, and comparison tests (GHI should be equivalent to the sum of DHI and the vertical component of the DNI). Subsequently, accurate minute-level solar radiation observations were obtained spanning from 1 January 2020 to 22 March 2022. The evaluation of H8 and MODIS satellite products against ground-based GHI observations revealed strong correlations with R-squared (R²) values of 0.89 and 0.81, respectively. However, both satellite products exhibited a tendency to overestimate solar radiation, with H8 overestimating by approximately 21.05% and MODIS products by 7.11%. Additionally, solar zenith angles emerged as a factor influencing the accuracy of satellite products. This dataset serves as crucial support for investigations of surface solar radiation variation mechanisms, future energy utilization prospects, environmental conservation efforts, and related studies in urban areas such as Beijing.

1. Introduction

Surface solar radiation refers to the solar radiation reaching the Earth’s surface, also known as shortwave radiation. It provides the primary energy for atmospheric circulation and the water cycle and plays a crucial role in surface radiation balance, energy exchange at the Earth’s surface, hydrological cycles, and the formation of weather and climate [1,2,3,4]. During its passage through the atmosphere, solar radiation is scattered by atmospheric molecules, clouds, and aerosols, and reaches the Earth’s surface in the form of diffuse sky radiation, known as Diffuse Horizontal Irradiance (DHI). Solar radiation directly from the solar facula, after absorption and scattering by atmospheric molecules, clouds, and aerosols, is referred to as direct radiation, or Direct Normal Irradiance (DNI). The sum of direct and diffuse radiation is called total solar radiation or surface global solar radiation, commonly referred to as Global Horizontal Irradiance (GHI).

Measurements of solar radiation and its components (GHI, DNI, and DHI) generally yield accurate products, which can be obtained through two fundamental methods: ground-based instrumentation or remotely using satellites [5,6,7]. These measurements are widely used to validate reanalysis products [8,9,10,11] or machine learning products [12,13]. Ground-based observations provide the most accurate and direct means of obtaining surface solar radiation [11,14,15,16] and are used to validate all other types of solar radiation datasets [17,18,19,20]. Currently, ground-based observation stations for solar radiation measurement are unevenly distributed globally and relatively sparse because of their complex and expensive installation and maintenance of measuring instruments, making it difficult to depict the spatial distribution characteristics of surface solar radiation in local areas at higher spatial resolutions [9,21,22]. Widely used ground-based observations include the Baseline Surface Radiation Network (BSRN) [23], the SURFace RADiation Budget Network (SURFRAD) [24,25], the World Radiation Data Center (WRDC) [26,27], and the Global Energy Balance Archive (GEBA) [28,29]. In China, there are a total of 122 observation stations capable of directly measuring surface solar radiation. However, due to the high cost of measurement instruments and inadequate basic maintenance, the number of observation stations capable of directly measuring direct and diffuse radiation has decreased to less than 20 since the 1990s [30,31,32]. Issues such as instrument replacement, aging, and relocation have led to data inhomogeneities in solar radiation observations in China, increasing the uncertainty in the evaluation of the data [33,34].

Compared with ground-based observations, satellite retrieval of solar radiation products offer superior spatial coverage, temporal resolution, and cost-effectiveness, although the accuracies are generally lower [11,35]. They are developed employing either a semi-empirical or physical methodology, leveraging cloud or aerosol data acquired from geostationary satellites [36,37]. They can provide consistent, continuous data over large areas, enabling comprehensive analyses of solar radiation patterns with minimal infrastructure and logistical requirements. A series of satellite products have been developed and are widely used in related research, such as International Satellite Cloud Climatology Project-Flux Data (ISCCP-FD) [38], Moderate Resolution Imaging Spectroradiometer (MODIS) level-3 products [39], the Clouds and Earth’s Radiant Energy System (CERES) radiation products [40], Himawari-8(H8) [41,42], Global Land Surface Satellite (GLASS) [43], and Fengyun-4A [44]. These satellite products have been widely validated and used in many large-scale analyses [11,15,45,46]. Jia et al. [37] observed a strong correlation between satellite products and in situ observations, although they noted a tendency for the satellite data to overestimate or underestimate radiation magnitude over mainland China. These discrepancies were attributed to limitations in cloud, aerosol, and water vapor parameterization, as well as deficiencies in retrieval algorithms. Similarly, Yang et al. [47] conducted a comprehensive evaluation of multiple satellite products, revealing an overestimation of solar radiation and poor accuracy, particularly in high-latitude regions worldwide. To comprehensively ascertain their accuracy, a thorough evaluation of satellite products is essential, incorporating in situ observations.

Beijing, the capital and political–cultural center of China, has been among the fastest-growing cities in the world in recent decades [48]. It has abundant solar energy resources, with an annual mean radiation of up to 158.5 W/m² and total sunshine hours reaching or exceeding 2700 h [49]. In recent years, despite significant improvements in environmental quality, the complex characteristics of aerosols in the Beijing area during the winter and spring coal combustion, as well as the frequent spring dust weather, have had quite complex impacts on solar radiation [50,51,52,53,54]. While there is only one in situ station operated in Beijing, as reported in the China Meteorological Data Service Center (http://data.cma.cn (accessed on 14 May 2023)) of the China Meteorological Administration (CMA), it has been utilized in prior research [55,56,57]. However, this singular data point fails to fulfill the requirements for fine-scale analysis. Some studies detected a significant decrease in solar radiation from 1960 to 1990, consistent with the findings of most studies in China [58,59,60] and the global dimming trend [2,61,62]. He et al. [58] and Wang et al. [56] found a significant decrease in solar radiation in Beijing from 1961 to 2016 and attributed it to a significant reduction in direct radiation. Tang et al. (2011) [63] found that although solar radiation decreased in Beijing from 1961 to 1989 and from 1989 to 2006, the decrease was not significant. Hence, for a more comprehensive analysis of solar radiation dynamics in Beijing, high-temporal and high-spatial resolution satellite imagery will serve as an invaluable remote sensing tool.

This study undertook an assessment of the accuracy of two high-resolution satellite products, namely H8 and MODIS, utilizing data from a novel solar radiation observation system established at the Beijing Normal University (BNU) station in Beijing in 2017. The newly acquired solar radiation dataset encompassed three primary components: GHI, DNI, and DHI. Stringent quality control protocols were implemented to generate dependable in situ observation datasets. These in situ observations were subsequently leveraged to evaluate the performance of MODIS and H8 products. The findings of this study provide valuable practical insights for the advancement of future satellite remote sensing retrieval products in atmospheric science.

The paper is organized as follows: Section 2 outlines the establishment of the solar radiation observation system and the acquisition of satellite data. In Section 3, the quality of the observational data is assessed. Section 4 presents the evaluation results of MODIS and H8 datasets. Finally, the discussion and conclusion sections synthesize the results.

2. Data and Methods

2.1. Site Observations

2.1.1. Site Description and Measure Instruments

The newly constructed BNU site, depicted in Figure 1a, was established in 2017 atop an office building within Beijing Normal University (39.956°N, 116.359°E), situated at an altitude of 150 m. The location boasts minimal obstructions and lacks noticeable sources of pollution nearby, meeting the environmental criteria for installing solar radiometers and sun photometers from an atmospheric science perspective.

At the BNU site, a set of solar radiometers was installed (Figure 1b). Solar radiation measurements were conducted following the BSRN principles [23], enabling the measurement of GHI, DNI, and DHI. As shown in

Table 1. Instrument attribute information for measuring solar radiation components.

Component	Instrument	Spectral Range (nm)	Sensitivity (μV/W·m−2)	Time Period	Manufacturer
GHI	CMP	200–4000	7~14	1 January 2020–22 March 2022	Kipp & Zonen
DNI	CHP1
DHI	CMP

, GHI and DHI were recorded using the CMP series radiation meter, while DNI was captured by the CHP1 direct radiation meter manufactured by Kipp and Zonen, the Netherlands (https://www.kippzonen.com/ (accessed on 6 October 2023)). Comprehensive technical details of this system can be referenced in the study by Driemel et al. [23]. All radiation components were sampled every second and recorded as 1 min statistics. Solar radiation data spanning from 1 January 2020 to 22 March 2022 were utilized for this study. Regular inspection and maintenance procedures, including annual calibration, were performed on the solar radiometers to ensure the quality of measurements. The calibration of solar instruments was achieved by regularly sending them to the manufacturer each year. This process includes a thorough inspection and assessment of all performance parameters. The basic procedure involves comparing the instrument to be calibrated with a reference instrument to obtain its sensitivity parameters. Based on these parameters, the observed signals from the instrument can be accurately converted into radiation energy.

2.1.2. Data Quality Control Procedure

Although the observation instruments were diligently maintained and regularly calibrated, data records may still contain inaccuracies due to adverse weather conditions or human factors, such as potential operational errors during the manual replacement of memory cards and data copying. Quality control for solar radiation observations primarily follows the procedures recommended by the BSRN protocol [64,65], which have been widely adopted in previous studies [66,67]. Key procedures include the missing data test, physically possible limits test, solar tracker malfunction test, and comparison test.

• Missing test

During the actual observation process, occurrences such as malfunctioning solar radiation sensors, adverse weather conditions, instrument repairs, or maintenance periods may result in “NAN” values in the observation output. Therefore, upon reviewing the observation time series, “NAN” values were identified, signifying missing measurements during those instances.

2.

Physically possible limits test

The “Physically possible limits” test aims to detect significant observational errors and random errors related to data processing [65,68]. Radiation data exceeding these limit values typically occur under extremely rare conditions and for very short periods of time. The specific testing method involves comparing the minute observational values of radiation components with the threshold values for each component, which are generally defined by the relationship established between the solar constant corrected for the distance between the Earth and the Sun and the solar zenith angle (SZA).

The threshold of each solar radiation component, i.e., GHI, DNI, and DHI, was derived from Long and Dutton [69] and Liu et al. [66], as shown in

Table 2. Lower and upper limits for “Physically Possible Limits”. μ represents the cosine of the solar zenith angle, while SC denotes the solar constant.

Components	Lower Limit (W m−2)	Upper Limit (W m −2) (μ > 0)	Upper Limit (W m −2) (μ < 0)
GHI	−4	1.5 × SC × μ1.2 + 100	50
DNI	−4	SC × μ	50
DHI	−4	0.95 × SC × μ1.2 + 50	50

. By comparing the minute observational values of each radiation component with their respective thresholds, unreasonable radiation values exceeding the thresholds were removed.

3.

Solar tracker malfunction test

Conditions with high aerosol loading and cloudiness will lead to the malfunction of the solar tracker. The threshold used to test for solar tracker malfunction was that recommended by Liu et at. [66], which is proven to be effective in detecting potential tracking problems.

First, the potential clear sky GHI (i.e., GHI_clear) is calculated using the following formula:

GHI_clear = 850 μ^1.2

(1)

To calculate Rayleigh DHI(DHI_R) using the method proposed by Long and Shi [68], we first calculate atmosphere pressure P_S following Formula (2):

P_s = 1013 (1 − Altitude/44,300)^5.256

(2)

DHI_R = 209.3μ − 708.3μ² + 1128.7μ³ − 911.2μ⁴ + 287.85μ⁵ +0.046725μ × P_s

(3)

where Altitude is elevation value. When SZA < 87.5° and GHI/GHI_clear > 0.85, DNI should exceed 2 W·m⁻². When SZA < 87.5°, GHI/GHI_clear > 0.85, and DHI/GHI > 0.85, DNI should exceed 15 W·m⁻², and DHI should be less than DHI_R.

4.

Comparison test

Previous studies have indicated that a stable relationship exists among various radiation components [24,70,71]. For instance, GHI is equivalent to the sum of DHI and the vertical component of the DNI. Therefore, to assess the consistency among GHI, DNI, and DHI, and to evaluate the deviation from an ideal closure, a surface reference radiation network closure test was employed following the formulas below [66]:

When SZA is less than or equal to 75° and DNI·μ + DHI is greater than 50 W·m⁻², the ratio of GHI to (DNI·μ + DHI) should fall within the range of 0.92 to 1.08. When the SZA exceeds 75° and DNI·μ + DHI is greater than 50 W·m⁻², the ratio of GHI to (DNI·μ + DHI) should be between 0.85 and 1.15. When the SZA is less than 75° and GHI exceeds 50 W·m⁻², the ratio of DHI to GHI should be less than 1.05. Similarly, when the SZA exceeds 75° and GHI is greater than 50 W·m⁻², the ratio of DHI to GHI should be less than 1.10.

2.2. Satellite Products

2.2.1. Himawari-8(H8) Satellite Products

The H8 satellite, operated by the Japan Meteorological Agency (JMA), is a geostationary meteorological satellite equipped with an advanced Himawari Images (AHIs) system, which provides measurements in sixteen bands comprising three for visible wavelengths, three for near-infrared wavelengths, and ten for infrared wavelengths. This satellite offers global solar radiation products with a remarkable temporal resolution of 10 min (and every 2.5 min over Japan) and spatial resolutions ranging from 0.5 to 2 km, facilitating the study of solar radiation in localized regions [72,73,74]. Its high temporal resolution enables the quasi-real-time monitoring of solar radiation, facilitating timely assessments of solar energy availability [1,41,75,76,77].

The solar radiation retrieval algorithm that was employed originates from Frouin and Murakami [78], which utilizes a plane-parallel radiation-transfer theory and distinguishes between the effects of a clear atmosphere and clouds. This algorithm operates under the assumption that the planetary atmosphere can be represented as a clear atmosphere situated above a cloud layer [73]. The solar radiation reaching the land surface (E) can be derived using the following formula:

$$\mathrm{E}=E_{clear}(1-A)({1-A_s)}^{-1}{(1-S_{a}A)}^{-1}$$

(4)

where E_clear represents the solar flux arriving at the surface when there is no reflection or absorption by the cloud/surface layer; A denotes the albedo of the cloud/surface layer; A_s signifies the albedo of the ocean surface; S_a is the spherical albedo. For a detailed description of the calculation procedure, please refer to Frouin and Murakami [78].

The H8 Level-3 hourly products offer two types of solar radiation products at spatial resolutions of 1 and 5 km. The former covers latitudes ranging from 24°N to 50°N and longitudes from 123°E to 150°E, while the latter extends from 60°S to 60°N and from 80°E to 160°W. This study utilizes a spatial resolution of 5 km. The adopted products encompass GHI, DNI, DHI, and other related parameters such as SZA.

2.2.2. MODIS Products

The MODIS solar radiation product MCD18 provides high-spatiotemporal-resolution satellite data, offering continuous observational data globally [79,80,81,82]. The principle behind MCD18 solar radiation computation involves utilizing MODIS product data with prototype algorithms to derive top of atmosphere (TOA) reflectance through multi-temporal features and subsequently calculate solar radiation using lookup tables [39,83]. The lookup table considers different types of aerosol and cloud loadings at a variety of illumination/viewing geometries. Two major steps in the lookup table are the estimation of TOA spectral reflectance of a given value of atmospheric optical depth and the calculation of incident solar radiation. The atmospheric optical depth refers to the aerosol optical depth for the clear-sky cases and the cloud optical depth for the cloudy-sky cases. The TOA spectral reflectance (R(λ)) for a given wavelength λ can be calculated using the following formula:

$$R\left(\lambda \right)=R_0\left(\lambda \right)+\frac{r\left(\lambda \right)}{1-r\left(\lambda \right)\rho \left(\lambda \right)}\cos ({\theta }_s)\gamma (\lambda )/\pi$$

(5)

where r(λ) indicates surface spectral reflectance, θ_s is SZA, and R₀(λ), ρ(λ), and γ(λ) are path reflectance, atmospheric spherical albedo, and transmittance for the spectral band, respectively.

The surface incident solar radiation (F) depends mainly on atmospheric properties, but also, to a lesser extent, on surface reflectance, demonstrated by the formula as a function of atmospheric and surface parameters, as follows:

$$F=F_0+\frac{r\rho }{1-r\rho }{E}_0\mathrm{c}\mathrm{o}\mathrm{s}({\theta }_s)$$

(6)

where r, θ_s, ρ, and γ are the same as in Formula (5), F₀ is path irradiance, and E₀ is extraterrestrial solar broadband irradiance.

The parameters in the above two equations can be simulated offline with the numerical atmospheric transfer models. More details on this satellite retrieval algorithm can be found in [39,84]. In this study, MCD18A1 version 6, a MODIS Terra and Aqua combined gridded Level-3 product with a temporal resolution of 3 h and a spatial resolution of 5 km is utilized.

2.3. Evaluation Methods

To ensure comparability between ground observations and satellite-derived data, we excluded ground observation data for hours without corresponding satellite data. This approach ensured our analysis focused only on overlapping time periods between the ground observations and satellite datasets. The overlapping hours between ground observations and satellite products were 8 a.m., 11 a.m., 2 p.m., and 5 p.m. The available hourly average values primarily reflect daytime solar radiation. To obtain daily average values, we calculated the day length for each day and converted the daytime solar radiation to a daily value by multiplying the daytime solar radiation by the day length and then dividing this by 24 h.

Three statistical indicators, R-squared (R²), Root Mean Square Error (RMSE), and Bias, were employed to evaluate the accuracy of satellite-derived solar radiation products by assessing the discrepancy between satellite datasets and ground-based observations. R² ranges from 0 to 1 and quantifies the goodness of fit between satellite datasets and ground observations, with a higher R² value indicating a greater level of agreement, implying that the satellite datasets effectively capture much of the variability present in the observed data. RMSE offers a quantitative measure of the overall accuracy of the satellite data relative to the observed data, with a lower RMSE indicating a closer match between satellite data and observations. Bias represents the average difference between the satellite data and observed values, with a positive bias indicating that the satellite data are higher than the observed values, and vice versa. These methods are extensively employed in evaluation research and further details can be found in the relevant literature [76,77,78].

3. Establishment of Ground Measurements

3.1. Data Quality of BNU Site Measurement

The data quality assessment results for solar radiation measurements at the BNU site are presented in

Table 3. Quality control results of minute-level radiation observational data from 2020 to 2022. The ratios represent the data coverage within all available data that did not pass the test.

Year	Components	Sample Size	Missing Rate	Physically Possible Limits Test	Solar Tracker Malfunction Test	Comparison Test
2020	GHI	527,040	0	0	0.92%	0.98%
	DNI			0
	DHI			0
2021	GHI	525,527	0.01%	0.02%	1.49%	9.78%
	DNI			0
	DHI			0
2022	GHI	116,640	77.69	0.01%	0.61%	18.76
	DNI			0
	DHI			0

. It is apparent that the missing data rate in 2022, with respect to the data extraction date, is notably high, reaching a ratio of 77.69%, while both 2020 and 2021 demonstrate negligible missing values. Regarding physically possible limits, both DNI and DHI passed this assessment. However, a small portion of GHI, constituting 0.02% of the available data in 2021 and 0.01% in 2022, did not meet the physically possible limits. Table 3 further indicates that 0.92% of data in 2020, 1.49% in 2021, and 0.61% in 2022 did not pass the solar tracker malfunction test of the available data.

In terms of the comparison test, Table 3 indicates that, within the available data, 0.98% in 2020, 9.78% in 2021, and 18.76% in 2022 did not pass the comparison test. Figure 2 illustrates scatter plots of minute-level GHI and the sum of DNI multiplied by the cosine of solar zenith angle (μ) plus DHI (DNI·μ + DHI) data, in both their raw and quality-controlled forms, spanning from 2020 to 2022. It is evident that although there is no significant change in the R2 of GHI and DNI·μ + DHI after the aforementioned quality control process, there is a notable decrease in the RMSE.

In 2020, the RMSE of GHI and DNI·μ + DHI in the original data was relatively large, reaching 12.6 W·m⁻², with some scattered values deviating noticeably from the 1:1 line. The quality control process effectively eliminated these scattered values, resulting in a decrease in RMSE. In 2021, the quality of the originally observed solar radiation data was poor, as evidenced by a large number of observed values deviating from the 1:1 line, predominantly manifesting as GHI values smaller than the sum of DNI·μ + DHI. The RMSE reached 38.41 W·m⁻². However, the quality control process successfully rectified a large number of deviations, significantly reducing the RMSE to only 6.8 W·m⁻². Similarly, in 2022, the quality of the original observed solar radiation data was suboptimal, with an RMSE of 30.88 W·m⁻². However, the quality control process notably reduced the RMSE to 6.19 W·m⁻², thereby ensuring the accuracy and reliability of the data.

3.2. Establishment of Ground Measurements at Different Time Scales

Based on the quality-controlled, minute-level, solar radiation data, hourly solar radiation data were calculated, and subsequently, daily average solar radiation was computed based on the hourly solar radiation data. First, an assessment was made of the hourly observations contained in each date with available data from 2020 to 2022, as shown in Figure 3. From the figure, it can be seen that the data in 2020 are relatively complete, with almost every day ensuring a full 24 h observation, and there are few days with fewer than 24 h of observation, mainly concentrated in April. The data in 2021 are relatively complete, but in the first half of the year, there are many dates with fewer than 24 h of hourly observations, with some dates having only a dozen hours of observation. The data in 2022 are limited to the first three months, with fewer observations per day ranging from 18 to 24 h.

To further calculate the daily average solar radiation, considering the distribution of hourly data volume (Figure 3), we determined that when the hourly observations exceed 80% (approximately 19 h) per day, we can compute the average daily solar radiation based on hourly averages. The obtained time series of daily average solar radiation from 1 January 2020 to 22 March 2022 is shown in Figure 4. From Figure 4, it can be observed that all radiation variables exhibit significant seasonal variations. The daily GHI ranges from a minimum of only a few W·m⁻² to a maximum of around 350 W·m⁻² from 2020 to 2022. The DNI ranges from a minimum of 0 to a maximum of around 280 W·m⁻², while the DHI shows a smaller range of variation, with a maximum of only around 170 W·m⁻².

4. Evaluation of Satellite Retrieval Product Using Ground Measurements

Based on the solar radiation observational data from 1 January 2020 to 22 March 2022, we conducted an evaluation of the widely used solar radiation satellite products, with the focus on hourly and daily solar radiation data retrieved from the H8 and MODIS MCD18 satellite products. Figure 5 illustrates the high R² values between in situ observed hourly GHI and H8, as well as the MCD18 datasets, suggesting a strong linear relationship among the three datasets. Specifically, the R² between observed GHI and H8 is 0.89, indicating a robust linear correlation between the solar radiation data from H8 and the observed GHI. Meanwhile, the R² value between the observed GHI and MCD18 is slightly lower, at 0.81, though still demonstrating a strong correlation with a tendency towards overestimation. The RMSE between H8 and the observed GHI is calculated at 86.58 W·m⁻², with a bias of 51.79 (21.05%) W·m⁻². In comparison, the RMSE between the MCD18 and observed GHI is higher, at 113.93 W·m⁻², with a bias of 17.49 (7.11%) W·m⁻². The smaller RMSE between the observed GHI and H8 suggests a higher accuracy in H8 data compared to MCD18, indicating a closer fit to the observed GHI.

The scatterplot in Figure 6 illustrates the daily solar radiation comparisons among the observed GHI, H8, and MCD18. Compared to the hourly analysis, there is a significant decrease in RMSE values. For instance, the RMSE value between H8 and the observed GHI decreases from 86.58 W·m⁻² at the hourly scale to 37.68 W·m⁻² at the daily scale. Similarly, bias decreases notably from 51.79 W·m⁻² at the hourly scale to 27.20 W·m⁻² at the daily scale. Additionally, the RMSE value between MCD18 and the observed GHI decreases from 113.93 W·m⁻² to 40.96 W·m⁻². Likewise, bias decreases from 17.49 W·m⁻² at the hourly scale to 11.30 W·m⁻² at the daily scale. The reason why MODIS MCD18 has a larger RMSE than H8 is as follows. H8 is based on observations from a geostationary satellite, providing a high temporal resolution. In contrast, MODIS MCD18 is based on data from polar-orbiting satellites (Aqua and Terra), which pass over the Beijing region, at most, twice in 24 h. This limited frequency means that MODIS MCD18 cannot account for changes in cloudiness between the observation times. According to the documentation (https://lpdaac.usgs.gov/documents/106/MCD18_ATBD.pdf (accessed on 20 October 2023), the three-hourly MODIS MCD18 product is generated by keeping AOD and cloud properties fixed in the time interpolation. Notably, both H8 and MCD18 exhibit positive biases when compared to the observed GHI, suggesting overestimation errors in both satellite datasets.

Through Figure 7, the hourly distribution of solar radiation in each month can be obtained. Overall, the observational GHI, H8, and MCD18 exhibit similar data distributions in most months. For the majority of months, the three datasets demonstrate similar distribution characteristics. For instance, in January, the median values for observational GHI, H8, and MCD18 are 128.84 W·m⁻², 169.26 W·m⁻², and 117.43 W·m⁻², respectively. Throughout all months, H8 consistently has the highest median value, while MCD18 generally has the lowest. It is notable that, during certain months, such as April, May, and August, H8 and MCD18 show substantial deviations in median values compared to observations. For example, in August, the median values for the observed GHI, H8, and MCD18 are 354.76 W·m⁻², 432.01 W·m⁻², and 326.26 W·m⁻², respectively. These differences may be attributed to seasonal biases in satellite data and the geographical location of ground observation stations. Overall, H8 and MCD18 satellite data demonstrate good consistency with ground-based GHI observations across multiple time scales.

Figure 8 illustrates the differences between the observed hourly GHI and the radiation products from H8 and MCD18 datasets at different SZA values. Across all SZA categories, the range of differences between observed hourly GHI and H8 is relatively small compared to that between observed GHI and MCD18, but the median differences deviate significantly. Especially within the categories of 0°–30° and 30°–60° SZA, the median differences between H8 and the observed GHI are notably higher than those in other categories. This suggests that there are positive systematic biases in H8 predictions within these SZA ranges. This discrepancy may be related to the radiation transmission models and input parameters used in H8 radiation products. In reality, factors such as SZA and atmospheric pollution can affect the scattering and absorption of solar radiation in the atmosphere.

Regarding the differences between the observed GHI and MCD18 within the SZA range of 75°–90°, the differences are relatively small, with the median close to zero, considering the rather small GHI when the solar elevation is low. However, as the SZA increases from 0°, the differences also increase. Especially within the ranges of 30°–60° and 60°–75°, the range of differences widens, indicating significant discrepancies between MCD18 predictions and the observed GHI at these angles. This increase in differences may be related to the increase in SZA. As the SZA increases, the angle at which solar radiation reaches the Earth’s surface decreases, increasing the likelihood of cloud cover. This is due to the fact that, at a higher SZA, sunlight traverses a longer path through the atmosphere, making it more susceptible to cloud obscuration or scattering. Additionally, algorithms for deriving downwelling solar radiation from satellite data rely on plane-parallel radiative transfer models, treating radiative transfer as a one-dimensional process (vertical direction only). Consequently, these algorithms may not accurately account for complex 3D cloud effects, such as situations where clouds in neighboring pixels block the sun. This can lead to significant differences between the observed GHI and the predicted values from satellite-derived radiation models.

Figure 9 depicts the relative differences between the observed hourly GHI and the corresponding estimates from the H8 and MCD18 datasets, categorized by SZA. These relative differences, expressed as percentages, were computed by dividing the absolute differences illustrated in Figure 9 by the mean observed GHI within each SZA category. Within the SZA ranges of 0°–30° and 30–60°, both H8 and MCD18 datasets exhibit relatively stable relative differences, as depicted in Figure 8. However, notable variations emerge within the SZA range of 75°–90°, particularly for the H8 data, where the relative differences escalate significantly. Similarly, the MCD18 dataset demonstrates substantial increases in relative differences within the SZA intervals of 60°–75° and 75°–90°. The broader range of relative differences observed in both datasets indicates a decline in the accuracy of solar radiation estimates within these SZA intervals, despite the seemingly small absolute differences depicted in Figure 9. Furthermore, the expanded range of relative differences in the MCD18 dataset corroborates its lower accuracy when compared to the H8 dataset.

5. Discussion and Conclusions

In this study, the accuracy of two high-resolution satellite products, namely H8 and MODIS MCD18, were assessed, utilizing data from a novel solar radiation observation system established at the Beijing Normal University (BNU) station in the urban area of Beijing in 2017. Rigorous quality control protocols were implemented to generate reliable in situ observation datasets at a temporal resolution of one minute, covering the period from 1 January 2020 to 22 March 2022. Utilizing minute-level observational data as a foundation, datasets of surface solar radiation at different temporal scales (hourly, daily, monthly, and yearly) were computed and subsequently leveraged to evaluate the performance of H8 and MODIS products. The evaluation results demonstrated that both H8 and MODIS MCD18 solar radiation products exhibited strong consistency with observational data across multiple time scales, with R² values of 0.89 and 0.81, respectively. The RMSE between H8 and observed GHI was calculated at 86.58 W·m⁻², with a bias of 51.79 W·m⁻². In comparison, the RMSE between MCD18 and observed GHI was higher, at 113.93 W·m⁻², with a bias of 17.49 W·m⁻². The smaller RMSE between observed GHI and H8 suggested a higher accuracy in H8 data compared to MCD18, indicating a closer fit to the observed GHI. However, both satellite datasets showed certain overestimation biases, with H8 overestimating by approximately 21.05% and MCD18 by approximately 7.11%, with H8 exhibiting a more pronounced overestimation tendency. Additionally, solar zenith angles significantly influenced the accuracy of satellite products. Notably, at solar zenith angles of 0°–60°, significant differences compared to the surface measurements were observed for both MCD18 and H8 predictions, which were particularly evident for H8.

The inadequate estimation of satellite solar radiation products has been widely reported [39,82], with various underlying reasons for this having been identified. Tong et al. [80] attributed the overestimation of MCD18 in China to the 3D effects of clouds and the spatial scale impact, as evidenced by a strong correlation between RMSE and spatial scale, ranging from 160.43 to 146.02 W m⁻². Shi et al. [75] linked the positive bias of H8 solar radiation in the North China Plain to the underestimation of aerosol optical depth, and the negative bias in high elevation stations to improper atmospheric profile inputs. Damiani et al. [72] similarly observed H8 overestimation compared to ground observations conducted in Japan. They identified cloud variability as the primary source of uncertainty in satellite radiation estimates, followed by direct effects attributed to aerosols and bright albedo. A study evaluating surface downward shortwave radiation in the Tibetan Plateau attributed the insufficient performance of H8 and MCD18 products to the cloud parameterization, rather than aerosol impact [85]. Additionally, complex terrain [86] and climate zones [87] may also influence satellite-based solar radiation estimations. Other deficiencies in the parameterizations of aerosols and water vapor, as well as errors in atmospheric and surface properties for the retrieval algorithms, may also contribute to the estimation inaccuracies [37]. The findings of this study underscore the importance of validating satellite remote sensing-derived solar radiation data using ground observations, as satellite data may be subject to algorithmic limitations and exhibit variations under different atmospheric conditions. Integrating multiple ground-based observations for the comparative validation of satellite products is crucial for enhancing their accuracy. Furthermore, to improve the accuracy of satellite products, incorporating additional geostationary data capable of capturing solar radiation diurnal variability, as exemplified by the Clouds and Earth Radiant Energy System (CERES) product, is also essential [39].

The obtained observational data obtained firsthand in this study underwent rigorous quality control, leading to the creation of a high-quality dataset spanning multiple temporal scales, encompassing three surface solar radiation components (GHI, DNI, and DHI). This dataset provides fundamental data support for research on surface solar radiation variation mechanisms, future energy utilization, environmental protection, and related studies in the Beijing urban area. Furthermore, this research has enhanced our scientific comprehension of the characteristics of surface solar radiation fluctuations in the Beijing urban area and evaluated commonly employed satellite retrieval products. The foundational data and insights derived from this study are instrumental in assessing the influence of human activities and urban development on solar radiation dynamics. They also facilitate research into solar energy utilization in the Beijing region, the formulation of climate change adaptation strategies, and the development of pertinent policies aimed at mitigating environmental degradation, all grounded in robust scientific evidence.

It is crucial to acknowledge that the uncertainties inherent in observational experiments may also influence the evaluation outcomes of satellite products. Throughout our observational experiments, several objective constraints were encountered, including unstable wired network connections, COVID-19 lockdown measures, the premature aging of equipment distribution boxes, and extended maintenance cycles. These challenges impeded real-time equipment observation, resulting in occasional data gaps and hindrances to achieving satisfactory results for the planned long-term data analysis. Future endeavors will prioritize the continuous enhancement of equipment management methods and the integration of additional observational parameters. These efforts aim to obtain more accurate and continuous observational data, building upon the groundwork laid in this study.