Surface ALbedo VALidation (SALVAL) Platform: Towards CEOS LPV Validation Stage 4—Application to Three Global Albedo Climate Data Records

Abstract

The Surface ALbedo VALidation (SALVAL) online platform is designed to allow producers of satellite-based albedo products to move to operational validation systems. The SALVAL tool integrates long-term satellite products, global in situ datasets, and community-agreed-upon validation protocols into an online and interactive platform. The SALVAL tool, available on the ESA Cal/Val portal, was developed by EOLAB under the framework outlined by the Committee on Earth Observation Satellites (CEOS) Working Group on Calibration and Validation (WGCV) Land Product Validation (LPV) subgroup, and provides transparency, consistency, and traceability to the validation process. In this demonstration, three satellite-based albedo climate data records from different operational services were validated and intercompared using the SALVAL platform: (1) the Climate Change Service (C3S) multi-sensor product, (2) the NASA MODIS MCD43A3 product (C6.1) and (3) Beijing Normal University’s Global LAnd Surface Satellites (GLASS) version 4 products. This work demonstrates that the three satellite albedo datasets enable long-term reliable and consistent retrievals at the global scale, with some discrepancies between them associated with the retrieval processing chain. The three satellite albedo products show similar uncertainties (RMSD = 0.03) when comparing the best quality retrievals with ground measurements. The SALVAL platform has proven to be a useful tool to validate and intercompare albedo datasets, allowing them to reach stage 4 of the CEOS LPV validation hierarchy.

1. Introduction

Land Surface Albedo (SA), defined as the ratio of the radiant flux reflected from the Earth’s land surface to the incident flux on it, is a parameter of critical importance in understanding both climate and vegetation dynamics [1], and plays a significant role in quantifying the surface energy balance and parameterizing global and regional climate models [2]. SA, established as an Essential Climate Variable (ECV) by the Global Climate Observing System (GCOS) [3], connects the land surface and climate system through the regulation of the shortwave energy exchange [4,5,6].

Many SA products derived from satellites have been developed and made available to the user community over the last 40 years [7,8] thanks to the availability of well-characterized and calibrated satellite data, and pressing demands from the user community to have access to consistent and ready to use products. The current state-of-the-art method for the computation of surface albedo is to use a Bidirectional Reflectance Distribution Function (BRDF) kernel-based approach for the development of hemispherical albedo products from a number of multi-angular space-borne sensors. This approach is a pragmatic and cost-effective solution to the surface brightness inversion problem of operational programs that are constrained by the need to process extensive amounts of satellite data in near real time. Such an approach was adopted for near real time retrieval of albedo from ADEOS/POLDER [9], Terra + Aqua/MODIS [10,11], MSG/SEVIRI, and MetOp/AVHRR [12,13,14,15,16] in the framework of the Land Surface Analysis Satellite Application Facility (LSA SAF) [17,18], from SPOT/VGT and PROBA-V in Copernicus Global Land Service (CGLS) [19] and Copernicus Climate Change Service (C3S) [14,20], and recently adopted for Sentinel-3 data [21] in the C3S.

Due to the multitude of albedo products available, users face a complex situation because of the spatial and temporal discrepancies among them [13,22,23,24]. Therefore, product quality needs to be assessed, the product’s compliance with requirements must be known, and the user should know to what extent a product is suitable for their specific application [25]. Thus, the availability of reliable in situ data for direct validation of remote sensing ECV products is a key scientific requisite for users to make effective decisions [26] about the utility of a given ECV.

The Land Product Validation (LPV) subgroup [27] of the Committee for Earth Observation Satellites (CEOS) Working Group on Calibration and Validation (WGCV) (the so-called CEOS LPV) coordinates the quantitative validation of satellite-derived land products, focusing on standardized intercomparisons and evaluation across products from different satellites, algorithms, and agency sources. Validation is defined as the process of independently assessing and evaluating the quality of the data products from the system outputs [28]. In terms of satellite-based land products, validation refers to the assessment and quantification of their accuracy and uncertainties via analytical comparisons with reference datasets. In 2019, the CEOS LPV subgroup compiled a global surface albedo product validation best practices protocol [29], a community-agreed-upon document on validation best practices for satellite-based albedo products. The validation protocol is mainly based on two strategies: comparison of satellite products versus in situ data (direct validation) and intercomparison of satellite products (indirect validation).

A hierarchical approach that identifies four land product validation stages (

Table 1. The CEOS LPV validation stages [27].

Stage	Description
0	No validation. Product accuracy has not been assessed. Product considered beta.
1	Product accuracy is assessed from a small (typically < 30) set of locations and time periods by comparison with in situ or other suitable reference data.
2	Product accuracy is estimated over a significant (typically > 30) set of locations and time periods by comparison with reference in situ or other suitable reference data. Spatial and temporal consistency of the product, and its consistency with similar products, has been evaluated over globally representative locations and time periods. Results are published in the peer-reviewed literature.
3	Uncertainties in the product and its associated structure are well quantified over a significant (typically > 30) set of locations and time periods representing global conditions by comparison with reference in situ or other suitable reference data. Validation procedures follow community-agreed-upon good practices. Spatial and temporal consistency of the product, and its consistency with similar products, has been evaluated over globally representative locations and time periods. Results are published in the peer-reviewed literature.
4	Validation results for stage 3 are systematically updated when new product versions are released or as the inter-annual time series expands. When appropriate for the product, uncertainties in the product are quantified using fiducial reference measurements over a global network of sites and time periods (if available).

) was adopted by CEOS LPV, following CEOS validation principles [30]. Stage 3 implies that uncertainties are evaluated over a significant set of locations (>30), using community consensus protocols, where spatial and temporal consistency is evaluated over globally representative locations. Stage 4 includes systematic and regular update of stage 3 validation results when new product versions are released, or as the time series expands. Stage 4 implies an operational validation that ensures time series are systematically validated.

A current review of the literature revealed only a few long-term global SA validation exercises [31]. Among them, global albedo products from MODIS observations [22,32,33], PROBA-V [24], SPOT/VGT, and EPS/AVHRR [15] were validated to the CEOS LPV stage 3 hierarchy level. However, in most cases, the existing validation results are not directly comparable, as validation practices are rather diverse in terms of methods, reference data and locations, lack of traceability and transparency, spatiotemporal coverage, scaling, metrics and target accuracies, resulting in a considerable lack of consistency in validation outputs [8]. These inconsistencies support the movement of the future validation activities into an operational validation workflow that would align all the various aspects, thus allowing for consistent and traceable validation of the existing satellite-based albedo products.

An operational validation workflow to validate remote sensing global terrestrial ECV products should consider, at least, four key components [8]: (1) the long-term Climate Data Records (CDRs) of satellite-based ECVs; (2) a set of representative, reliable, and globally distributed in situ measurements; (3) a suitable standard assessment framework based on community-agreed-upon validation best practice protocols; and (4) an online validation platform that provides open-access tools to generate standardized validation reports.

In this context, the Surface ALbedo VALidation (SALVAL) web tool [34] has been developed, integrating these four key components, in order to facilitate the evaluation of global albedo products derived from satellite data in accordance with the CEOS LPV validation good practice [29]. The main objectives of the SALVAL tool are: (i) to provide transparency and traceability (i.e., reproducibility) to the validation process; (ii) to integrate the protocols and metrics from the CEOS LPV albedo product validation best practices document [29] into a tool within which the user can analyze the different validation criteria; (iii) to provide a platform where new versions or new products can be consistently evaluated; and (iv) to facilitate the update of validation results and to achieve CEOS LPV validation stage 4. The SALVAL tool currently includes existing CDRs and a representative network of ground observations, and, additionally, it also allows for user-friendly result updates for new products or periods.

The objective of this paper is two-fold: (1) to introduce the SALVAL tool and functionalities; and (2) to show an application of validation and intercomparison of three existing surface albedo CDRs: the NASA’s MCD43A3 Collection 6.1 (C6.1), C3S SPOT/VGT and PROBA-V V2, and Beijing Normal University’s (BNU) GLASS albedo products. The remainder of this paper is structured as follows: Section 2 describes the functionalities of the SALVAL tool; Section 3 presents the specifications and validation results from an intercomparison and validation exercise including these three remote sensing SA products; and Section 4 and Section 5 discuss the results and provide the conclusions, respectively.

2. Methods and Datasets: The SALVAL Tool

2.1. Validation Methodology

The validation methodology follows the CEOS LPV best practices protocol for the validation of satellite-derived albedo products [29]. The methods are divided into four categories: product intercomparison, direct validation, precision, and stability. Each category has several sub-criteria, including completeness, spatial consistency, temporal consistency, accuracy, precision, and stability.

Table 2. Summary of SALVAL validation methods.

Category	Criteria	Methods
Product inter- comparison	Completeness	Gap size distribution (spatial and temporal) and gap length.
	Spatial consistency	Mean residual and mean difference maps, percentage of cases within requirements
	Temporal consistency	Temporal profiles and histograms of cross-correlation
	Overall analysis	Product histograms, difference histograms, scatterplots (APU validation metrics) and box plots of bias and Root Mean Square Deviation (RMSD) per bin
Direct validation	Temporal realism	Temporal evolution of the satellite-derived products vs ground data
	Overall analysis	Scatterplots (APU validation metrics)
Precision	Intra-annual precision	Median 3-point difference (smoothness)
	Inter-annual precision	Median absolute deviation over desert calibration sites
Stability	Stability	Slope of the 10-year linear regression over desert calibration sites

describes the validation methods associated with each criterion. The definitions of the completeness, precision, uncertainty, and accuracy are drawn from the experimental recommendations of the Joint Committee for Guides in Metrology (JCGM) regarding the expression of uncertainty in measurement [35] and from the GCOS [36]. The summary of accuracy, precision, and uncertainty (APU) validation metrics is provided in

Table 3. APU validation metrics.

Statistics	Comment
N	Number of samples. Indicative of the strength of the validation.
B	Mean Bias. Difference between average values of x and y. Indicative of accuracy and offset. Bias (%) is the relative mean bias between the average of x and y.
MD	Median (i.e., 50th percentile) deviation between x and y. MD is the CEOS LPV good practice reporting of the accuracy. MD (%) is the relative MD between the average of x and y.
STD	Standard deviation of the pair differences. Indicates precision. STD (%) is the relative STD between the average of x and y.
MAD	Median (i.e., 50th percentile) absolute deviation between x and y. MAD is the CEOS LPV good practice indicator of precision. MAD (%) is the relative MAD between the average of x and y.
RMSD	Root Mean Square Deviation. RMSD is the square root of the average of squared errors between x and y. The RMSD is the CEOS LPV good practice reporting of uncertainty. RMSD (%) is the relative RMSD between the average of x and y.
R	Correlation coefficient. Indicates descriptive power of the linear accuracy test. Pearson coefficient is used.
MAR	Slope and offset of the Major Axis Regression (MAR) linear fit. Indicates possible bias.

. The SALVAL tool allows users the possibility of directly visualizing the results by category and criteria, or they can generate a validation report document of the results.

The validation exercise is based on two main approaches: indirect validation (i.e., satellite product intercomparison) and direct validation. The indirect validation offers a means of assessing the discrepancies (systematic or random) between products and allows the evaluation of the metrics at a global scale due to the limited availability of ground measurements. For satellite-derived albedo, direct validation involves comparison of satellite products with the albedo measured from in situ tower-based instruments. Direct validation enables the assessment of uncertainties, and it may be argued that only such methods can be considered actual validation in the field of remote sensing [23]. SALVAL uses two different sampling strategies for product intercomparison and direct validation, as described below. The list of sites is available at the CEOS Cal/Val portal [37].

The quantitative and qualitative product-to-product intercomparisons are performed over the 720-site land validation (LANDVAL) network [17,28] (shown in Figure 1), which was designed to globally represent the variability of land surface types. This network also includes 19 well-known desert calibration sites [29] for the precision and stability evaluation. Product intercomparison of different satellite products needs a common spatial and temporal sampling. The comparison is performed at 3 km × 3 km spatial support area with the aim of reducing the co-registration errors between products and differences in their sensor point spread function which determines the actual footprint of the data [38,39]. The temporal frequency used corresponds to that of the products under evaluation, which are compared with the closest date of the reference satellite products.

The 99 sites of the Representativeness-Evaluated ALbedo Stations (REALS) dataset (also depicted in Figure 1) are used for the direct point-to-pixel validation (see Section 2.3). In order to compare tower in situ measurements to satellite-based albedo products, the generation of clear blue-sky satellite albedo [40] is performed as the weighted average of black-sky and white-sky retrieved albedos by the fraction of diffuse downwelling shortwave radiation from the ground station at a particular illumination and atmospheric condition. The test of spatial representativeness of the in situ albedometer footprints was performed for the satellite pixel resolution of interest according to in situ measurement standards [41,42]. This exercise is usually performed at a resolution of 1 km considering only sites that are homogeneous over a footprint area of at least a 1 km² around the albedo in situ station. More details about the test of the spatial representativeness can be found below in Section 2.3. Temporal averages of daily ground data are computed to allow comparison with satellite products generated in composited time intervals.

It is important to highlight the fact that the sampling of LANDVAL and REALS sites allows the validation exercise to reach CEOS LPV stage 3, as uncertainties can be quantified over a significant set of in situ stations (99 REALS) and the spatial and temporal consistency with similar satellite datasets can be evaluated over globally representative locations (LANDVAL).

2.2. Satellite Datasets

Currently, the SALVAL platform incorporates satellite-based albedo datasets from existing programs, such as NASA MODIS [43], Copernicus C3S [20], BNU GLASS [44], or GlobAlbedo [45]. Additionally, the tool also allows for importing either a new product dataset or expanding the temporal coverage of existing products.

In this work, we selected three satellite-based CDRs with around 20 years of data from existing operational programs: NASA MCD43A3 C6.1, C3S multi-sensor V2, and BNU GLASS. These products include the total shortwave domain [0.3 μm, 4 μm], which is the most relevant albedo quantity in terms of energy budget. The total shortwave domain also includes visible [0.4 μm, 0.7 μm] and near-infrared [0.7 μm, 4 μm]. Additionally, different definitions of satellite albedo products exist according to the domain of directional integration [46]: the directional-hemispherical reflectance (DHR) or black-sky albedo (BSA or AL-DH), and the bi-hemispherical reflectance (BHR) or white-sky albedo (WSA or AL-BH). BSA is defined as the ratio of the radiant flux for light reflected by a unit surface area into the view hemisphere to the illumination radiant flux, when the surface is illuminated with a parallel beam of light from a single direction [47]. WSA is the ratio of the radiant flux reflected from a unit surface area into the whole hemisphere to the incident radiant flux of hemispherical angular extent [48]. The combination of both BSA and WSA in relation to the proportion of sky irradiance provides the actual albedo value, also called blue-sky albedo [40].

The three satellite albedo products provide both BSA at local solar noon and WSA for three broadband ranges (visible, NIR, and total shortwave).

2.2.1. NASA MCD43A3 C6.1

The MODIS BRDF/Albedo MCD43A3 C6.1 dataset, available from the LPDAAC [49], produces albedo quantities at a resolution of 500 m in a sinusoidal projection. These quantities have been produced daily since 2000 with a synthesis period of 16 days, using data from both the Terra and Aqua satellites. The MODIS albedo algorithm uses atmospherically corrected cloud-free reflectance data (the MOD/MYD09 product) to establish the best fit to a linear kernel-driven BRDF model. Observations flagged as “cloud”, “cirrus high” or “aerosol high”, or “very high solar zenith angles” are not utilized. The parametric BRDF model uses the Ross_Thick kernel for volumetric scattering and the Li_Sparse_Reciprocal kernel for geometrical scattering [47,50]. A full retrieval of the model is attempted if there are at least seven or more high-quality observations that are well-distributed over the viewing hemisphere during the 16-day synthesis period. When the number of observations is strictly less than 7 and strictly greater than 2, or if observations are not well sampled or do not fit the BRDF model well, though the number of observations is larger than 7, a backup algorithm (magnitude inversion) with prior information is used and the values are designated with a lower quality flag. A fill value is stored if the number of observations is strictly less than 3. Snow and snow-free albedos are processed separately depending on the ground condition of the day of interest. In addition, products at 30 arc second and 0.05 degree resolutions on a geographic lat/lon projection are also available for ease of use by modelers. Separate 30 arc second snow-free gap-filled products (MCD43GF) are also accessible [51]. The BRDF model parameters are then used for estimating spectral albedos from angular integration. The broadband albedos are then computed using the spectral to broadband conversion approach [52]. The MCD43 C6.1 products use an improved backup database [53], which is pixel-based, updated from the latest high quality full inversion, as opposed to the land cover-based BRDF database used in the previous Collection 5. In this work, the updated C6.1 version is being used, which incorporates the latest improved calibration coefficients and surface reflectance values.

MCD43A3 C6.1 SA products have reached CEOS LPV validation stage 3 [53]. Existing studies of the previous MCD43 Collections 5 and 6 indicate that the accuracy of the MODIS shortwave broadband albedo met the GCOS accuracy requirements (Max [5%, 0.0025]) for both snow-free and snow-covered surfaces [41,42,54,55].

2.2.2. C3S Multi-Sensor V2

The C3S V2 products, which are available in the C3S Climate Data Store (CDS, [56]), provide a CDR of global albedo estimates in a nearly 40-year record of satellite observations from 1981 to 2020, using multiple input datasets: NOAA/AVHRR from 1981 to 2005 (around 4 km pixel size), SPOT/VGT from 1998 to 2014 (1 km pixel size), and PROBA-V from 2014 to 2020 (1 km pixel size). The temporal frequency of the products is 10 days, built with a compositing window of 20 days. The dates of production are the 10th, 20th, and final days of each month.

C3S V2 version builds on V1 [57] and adds a multi-sensor aspect to the albedo products delivered so far. The retrieval algorithm [14,58] uses Top-of-Canopy (TOC) reflectance resulting from both the harmonized pixel classification approach (cloud, snow, and shadow pixels) and the Simplified Method for Atmospheric Correction (SMAC) algorithm [59], to improve satellite cross-consistency. Additionally, a spectral harmonization is performed, by creating TOC reflectance values as if they had been acquired with SPOT/VGT. The harmonized TOC reflectance values are processed to determine the coefficients of a semi-empirical kernel-based reflectance model, which accounts for the complete angular dependence of the bi-directional reflectance factor. This inversion step is performed using prior information and a climatology of surface BRDF. To estimate albedo using a multi-sensor time series, a second harmonization was conducted, using BRDF climatology data from SPOT/VGT. The algorithm [14] relies on a similar method to that of previous versions (Kalman filters and BRDF model fit) but with the addition of a reference BRDF (climatology derived from VGT BRDF) to (i) reduce the gaps in the time series, and (ii) introduce multi-sensor information in each albedo estimation to increase homogeneity among the datasets derived from each sensor. In the final steps, the spectral albedo values are determined from the angular integrals of the model functions using the retrieved parameter, and the narrow-to-broadband conversion is performed with a linear regression formula. A different set of narrow-to-broadband conversion coefficients is applied for snow-free pixels and for pixels flagged as “snow/ice” in the input data status map.

An independent scientific quality assessment of the C3S multi-sensor V2 products was performed [60], considering the whole CDR (1981–2020), achieving CEOS LPV validation stage 2. The V2 time series was demonstrated to be more consistent compared with V1 when changing input data in the transitions from AVHRR to SPOT/VGT and from SPOT/VGT to PROBA-V. Direct validation results of SPOT/VGT SA V2 showed positive bias (12.5%) and overall uncertainty (RMSD) of 0.048 in the comparison with albedo measurements from 15 homogeneous FLUXNET stations (2000–2005 period). The comparison of C3S PROBA-V SA V2 with field data for 20 homogeneous sites from the Copernicus Ground-Based Observations for Validation (GBOV) program (2014–2018 period) also showed positive bias (9.1%) and RMSD of 0.039

2.2.3. BNU GLASS V4

The Global LAnd Surface Satellites [61] (GLASS) product suite provides albedo products from 1981 to 2019 with a temporal resolution of 8 days, and is available from BNU [44]. Products from 1981 to 1999 are derived from AVHRR data, with a spatial resolution of 0.05° in the global Climate Modelling Grid (CMG) projection. The GLASS albedo products from 2000 to 2019 are derived from MODIS data, with a spatial resolution of 1 km in a tile-based sinusoidal projection.

The GLASS albedo products are generated in two steps. First, the albedo is directly retrieved from remote sensing data by employing the second simulation of a satellite signal in the solar spectrum (6S) atmospheric radiative transfer model (RTM) to simulate the TOA directional reflectance, calculating the broadband albedos based on the POLDER BRDF dataset, and then establishing a relationship between the TOA reflectance and surface broadband albedo using an angular bin regression method [62]. Intermediate products from this first step are merged to generate a unique and gap-filled final product based on the Statistics-based Temporal Filtering (STF) algorithm [63]. Additionally, the current version 4 data have been improved with respect to previous versions: the snow/ice BRDF model has been updated [64] and a water surface BRDF model has been adopted for the ocean surface as well as for mixed pixels of water/sea ice [65].

The preliminary evaluation of the GLASS albedo product [22] showed that it is a gapless, long-term, continuous and self-consistent dataset with an accuracy similar to that of the MODIS MCD43 C5 product. Recent validation efforts for version 4 [61] showed that products are consistent with MODIS MCD43A3 C6 over snow-free pixels, and overall uncertainty (RMSD) of 0.052 was found in comparison with tower-based observations from 53 spatially homogeneous global sites. The RMSD was reduced to 0.037 for snow-free pixels.

2.2.4. Summary and Quality Flags

Table 4. Characteristics of the global remote sensing SA products under study. GSD stands for the Ground Sampling Distance.

Product	Satellite /Sensor	Methodology	Broadband Definition	Frequency /Period	GSD /Projection	Reference
NASA MCD43A3 C6.1	TERRA + AQUA /MODIS	BRDF model inversion and angular/spectral integration	visible [0.3–0.7 μm] NIR [0.7–5.0 μm] total SW [0.3–5.0 μm]	Daily (*) /16 days	500 m /Sinusoidal	[10]
C3S V2	SPOT /VGT PROBA /VGT	BRDF model inversion and angular/spectral integration	visible [0.4–0.7 µm] NIR [0.7–4 µm] total SW [0.3–4.0 µm]	10 days /20 days using prior climatology BRDF	1 km /Plate Carrée	[14,58]
GLASS V4	TERRA + AQUA /MODIS	RTM + gap-filling	visible [0.3–0.7 μm] NIR [0.7–5.0 μm] total SW [0.3–5.0 μm]	8 days /16 days	1 km /Sinusoidal	[61]

(*) MCD43A3 C6.1 products are produced daily but are ingested into the SALVAL tool at a temporal frequency of 5 days.

summarizes the main features of the three SA products used in this study. Both MCD43A3 and C3S are based on a BRDF model resulting from the combination of two of the same models for defining the volumetric (Ross_Thick) and geometric (Li_Sparse_Reciprocal) kernels [47]. Unlike the MCD43A3 and C3S albedo products, which are based on inversions of BRDF model parameters, the GLASS albedo products are based on the direct-estimation method and represent surface albedo under general clear-sky atmospheric conditions.

The production frequency of C3S V2 and GLASS V4 is 10 and 8 days, respectively. MCD43A3 C6.1 products are originally produced daily, but are ingested into the SALVAL platform at a temporal frequency of 5 days, due to the limitations in memory storage and processing capabilities of the server.

The quality flag information for each product was used to filter low quality pixels (

Table 5. Quality flag information used to filter pixels flagged as “low quality”.

Product	Quality Control Used as “Best Quality”	Quality Control Used to Discard Pixels
MCD43A2 C6.1	Full BRDF inversion	Magnitude inversion
C3S V2	Land (bits 0–1 QFLAG) Normally processed (bit 7 QFLAG) ERR $\le$ 0.2 AGE $\le$ 20	Sea and continental water (bits 0–1 QFLAG) Algorithm Failed (bit 7 QFLAG) ERR > 0.2 AGE > 20
GLASS V4	Overall uncertainty ‘best quality’	Overall uncertainty ‘acceptable’, ‘with uncertainty’ or ‘fill value’

), and the SALVAL tool provides the results for both cases of “best quality” retrievals, and all retrieved valid pixels (quality flag not considered).

For MCD43A3 only “best quality” full BRDF inversions were considered, and magnitude BRDF inversions were discarded. Higher confidence is expected for a “full inversion” retrieval that is performed under good sampling of the viewing and illumination geometry for a grid location. The “magnitude inversion” is a backup algorithm, which performs generally well, but relies on prior estimates of the BRDF when insufficient observations are available to fully sample the viewing and illumination geometry. On the other hand, the C3S V2 pixels where the algorithm failed were not considered in this validation exercise. Additionally, two ancillary variables were also considered: the uncertainty (ERR) and the mean age (AGE, in number of days) of the observations used to produce the SA. The C3S V2 pixels with associated uncertainty of greater than 0.2 and an AGE greater than 20 were discarded, indicating excessive use of prior information [60]. Finally, in the case of GLASS V4, only pixels classified as “good overall uncertainty” were considered as “best quality”, while pixels flagged as “acceptable”, “with uncertainty” or “fill value” resulting from gap-filling methods were not considered.

2.3. Representativeness-Evaluated ALbedo Stations (REALS) Dataset

REALS is a database of sites with the objective to generate an extensive in situ dataset for direct validation purposes. The database has been defined as a combination of 99 sites with availability of ground data in the 2000–2020 period from existing networks and initiatives, such as Ground-Based Observations for Validation (GBOV) [66], Flux Network (FLUXNET) [67], the National Science Foundation’s National Ecological Observatory Network (NEON) [68], European Fluxes Database Cluster (EFDC) [69], Integrated Carbon Observation System (ICOS) [70], and Australia’s Land Ecosystem Observatory or Terrestrial Ecosystem (TERN) [71]. Some GBOV, FLUXNET, and EFDC sites incorporate measurements from the Baseline Surface Radiation Network (BSRN) [72] and its U.S. component known as the Surface Radiation Budget (SURFRAD) Network [73]. BSRN is considered the gold standard of albedo measurements according to the GCOS [36] and CEOS LPV albedo best practices protocol [29]. It is worth noting that 23 of these REALS sites are considered “Super Sites” endorsed by the CEOS LPV subgroup, meaning that they are well-characterized (canopy structure and biogeophysical variables) following well-established protocols, and are active in long-term operation, supported by appropriate funding and infrastructural capacity.

The albedo measured from a tower covers a circular footprint (dependent upon the tower height) that should be ideally equivalent to the pixel size of satellite estimation. However, satellite footprints are often much larger than the tower footprints. For that reason, the representativeness of the measurement, both within the tower footprint and in the surrounding landscape, is evaluated through geostatistical indices based on a semivariogram model [74,75], following current state-of-the-art protocols [33,41,42] and CEOS LPV recommendations. Four different geostatistical attributes have been used for this evaluation: relative coefficient of variation (R_CV), scale requirement index (R_SE), relative strength of the spatial correlation (R_ST), and relative proportion of structural variation (R_SV). They are combined in a condensed indicator of spatial representativeness: the standard score (ST, see Equation (1)). For those situations where the semivariogram estimator does not provide a good fit with a semi-spherical variogram, the first order score (RAW, see Equation (2)) can be adopted to evaluate the spatial representativeness [41]. Both scores are directly proportional to the representativeness or relative homogeneity of a site, so a higher score means that a ground site (point) is more suitable to be comparable to satellite-based measurements (pixel). Note that the cover does not have to be uniform, and can be a heterogeneous landscape, as long as that heterogeneous landscape is similar both within the tower footprint and the surrounding landscape.

$$\mathrm{ST}={\left(\frac{\left|{\mathrm{R}}_{\mathrm{CV}}\right|+\left|{\mathrm{R}}_{\mathrm{ST}}\right|+\left|{\mathrm{R}}_{\mathrm{SV}}\right|}{3}+{\mathrm{R}}_{\mathrm{SE}}\right)}^{-1}$$

(1)

$$\mathrm{RAW}={\left|2{ \mathrm{R}}_{\mathrm{CV}}\right|}^{-1 }$$

(2)

The methodology adopted for the evaluation of the representativeness of the sites is based on the estimation of the spherical semivariogram for different spatial resolutions (1, 1.5, and 3 km²). When the semivariogram has been estimated, geostatistical indices are calculated in order to quantify the level of representativeness of a site.

The spatial representativeness is estimated for each site of the REALS sites in different temporal conditions (leaf-off season and leaf-on season) using high-resolution Sentinel-2 imagery [76] for the B8 band, which is the most spectrally representative of the total shortwave [77]. Figure 2 shows an example of variogram fitting and ST estimation over two different sites of the REALS database, Desert Rock (DRAK) and Talladega National Forest (TALL). These results show more homogeneity or spatial representativeness in the case of TALL (ST = 8) in the leaf-on period than in DRAK (no seasonality) (ST = 0.96). Appendix A describes the ST summary of the REALS sites.

In order to choose a ST threshold for filtering non-representative or differing heterogeneous sites, an analysis of the variation in RMSD (uncertainty) of number of sites and samples between the NASA MCD43A3 C6.1 product and REALS sites was performed for the available period (2000–2020). Figure 3 shows the evolution of number of sites, number of samples, and RMSD as function of the ST score for the comparison between the MCD43A3 C6.1 satellite-derived product and REALS in situ measurements for the 2000–2020 period. According to the results, the RMSD tends to decrease when the ST threshold increases, but the numbers of sites and samples decrease. For this reason, a threshold of 1.5 for ST was selected as the filter in the REALS database because the RMSD tends to be stable at this score and the number of sites and samples discarded is reasonable. This threshold is similar to that used in previous works [24,33].

The main characteristics of REALS sites and ST scores for each site are summarized in Appendix A.

2.4. SALVAL Functionalities and Configuration

The SALVAL tool has three main functionalities: (i) to select both the product to be evaluated and the reference products based on existing datasets or importing a new test candidate product; (ii) to configure the validation exercise by selecting a set of user requirements, spectral region, spatial domain, and temporal domain of the study; and (iii) to run the validation exercise according to the previously set configuration, based on the protocols and metrics implemented from the CEOS LPV albedo protocol [29]. More information about the use of the tool can be found in the SALVAL user guide [34] or in Appendix B.

After the selection of the products to be evaluated, SALVAL requires the configuration of the validation exercise. Different sets of configurations should be introduced by the user, such as the period, the albedo type (black-sky or white-sky albedo) for product intercomparison, the user requirements for the evaluation of stability and accuracy, and the spatial region of the study (global or continental region).

The stability and accuracy results were evaluated against three predefined requirement levels (optimal, target, and threshold), which are used for both BSA and WSA products in all spectral broadband domains. We used default values within the SALVAL tool (

Table 6. Predefined accuracy and stability requirement levels used for SA validation.

	Optimal	Target	Threshold
Accuracy requirement	Max [5%, 0.0025]	Max [10%, 0.01]	Max [15%, 0.015]
Stability requirement	Max [1%, 0.001]	Max [2%, 0.002]	Max [3%, 0.003]

), which are based on a review of the existing user requirements for measuring global climate change [78], and from the Global Climate Observing System (GCOS) [36] and the World Meteorological Organization (WMO) [79]. The optimal accuracy level (Max [5%, 0.0025]) was selected according to the GCOS uncertainty threshold and is equivalent to the WMO goal. The target level is equivalent to the WMO breakthrough level, which is an intermediate level between “goal” and “threshold” (i.e., if achieved, would result in a significant improvement for the targeted application). The breakthrough level may be considered as an optimum, from a cost–benefit point of view, when planning or designing observing systems. Poor performances of the product correspond to values above the threshold levels (WMO minimum requirement). Figure 4 displays the selected uncertainty levels as a function of the product values.

In the recent update of GCOS requirements [80], a new goal uncertainty level of 3% was defined. The SALVAL platform allows modifying the predefined requirements as a function of the user needs.

The defined temporal domain is the 20-year period from 2000 to 2019. For this period, NASA MCD43A3 C6.1 and GLASS are based on MODIS data whereas C3S multi-sensor V2 products are based on input data from two different platforms (SPOT/VGT for 2000 to May 2014, and PROBA-V from May 2014 to 2019). The whole period is used for direct validation with ground data, and 10 years (2001–2010) is used for the stability evaluation. Product intercomparison and completeness are based on five years (2001–2005) of data, as the SALVAL platform is restricted for those exercises due to computational constraints. The spatial domain covers the whole globe and retrievals from all LANDVAL and REALS sites are included in the analysis.

Results are displayed for total shortwave black-sky albedo (AL-DH-BB) for product intercomparison. For the direct evaluation with in situ measurements, the SALVAL platform computes satellite blue-sky albedos as the weighted average of total shortwave black-sky and white-sky albedos by the fraction of diffuse downwelling shortwave radiation from the ground station.

3. Results

3.1. Product Completeness

The three products (C3S V2, GLASS V4, and MCD43A3 C6.1) show a similar spatial distribution of missing data when all pixels are considered (left side in Figure 5), with gaps mainly located at northern regions. However, when quality flags (see Table 5) are used (right side in Figure 5), MCD43A3 C6.1 is more restrictive, if only pixels based on high-quality full BRDF inversions are considered, compared to C3S V3 and GLASS V4.

The temporal evolution of missing values (Figure 6) is displayed at the different temporal resolutions being used in this effort (i.e., 10, 8, and 5 days for C3S, GLASS, and MCD43A3, respectively) showing similar trends, with the highest percentage of missing data during wintertime in the northern hemisphere (December and January). Products based on MODIS (MCD43A3 C6.1 and GLASS V4) and VGT (C3S V2) show maximum percentages of LANVAL missing data, typically around 20% and 10%, respectively. When only best quality observations are considered according to quality flags, the most (around 80%) missing values are found for MCD43A3 C6.1, and around 30% missing values are found for the other products (C3S V2 and GLASS V4) that incorporate gap-filling techniques in their algorithms.

3.2. Spatial Consistency

The spatial consistency is quantitatively assessed through the global distribution of residuals between pair of products. The residual represents the remaining discrepancies regarding the general trend between products, which means that systematic differences are not considered, depicting more clearly the patterns associated with the spatial distribution of retrievals [81]. Two products are considered spatially consistent when the residual lies within predefined uncertainty requirements. Figure 7 shows the spatial distribution of residuals (average value for 2001–2005 period) between pairs of evaluated products (C3S V2, GLASS V4 and MCD43A3 C6.1). The percentages of residuals within the predefined requirements are summarized in

Table 7. Percentage of mean residuals between pair of products (C3S V2, GLASS V4 and MCD43A3 C6.1) within each level of uncertainty requirements. Evaluation in the 2001–2005 period over LANDVAL sites using best quality total shortwave black-sky albedo (AL-DH-BB) retrievals.

Residual	Optimal	Target	Threshold	Non-Compliance
C3S V2 vs. GLASS V4	83.5%	98.1%	99.7%	0.3%
C3S V2 vs. MCD43A3 C61	72.8%	91.8%	95.4%	4.6%
GLASS V4 vs. MCD43A3 C61	89.6%	94.5%	94.9%	5.1%

.

Most residuals between pairs of products are within ±0.015, demonstrating overall good spatial consistency among them. The comparison between GLASS and MCD43A3 shows the higher percentage (around 90%) of evaluated samples within optimal requirements. The comparison of C3S and GLASS seems to display the better spatial consistency, with 98% of cases within target level and almost no non-compliant cases (0.3%). The results show that, in all comparisons, more than 90% of residuals are within target requirements, and typically less than 5% of cases are non-compliant.

3.3. Temporal Consistency

To assess the realism of the seasonal and inter-annual temporal variations, examples of temporal profiles of C3S V2, GLASS V4, and MCD43A3 C6.1 satellite products are qualitatively compared with ground data over REALS sites. It should be noted that two different time periods are displayed due to the different input data used to retrieve C3S V2 products: 2001–2005 (Figure 8, SPOT/VGT) and 2014–2019 (Figure 9, PROBA-V).

Good temporal agreement is found between satellite products and ground data over forest sites in the 2001–2005 period, properly reproducing the different situations: rapid changes due to snow events (CA_Oas, RU_Fyo), stable values over long periods (NL_Loo, RU_Fyo), and variation due to phenological changes (AUS_TUMB). C3S V2 products based on SPOT/VGT typically tend to provide higher values than ground data (and other satellite products) for most snow-free cases. GLASS V4 shows, on the other hand, some unexpected peaks (e.g., NL_Loo in January 2004) which are not captured by the other satellite products and ground data.

Good temporal agreement is also found between satellite retrievals and in situ data in the 2014–2019 period for different biome types, following similar temporal trajectories. C3S V2 (based on PROBA-V for this period) shows an overestimation compared with ground truth and the other satellite products for USA_SFSD (cropland) and SCBI (forest) sites, but better accordance with ground data over AUS_CPRM (grassland) and AU_ASM (forest) than GLASS V4 and MCD43A3 C6.

The snow episodes were correctly reported by the three satellite products in most cases but MCD43A3 C6.1 reaches typically higher values, which are more consistent with daily ground observations. Some spurious events (e.g., February 2015 and 2016 in SCBI) were not captured by C3S V2, which could be attributed to its larger temporal composite and lower production frequency, and to the more conservative approach of the PROBA-V cloud-masking algorithm [24,82].

3.4. Intra-Annual Precision

Intra-annual precision (so-called smoothness) corresponds to temporal noise assumed to have no serial correlation within a season and is quantitatively assessed as the anomaly between the product value for one date and the linear estimate based on its neighbors [83]. Figure 10 shows the Probability Density Function (PDF) of the intra-annual precision for C3S V2, GLASS V4, and MCD43A3 C6.1 products. The median values (indicative of intra-annual precision) of each product are summarized in

Table 8. Summary of median δ for C3S V2, GLASS V4, and MCD43A3 C6.1 total shortwave black-sky albedo products. Evaluation in the 2001–2005 period over LANDVAL sites considering all pixels.

Satellite Product	Median δ
C3S V2	0.0022
GLASS V4	0.0014
MCD43A3 C6.1	0.0008

.

Smoothness histograms reveal that most values are below 0.005, which demonstrates that the three satellite products show high precision. The temporal resolution of each product directly impacts their smoothness, as MCD43A3 C6.1 (smoothness calculated at 5 days temporal step rather than the actual daily temporal resolution) shows better intra-annual precision than GLASS V4 (8 days) and C3S V2 (10 days).

3.5. Inter-Annual Precision

Inter-annual precision (Figure 11,

Table 9. Inter-annual precision indicator (median absolute deviation between two consecutive years) of C3S V2, GLASS V4, and MCD43A3 C6.1 products. Evaluation for total shortwave black-sky albedo retrievals over LANDVAL sites in the 2001–2005 period considering all pixels.

	C3S V2	GLASS V4	MCD43A3 C6.1
Inter-annual precision: median absolute deviation	0.007 (1.64%)	0.002 (0.55%)	0.004 (0.84%)

) for each satellite product under study is assessed by comparison of retrievals for consecutive years over 19 desert calibration sites [84] during 5 years of data (2001–2005). The best inter-annual precision (i.e., MAD, CEOS LPV best practice) is observed for GLASS V4 (MAD = 0.002, 0.55%), while the C3S VGT V2 product provides the worst results (MAD = 0.007, 1.64%), with median absolute deviations of 1.64%. MCD43A3 C6.1 shows intra-annual precision better than 1% (MAD = 0.004, 0.84%).

3.6. Overall Spatio-Temporal Consistency

The overall consistency between a pair of satellite products is evaluated by reporting several metrics indicative of the goodness of the fit, such as accuracy (mean bias, B, and median deviation, MD), precision (standard deviation, STD, and median absolute deviation, MAD) and uncertainty (RMSD). Scatterplots (Figure 12) and validation metrics between products are computed over LANDVAL sites for the period of 2001–2005 and two different postulates: taking into account all pixels (

Table 10. Summary of the main statistics for scatterplots between C3S V2 vs. GLASS V4, C3S V2 vs. MCD43A3 C6.1, and GLASS V4 vs. MCD43A3 C6.1 for black-sky total shortwave retrievals in the 2001–2005 period over LANDVAL sites. All quality pixels are taken into account.

	C3S V2 vs. GLASS V4	C3S V2 vs. MCD43A3 C6.1	GLASS V4 vs. MCD43A3 C6.1
N	122086	115912	145694
R	0.91	0.89	0.94
MAR	y = 0.87x + 0.04	y = 0.83x + 0.05	y = 0.96x + 0.01
B	0.017 (8.1%)	0.015 (7.1%)	−0.001 (−0.3%)
MD	0.024 (11.4%)	0.024 (11.2%)	<0.001 (0.2%)
STD	0.052 (24.8%)	0.062 (28.7%)	0.043 (21.0%)
MAD	0.027 (12.7%)	0.027 (12.8%)	0.006 (3.0%)
RMSD	0.055 (26.1%)	0.064 (29.6%)	0.043 (21.0%)
%Optimal	10.2	9.6	61.8
%Target	28.4	29.4	83.7
%Threshold	49.4	49.7	90.9

) and only considering best quality pixels (

Table 11. Summary of the main statistics for scatterplots between C3S V2 vs. GLASS V4, C3S V2 vs. MCD43A3 C6.1, and GLASS V4 vs. MCD43A3 C6.1 for black-sky total shortwave retrievals in the 2001–2005 period over LANDVAL sites. Only best quality pixels are taken into account.

	C3S V2 vs. GLASS V4	C3S V2 vs. MCD43A3 C6.1	GLASS V4 vs. MCD43A3 C6.1
N	102857	52954	69280
R	0.90	0.99	0.97
MAR	y = 0.94x + 0.03	y = 1.03x + 0.02	y = 1.01x + 0.00
B	0.021 (11.0%)	0.022 (10.6%)	>−0.001 (−0.0%)
MD	0.025 (13.0%)	0.022 (10.3%)	−0.001 (−0.5%)
STD	0.037 (19.5%)	0.013 (6.2%)	0.026 (12.4%)
MAD	0.026 (13.5%)	0.022 (10.3%)	0.005 (2.3%)
RMSD	0.042 (22.4%)	0.026 (12.3%)	0.026 (12.4%)
%Optimal	9.8	12.4	77.5
%Target	28.7	43.1	95.8
%Threshold	50.6	70.4	98.7

).

The best agreement in terms of accuracy and uncertainty is found between MCD43A3 C6.1 and GLASS V4, with almost no bias and RMSD of 0.043 (21%). Improved results are found when considering best quality retrievals (RMSD of 0.026 (12.4%)). The C3S V2 product tends to provide systematically higher values compared to MODIS-based products (MCD43A3 C6.1 and GLASS V4), with mean bias of 7–8% (0.015–0.017) and MD of 11% (0.024) when all pixels are considered. The uncertainty between C3S V2 and the MODIS-based products significantly improves when only best quality retrievals are considered (mainly in the comparison with MCD43A3 C6.1) but a large bias is found, indicating systematic positive differences.

3.7. Stability

Stability is the extent to which a product remains constant over a long period, typically a decade or more [36]. Temporal stability can be also defined as the change in bias over a predefined time period [85], and stability can be estimated as the slope of a linear regression for the bias over time [86]. In SALVAL implementation, pseudo-invariant desert calibration sites [84] are used for stability evaluation, and the slope of albedo values per decade is provided as an indicator of stability. As desert sites are supposed to experience very little temporal variation, variation in albedo time series can be considered to be equivalent to evaluation of the bias over time.

Figure 13 displays some examples of C3S V2, GLASS V4, and MCD43A3 C6.1 temporal profiles in the 2001–2010 period for some selected desert calibration sites. The box plots of the decadal slopes over all calibration sites are displayed in Figure 14. The three products show some seasonality that could be attributed to changes in illumination, but no deviations at the long-term scale. Furthermore, C3S V2 shows more noise in the signal compared with GLASS V4 and MCD43A3 C6.1, as previously revealed in its worst intra-annual (see Section 3.4) and inter-annual (see Section 3.5) precision. This noise could be partly attributed to the BRDF climatology data used as prior information in the C3S V2 algorithm, which presents the same behavior. Median slopes in all calibration sites revealed no deviation for C3S V2 and GLASS V4 and a very slight positive slope (i.e., 0.003, which is equivalent to 0.6%) for MCD43A3 C6.1, largely fulfilling the GCOS requirements in terms of stability (1%).

3.8. Direct Validation

Direct validation involves the comparison of satellite retrievals with albedo measured from tower-based instruments (REALS dataset). Figure 15 shows the scatterplots between blue-sky satellite albedo quantities and REALS during the 2000–2019 period, taking into account all quality pixels (left side), and only best quality pixels (right side), where most of the outliers are removed. The main statistics resulting from the direct validation are summarized in

Table 12. Summary of the main statistics for the direct validation of C3S V2, GLASS V4, and MCD43A3 C6.1 satellite blue-sky albedo products vs. blue-sky albedo ground values from the REALS dataset in the 2000–2019 period. All pixels were taken into account.

	C3S V2	GLASS V4	MCD43A3 C6.1
N	12067	12067	12067
R	0.63	0.61	0.60
MAR	y = 0.68x + 0.06	y = 0.93x + 0.01	y = 1.05x−0.01
B	0.017 (12.2%)	−0.003 (−2.5%)	−0.003 (−2.5%)
MD	0.021 (14.9%)	<0.001 (−0.1%)	−0.002 (−1.2%)
STD	0.040 (27.8%)	0.043 (32.7%)	0.047 (35.2%)
MAD	0.029 (20.4%)	0.017 (13.2%)	0.017 (13.2%)
RMSD	0.043 (30.4%)	0.043 (32.8%)	0.047 (35.3%)
%Optimal	12.6	20.6	18.1
%Target	24.5	38.3	37.0
%Threshold	46.8	63.1	64.7

(all pixels) and

Table 13. Summary of the main statistics for the direct validation of C3S V2, GLASS V4, and MCD43A3 C6.1 satellite blue-sky albedo products vs. blue-sky albedo ground values from the REALS dataset in the 2000–2019 period. Only best quality pixels were taken into account.

	C3S V2	GLASS V4	MCD43A3 C6.1
N	4598	4598	4598
R	0.68	0.74	0.76
MAR	y =0.66x + 0.06	y = 0.61x + 0.04	y = 0.65x + 0.04
B	0.014 (9.7%)	−0.008 (−6.2%)	−0.008 (−5.7%)
MD	0.017 (11.7%)	−0.004 (−2.9%)	−0.005 (−3.8%)
STD	0.032 (22.2%)	0.030 (22.3%)	0.029 (21.6%)
MAD	0.024 (16.7%)	0.013 (10.1%)	0.015 (11.3%)
RMSD	0.035 (24.2%)	0.031 (23.2%)	0.030 (22.4%)
%Optimal	16.8	27.5	20.9
%Target	32.2	48.1	43.3
%Threshold	56.8	72.0	73.4

(best quality pixels).

The overall accuracy (median error) of C3S V2 is 15%, with a systematic tendency to provide higher values than ground measurements (mean bias of 12.2%). GLASS V4 and MCD43A3 C6.1 show betters results, and an opposite sign of differences (mean bias of −2.5%). In terms of overall uncertainty, the three satellite products provide similar results in the comparison with in situ data, with RMSD of around 0.04 when all pixels are considered in the comparison, and RMSD around 0.03 when only best quality retrievals are contemplated. However, C3S V2 shows a lower percentage of cases within the predefined accuracy levels than GLASS V4 and MCD43A3 C6.1.

The MAR relationship of the best quality pixels indicates a tendency of all satellite products to overestimate ground data for the lowest albedo values (mainly dominated by forests) and the opposite trend for higher albedo values (sparse vegetation), with slopes lower than 0.7 in the line comparing results to other validation studies over a significant set of locations [15,24,33].

4. Discussion

The three products under study (MCD43A3 C6.1, C3S V2, and CLASS V4) show remarkably good completeness, with missing data mainly located over northern regions and wintertime, typically affected by persistent clouds. The three products introduce different techniques to improve the spatiotemporal continuity: a poorer quality back-up algorithm is used in the case of MCD43A3 C6.1, a prior climatology of BRDF data is used in the case of C3S V2, and gap-filling techniques are used in the case of GLASS. When considering best quality pixels, MCD43A3 C6.1 is the most restrictive product, as only full retrievals of the model are provided when at least 50% of high-quality observations are well-distributed over the viewing hemisphere during the 16-day synthesis period.

In terms of spatial consistency (i.e., residuals), all combinations between pairs of products largely meet uncertainty requirements, with more than 70% of global cases achieving optimal level of consistency (residuals typically lower than 0.015). As expected, the best spatial consistency between pairs of products was found between MCD43A3 C6.1 and GLASS, as they are based on data from the same MODIS instruments on board Terra and Aqua, whereas C3S products are retrieved using SPOT/VGT or PROBA-V depending on the temporal range. Different spectral response functions among the instruments show dissimilarities in band location, band width, and response percentage of input signal over similar spectral channels [87]. The main discrepancies are typically located over equatorial areas and northern regions, which can be explained by cloud contamination and differences in the pre-processing chain. The underestimation of C3S products over snow targets [60] is another reason for the discrepancies over northern areas.

The three satellite products provide good temporal agreement among them and in comparison with in situ data. The better temporal resolution of MCD43A3 C6.1 allows us to capture smoother temporal variations than is possible from C3S V2 and GLASS V4, despite the incorporation of gap-filling techniques in the C3S and GLASS algorithms, and the inclusion of a temporal smoothing method in GLASS V4.

The evaluation of the APU metrics from the indirect evaluation also indicated that the best agreement is found between GLASS V4 and MCD43A3 C6.1. More than 77.5% of best quality observations lie within the optimal uncertainty level (Max [5%, 0.0025]) when the same sensor is used (i.e., GLASS V4 versus MCD43A3 C6.1). C3S V2 provides larger differences from both MODIS-based products, but satisfactory results, with systematically higher values of around 10%.

As a corollary, we can conclude that the use of a different sensor is the most important factor contributing to discrepancies among products. Nevertheless, the inversion algorithm is another contributing factor to differences between products. C3S V2 and MCD43A3 C6.1 make use of semi-empirical linear kernel-driven models to first retrieve BRDF coefficients and then compute surface albedo by angular and spectral integration. By comparison, GLASS adopts the angular bin and STF algorithm, and incorporates improvements in the inversion of snow and ice using an asymptotic radiative transfer model [88]. Additionally, the different spectral integration approach also contributes to differences between products. MODIS and GLASS adopt the same broadband albedo range and narrow-to-broadband conversion algorithm [89]. C3S products are computed over slightly different broadband albedo intervals and a different conversion algorithm [14].

The direct validation showed systematic positive bias of around 10% for C3S (SPOT/VGT and PROBA-V) V2 products for the period under study (2000–2019), in line with that found for previous C3S V1 versions [15,24], where positive bias of 11.5% was also reported. GLASS V4 and MCD43A3 C6.1 showed the opposite sign of differences, but improved results (with mean bias of around 6% and median deviation of 3%).

The comparison of satellite-based surface albedo estimates versus ground measurements indicates the difficulty in complying with existing user uncertainty requirements. Typically, less than 20% of satellite-based best quality retrievals actually achieve the GCOS target (Max [5%, 0.0025]) and the WMO goal requirements in terms of accuracy. By comparison, the three satellite products investigated largely accomplished stability optimal requirements (Max [1%, 0.001]).

5. Conclusions

This paper demonstrates the functionality of the SALVAL online platform to validate currently available operational albedo products. A validation and intercomparison exercise was conducted on three long-term global products generated by C3S, MODIS, and GLASS. Completeness, spatiotemporal consistency, precision, and accuracy were evaluated. Results from the SALVAL tool indicate that the three datasets under evaluation provide long-term reliable and highly consistent retrievals at a global scale. Discrepancies between products are primarily associated with differences in the retrieval processing chain: different input data sensors, pre-processing and atmospheric corrections, and inversion algorithms.

The CEOS LPV validation stage assigned to these global satellite albedo products is currently stage 3, which means that direct validation with in situ data or other reference datasets is performed over a significant set of locations and time periods representing global conditions. Thanks to the availability of the SALVAL online platform, the four main components [8] for an operational validation system of satellite-based surface albedo products have been integrated: long term satellite products, a global in situ dataset, the CEOS LPV validation best practices protocol, and an online validation platform. The SALVAL tool provides the potential functionality to achieve CEOS LPV validation stage 4 as it is also designed to accommodate regular updates of the validation results, providing albedo ECVs the readiness level for ongoing operational validation.

SALVAL provides transparency, consistency, and traceability to the validation process. The tool is available within the CEOS Cal/Val Portal [90], and offers a way to contribute to and collaborate with the greater the scientific community, thus allowing new products or ground reference datasets to be incorporated into the tool.