“Regression-then-Fusion” or “Fusion-then-Regression”? A Theoretical Analysis for Generating High Spatiotemporal Resolution Land Surface Temperatures

Abstract

The trade-off between spatial and temporal resolutions in satellite sensors has inspired the development of numerous thermal sharpening methods. Specifically, regression and spatiotemporal fusion are the two main strategies used to generate high-resolution land surface temperatures (LSTs). The regression method statically downscales coarse-resolution LSTs, whereas the spatiotemporal fusion method can dynamically downscale LSTs; however, the resolution of downscaled LSTs is limited by the availability of the fine-resolution LSTs. Few studies have combined these two methods to generate high spatiotemporal resolution LSTs. This study proposes two strategies for combining regression and fusion methods to generate high spatiotemporal resolution LSTs, namely, the “regression-then-fusion” (R-F) and “fusion-then-regression” (F-R) methods, and discusses the criteria used to determine which strategy is better. The R-F and F-R have several advantages: (1) they fully exploit the information in the available data on the visible and near infrared (VNIR) and thermal infrared (TIR) bands; (2) they downscale the LST time series to a finer resolution corresponding to that of VNIR data; and (3) they inherit high spatial reconstructions from the regression method and dynamic temporal reconveyance from the fusion method. The R-F and F-R were tested with different start times and target times using Landsat 8 and Advanced Spaceborne Thermal Emission and Reflection Radiometer data. The results showed that the R-F performed better than the F-R when the regression error at the start time was smaller than that at the target time, and vice versa.

1. Introduction

Land surface temperature (LST) is a key parameter in environmental applications extending from local to global scales. This physical parameter is a key input of the surface energy balance model [1,2], which can be used to monitor evapotranspiration [3] and map land surface carbon [1]. In addition, LST is also used to study the urban thermal environment [4,5] and detect thermal anomalies before earthquakes [6].

Remotely sensed thermal images with high spatial and temporal resolution facilitate these environmental monitoring efforts. However, the available satellite thermal images yield a trade-off between spatial and temporal resolutions that limits the applications of LSTs. For example, thermal images of Landsat 8 or Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) have a spatial resolution of ~100 m but the repeat cycle is 16 days. In contrast, Moderate Resolution Imaging Spectroradiometer (MODIS) can generate 1 km-resolution LSTs four times per day. Thermal sharpening methods provide a solution to the resolution limitations.

The most widely used thermal sharpening method uses the relationship between LST and the normalized difference vegetation index (NDVI) to predict high-resolution LSTs [7], and it is efficient when used over a vegetated region. In addition to the NDVI, vegetation indices, such as the leaf area index, temperature vegetation dryness index, and ratio vegetation index, are also used as predictors to downscale LSTs [8]. To better adapt to an urban area, Dominguez developed the high resolution urban thermal sharpener (HUTS) to predict high-resolution LSTs using the relationship among the LST, NDVI, and surface albedo (α) [9]. In addition, the impervious surface index is applied in the disaggregation procedure for radiometric surface temperature (DisTrad) [7] and the TsHARP algorithm [10] to downscale urban LSTs as well [11,12]. These thermal sharpening methods are based on the assumption that the relationship between the LST and predictors is unique at different spatial resolutions [10]. Hence, we can obtain this relationship from low-resolution data and then predict high-resolution LST with the obtained relationship and predictors. These methods using predictors such as panchromatic images [13], emissivity [14], and surface albedo [9], can be considered as regression methods. Traditional regression tools, e.g., least-squares regression, build unique regression functions between LSTs and predictors [10]. However, when we use machine learning methods (e.g., artificial neural networks (ANN) [8], random forest (RF) [15], and support vector machine (SVM) [16]) as the regression tools, the details of the regression function are unknown. Specifically, compared with traditional regression tools, the RF regression yields better results and is more effective over heterogeneous regions [15,17], which indicates that the RF is a satisfactory regression tool for thermal sharpening.

Compared with the regression methods, which instantaneously downscale LSTs, spatiotemporal fusion methods use coarse/fine LST pairs to predict fine-resolution LSTs and thus can provide better data with high spatial and temporal resolution. Spatiotemporal fusion methods are different from traditional fusion methods, such as the intensity-hue-saturation (IHS) transformation [18], and wavelet decomposition [19,20], which combine high-resolution panchromatic data to obtain fine LSTs. For instance, the spatial and temporal adaptive reflectance fusion model (STARFM) [21] assumes that the land cover type and system error do not change over time. Combining details from neighboring pixels and temperature variation over time, the STARFM can effectively predict fine LST time series. The widely used STARFM has been the basis for many other spatiotemporal fusion methods, such as the unmixing-based, the weight function-based, and the dictionary-pair learning-based methods [22]. Limited in the pure temporal information obtained from the homogeneous pixels, the STARFM is less effective over heterogeneous regions [21]. Spatiotemporal fusion methods have been developed to solve the problems of the STARFM, such as the enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM) [23], the spatio-temporal integrated temperature fusion model (STITFM) [24], and the spatio-temporal adaptive data fusion algorithm for temperature mapping (SADFAT) [25].

The aforementioned regression methods and the spatiotemporal fusion methods have their own advantages and disadvantages. Specifically, the regression methods downscale coarse LSTs to a fine resolution corresponding to the predictors obtained from the visible and near infrared (VNIR) bands. However, regression methods can only downscale LSTs instantaneously and have poorer performance when the ratio of the fine/coarse resolution is too large [26]. In addition, the spatiotemporal fusion methods can dynamically downscale LST time series but are limited by the resolution of the available fine LSTs. A combination of regression methods and spatiotemporal fusion methods is ideal to counteract their limitations and inherit their strengths. Bai [27] used an extreme learning machine (ELM) algorithm to downscale the Landsat ETM+ thermal infrared (TIR) data to a 30-m resolution then used the downscaled LST as input for the SADFAT to sharpen the MODIS LSTs. This method can be considered as the regression-then-fusion (R-F) method, i.e., using the regression method to first downscale the medium-resolution (~100 m) LST into fine resolution and then using the spatiotemporal fusion method to fuse the low-resolution LST time series and the downscaled LST obtained in the previous step. Accordingly, the fusion-then-regression (F-R) method, which has seldom been employed, uses the spatiotemporal fusion method to first fuse the low-resolution LSTs into medium resolution then downscales the fused medium-resolution LSTs into high-resolution via the regression method.

Because few studies have combined regression methods and spatiotemporal fusion methods together to generate high spatiotemporal LSTs, strategies for selecting from combinations of the two methods have not been discussed. The objective of this study is to present these two combinations (i.e., R-F and F-R) and discuss the strategies by which they have selected. The details of the two combinations and the strategies for selection are further described in Section 3. The results are tested and evaluated using the Landsat 8 data and ASTER data and are presented in Section 4. The discussion and conclusions are provided in Section 5 and Section 6, respectively.

2. Data and Study Area

Landsat 8 datasets (downloaded from https://glovis.usgs.gov/) are used in this paper, of which the overpass time is approximately 10:50 local solar time. The thermal infrared (TIR) channel has a spatial resolution of 100 m, and the VNIR channel has a spatial resolution of 30 m. In addition, the LSTs are retrieved via a single channel algorithm using the TIR band 10 [28], of which the mean errors are below 1.5 K [28]. Landsat 8 datasets, collected on 4 September 2014 and 6 October 2014 with no clouds, are used for the predictions. The study area (see Figure 1b) is located in the north of Beijing and the land cover type is mainly forest and grassland, and it was selected because of its homogeneity.

ASTER data containing VNIR, short wave infrared (SWIR), and TIR channel are downloaded from https://glovis.usgs.gov/ and are used in Section 4. The spatial resolution is 15 m for the VNIR channel, 30 m for the SWIR channel, and 90 m for the TIR channel. We use the two-channel algorithms [29] to derive the LST, of which the accuracies are within 1.5 K [29]. Two ASTER TIR images under clear sky collected on 18 October 2004 (in the daytime and at night-time) and showing the same region (see Figure 1c) are used.

3. Methodology

This section first provides an overview of the regression and fusion methods (Section 3.1) and then presents the implementation details of the R-F and F-R methods (Section 3.2). Subsequently, we analyze the errors of the R-F and F-R method (Section 3.3) and then summarize the criteria used to determine the optimal strategy by comparing the errors of the R-F and F-R methods (Section 3.4). In addition, Section 3.5 introduces the implementation strategies with Landsat 8 data and ASTER data, respectively.

3.1. Overview of the Regression and Spatiotemporal Fusion Methods

3.1.1. Overview of the Regression Method

The regression method relies on the relationship between LST and predictors. The relationship is assumed to be invariant at different spatial resolutions [10]. Hence, the low-resolution LST and the corresponding predictors can be expressed as follow:

$$T_{low}=f\left({\rho }_{low}\right)+{\epsilon }_{low}$$

(1)

where $T_{low}$ is the low-resolution LST, $f(·)$ represents the function relationship between the LST and predictors, ${\rho }_{low}$ represents the low-resolution predictors, and ${\epsilon }_{low}$ indicates the residual of the true LST and the predicted LST via $f(·)$ at low resolution. Here, we apply the function relationship in the high-resolution data. The predicted LST at high resolution is obtained via the following equation:

$${\widehat{T}}_{high}=f\left({\rho }_{high}\right)+{\epsilon }_{low}$$

(2)

where ${\widehat{T}}_{high}$ represents the predicted high-resolution LST, and ${\rho }_{high}$ represents the high-resolution predictors.

Specifically, the function relationship is obtained using the regression tools, such as linear or non-linear regression [9,10], global or local regression, piecewise regression [30], and geographically weighted regression [31]. Since the RF regression has been satisfactorily tested in combination with a residual correction [15], we adopt the RF regression as the regression tool for this study. In addition, the reflectance of the VNIR channels is used as the predictors.

3.1.2. Overview of the Spatiotemporal Fusion Method

The spatiotemporal fusion method takes several fine LSTs and time series coarse LSTs to predict the fine LST time series. The STARFM, which requires minimum LST inputs, uses one fine LST and two coarse LSTs as inputs [21]. The similar pixels, which are determined using the spatial difference and temporal difference, are used to predict the fine LST. The equation used to predict the fine LST via the STARFM is given as follows:

$${\widehat{T}}_{high}\left(t_1\right)=T_{high}\left(t_0\right)+{\displaystyle \sum }_{k=1}^w{W}_k\times \left(T_{low}\left(t_1\right)-T_{low}\left(t_0\right)\right)$$

(3)

where $t_0$ is the start time, $t_1$ is the target time; ${\widehat{T}}_{high}\left(t_1\right)$ is the predicted high-resolution LST at $t_1$; $T_{high}\left(t_0\right)$ represents the high-resolution LST at $t_0$; $T_{low}\left(t_1\right)$ and $T_{low}\left(t_0\right)$ represent the low-resolution LSTs at $t_1$ and $t_0$, respectively; $w$ indicates the number of the similar pixels within the moving window; and $W_k$ represents the weight that determines how much the similar pixels contribute to the central pixel. In addition, the similar pixels in a moving window are shown in Figure 2. The detailed principle for selecting the similar pixels is referenced in [21].

3.2. Implementations of the R-F and F-R Methods

The R-F and F-R methods are combinations of the regression and spatiotemporal fusion methods, and they downscale low-resolution LSTs into high-resolution LSTs based on the resolution of the VNIR data. Both regression and fusion methods involve two scales of resolution, i.e., low resolution and high resolution. However, in the R-F and F-R methods, we introduce the use of medium resolution. Hence, at least one medium-resolution LST at the start time and corresponding high-resolution VNIR data and two low-resolution LSTs at the start and target time are required in both strategies. For example, a 100 m resolution LST from Landsat 8 and corresponding 30 m resolution VNIR data and two 1 km resolution LSTs from MODIS can be used to generate 30 m diurnal LSTs via the R-F or F-R method. The implementation details of the R-F and F-R methods are shown in Figure 3, in which the notations correspond to the descriptions in Section 3.2.1 and Section 3.2.2.

3.2.1. Implementation Details of the R-F Method

The R-F method includes two steps (see Figure 3a). First, the medium-resolution LST and the corresponding VNIR data are used to obtain the regression relationship as follows:

$$T_{medium}\left(t_0\right)=f_{medium,t0}\left({\rho }_{medium}\left(t_0\right)\right)+{\epsilon }_{medium}\left(t_0\right)$$

(4)

where $T_{medium}\left(t_0\right)$ represents the medium-resolution LST at $t_0$, ${\rho }_{medium}\left(t_0\right)$ represents the predictors at $t_0$, which is obtained from the VNIR data, $f_{medium,t0}(·)$ represents the relationship between $T_{medium}\left(t_0\right)$ and ${\rho }_{medium}\left(t_0\right)$; and ${\epsilon }_{medium}\left(t_0\right)$ indicates the residual between the true medium-resolution LST and the predicted medium-resolution LST via $f_{medium,t0}(·)$ at $t_0$.

Here, we apply the relationship between high-resolution predictors to predict the high-resolution LST at $t_0$ using following equation:

$$T_{high,re}\left(t_0\right)=f_{medium,t0}\left({\rho }_{high}\left(t_0\right)\right)+{\epsilon }_{medium}\left(t_0\right)$$

(5)

where $T_{high,re}\left(t_0\right)$ represents the predicted high-resolution LST via the regression method at $t_0$ and ${\rho }_{high}\left(t_0\right)$ represents the high-resolution predictors at $t_0$. The regression method error can be expressed as follows:

$${\epsilon }_{high,re}\left(t_0\right)=T_{high,re}\left(t_0\right)-T_{high}\left(t_0\right)$$

(6)

We input the result of the regression method into the fusion method to obtain the high-resolution LST at the target time. The predicted fine LST is expressed as follows:

$$T_{high,fu}\left(t_1\right)=T_{high,re}\left(t_0\right)+{\displaystyle \sum }_{k=1}^w{W}_k\times \left(T_{low}\left(t_1\right)-T_{low}\left(t_0\right)\right)$$

(7)

where $T_{high,fu}\left(t_1\right)$ indicates the predicted high-resolution LST via the fusion method at $t_1$.

3.2.2. Implementation Details of the F-R Method

Similar to the R-F method, the F-R method includes two steps (see Figure 3b). First, a pair of medium-resolution and low-resolution LSTs at the start time and a low-resolution LST at the target time are used to obtain the medium-resolution LST at the target time. The predicted LST can be obtained using the following equation:

$$T_{medium,fu}\left(t_1\right)=T_{medium}\left(t_0\right)+{\displaystyle \sum }_{k=1}^w{W}_k\times \left(T_{low}\left(t_1\right)-T_{low}\left(t_0\right)\right)$$

(8)

where $T_{medium,fu}\left(t_1\right)$ represents the predicted medium-resolution LST via the fusion method at $t_1$.

We then use the results of the fusion method as input for the regression method to obtain the regression relationship between the LSTs and predictors as follow:

$$T_{medium,fu}\left(t_1\right)=f_{medium,t1}\left({\rho }_{medium}\left(t_1\right)\right)+{\epsilon }_{medium}\left(t_1\right)$$

(9)

where ${\rho }_{medium}\left(t_1\right)$ indicates the medium-resolution predictors at $t_1$, $f_{medium,t1}(·)$ represents the relationship between $T_{medium,fu}\left(t_1\right)$ and ${\rho }_{medium}\left(t_1\right)$, and ${\epsilon }_{medium}\left(t_1\right)$ indicates the residual between the fused medium-resolution LST and the predicted medium-resolution LST via $f_{medium,t1}(·)$ at $t_1$.

We apply the regressed relationship to the high-resolution predictors, and the fine LST can be predicted using the following equation:

$$T_{high,re}\left(t_1\right)=f_{medium,t1}\left({\rho }_{high}\left(t_1\right)\right)+{\epsilon }_{medium}\left(t_1\right)$$

(10)

where $T_{high,re}\left(t_1\right)$ represents the predicted high-resolution LST via the regression method at $t_1$.

3.3. Error Analysis of the R-F and F-R Methods

3.3.1. Error Analysis of the R-F Method

The R-F method error is the difference between the predicted fine LST and the actual high-resolution LST at $t_1$, which can be written as follows:

$${\delta }_{RF}=T_{high,fu}\left(t_1\right)-T_{high}\left(t_1\right)$$

(11)

Combined with Equations (6) and (7), the R-F method error can be expressed as follows:

$$\begin{array}{ll}{\delta }_{RF}=T_{high,re}\left(t_0\right)& +{\displaystyle {\displaystyle \sum }_{k=1}^w}{W}_k\times \left(T_{low}\left(t_1\right)-T_{low}\left(t_0\right)\right)-T_{high}\left(t_1\right)\\ & =T_{high}\left(t_0\right)+{\epsilon }_{high,re}\left(t_0\right)+{\displaystyle {\displaystyle \sum }_{k=1}^w}{W}_k\times \Delta T_{low}-T_{high}\left(t_1\right)\\ & =\Delta T_{medium}-\Delta T_{high}+{\epsilon }_{high,re}\left(t_0\right)\end{array}$$

(12)

where $\Delta T_{low}=T_{low}\left(t_1\right)-T_{low}\left(t_0\right)$, $\Delta T_{medium}=T_{medium}\left(t_1\right)-T_{medium}\left(t_0\right)$, and $\Delta T_{high}=T_{high}\left(t_1\right)-T_{high}\left(t_0\right)$. The R-F method error contains the errors caused by the regression process and the fusion process. Here, when we ignore the error caused by the fusion process, the error of the R-F method can be simplified as follows:

$${\delta }_{RF}={\epsilon }_{high,re}\left(t_0\right)$$

(13)

3.3.2. Error Analysis of the F-R Method

The F-R method error is the difference between the predicted fine LST and the true high-resolution LST at $t_1$, which can be written as follows:

$${\delta }_{FR}=T_{high,re}\left(t_1\right)-T_{high}\left(t_1\right)$$

(14)

In addition, the regression method error at $t_1$ can be written as follows:

$${\widehat{\epsilon }}_{high,re}\left(t_1\right)=T_{high,re}\left(t_1\right)-T_{high}\left(t_1\right)$$

(15)

This regressed error contains the error of the fusion process, since we use the fused result as the input in the regression process. Here, we assume the error of the fusion process is small enough to be ignored. Hence, the F-R method error can be simplified as follows:

$${\delta }_{FR}={\epsilon }_{high,re}\left(t_1\right)$$

(16)

This regressed error is different to that in Equation (15) that does not contain the error of the fusion process.

3.4. Comparisons of the R-F and F-R Method Errors

The squared error (SE) [32] is used to describe the performance of the R-F and F-R methods. The SE of the predicted LST is expressed as follows:

$$\mathrm{SE}={\left(T_{pre}-T_{true}\right)}^2$$

(17)

where $T_{pre}$ represents the predicted LST, and the $T_{true}$ represents the actual LST. Hence, the error difference between the R-F and F-R is as follows:

$$S{E}_{RF}-S{E}_{FR}={\delta }_{RF}{}^2-{\delta }_{FR}{}^2$$

(18)

Combined with Equations (13) and (16), Equation (18) can be expressed as follows:

$$S{E}_{RF}-S{E}_{FR}={\epsilon }_{high,re}{\left(t_0\right)}^2-{\epsilon }_{high,re}{\left(t_1\right)}^2$$

(19)

Both R-F method error and F-R method error neglect the error of the fusion process, of which the difference is the resolution difference. Specifically, the fusion process of the R-F method sharpens the low-resolution LST into high resolution, while that of the F-R method sharpens the low-resolution LST into medium resolution. Therefore, when we compare the R-F method error and F-R method error, the fusion process error can be ignored when the fusion process error is almost undifferentiated at different downscaling ratios. Here, we compare the SEs of the R-F and F-R methods according to Equation (19). The SE of the R-F method is larger than that of the F-R method when the regression error satisfies Equation (20).

$${\epsilon }_{high,re}{\left(t_0\right)}^2>{\epsilon }_{high,re}{\left(t_1\right)}^2$$

(20)

In contrary, the SE of the R-F method is smaller than that of the F-R method when the regression error satisfies Equation (21).

$${\epsilon }_{high,re}{\left(t_0\right)}^2<{\epsilon }_{high,re}{\left(t_1\right)}^2$$

(21)

In conclusion, we obtain the criteria to determine which method is better: when the performance of the regression at $t_0$ is better than that at $t_1$, the R-F method is better than the F-R method, and when the performance of the regression at $t_1$ is better than that at $t_0$, the F-R method is the better choice.

3.5. Implementation Strategies with Landsat 8 Data and ASTER Data

We use the Landsat 8 data collected over two days to implement the R-F and F-R methods. One date is used as the start time data while the other is used as the target time. First, we aggregate the 100 m data (both LST and reflectance of the VNIR bands) to 300 m resolution as the medium-resolution data, and then the 300 m resolution LSTs are aggregated to 3 km as the low-resolution data. We then use the R-F and F-R methods to sharpen the low-resolution LST to high-resolution LST. The implementation details are shown in Figure 4. The results of step 1 are the products of the middle process. To evaluate the transmitted errors caused by step 1, we use the actual LSTs to generate 100 m resolution LSTs using the single regression method or fusion method. The original 100 m resolution LST is compared with the sharpened LSTs. Here, we also use the root mean square error (RMSE), mean absolute error (MAE), Pearson’s correlation coefficient (r), and Co-Occurrence Root Mean Square Error (CO-RMSE) [33] to evaluate the accuracy of the thermal sharpening methods.

We also use ASTER data collected in one day to implement the R-F and F-R methods. TIR data collected in the daytime is used as the start time data while that at night-time is used as the target time. First, we aggregate the 90 m data (both LST and reflectance of the VNIR bands) to 270 m resolution as the medium-resolution data, and then the 270 m resolution LSTs are aggregated to 2.7 km as the low-resolution data. We then use the R-F and F-R methods to sharpen the low-resolution LSTs. The implementation details are shown in Figure 5. Here, we have not analyzed the transmitted errors caused by step 1 using the single regression method or fusion method, because the implementation with ASTER data is intended to evaluate the performance of the F-R and R-F methods in one day.

4. Results

4.1. Tests with Landsat 8 Data on Different Days

4.1.1. Results of the R-F and F-R Methods When ${\epsilon }_{high,re}{\left(t_0\right)}^2>{\epsilon }_{high,re}{\left(t_1\right)}^2$

Landsat 8 data collected over two days were used to implement the R-F and F-R methods. Specifically, data collected on 4 September 2014 were used as the start time data while data collected on 6 October 2014 were used as the target time data, because the error of the regressing process on 4 September 2014 was larger than that on 6 October 2014. The implementation details are introduced in Section 3.5.

The results of the sharpened LSTs are shown in Figure 6. A visual comparison shows that the results of the F-R method are better than that of the R-F method with more similar details to the actual LST. A quantitative comparison of the results in Figure 6g,h indicate that the F-R method performs better than the R-F method and presents lower RMSE, MAE, and CO-RMSE values and a higher r value. In addition, we also used the SEs to evaluate the F-R and R-F methods. As shown in Figure 7a,b, the spatial distribution of the difference between the regressed errors in two days along with the difference between the SEs of the R-F and of the F-R methods are similar. The similar spatial distribution illustrates that when the error of the regression method at the target time is smaller than that at the start time, the F-R method is better than the R-F method for sharpening the LST. That is because the larger difference of the regression errors between two different days indicates poorer performance of the regression method on 4 September 2014, and larger difference of the SEs indicates poorer performance of the R-F. In addition, the scatter plot representing the SEs of the R-F and F-R methods also shows a higher performance of the F-R method because more spots are located on the one-to-one dividing line (Figure 7c).

Since the results of the regression/fusion method are used as the input of the other method, errors transmitted during the middle process cannot be avoided. Here, we compared the accuracies of the F-R and R-F methods with that of the single method (i.e., regression/fusion method) to analyze the error propagation. As shown in Figure 8a, the accuracy of the F-R method (step 2) is lower than that of the middle process (step 1) and presents a higher RMSE value and a lower r value, which is partly due to the differences in scale that are applied in the two methods, namely, the fusion process downscaling the 3 km resolution LST to 300 m, and F-R downscaling the 300 m fused LST to 100 m. In addition, the F-R method takes the result of the fusion process as the input of the regression process, which means that the final result contains errors from both the fusion and regression methods. To analyze the performance of the transmitted error, we compared the accuracy of the F-R method with that of the regression method (contrast), which used the actual 300 m resolution LST for the regression. Compared with the regression method, the F-R method increased the RMSE value by 0.27 K, and decreased the r value by 0.05 (Figure 8a).

As for the R-F method, the error is larger than that of the middle process (step 1 in Figure 8b) because the error of the R-F method contains the error transmitted from the fusion process. In addition, compared with the fusion method (contrast), the R-F method increases the RMSE value by 0.15 K, and decreases the r value by 0.06 (Figure 8b). The increased error of the R-F method caused by the middle process is smaller than that of the F-R method, which means that the transmitted error in the F-R method impacts the final result more than that in the R-F method. However, the F-R method still has a higher performance than the R-F method when the error of the regression method is smaller at the target time than the start time.

4.1.2. Results of the R-F and F-R Methods When ${\epsilon }_{high,re}{\left(t_0\right)}^2<{\epsilon }_{high,re}{\left(t_1\right)}^2$

Here, we used the same data as that in Section 4.1.1. However, the data collected on 6 October 2014 were used as the start time data, whereas the data collected on 4 September 2014 were used as the target time data. As shown in Figure 9, the spatial details of the R-F method are more abundant than those of the F-R method and a clearer boundary is observed. The accuracy of the R-F method is greater than that of the F-R method and presents RMSE values of 0.88 K and 1.09 K and r values of 0.89 and 0.84 for the R-F and F-R methods, respectively (Figure 9g,h). In addition, the spatial distribution of the differences in regression error and SEs is similar, whereas larger differences in the regression error accompany larger differences in the SEs (see Figure 10a,b). This finding illustrates that when the regression error is larger at the target time (on 4 September 2014), the SE of the F-R method is larger than that of the R-F method. With fewer spots located over the one-to-one dividing line, the R-F method presents higher performance when the regression error is larger at the target time than at the start time (Figure 10c).

To evaluate the transmitted errors in the F-R and R-F methods, their accuracies and their middle processes accuracies were compared. As shown in Figure 11a, the accuracy of the F-R method is lower than that of its previous process (step 1), which can be explained by reasons similar to those provided in Section 4.1.1. Compared with the regression method which used the actual 300 m resolution LST for the regression (contrast), the F-R method increased the RMSE value by 0.27 K and decreased the r value by 0.08 (Figure 11a). In the R-F method, the accuracy is also lower than that of its previous process (step 1). Compared with the fusion method, which used the actual 300 m resolution LST at the start time as the input (contrast), the R-F method increased the RMSE value by 0.13 K and decreased the r value by 0.05 (Figure 11b). The transmitted error of the F-R method is larger than that of the R-F method, which is similar to that in Section 4.1.1. However, the performance of the R-F method is better than that of the F-R method when the error of the regression method is better at the start time than at the target time (see in Figure 11).

4.2. Tests with ASTER Data Collected in One Day

In Section 3, we introduce the strategy used to compare the errors of the R-F and F-R methods via simplified errors, which do not consider the influence of the fusion method because we consider the fusion process error is almost undifferentiated at different downscaling ratios. However, when we use the LST in the daytime to predict the LST at night-time, the error of the fusion method is much larger and cannot be ignored. Hence, we need to analyze the performance of the F-R and R-F methods when the errors of fusion method are larger than that of regression method and evaluate whether the conclusion provided in Section 3.4 is still valid.

Here, we tested the F-R and R-F methods using ASTER data collected in one day. Since VNIR data are not collected at night-time, we use the same predictors derived from the VNIR data in the daytime to predict the LST at different times. The implementation details of ASTER data are introduced in Section 3.5. The regression error of the start time is slightly lower than that of the target time. The SE value in the regression method is 0.25 at 11:10 and 0.28 at 22:15, which means that the R-F method is better than the F-R method according to the criteria provided in Section 3.4.

As shown in Figure 12g,h, the accuracy of the R-F method is slightly higher than that of the F-R method and presents lower CO-RMSE and MAE values and a higher r value, which corresponds to the criteria referred to in Section 3.4. However, the R-F method’s better performance is not as evident as that in Section 4.1.2. Moreover, the results of the F-R and R-F methods are only slightly relevant to the actual LST with coarse spatial details (see Figure 12d–f). This result is due to the poor performance of the fusion method when using the daytime LST to predict night-time LST. As shown in Figure 12a, the result of the fusion process using 270 m resolution LST at the start time and 2.7 km LSTs at the start and target time to predict 270 m resolution LST at the target time, is quite different from the actual LST (Figure 12f). Differently, the result of the regression process (Figure 12b) using the 90 m resolution VNIR data and 270 m resolution LST to predict 90 m resolution LST is similar to the actual LST (Figure 12c). This indicates the poor performances of the F-R and R-F methods are affected more by the fusion process than by the regression process. In addition, as shown in Figure 12a, the low temperature details in the black circle are similar to those in the actual LST at 11:10 (Figure 12c). However, the temperatures in the black circle at 22:15 (Figure 12f) are higher than the neighboring area while those at 11:10 (Figure 12c) are not. This indicates the result of the fusion process in the black circle (Figure 12a) is anamorphic and the reason is that the inherited spatial details from the start time are quite different from that at the target time.

In conclusion, the poor performance of the fusion method affects the performance of the R-F and F-R methods but it does not influence the selection strategy for the R-F and F-R methods, which is determined via a comparison of the regression errors.

5. Discussion

5.1. Comparisons of the Regression Method and the Fusion Method

The R-F and F-R methods are combinations of the regression and fusion methods and contain the characteristics of both. Hence, the poor performance of the regression method or of the fusion method might result in poor performance of the R-F and F-R methods. Here, we compared the regression method and the fusion method at different downscaling ratios. First, we aggregated the Landsat 8 data into resolutions of 300 m, 600 m, and 1 km. Then, we downscaled the coarse LSTs to 100 m resolution via the regression and fusion methods. As shown in Figure 13a, the accuracies of both methods declined as the downscaling ratios increased, which indicates that a larger downscaling ratio corresponds to poorer performance for the single regression or fusion methods. Hence, a combination of different methods is required to reduce the error caused by the large downscaling ratio. A visual comparison shows that the results of the regression and the fusion method are highly similar to the actual LST, which indicates that both methods are satisfactory for use in the R-F and F-R methods (see Figure 14a–c). In addition, the accuracy of the fusion method is higher than that of the regression method at all three downscaling ratios (Figure 13a), which means that the error of the fusion method can be ignored when comparing the performance of the R-F and F-R methods.

The comparison of the regression and fusion methods in Figure 14a is for the daytime, i.e., the start time and the target times in the fusion method are the same time on different days. To further explore the differences between these two methods, we compared the regression and fusion methods in one day. Here, we aggregated ASTER data into resolutions of 270 m, 540 m, and 900 m. Then, we downscaled the aggregated LSTs to a 90 m resolution via the regression and fusion methods. The start time and target time of the LSTs used in the fusion method are 11:10 and 22:15 on 18 October 2004, respectively. The results in Figure 13b show that the accuracy of the fusion method is lower than that of the regression method at all three downscaling ratios, which is different from the results shown in Figure 13a. In addition, the spatial details of the fusion method are much simpler than those of the regression method, which are less similar to the actual LST (see Figure 14d,e). This finding indicates that the fusion method is less effective when using the LST in daytime to predict LST at night-time. The results in Section 4.2 also show the effect of the fusion method on the R-F and the F-R methods.

In summary, the higher performance of one method (regression or fusion) does not vary with the changes in the downscaling ratios; therefore, we can utilize the comparisons at a coarse resolution to determine the optimal strategy. In addition, poor performance of the fusion method in one day (day and night) might limit the applications of the R-F and F-R methods at a diurnal scale. Hence, a better spatiotemporal fusion method, which is able to use daytime LSTs to predict nighttime LSTs can be applied in the F-R method or R-F method to obtain a diurnal cycle of LSTs. For example, the integrated framework provided by Quan [34], which can generate hourly Landsat-like LSTs can be applied in the F-R and R-F methods to generate high-resolution LSTs of diurnal cycles.

5.2. Advantages, Prospects and Limitations of the F-R and R-F Methods

The F-R and R-F methods are combinations of the regression and fusion methods, which adopt the spatial prediction from the regression method and the temporal prediction from the fusion method. These two combined methods have several advantages. On the one hand, they fully exploit the available VNIR data and the TIR data, which allows us to obtain fine spatial details from the VNIR data and temporal information from the LST time series. On the other hand, with the regression and fusion methods, the F-R and R-F methods can downscale the coarse LST time series to a fine resolution that corresponds to the resolution of the VNIR data, whereas the single fusion method can only downscale the coarse LST time series to fine resolution that corresponds to the resolution of the TIR data [21].

The F-R and R-F methods present a number of potential applications. First, the R-F and F-R methods’ ability to perform spatiotemporal predictions can be applied to replace the single regression or fusion methods when the single method cannot meet the requirements. Second, the F-R and R-F methods can generate high spatiotemporal LSTs, which can then be used in wide-ranging environmental applications, such as urban heat island effect predictions [35], hydrological simulations [36], and climate change predictions [37].

There are also a few limitations of the F-R and R-F methods. Specifically, the spatiotemporal fusion method used in this paper is designed for clear-sky conditions [21], the predicted results of which are without cloud contaminations. Hence, the F-R and R-F methods combining the spatiotemporal fusion method are also limited by the cloud contaminations of the images. When the pixels of the LSTs are polluted by clouds, they will not be dealt with by the F-R or R-F method.

6. Conclusions

This study proposed a combination of two strategies adopted from regression and fusion methods for thermal sharpening, namely, the R-F and F-R methods, and then provided the criteria for determining which strategy is better. The R-F method first sharpens the medium-resolution LSTs at the start time, and then the sharpened LSTs are used as input for the fusion method to predict fine-resolution LSTs at the target time. In contrast, the F-R method first fuses the coarse LSTs to a medium resolution and then downscales the fused medium-resolution LSTs to fine-resolution LSTs via the regression method. As the criteria suggested, the R-F method is better than the F-R method when the performance of the regression at the start time is better than that at the target time, and when the performance of the regression at the target time is better than that at the start time, the F-R method is the better choice.

The R-F and F-R methods were tested with Landsat 8 data and ASTER data under different conditions. One was tested when the regression error was larger at the start time than the target time; and the other was when the regression error was smaller at the start time than the target time. The results show that the F-R method performs better than the R-F method when the regression error at the start time is larger than that at the target time, and vice versa, which corresponds to the criteria referred to in Section 3. In addition, comparisons of the regression and fusion methods indicate that the higher performance of one method does not change with variations in the downscaling ratios. Hence, we can use the performance of the regression method at a coarse resolution to determine which strategy is better.

The R-F and F-R methods are proposed as strategies to compensate for the limitations of the single thermal sharpening methods. By combining the advantages of the regression and fusion methods, the R-F and F-R methods represent promising methods of generating high spatiotemporal resolution LSTs. Using the criteria outlined in this study, we can easily choose a satisfactory strategy under specific conditions.