A Stepwise Calibration of Global DMSP/OLS Stable Nighttime Light Data (1992–2013)

Abstract

The Defense Meteorological Satellite Program (DMSP)/Operational Linescan System (OLS) stable nighttime light (NTL) data provide a wide range of potentials for studying global and regional dynamics, such as urban sprawl and electricity consumption. However, due to the lack of on-board calibration, it requires inter-annual calibration for these practical applications. In this study, we proposed a stepwise calibration approach to generate a temporally consistent NTL time series from 1992 to 2013. First, the temporal inconsistencies in the original NTL time series were identified. Then, a stepwise calibration scheme was developed to systematically improve the over- and under- estimation of NTL images derived from particular satellites and years, by making full use of the temporally neighbored image as a reference for calibration. After the stepwise calibration, the raw NTL series were improved with a temporally more consistent trend. Meanwhile, the magnitude of the global sum of NTL is maximally maintained in our results, as compared to the raw data, which outperforms previous conventional calibration approaches. The normalized difference index indicates that our approach can achieve a good agreement between two satellites in the same year. In addition, the analysis between the calibrated NTL time series and other socioeconomic indicators (e.g., gross domestic product and electricity consumption) confirms the good performance of the proposed stepwise calibration. The calibrated NTL time series can serve as useful inputs for NTL related dynamic studies, such as global urban extent change and energy consumption.

1. Introduction

The Defense Meteorological Satellite Program (DMSP)/Operational Linescan System (OLS) stable nighttime light (NTL) data have been extensively used in a range of study fields, including socioeconomic research, electricity generation activity, population distribution, urban extent mapping, and environmental assessment (e.g., light pollution) [1,2,3,4,5,6,7,8,9]. The global coverage and long-temporal record (i.e., 1992–2013) advances the application of NTL datasets in related dynamic studies, e.g., the global dynamics of urban expansion, electricity consumption, and economy growth [10,11]. Acquiring the pathway (or trend) of historical socioeconomic dynamics is of great importance to future projections for sustainable development as well as deepening our understanding of drivers (or reasons) behind these changes [4,12,13,14]. However, due to the lack of on-board calibration, varied atmospheric conditions, satellite shift or sensor degradation, the obtained digital number (DN) of NTL time series cannot be directly used across years to detect these dynamics [15,16]. Therefore, a calibration procedure should first be implemented for the NTL datasets, before applying these to relevant dynamic studies [11,17,18].

A variety of studies have been carried out to calibrate the NTL dataset for achieving a temporally consistent and comparable time series. Elvidge et al. [16] proposed a general framework for calibrating NTL series using the second order regression model. In particular, a reference region that is relatively stable over years and a reference image derived in a particular year and satellite, are defined when implementing the calibration procedure. More specifically, the widely known Sicily, Italy was regarded as an invariant area in terms of socioeconomic activities over recent decades, and the image (featured as year plus satellite) with the maximum luminance (i.e., sum of lit DN values) over this island was selected as a reference for calibration [17,19,20]. This framework has been extensively adopted over the world due to its easy implementation and robust performance [21]. Moreover, it is flexible and can therefore be adjusted according to different study areas for particular applications, with reference regions such as Jixi county in China [22], Swain county in the United States [23], Lucknow and Nawabganj in India [24], and Okinawa in Japan [25]. The selection of reference region should meet the following requirements: (1) relative stability over time without dramatic changes; and (2) with a wide range of DN values to build the model [25,26]. In addition, several studies were performed based on the calibration framework of Elvidge et al. [16] with further modification. For instance, Zhang et al. [15] used the image that lies in the middle of the NTL temporal interval as the reference to minimize the bias caused by images that are temporally far away. Moreover, instead of a reference region, a ridgeline sampling approach was adopted to manually select representative samples for model calibration in their study. Wu et al. [25] employed a power function to calibrate the NTL time series using a radiance-calibrated image as a reference. The power function used is different with the conventional second order regression model, and the reference region was manually selected based on the luminance change of NTL time series in 1990, 2000, and 2010.

Until now, many calibration studies focused on regional or national scales (e.g., China and USA), and there have only been limited attempts at a global coverage and the whole temporal range (1992–2013) [15,17,25,27]. Therefore, there are several issues that remain to be addressed for NTL calibration at the global scale. First is the sensitivity of the selected reference image, including the selection of a stable region and the appropriate year (and satellite). There are two assumptions in the widely used framework of Elvidge et al. [16]: (1) the selected reference regions are invariant over time in terms of the nighttime luminosity; and (2) the selected reference image is unbiased and serves as a benchmark for calibration. However, these are always subjectively determined based on local knowledge, with diverse spatial and temporal references [15,17,22,23,25,28], which may result in systematic biases, and may therefore be incomparable among results derived from different reference datasets. Second, compared to the raw NTL time series, there is likely a notable modification in the calibrated NTL time series, because only one reference image is being used to calibrate the whole time series. For example, after the calibration using the approaches proposed by Elvidge et al. [16] and Zhang et al. [15], the range of total lights (DN values) in the world over the last two decades was narrowed down from an initial 242~470 (million) to around 280~380 (million). The range of total NTL DN values was considerably reduced with a flattened change of NTL time series for each satellite compared to the initial pattern [25]. Due to the unknown actual trend of NTL, the calibration with large modifications may lose the considerable amount information we can gain from the raw NTL series. In fact, these issues are crucial for achieving a reliable and consistent DMSP/OLS NTL time series to support subsequent dynamic studies.

To address the challenging issues (e.g., difficulties in selecting reference regions) in previous studies, we developed a stepwise calibration scheme for NTL data derived from different satellites and years. This study aims to provide a calibrated DMSP/OLS NTL time series (1992–2013) with the following criteria: (1) to improve the systematic consistency of NTL time series derived from different satellites (or periods) instead of the whole NTL dataset; (2) to make full use of the temporally neighbored image as a reference for calibration at the global level; and (3) to generate a temporally consistent long-term NTL time series with less modifications of the original DN values. The remaining parts of this paper are organized as: Section 2 briefly introduces the DMSP/OLS NTL dataset; Section 3 provides a detailed introduction of the stepwise calibration procedure; Section 4 presents calibrated results for comparison and validation over global and selected regions; the concluding remarks are given in Section 5.

2. DMSP/OLS NTL Time Series

The NTL time series used in this study is the stable DMSP/OLS NTL (Version 4) from NOAA (National Oceanic and Atmospheric Administration) (http://ngdc.noaa.gov/eog/dmsp/downloadV4composites.html). This dataset is a composite of annual cloud-free observations with six satellites including F10, F12, F14, F15, F16, and F18, spanning from 1992 to 2013 (see

Table 1. Annual composites of the stable Defense Meteorological Satellite Program/Operational Linescan System Nighttime Light (DMSP/OLS NTL) series.

Year	Satellite
1992	F10
1993	F10
1994	F10	F12
1995		F12
1996		F12
1997		F12	F14
1998		F12	F14
1999		F12	F14
2000			F14	F15
2001			F14	F15
2002			F14	F15
2003			F14	F15
2004				F15	F16
2005				F15	F16
2006				F15	F16
2007				F15	F16
2008					F16
2009					F16
2010						F18
2011						F18
2012						F18
2013						F18

). It measures lights from cities, towns, and other continuous lighting areas at night, in a form of DN ranging from 0 to 63. The spatial resolution of stable DMSP/OLS NTL is a 30 arc second (equals 1 km in equator), with a near global coverage of −180° to 180° in longitude and −65° to 65° in latitude [11]. In addition, for some years, there are two annual composites of NTL images derived from two satellites (see temporally overlaid images in Table 1). In this study, we used all images of the DMSP/OLS NTL time series for calibration to achieve a temporally consistent dataset.

3. Methodology

The raw DMSP/OLS NTL time series are temporally inconsistent with multiple trajectories among different satellites (Figure 1). Overall, the sum of NTL (SNTL) is increasing over time, which is more consistent with NTL images derived from satellites F10, F12, and F14. However, for the remaining three satellites (i.e., F15, F16, and F18), their temporal dynamics are more complicated without a clear trajectory. Therefore, we developed a stepwise calibration scheme for the continuous DMSP/OLS NTL time series, based on images derived from different satellites. This scheme includes four steps in a sequential order: (1) systematic adjustment of F14 for underestimation; (2) period-based adjustment of F15; (3) a hybrid calibration of F16; and (4) adjustment of F182010 (labeled as satellite plus year) for overestimation. This stepwise calibration aims to improve temporally inconsistent NTL time series separately for each satellite; thus, the information in the original NTL data can be maximally maintained with less modification. The calibrated NTL time series with a temporally more consistent trend and fewer modifications of DN values are crucial for further dynamic studies, such as urban expansion and electricity consumption. Each step of NTL calibration is described, in detail, in the following subsections.

3.1. Step 1: Systematic Adjustment of F14

We adjusted the systematic underestimation of F14 to make it consistent with the trajectory of F10 and F12. Based on the trajectories of NTL time series derived from different satellites (step 1 in Figure 1), we assumed that the temporal patterns of NTL images derived from F10 and F12 are reasonable since (1) they share the same increasing trajectory and (2) the temporally consistent observations (i.e., F101994 and F121994) are connected. Compared to NTL time series of F12 and F14, the temporal trend of F14 is reasonable, but there exists a systematic underestimation of F14 with a lower sum of DN (Figure 1, yellow ellipse). This inconsistency is likely attributed to the satellite change, which can be systematically adjusted [15]. Given that F12 and F14 have composites for the overlaid years (1997, 1998 and 1999), we used the second order regression model to build the relationship between these two satellites, see Equation (1) [16]:

$$D{N}_{ref}=a_0+a_{1}DN+a_{2}D{N}^2$$

(1)

where $D{N}_{ref}$ and $DN$ represent NTL time series derived from F12 and F14, respectively; $a_0$, $a_1$ and $a_2$ are three coefficients in the second order regression model. All lit pixels derived from satellite F12 and F14 in the overlaid years (1997, 1998 and 1999) confirm the robust relationship between these two satellites (Figure 2).

3.2. Step 2: Period-Based Adjustment of F15

We adjusted the underestimation of the sub-segments (2003–2007) of F15 to make it consistent with the trajectory of F14 and the earlier years (2000–2002) of F15. There is a notable decrease in the total DN value since 2003 in NTL time series derived from satellite F15. Thereafter, it increases slightly, since 2003 (step 2 in Figure 1, purple ellipse). There is also a systematic underestimation in the F15 NTL time series from 2003 to 2007 compared to the years of 2000–2002. We compared two temporally overlaid NTL images (i.e., calibrated F142003 in step 1 and F152003), and found that the differences of DN mainly occur around the urban fringe areas. Thus, we adjusted the NTL time series of 2003–2007 to be consistent with the historical period. We used all the lit pixels in calibrated F142003 and F152003 to modify the systematic underestimation of sub-segment (2003–2007) of F15. The relationship between NTL images of calibrated F142003 and F152003 (Figure 3) fits well with the second order regression model. Eventually, we applied this relationship to adjust NTL time series from F152003 to F152007.

3.3. Step 3: A Hybrid Calibration of F16

We adopted a widely used approach based on that used for Sicily, Italy to calibrate the NTL time series derived from F16 (Elvidge et al. [16]) because the trajectory of NTL time series derived from F16 does not show a temporally consistent pattern as other satellites do (step 3 in Figure 1, green ellipse), for which we then used the systematic adjustment. In this step, we chose F162007 as the reference for calibration due to its highest total DN value. The coefficients for the second order regression model are listed in

Table 2. Coefficients adopted for calibration of NTL series of F16.

Year	${\mathit{a}}_0$	${\mathit{a}}_1$	${\mathit{a}}_2$	${\mathit{R}}^2$
2004	0.1194	1.2265	−0.0041	0.9459
2005	−0.3209	1.4619	−0.0072	0.9765
2006	0.0877	1.1616	−0.0021	0.9733
2007	0	1	0	1
2008	0.1100	1.0513	−0.001	0.9820
2009	0.6294	1.1188	−0.0024	0.9548

.

We further conducted a systematic adjustment of NTL time series of F16. While calibration among all the NTL time series of satellite F16 reduces the temporal inconsistency (F162004~F162009), it still shows an overall underestimation compared to the temporally neighbored satellite F15 (after step 2). To address this issue, we adopted all the lit pixels in NTL images of F152007 and F162007 to build their relationship. We chose F162007 as a reference because its DN values did not change during the previous calibration step for NTL images of F16. The relationship of NTL images of calibrated F152007 and F162007 (Figure 4) fits well with the second order regression model. This relationship was then applied to calibrate the whole NTL time series of satellite F16 again.

3.4. Step 4: Adjustment of F182010

We adjusted the extremely high observations of NTL in F182010 to be consistent with the overall trend. Through checking the whole NTL time series, F182010 was regarded as an abnormal observation due to its magnitude of the total NTL DN values [15] (see step 4 in Figure 1, blue ellipse). Considering that there is only one NTL image in 2010, we here used the reference F162009, which had been calibrated in the previous step, to adjust the overestimated DN values of F182010. In this step, we used Sicily as a reference to build the second order regression model. The relationship between temporally neighbored NTL images of calibrated F162009 and F182010 (Figure 5) fits well with the second order regression model.

3.5. Evaluation of the Calibrated NTL Time Series

We evaluated the calibrated NTL time series from three different aspects for a comprehensive assessment. The most direct way is the visual comparison of NTL time series before and after calibration. Additionally, we used other two assessments for a quantitative evaluation. The first assessment we used is the comparison of indicators before and after calibration. We used the index of SNTL at the global and regional levels to compare the magnitudes of DN values before and after calibration. We also evaluated the agreement of NTL images derived from different satellites in the same year after calibration using an indicator of normalized difference index (NDI) [25]. These two indicators were calculated in Equations (2) and (3).

$$NTL={\displaystyle \sum }_{i}D{N}_i\times C_i$$

(2)

$$SNDI=\left|SNT{L}_1-SNT{L}_2\right|/SNT{L}_1+SNT{L}_2$$

(3)

where $D{N}_i$ indicates the $DN$ value at the $i$ level (i.e., ranges from 0–63) and $C_i$ is the total pixel number of corresponding $DN$ values; and $SNT{L}_1$ and $SNT{L}_2$ represent two $SNTL$ indices derived from two satellite images at the same year. A lower NDI indicates a higher degree of convergence of NTL images derived from different satellites. We compared the NDI (with or without calibration) using NTL images that were temporally overlaid from satellites of F12, F14, F15, and F16 (see Figure 1). The pair of F101994 and F121994 was not compared because there was no calibration applied for them.

In the second assessment, we investigated the Pearson correlation between the calibrated NTL time series and ancillary socioeconomic datasets (e.g., Gross Domestic Product (GDP) and electricity consumption, (EC)) [15,25]. The GDP datasets were obtained from the World Bank (http://data.worldbank.org/) and the EC records were derived from International Energy Statistics (http://www.eia.gov/beta/international/data/browser/#). We calculated correlation coefficients between the calibrated NTL with GDP and EC at the country level from 1992 to 2012. For the same year with two NTL satellite observations, we used their mean when comparing with GDP and EC. Additionally, we also compared our calibrated results with other two global NTL datasets, which were derived from Zhang et al. [15] and Elvidge et al. [17].

4. Results and Discussion

4.1. Improvement of NTL Data in Each Step

There is a notable improvement in terms of temporal consistency in the NTL time series after the stepwise calibration (Figure 6). Following the trend of F10 and F12, the systematic underestimation of NTL time series in the satellite F14 was adjusted after the step 1 calibration (Figure 6b). The trend of increasing SNTL of satellite F14 was maintained and the consistency with the NTL time series of satellite F12 was improved. After the step 2 calibration for satellite F15 (2003–2007), the calibrated time series with low SNTL values was improved and showed an increasing and consistent trajectory with the historical data (Figure 6c). In step 3, first, the annual fluctuations of NTL time series were reduced after the calibration of satellite F16 (see Figure 6c,d). However, there is an underestimation of SNTL compared to the NTL time series of satellites F15 and F16 (Figure 6d). Second, this underestimation was addressed through a systematic adjustment of satellite F16 based on the relationship between F152007 and F162007 (Figure 6e). In step 4, after the adjustment of F182010, the calibrated NTL was temporally consistent with its two neighboring satellites F162009 and F182011 (Figure 6f). Overall, after a series of stepwise calibration processes, the resulting NTL time series (1992–2013) (Figure 6f) are temporally more consistent, as compared to the raw NTL data (Figure 6a). In addition, the calibrated NTL time series maintain a similar range of SNTL (242~407 million) compared to the raw NTL data.

4.2. The Principle of Stepwise Calibration (Case of Satellite F15)

The stepwise calibration works by reducing the systematic over- and under- estimation in the NTL time series. We took Syria and the NTL time series of F15 as an example to evaluate the change of DN distribution before and after calibration (Figure 7). It is helpful to understand the gap generation of SNTL over different years as well as how the stepwise calibration worked for mitigating this gap. We selected the NTL image F152004 as an example and compared its DN distributions (with and without calibration) with the reference image F152002 (Figure 7a). There is a significant decrease (2003–2007) of SNTL for images derived from satellite F15, which has been systematically calibrated using calibrated F142003 and F152003. Therefore, we chose two temporally closest images—F152002, the raw data without any calibration, and F152004, the calibrated data after stepwise calibration—which were not involved in the procedure of obtaining coefficients as F152003, in addition to the raw data of F152004 for comparison (Figure 7a). Thus, through comparing the DN distributions derived from the raw, calibrated and referred images, we found that the distinctive differences in terms of SNTL in the raw F152004 and referred image F152002 occur at the urban fringe areas with relatively low luminance (see distribution differences colored as blue and green in Figure 7b), i.e., there exists a systematic underestimation of DN in raw F152004 when compared to F152002. This underestimation is depicted well by Equation (1) and Figure 3. After the stepwise calibration, the DN distribution was slightly adjusted with overall increased DN values (see the red line in Figure 7b), which was consistent in DN magnitude with the reference F152002. The change of DN distributions before and after calibration reveals the systematic correction (i.e., over- or under- estimation) of DN values caused by individual NTL images for achieving a consistent trend. Figure 7c,d shows the change of NTL images before and after calibration. Overall, their general patterns of luminance were similar, whereas those pixels with lower DN values were slightly increased after calibration (see red ellipses in Figure 7c,d).

4.3. Evaluation of the Stepwise Calibration Based on SNTL

The stepwise calibration reduced the temporal inconsistency (i.e., trend and SNTL range) in the raw NTL time series and the calibrated SNTL is in good agreement with the global EC (Figure 8). Compared to the results from Zhang et al. [15] and Elvidge et al. [17] with narrower ranges of SNTL (two bounding lines in Figure 8c,d) after calibration, the SNTL of our calibrated NTL data maintained a similar range as that which was in the raw NTL data. That is, the globally increasing trajectory of SNTL over the last two decades is more distinctive in our results; using the general calibration framework, the range of raw NTL time series is considerably reduced. In addition, when regarding the total electricity consumption, our results also achieved a better performance with a more consistent temporal trend, particularly in recent years (2010–2013) (see Figure 8b).

The comparison of SNTL before and after the calibration at the country level shows similar improvement in our calibrated NTL data as those at the global scale. We selected five representative countries (Brazil, China, Egypt, India, and South Africa) to examine their temporal trends of calibrated NTL time series (Figure 9). These regions experienced a rapid urbanization over the last two decades [29]; thus, their temporal trends of NTL DN were increasing as well. Overall, the calibrated NTL time series were more consistent, as compared to the raw data. Results derived from Elvidge et al. [17] and Zhang et al. [15] were similar in terms of their temporal patterns, whereas the ranges of SNTL were slightly reduced compared to the raw NTL data. This issue was well addressed using the proposed stepwise calibration, resulting in a similar range as the raw NTL data. In summary, all three calibration methods can achieve a temporally consistent increasing trend. However, the calibrated NTL time series for F16 and F18 are more consistent using the stepwise calibration, as compared to the other two approaches, with a noticeable gap between these two satellites in the calibrated NTL data.

4.4. Comparison between Different Satellites in the Same Year

Our calibrated NTL time series also shows a distinctive decrease of NDI for NTL images that derived from different satellites in the same year (Figure 10). This is because our stepwise calibration used temporally overlaid NTL images for different satellites in each step, except for the last step (F182010). The mean NDI in the raw NTL time series is 1.2, and this number was reduced to 0.6 after implementing the stepwise calibration (Figure 10a), i.e., most scatters are centered below the 1:1 line, which indicates the calibrated results from different satellites are closer in terms of the DN values. In addition, the spatial distribution of reduced NDI at the country-level also confirmed the narrowed disagreements of NTL images with different satellites in the same year (Figure 10b), particularly for most developing countries in such regions as South America, Africa and Asia. Therefore, the calibrated results are helpful to trace the NTL change over these regions with more reliable observations.

4.5. Comparison of NTL with Other Data

The calibrated NTL time series using the stepwise calibration shows a relatively high correlation (Pearson coefficient) with other socioeconomic indicators (GDP and EC) (see Figure 11). For our results and Zhang et al. [15], most scatters are above the 1:1 line when regarding the GDP or EC., whereas for the results of Elvidge et al. [17], this correlation has been weakened, with more scatters below the 1:1 line. The mean correlation coefficient of GDP over different countries for the stepwise based calibration is 0.53, which is higher than Zhang et al. [15] (0.48) and outperformed Elvidge et al. [17] (0.32) (see Figure 11a–c). When compared to EC, the correlation coefficients are higher with values of 0.59, 0.41 and 0.54 in terms of the mean correlations for the stepwise based results, Elvidge et al. [17] and Zhang et al. [15], respectively (see Figure 11d–f). That is, our calibrated results can contribute more to explaining the dynamics of socioeconomic activates (e.g., energy consumption or urban expansion), using the improved stable NTL time series [4,30,31].

5. Conclusions

In this study, we proposed a stepwise approach to calibrate the NTL time series for achieving a temporally more consistent NTL observation. Each step was designed to calibrate particular satellites (or years) with systematic over- or under-estimation. First, the overall underestimation of F14 was adjusted using referred NTL images derived from F12. Then, we modified the subset (2003–2007) of NTL time series of F15 to calibrate its inconsistent trend. Thereafter, a hybrid calibration scheme was implemented on F16, which used the calibration framework proposed by Elvidge et al. [16]. Furthermore, we systematically adjusted them to make the trend as consistent as the historical trajectory. Finally, we specifically improved the abnormal NTL image of F182010 to mitigate its overly-high DN values. Our calibrated results show a temporally more consistent pattern of NTL time series, as well as a good agreement with socioeconomic data, such as GDP and EC.

Different from the general calibration approaches using a reference image and same scheme for the entire NTL time series [15,16,25], the proposed stepwise calibration approach is aimed at calibrating systematic biases in each satellite (or period), with less modification of raw DN values. The calibrated results are improved significantly at both global and country scales, when compared to the raw NTL time series. Moreover, it also maintains a similar range of SNTL as the raw NTL data, which is different from other widely used approaches with narrowed ranges of SNTL. In addition, the stepwise calibration approach is more successfully aligned with socioeconomic indicators (e.g., GDP and EC), as compared to other approaches. Equally important, the stepwise calibration approach can be repeated without rigorous requirements on reference regions, which were always subjectively selected in different studies [15,25]. For instance, we used global references derived from different satellites at the same year mark for calibration, and only incorporated the commonly used example of Sicily in the final step of calibration for F182010.

The calibrated NTL time series using the stepwise calibration approach provides a useful input for future dynamic studies such as urbanization and electricity consumption, which highly rely on the calibrated NTL results to reflect the actual change [10,32]. It should be noted that our stepwise calibration scheme follows the order from the 1990s to 2010s. That means we assumed that the night light brightness increased from the 1990s to 2010s, and NTL series of F10 and F12 serve as a reference in the sequential stepwise calibration. Although we found that our calibrated results are reliable with regard to the change of SNTL, reduced NDI, and correlation with GDP and EC at a global scale, the trends of calibrated NTL series (e.g., F10 and F12) may not show a consistent increase in some countries. In addition, our results using the stepwise calibration still keep the dimming trend of nighttime light brightness in some countries (e.g., some of the former Soviet Union countries and Syria). This is because our calibration is a systematic adjustment of NTL data at the global level, i.e., the calibration is consistent for countries with increasing or decreasing NTL. With the newly launched Visible/Infrared Imager/Radiometer Suite (VIIRS) [33], the improved DMSP/OLS NTL time series can be further extended, with a more consistent trend and a longer temporal coverage.