GF-1 Satellite Imagery Data Service and Application Based on Open Data Cube
Abstract
:1. Introduction
2. Methods
2.1. Data Preprocessing Based on the FLASSH and RPC Methods
2.1.1. Radiometric Correction Based on the FLASSH Method
2.1.2. Geometric Normalization Based on the HighImgCorrect Method
2.2. Data Management Based on the Pixel-Level Grid
2.2.1. Database Connection and Initialization
2.2.2. Product Definition and GiST Index
- (I)
- The imagery data product definition focused on defining and describing the type of prestored imagery data. Considering the characteristics of GF-1 imagery metadata, the defined information included product name, basic metadata information, spatial reference information, and band information, which were saved in YAML format.
- (II)
- The image data format conversion was mainly the imagery data single-band extraction and YAML format metadata generation. Since ODC is based on single-band storage and the GF-1 imagery data are in a composite band format, a single-band separation operation is required to extract single-band data from the original images. To avoid the case of NoData in metadata documents of XML and other formats, we unified the NoData values of the original images to the parameters defined in YAML documents using the Geospatial Data Abstraction Library (GDAL) tool.
- (III)
- Index construction focused on storing the above YAML format data products in the PostgreSQL database with the GiST indexing method. GiST is a balanced tree structure access method, which can be used as a basic template to implement arbitrary indexing patterns such as b-trees and r-trees.
2.2.3. Data Access and Analysis
2.2.4. Data Ingestion
2.3. Comparative Analysis
3. Experiment and Results
3.1. Experiment Design and Environment
3.2. Study Area and Data
3.3. Experimental Results
3.3.1. Comparative Analysis of Time Efficiency for the Two Strategies
3.3.2. Comparative Analysis of Memory Efficiency for the Two Strategies
3.3.3. Spatiotemporal Analysis of NDVI and NDWI
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Overpeck, J.T.; Meehl, G.A.; Bony, S.; Easterling, D.R. Climate Data Challenges in the 21st Century. Science 2011, 331, 700–702. [Google Scholar] [CrossRef]
- Yao, X.; Li, G.; Xia, J.; Ben, J.; Cao, Q.; Zhao, L.; Ma, Y.; Zhang, L.; Zhu, D. Enabling the Big Earth Observation Data via Cloud Computing and DGGS: Opportunities and Challenges. Remote Sens. 2020, 12, 62. [Google Scholar] [CrossRef] [Green Version]
- Li, G.; Pang, L. A new age of public–oriented Earth observation development (In Chinese). Scientia Sinica Inf. 2017, 47, 193–206. [Google Scholar] [CrossRef]
- Anderson, K.; Ryan, B.; Sonntag, W.; Kavvada, A.; Friedl, L. Earth observation in service of the 2030 Agenda for Sustainable Development. Geo-Spat. Inf. Sci. 2017, 20, 77–96. [Google Scholar] [CrossRef]
- Giuliani, G.; Camara, G.; Killough, B.; Minchin, S. Earth Observation Open Science: Enhancing Reproducible Science Using Data Cubes. Data 2019, 4, 147. [Google Scholar] [CrossRef] [Green Version]
- Casu, F.; Manunta, M.; Agram, P.; Crippen, R. Big Remotely Sensed Data: Tools, applications and experiences. Remote Sens. Environ 2017, 202, 1–2. [Google Scholar] [CrossRef]
- Ross, J.; Killough, B.; Dhu, T.; Paget, M. Open Data Cube and the Committee on Earth Observation Satellites Data Cube Initiative; IAC: New York, NY, USA, 2017. [Google Scholar]
- Augustin, H.; Sudmanns, M.; Tiede, D.; Lang, S.; Baraldi, A. Semantic Earth Observation Data Cubes. Data 2019, 4, 102. [Google Scholar] [CrossRef] [Green Version]
- Vâjâială–Tomici, C.-M.; Filip, I.-D.; Pop, F. Landscape Change Monitoring using Satellite Data and Open Data Cube Platform. In Proceedings of the 2020 IEEE 16th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania, 3–5 September 2020; pp. 573–580. [Google Scholar]
- Lewis, A.; Oliver, S.; Lymburner, L.; Evans, B.; Wyborn, L.; Mueller, N.; Raevksi, G.; Hooke, J.; Woodcock, R.; Sixsmith, J.; et al. The Australian Geoscience Data Cube—Foundations and lessons learned. Remote Sens. Environ. 2017, 202, 276–292. [Google Scholar] [CrossRef]
- Dhu, T.; Dunn, B.; Lewis, B.; Lymburner, L.; Mueller, N.; Telfer, E.; Lewis, A.; McIntyre, A.; Minchin, S.; Phillips, C. Digital earth Australia—Unlocking new value from earth observation data. Big Earth Data 2017, 1, 64–74. [Google Scholar] [CrossRef] [Green Version]
- Krause, C.E.; Newey, V.; Alger, M.J.; Lymburner, L. Mapping and Monitoring the Multi-Decadal Dynamics of Australia’s Open Waterbodies Using Landsat. Remote Sens. 2021, 13, 1437. [Google Scholar] [CrossRef]
- Malthus, T.J.; Lehmann, E.; Ho, X.; Botha, E.; Anstee, J. Implementation of a Satellite Based Inland Water Algal Bloom Alerting System Using Analysis Ready Data. Remote Sens. 2019, 11, 2954. [Google Scholar] [CrossRef] [Green Version]
- Bishop–Taylor, R.; Nanson, R.; Sagar, S.; Lymburner, L. Mapping Australia’s dynamic coastline at mean sea level using three decades of Landsat imagery. Remote Sens. Environ. 2021, 267, 112734. [Google Scholar] [CrossRef]
- Lewis, A.; Lymburner, L.; Purss, M.B.J.; Brooke, B.; Evans, B.; Ip, A.; Dekker, A.G.; Irons, J.R.; Minchin, S.; Mueller, N.; et al. Rapid, high-resolution detection of environmental change over continental scales from satellite data—the Earth Observation Data Cube. Int. J. Digit. Earth 2016, 9, 106–111. [Google Scholar] [CrossRef]
- Brooke, B.; Lymburner, L.; Lewis, A. Coastal dynamics of Northern Australia–Insights from the Landsat Data Cube. Remote Sens. Appl. Soc. Environ. 2017, 8, 94–98. [Google Scholar] [CrossRef]
- Lucas, R.; Mueller, N.; Siggins, A.; Owers, C.; Clewley, D.; Bunting, P.; Kooymans, C.; Tissott, B.; Lewis, B.; Lymburner, L. Land cover mapping using digital earth Australia. Data 2019, 4, 143. [Google Scholar] [CrossRef] [Green Version]
- Honeck, E.; Castello, R.; Chatenoux, B.; Richard, J.-P.; Lehmann, A.; Giuliani, G. From a Vegetation Index to a Sustainable Development Goal Indicator: Forest Trend Monitoring Using Three Decades of Earth Observations across Switzerland. ISPRS Int. J. Geo–Inf. 2018, 7, 455. [Google Scholar] [CrossRef] [Green Version]
- Giuliani, G.; Chatenoux, B.; Piller, T.; Moser, F.; Lacroix, P. Data Cube on Demand (DCoD): Generating an earth observation Data Cube anywhere in the world. Int. J. Appl. Earth Obs. Geoinf. 2020, 87, 102035. [Google Scholar] [CrossRef]
- Chatenoux, B.; Richard, J.-P.; Small, D.; Roeoesli, C.; Wingate, V.; Poussin, C.; Rodila, D.; Peduzzi, P.; Steinmeier, C.; Ginzler, C. The Swiss data cube, analysis ready data archive using earth observations of Switzerland. Sci. Data 2021, 8, 1–11. [Google Scholar] [CrossRef]
- Poussin, C.; Guigoz, Y.; Palazzi, E.; Terzago, S.; Chatenoux, B.; Giuliani, G. Snow Cover Evolution in the Gran Paradiso National Park, Italian Alps, Using the Earth Observation Data Cube. Data 2019, 4, 138. [Google Scholar] [CrossRef] [Green Version]
- Giuliani, G.; Chatenoux, B.; De Bono, A.; Rodila, D.; Richard, J.-P.; Allenbach, K.; Dao, H.; Peduzzi, P. Building an Earth Observations Data Cube: Lessons learned from the Swiss Data Cube (SDC) on generating Analysis Ready Data (ARD). Big Earth Data 2017, 1, 100–117. [Google Scholar] [CrossRef] [Green Version]
- Killough, B. The impact of analysis ready data in the Africa regional data cube. In Proceedings of the IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, 28 July–2 August 2019; pp. 5646–5649. [Google Scholar]
- Yuan, F.; Repse, M.; Leith, A.; Rosenqvist, A.; Milcinski, G.; Moghaddam, N.F.; Dhar, T.; Burton, C.; Hall, L.; Jorand, C.; et al. An Operational Analysis Ready Radar Backscatter Dataset for the African Continent. Remote Sens. 2022, 14, 351. [Google Scholar] [CrossRef]
- Yuan, F.; Lewis, A.; Leith, A.; Dhar, T.; Gavin, D. Analysis Ready Data for Africa. In Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium, 11–16 July 2021; pp. 1789–1791. [Google Scholar]
- Mfundisi, K.; Mubea, K.; Yuan, F.; Burton, C.; Boamah, E. Acnalysing effects of drought on inundation extent and vegetation cover dynamics in the Okavango delta. Earth Space Sci. Open Arch 2021, 15, GC25B-0652. [Google Scholar] [CrossRef]
- Halabisky, A.M.; Mubea, K.; Mar, F.; Yuan, F.; Burton, C.; Birchall, E.; Moghaddam, N.F.; Adimou, G.; Mamane, B.; Ongo, D.O.; et al. Water Observations from Space: Accurate maps of surface water through time for the continent of Africa. Earth Space Sci. Open Archive 2022, 9. [Google Scholar] [CrossRef]
- Burton, C.; Yuan, F.; Ee-Faye, C.; Halabisky, M.; Ongo, D.; Mar, F.; Addabor, V.; Mamane, B.; Adimou, S. Co–Production of a 10-m Cropland Extent Map for Continental Africa using Sentinel-2, Cloud Computing, and the Open–Data-Cube. Earth Space Sci. Open Arch 2021, 10, GC451-0924. [Google Scholar] [CrossRef]
- Quang, N.H.; Tuan, V.A.; Hao, N.T.P.; Hang, L.T.T.; Hung, N.M.; Anh, V.L.; Phuong, L.T.M.; Carrie, R. Synthetic aperture radar and optical remote sensing image fusion for flood monitoring in the Vietnam lower Mekong basin: A prototype application for the Vietnam Open Data Cube. Eur. J. Remote Sens. 2019, 52, 599–612. [Google Scholar] [CrossRef] [Green Version]
- Luis, O.D.A.J.; Carlos, C.P.J.; Alfredo, S.M.H. Open Data Cube for Natural Resources Mapping in Mexico. In Proceedings of the 1st International Conference, Hangzhou, China, 20–21 April 2019; pp. 70–78. [Google Scholar]
- Ferreira, K.R.; Queiroz, G.R.; Vinhas, L.; Marujo, R.F.; Simoes, R.E.; Picoli, M.C.; Camara, G.; Cartaxo, R.; Gomes, V.C.; Santos, L.A. Earth observation data cubes for Brazil: Requirements, methodology and products. Remote Sens. 2020, 12, 4033. [Google Scholar] [CrossRef]
- Bravo, G.; Castro, H.; Moreno, A.; Ariza-Porras, C.; Galindo, G.; Cabrera, E.; Valbuena, S.; Lozano–Rivera, P. Architecture for a Colombian data cube using satellite imagery for environmental applications. In Proceedings of the Colombian Conference on Computing, Cali, Colombia, 19–22 September 2017; pp. 227–241. [Google Scholar]
- Gao, F.; Yue, P.; Jiang, L.; Zhipeng, C.; Liang, Z.; Shangguan, B.; Hu, L.; Zhao, S. GeoCube: A multi–source EO cube towards large-scale analysis. J. Remote Sens. 2021, 26, 1051–1066. [Google Scholar] [CrossRef]
- Liu, H.; Gong, P. 21st century daily seamless data cube reconstruction and seasonal to annual land cover and land use dynamics mapping–iMap (China) 1.0. Natl. Remote Sens. Bull. 2021, 25, 126–147. [Google Scholar] [CrossRef]
- Liu, H.; Gong, P.; Wang, J.; Wang, X.; Ning, G.; Xu, B. Production of global daily seamless data cubes and quantification of global land cover change from 1985 to 2020—iMap World 1.0. Remote Sens. Environ. 2021, 258, 112364. [Google Scholar] [CrossRef]
- San, A.B. Evaluation of different atmospheric correction algorithms for EO–1 Hyperion imagery. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2010, XXXVIII. [Google Scholar]
- Anderson, G.; Felde, G.; Hoke, M.; Ratkowski, A.; Cooley, T.; Chetwynd, J.; Gardner, J.; Adler-Golden, S.; Matthew, M.; Berk, A.; et al. MODTRAN4–Based Atmospheric Correction Algorithm: FLAASH (Fast Line–of-Sight Atmospheric Analysis of Spectral Hypercubes); SPIE: Bellingham, WA, USA, 2002; Volume 4725. [Google Scholar]
- Zhao, Y.; Shan, X.; Tang, P. Spatial Consistency Analysis and Relative Geometric Correction of Low Spatial Resolution Multi–source Remote Sensing Data. Remote Sens. Technol. Appl. 2014, 29, 155–163. [Google Scholar] [CrossRef]
- Zhao, Y. Principles and Methods of Remote Sensing Application Analysis (Chinese Edition); Science Press: Beijing, China, 2003; pp. 413–416. [Google Scholar]
- PostgreSQL: The World’s Most Advanced Open Source Database. Available online: https://www.postgresql.org/ (accessed on 1 March 2022).
- The Official YAML Web Site. Available online: https://yaml.org/ (accessed on 1 March 2022).
- Unidata|NetCDF. Available online: https://www.unidata.ucar.edu/software/netcdf/ (accessed on 1 March 2022).
- Rew, R.; Davis, G. NetCDF: An interface for scientific data access. IEEE Comput. Graph. Appl. 1990, 10, 76–82. [Google Scholar] [CrossRef]
- Li, M.; Wu, B.; Yan, C.; Zhou, W. Estimation of Vegetation Fraction in the Upper Basin of Miyun Reservoir by Remote Sensing. Resour. Sci. 2004, 26, 153–159. [Google Scholar]
- Zhang, B. Analysis on the change trend of water quality of Miyun reservoir introduced from the middle route of Sout-to–North water transfer Project. Beijing Water 2018, z2, 27–32. [Google Scholar] [CrossRef]
- Crippen, R.E. Calculating the vegetation index faster. Remote Sens. Environ. 1990, 34, 71–73. [Google Scholar] [CrossRef]
- Gao, B.-C. Normalized difference water index for remote sensing of vegetation liquid water from space. In Proceedings of the Imaging Spectrometry, Orlando, FL, USA, 12 June 1995; pp. 225–236. [Google Scholar]
- McFeeters, S.K. The use of the Normalized Difference Water Index (NDWI) in the delineation of open water features. Int. J. Remote Sens. 1996, 17, 1425–1432. [Google Scholar] [CrossRef]
- GitHub—Pixelb/ps_mem: A utility to Accurately Report the in Core Memory Usage for a Program. Available online: https://github.com/pixelb/ps_mem (accessed on 1 March 2022).
- Guoqing, L.; Zhenchun, H. Data infrastructure for remote sensing big data: Integration, management and on-demand service. J. Comput. Res. Dev. 2017, 54, 267. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Cao, Q.; Li, G.; Yao, X.; Jia, T.; Yu, G.; Zhang, L.; Xu, D.; Zhang, H.; Shan, X. GF-1 Satellite Imagery Data Service and Application Based on Open Data Cube. Appl. Sci. 2022, 12, 7816. https://doi.org/10.3390/app12157816
Cao Q, Li G, Yao X, Jia T, Yu G, Zhang L, Xu D, Zhang H, Shan X. GF-1 Satellite Imagery Data Service and Application Based on Open Data Cube. Applied Sciences. 2022; 12(15):7816. https://doi.org/10.3390/app12157816
Chicago/Turabian StyleCao, Qianqian, Guoqing Li, Xiaochuang Yao, Tao Jia, Guojiang Yu, Lianchong Zhang, Dan Xu, Hao Zhang, and Xiaojun Shan. 2022. "GF-1 Satellite Imagery Data Service and Application Based on Open Data Cube" Applied Sciences 12, no. 15: 7816. https://doi.org/10.3390/app12157816
APA StyleCao, Q., Li, G., Yao, X., Jia, T., Yu, G., Zhang, L., Xu, D., Zhang, H., & Shan, X. (2022). GF-1 Satellite Imagery Data Service and Application Based on Open Data Cube. Applied Sciences, 12(15), 7816. https://doi.org/10.3390/app12157816