Remote Sensing

Research

Jump to: Review

20 pages, 1537 KiB

Open AccessArticle

Decision Fusion Framework for Hyperspectral Image Classification Based on Markov and Conditional Random Fields

by Vera Andrejchenko, Wenzhi Liao, Wilfried Philips and Paul Scheunders

Remote Sens. 2019, 11(6), 624; https://doi.org/10.3390/rs11060624 - 14 Mar 2019

Cited by 16 | Viewed by 4429

Abstract

Classification of hyperspectral images is a challenging task owing to the high dimensionality of the data, limited ground truth data, collinearity of the spectra and the presence of mixed pixels. Conventional classification techniques do not cope well with these problems. Thus, in addition [...] Read more.

Classification of hyperspectral images is a challenging task owing to the high dimensionality of the data, limited ground truth data, collinearity of the spectra and the presence of mixed pixels. Conventional classification techniques do not cope well with these problems. Thus, in addition to the spectral information, features were developed for a more complete description of the pixels, e.g., containing contextual information at the superpixel level or mixed pixel information at the subpixel level. This has encouraged an evolution of fusion techniques which use these myriad of multiple feature sets and decisions from individual classifiers to be employed in a joint manner. In this work, we present a flexible decision fusion framework addressing these issues. In a first step, we propose to use sparse fractional abundances as decision source, complementary to class probabilities obtained from a supervised classifier. This specific selection of complementary decision sources enables the description of a pixel in a more complete way, and is expected to mitigate the effects of small training samples sizes. Secondly, we propose to apply a fusion scheme, based on the probabilistic graphical Markov Random Field (MRF) and Conditional Random Field (CRF) models, which inherently employ spatial information into the fusion process. To strengthen the decision fusion process, consistency links across the different decision sources are incorporated to encourage agreement between their decisions. The proposed framework offers flexibility such that it can be extended with additional decision sources in a straightforward way. Experimental results conducted on two real hyperspectral images show superiority over several other approaches in terms of classification performance when very limited training data is available. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Figure 1

25 pages, 3882 KiB

Open AccessArticle

SDF-MAN: Semi-Supervised Disparity Fusion with Multi-Scale Adversarial Networks

by Can Pu, Runzi Song, Radim Tylecek, Nanbo Li and Robert B. Fisher

Remote Sens. 2019, 11(5), 487; https://doi.org/10.3390/rs11050487 - 27 Feb 2019

Cited by 6 | Viewed by 4646

Abstract

Refining raw disparity maps from different algorithms to exploit their complementary advantages is still challenging. Uncertainty estimation and complex disparity relationships among pixels limit the accuracy and robustness of existing methods and there is no standard method for fusion of different kinds of [...] Read more.

Refining raw disparity maps from different algorithms to exploit their complementary advantages is still challenging. Uncertainty estimation and complex disparity relationships among pixels limit the accuracy and robustness of existing methods and there is no standard method for fusion of different kinds of depth data. In this paper, we introduce a new method to fuse disparity maps from different sources, while incorporating supplementary information (intensity, gradient, etc.) into a refiner network to better refine raw disparity inputs. A discriminator network classifies disparities at different receptive fields and scales. Assuming a Markov Random Field for the refined disparity map produces better estimates of the true disparity distribution. Both fully supervised and semi-supervised versions of the algorithm are proposed. The approach includes a more robust loss function to inpaint invalid disparity values and requires much less labeled data to train in the semi-supervised learning mode. The algorithm can be generalized to fuse depths from different kinds of depth sources. Experiments explored different fusion opportunities: stereo-monocular fusion, stereo-ToF fusion and stereo-stereo fusion. The experiments show the superiority of the proposed algorithm compared with the most recent algorithms on public synthetic datasets (Scene Flow, SYNTH3, our synthetic garden dataset) and real datasets (Kitti2015 dataset and Trimbot2020 Garden dataset). Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Figure 1

29 pages, 11763 KiB

Open AccessFeature PaperArticle

A Markovian Approach to Unsupervised Change Detection with Multiresolution and Multimodality SAR Data

by David Solarna, Gabriele Moser and Sebastiano B. Serpico

Remote Sens. 2018, 10(11), 1671; https://doi.org/10.3390/rs10111671 - 23 Oct 2018

Cited by 8 | Viewed by 3306

Abstract

In the framework of synthetic aperture radar (SAR) systems, current satellite missions make it possible to acquire images at very high and multiple spatial resolutions with short revisit times. This scenario conveys a remarkable potential in applications to, for instance, environmental monitoring and [...] Read more.

In the framework of synthetic aperture radar (SAR) systems, current satellite missions make it possible to acquire images at very high and multiple spatial resolutions with short revisit times. This scenario conveys a remarkable potential in applications to, for instance, environmental monitoring and natural disaster recovery. In this context, data fusion and change detection methodologies play major roles. This paper proposes an unsupervised change detection algorithm for the challenging case of multimodal SAR data collected by sensors operating at multiple spatial resolutions. The method is based on Markovian probabilistic graphical models, graph cuts, linear mixtures, generalized Gaussian distributions, Gram–Charlier approximations, maximum likelihood and minimum mean squared error estimation. It benefits from the SAR images acquired at multiple spatial resolutions and with possibly different modalities on the considered acquisition times to generate an output change map at the finest observed resolution. This is accomplished by modeling the statistics of the data at the various spatial scales through appropriate generalized Gaussian distributions and by iteratively estimating a set of virtual images that are defined on the pixel grid at the finest resolution and would be collected if all the sensors could work at that resolution. A Markov random field framework is adopted to address the detection problem by defining an appropriate multimodal energy function that is minimized using graph cuts. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

14 pages, 2829 KiB

Open AccessArticle

Feature Importance Analysis for Local Climate Zone Classification Using a Residual Convolutional Neural Network with Multi-Source Datasets

by Chunping Qiu, Michael Schmitt, Lichao Mou, Pedram Ghamisi and Xiao Xiang Zhu

Remote Sens. 2018, 10(10), 1572; https://doi.org/10.3390/rs10101572 - 1 Oct 2018

Cited by 73 | Viewed by 7778

Abstract

Global Local Climate Zone (LCZ) maps, indicating urban structures and land use, are crucial for Urban Heat Island (UHI) studies and also as starting points to better understand the spatio-temporal dynamics of cities worldwide. However, reliable LCZ maps are not available on a [...] Read more.

Global Local Climate Zone (LCZ) maps, indicating urban structures and land use, are crucial for Urban Heat Island (UHI) studies and also as starting points to better understand the spatio-temporal dynamics of cities worldwide. However, reliable LCZ maps are not available on a global scale, hindering scientific progress across a range of disciplines that study the functionality of sustainable cities. As a first step towards large-scale LCZ mapping, this paper tries to provide guidance about data/feature choice. To this end, we evaluate the spectral reflectance and spectral indices of the globally available Sentinel-2 and Landsat-8 imagery, as well as the Global Urban Footprint (GUF) dataset, the OpenStreetMap layers buildings and land use and the Visible Infrared Imager Radiometer Suite (VIIRS)-based Nighttime Light (NTL) data, regarding their relevance for discriminating different Local Climate Zones (LCZs). Using a Residual convolutional neural Network (ResNet), a systematic analysis of feature importance is performed with a manually-labeled dataset containing nine cities located in Europe. Based on the investigation of the data and feature choice, we propose a framework to fully exploit the available datasets. The results show that GUF, OSM and NTL can contribute to the classification accuracy of some LCZs with relatively few samples, and it is suggested that Landsat-8 and Sentinel-2 spectral reflectances should be jointly used, for example in a majority voting manner, as proven by the improvement from the proposed framework, for large-scale LCZ mapping. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Figure 1

22 pages, 6202 KiB

Open AccessArticle

A Prediction Smooth Method for Blending Landsat and Moderate Resolution Imagine Spectroradiometer Images

by Detang Zhong and Fuqun Zhou

Remote Sens. 2018, 10(9), 1371; https://doi.org/10.3390/rs10091371 - 29 Aug 2018

Cited by 17 | Viewed by 5373

Abstract

Landsat images have been widely used in support of responsible development of natural resources, disaster risk management (e.g., forest fire, flooding etc.), agricultural production monitoring, as well as environmental change studies due to its medium spatial resolution and rich spectral information. However, its [...] Read more.

Landsat images have been widely used in support of responsible development of natural resources, disaster risk management (e.g., forest fire, flooding etc.), agricultural production monitoring, as well as environmental change studies due to its medium spatial resolution and rich spectral information. However, its availability and usability are largely constrained by its low revisit frequency. On the other hand, MODIS (Moderate Resolution Imaging Spectroradiometer) images for land studies have much more frequent coverage but with a lower spatial resolution of 250–500 m. To take advantages of the two sensors and expand their availability and usability, during the last decade, a number of image fusion methods have been developed for generating Landsat-like images from MODIS observations to supplement clear-sky Landsat imagery. However, available methods are typically effective or applicable for certain applications. For a better result, a new Prediction Smooth Reflectance Fusion Model (PSRFM) for blending Landsat and MODIS images is proposed. PSRFM consists of a dynamic prediction model and a smoothing filter. The dynamic prediction model generates synthetic Landsat images from a pair of Landsat and MODIS images and another MODIS image, either forward or backward in time. The smoothing filter combines the forward and backward predictions by weighted average based on elapsed time or on the estimated prediction uncertainty. Optionally, the smooth filtering can be applied with constraints based on Normalized Difference Snow Index (NDSI) or Normalized Difference Vegetation Index (NDVI). In comparison to some published reflectance fusion methods, PSRFM shows the following desirable characteristics: (1) it can deal with one pair or two pairs of Landsat and MODIS images; (2) it can incorporate input image uncertainty during prediction and estimate prediction uncertainty; (3) it can track gradual vegetation phenological changes and deal with abrupt land-cover type changes; and (4) for predictions using two pairs of input images, the results can be further improved through the constrained smoothing filter based on NDSI or NDVI for certain applications. We tested PSRFM to generate a Landsat-like image time series by using Landsat 8 OLI and MODIS (MOD09GA) images and compared it to two reflectance fusion algorithms: STARFM (Spatial and Temporal Adaptive Reflectance Fusion Model) and ESTARFM (Enhanced version of STARFM). The results show that the proposed PSRFM is effective and outperforms STARFM and ESTARFM both visually and quantitatively. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Figure 1

28 pages, 10396 KiB

Open AccessArticle

HOMPC: A Local Feature Descriptor Based on the Combination of Magnitude and Phase Congruency Information for Multi-Sensor Remote Sensing Images

by Zhitao Fu, Qianqing Qin, Bin Luo, Hong Sun and Chun Wu

Remote Sens. 2018, 10(8), 1234; https://doi.org/10.3390/rs10081234 - 6 Aug 2018

Cited by 27 | Viewed by 5283

Abstract

Local region description of multi-sensor images remains a challenging task in remote sensing image analysis and applications due to the non-linear radiation variations between images. This paper presents a novel descriptor based on the combination of the magnitude and phase congruency information of [...] Read more.

Local region description of multi-sensor images remains a challenging task in remote sensing image analysis and applications due to the non-linear radiation variations between images. This paper presents a novel descriptor based on the combination of the magnitude and phase congruency information of local regions to capture the common features of images with non-linear radiation changes. We first propose oriented phase congruency maps (PCMs) and oriented magnitude binary maps (MBMs) using the multi-oriented phase congruency and magnitude information of log-Gabor filters. The two feature vectors are then quickly constructed based on the convolved PCMs and MBMs. Finally, a dense descriptor named the histograms of oriented magnitude and phase congruency (HOMPC) is developed by combining the histograms of oriented phase congruency (HPC) and the histograms of oriented magnitude (HOM) to capture the structure and shape properties of local regions. HOMPC was evaluated with three datasets composed of multi-sensor remote sensing images obtained from unmanned ground vehicle, unmanned aerial vehicle, and satellite platforms. The descriptor performance was evaluated by recall, precision, F1-measure, and area under the precision-recall curve. The experimental results showed the advantages of the HOM and HPC combination and confirmed that HOMPC is far superior to the current state-of-the-art local feature descriptors. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

19 pages, 7789 KiB

Open AccessArticle

Matching Multi-Sensor Remote Sensing Images via an Affinity Tensor

by Shiyu Chen, Xiuxiao Yuan, Wei Yuan, Jiqiang Niu, Feng Xu and Yong Zhang

Remote Sens. 2018, 10(7), 1104; https://doi.org/10.3390/rs10071104 - 11 Jul 2018

Cited by 8 | Viewed by 5560

Abstract

Matching multi-sensor remote sensing images is still a challenging task due to textural changes and non-linear intensity differences. In this paper, a novel matching method is proposed for multi-sensor remote sensing images. To establish feature correspondences, an affinity tensor is used to integrate [...] Read more.

Matching multi-sensor remote sensing images is still a challenging task due to textural changes and non-linear intensity differences. In this paper, a novel matching method is proposed for multi-sensor remote sensing images. To establish feature correspondences, an affinity tensor is used to integrate geometric and radiometric information. The matching process consists of three steps. First, features from an accelerated segment test are extracted from both source and target images, and two complete graphs are constructed with their nodes representing these features. Then, the geometric and radiometric similarities of the feature points are represented by the three-order affinity tensor, and the initial feature correspondences are established by tensor power iteration. Finally, a tensor-based mismatch detection process is conducted to purify the initial matched points. The robustness and capability of the proposed method are tested with a variety of remote sensing images such as Ziyuan-3 backward, Ziyuan-3 nadir, Gaofen-1, Gaofen-2, unmanned aerial vehicle platform, and Jilin-1. The experiments show that the average matching recall is greater than 0.5, which outperforms state-of-the-art multi-sensor image-matching algorithms such as SIFT, SURF, NG-SIFT, OR-SIFT and GOM-SIFT. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Figure 1

22 pages, 5314 KiB

Open AccessArticle

Combining Linear Pixel Unmixing and STARFM for Spatiotemporal Fusion of Gaofen-1 Wide Field of View Imagery and MODIS Imagery

by Jintian Cui, Xin Zhang and Muying Luo

Remote Sens. 2018, 10(7), 1047; https://doi.org/10.3390/rs10071047 - 3 Jul 2018

Cited by 24 | Viewed by 4626

Abstract

Spatiotemporal fusion of remote sensing data is essential for generating high spatial and temporal resolution data by taking advantage of high spatial resolution and high temporal resolution imageries. At present, the Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM) is one of the [...] Read more.

Spatiotemporal fusion of remote sensing data is essential for generating high spatial and temporal resolution data by taking advantage of high spatial resolution and high temporal resolution imageries. At present, the Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM) is one of the most widely used spatiotemporal fusion technologies of remote sensing data. However, the quality of data acquired by STARFM depends on temporal information from homogeneous land cover patches at the MODIS (Moderate Resolution Imaging Spectroradiometer) imagery, and the estimation accuracy of STARFM degrades in highly fragmentated and heterogeneous patches. To address this problem, we developed an innovative method to improve fusion accuracy, especially in areas of high heterogeneity, by combining linear pixel unmixing and STARFM. This method derived the input data of STARFM by downscaling the MODIS data with a linear spectral mixture model. Through this fusion method, the complement effect of the advantages of remote sensing information can be realized, and the multi-source remote sensing data can be realized for visual data mining. The developed fusion method was applied in Bosten Lake, the largest freshwater lake in China, and our analysis of results suggests that (1) after introducing the linear spectral mixture model, the fusion images illustrated improved spatial details to a certain extent and can be employed to identify small objects, as well as their texture distribution information; (2) for fragmented and highly heterogeneous areas, a stronger correlation between the predicted results and the real images was observed when compared to STARFM with small bias; and (3) the predicted red band and near infrared band can generate high-precision 16-m NDVI (Normalized Difference Vegetation Index) data with advantages in both spatial resolution and temporal resolution. The results are generally consistent with the Gaofen-1 wide field of view cameras (GF-1 WFV) NDVI in the same period and therefore can reflect the spatial distribution of NDVI in detail. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

18 pages, 4574 KiB

Open AccessArticle

An Enhanced Linear Spatio-Temporal Fusion Method for Blending Landsat and MODIS Data to Synthesize Landsat-Like Imagery

by Bo Ping, Yunshan Meng and Fenzhen Su

Remote Sens. 2018, 10(6), 881; https://doi.org/10.3390/rs10060881 - 5 Jun 2018

Cited by 21 | Viewed by 4228

Abstract

Landsat and MODIS data have been widely utilized in many remote sensing applications, however, the trade-off between the spatial resolution and temporal frequency has limited their capacities in monitoring detailed spatio-temporal dynamics. Spatio-temporal fusion methods based on a linear model that considers the [...] Read more.

Landsat and MODIS data have been widely utilized in many remote sensing applications, however, the trade-off between the spatial resolution and temporal frequency has limited their capacities in monitoring detailed spatio-temporal dynamics. Spatio-temporal fusion methods based on a linear model that considers the differences between fine- and coarse-spatial-resolution images as linear can effectively solve this trade-off problem, yet the existing linear fusion methods either regard the coefficients of the linear model as constants or have adopted regression methods to calculate the coefficients, both of which may introduce some errors in the fusion process. In this paper, we proposed an enhanced linear spatio-temporal fusion method (ELSTFM) to improve the data fusion accuracy. In the ELSTFM, it is not necessary to calculate the slope of the linear model, and the intercept, which can be deemed as the residual caused by systematic biases, is calculated based on spectral unmixing theory. Additionally, spectrally similar pixels in a given fine-spatial-resolution pixel’s neighborhood and their corresponding weights were used in the proposed method to mitigate block effects. Landsat-7/ETM+ and 8-day composite MODIS reflectance data covering two study sites with heterogeneous and homogenous landscapes were selected to validate the proposed method. Compared to three other typical spatio-temporal fusion methods visually and quantitatively, the predicted images obtained from ELSTFM could acquire better results for the two selected study sites. Furthermore, the resampling methods used to resample MODIS to the same spatial resolution of Landsat could slightly, but did not significantly influence the fusion accuracy, and the distributions of slopes of different bands for the two study sites could all be deemed as normal distributions with a mean value close to 1. The performance of ELSTFM depends on the accuracy of residual calculation at fine-resolution and large landscape changes may influence the fusion accuracy. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

23 pages, 41039 KiB

Open AccessArticle

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

by Jingxiang Yang, Yong-Qiang Zhao and Jonathan Cheung-Wai Chan

Remote Sens. 2018, 10(5), 800; https://doi.org/10.3390/rs10050800 - 21 May 2018

Cited by 171 | Viewed by 11832

Abstract

Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, [...] Read more.

Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

18 pages, 6142 KiB

Open AccessArticle

A Modified Spatiotemporal Fusion Algorithm Using Phenological Information for Predicting Reflectance of Paddy Rice in Southern China

by Mengxue Liu, Xiangnan Liu, Ling Wu, Xinyu Zou, Tian Jiang and Bingyu Zhao

Remote Sens. 2018, 10(5), 772; https://doi.org/10.3390/rs10050772 - 17 May 2018

Cited by 29 | Viewed by 4321

Abstract

Satellite data for studying surface dynamics in heterogeneous landscapes are missing due to frequent cloud contamination, low temporal resolution, and technological difficulties in developing satellites. A modified spatiotemporal fusion algorithm for predicting the reflectance of paddy rice is presented in this paper. The [...] Read more.

Satellite data for studying surface dynamics in heterogeneous landscapes are missing due to frequent cloud contamination, low temporal resolution, and technological difficulties in developing satellites. A modified spatiotemporal fusion algorithm for predicting the reflectance of paddy rice is presented in this paper. The algorithm uses phenological information extracted from a moderate-resolution imaging spectroradiometer enhanced vegetation index time series to improve the enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM). The algorithm is tested with satellite data on Yueyang City, China. The main contribution of the modified algorithm is the selection of similar neighborhood pixels by using phenological information to improve accuracy. Results show that the modified algorithm performs better than ESTARFM in visual inspection and quantitative metrics, especially for paddy rice. This modified algorithm provides not only new ideas for the improvement of spatiotemporal data fusion method, but also technical support for the generation of remote sensing data with high spatial and temporal resolution. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

24 pages, 14557 KiB

Open AccessArticle

Semantic Labeling of High Resolution Aerial Imagery and LiDAR Data with Fine Segmentation Network

by Xuran Pan, Lianru Gao, Andrea Marinoni, Bing Zhang, Fan Yang and Paolo Gamba

Remote Sens. 2018, 10(5), 743; https://doi.org/10.3390/rs10050743 - 11 May 2018

Cited by 64 | Viewed by 6583

Abstract

In this paper, a novel convolutional neural network (CNN)-based architecture, named fine segmentation network (FSN), is proposed for semantic segmentation of high resolution aerial images and light detection and ranging (LiDAR) data. The proposed architecture follows the encoder–decoder paradigm and the multi-sensor fusion [...] Read more.

In this paper, a novel convolutional neural network (CNN)-based architecture, named fine segmentation network (FSN), is proposed for semantic segmentation of high resolution aerial images and light detection and ranging (LiDAR) data. The proposed architecture follows the encoder–decoder paradigm and the multi-sensor fusion is accomplished in the feature-level using multi-layer perceptron (MLP). The encoder consists of two parts: the main encoder based on the convolutional layers of Vgg-16 network for color-infrared images and a lightweight branch for LiDAR data. In the decoder stage, to adaptively upscale the coarse outputs from encoder, the Sub-Pixel convolution layers replace the transposed convolutional layers or other common up-sampling layers. Based on this design, the features from different stages and sensors are integrated for a MLP-based high-level learning. In the training phase, transfer learning is employed to infer the features learned from generic dataset to remote sensing data. The proposed FSN is evaluated by using the International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam and Vaihingen 2D Semantic Labeling datasets. Experimental results demonstrate that the proposed framework can bring considerable improvement to other related networks. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

20 pages, 3596 KiB

Open AccessArticle

Automatic Discovery and Geotagging of Objects from Street View Imagery

by Vladimir A. Krylov, Eamonn Kenny and Rozenn Dahyot

Remote Sens. 2018, 10(5), 661; https://doi.org/10.3390/rs10050661 - 24 Apr 2018

Cited by 64 | Viewed by 14592

Abstract

Many applications, such as autonomous navigation, urban planning, and asset monitoring, rely on the availability of accurate information about objects and their geolocations. In this paper, we propose the automatic detection and computation of the coordinates of recurring stationary objects of interest using [...] Read more.

Many applications, such as autonomous navigation, urban planning, and asset monitoring, rely on the availability of accurate information about objects and their geolocations. In this paper, we propose the automatic detection and computation of the coordinates of recurring stationary objects of interest using street view imagery. Our processing pipeline relies on two fully convolutional neural networks: the first segments objects in the images, while the second estimates their distance from the camera. To geolocate all the detected objects coherently we propose a novel custom Markov random field model to estimate the objects’ geolocation. The novelty of the resulting pipeline is the combined use of monocular depth estimation and triangulation to enable automatic mapping of complex scenes with the simultaneous presence of multiple, visually similar objects of interest. We validate experimentally the effectiveness of our approach on two object classes: traffic lights and telegraph poles. The experiments report high object recall rates and position precision of approximately 2 m, which is approaching the precision of single-frequency GPS receivers. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

21 pages, 22099 KiB

Open AccessArticle

Urban Land Use Mapping by Combining Remote Sensing Imagery and Mobile Phone Positioning Data

by Yuanxin Jia, Yong Ge, Feng Ling, Xian Guo, Jianghao Wang, Le Wang, Yuehong Chen and Xiaodong Li

Remote Sens. 2018, 10(3), 446; https://doi.org/10.3390/rs10030446 - 12 Mar 2018

Cited by 63 | Viewed by 11400

Abstract

Land use is of great importance for urban planning, environmental monitoring, and transportation management. Several methods have been proposed to obtain land use maps of urban areas, and these can be classified into two categories: remote sensing methods and social sensing methods. However, [...] Read more.

Land use is of great importance for urban planning, environmental monitoring, and transportation management. Several methods have been proposed to obtain land use maps of urban areas, and these can be classified into two categories: remote sensing methods and social sensing methods. However, remote sensing and social sensing approaches have specific disadvantages regarding the description of social and physical features, respectively. Therefore, an appropriate fusion strategy is vital for large-area land use mapping. To address this issue, we propose an efficient land use mapping method that combines remote sensing imagery (RSI) and mobile phone positioning data (MPPD) for large areas. We implemented this method in two steps. First, a support vector machine was adopted to classify the RSI and MPPD. Then, the two classification results were fused using a decision fusion strategy to generate the land use map. The proposed method was applied to a case study of the central area of Beijing. The experimental results show that the proposed method improved classification accuracy compared with that achieved using MPPD alone, validating the efficacy of this new approach for identifying land use. Based on the land use map and MPPD data, activity density in key zones during daytime and nighttime was analyzed to illustrate the volume and variation of people working and living across different regions. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

23 pages, 20794 KiB

Open AccessArticle

Matching of Remote Sensing Images with Complex Background Variations via Siamese Convolutional Neural Network

by Haiqing He, Min Chen, Ting Chen and Dajun Li

Remote Sens. 2018, 10(2), 355; https://doi.org/10.3390/rs10020355 - 24 Feb 2018

Cited by 74 | Viewed by 8086

Abstract

Feature-based matching methods have been widely used in remote sensing image matching given their capability to achieve excellent performance despite image geometric and radiometric distortions. However, most of the feature-based methods are unreliable for complex background variations, because the gradient or other image [...] Read more.

Feature-based matching methods have been widely used in remote sensing image matching given their capability to achieve excellent performance despite image geometric and radiometric distortions. However, most of the feature-based methods are unreliable for complex background variations, because the gradient or other image grayscale information used to construct the feature descriptor is sensitive to image background variations. Recently, deep learning-based methods have been proven suitable for high-level feature representation and comparison in image matching. Inspired by the progresses made in deep learning, a new technical framework for remote sensing image matching based on the Siamese convolutional neural network is presented in this paper. First, a Siamese-type network architecture is designed to simultaneously learn the features and the corresponding similarity metric from labeled training examples of matching and non-matching true-color patch pairs. In the proposed network, two streams of convolutional and pooling layers sharing identical weights are arranged without the manually designed features. The number of convolutional layers is determined based on the factors that affect image matching. The sigmoid function is employed to compute the matching and non-matching probabilities in the output layer. Second, a gridding sub-pixel Harris algorithm is used to obtain the accurate localization of candidate matches. Third, a Gaussian pyramid coupling quadtree is adopted to gradually narrow down the searching space of the candidate matches, and multiscale patches are compared synchronously. Subsequently, a similarity measure based on the output of the sigmoid is adopted to find the initial matches. Finally, the random sample consensus algorithm and the whole-to-local quadratic polynomial constraints are used to remove false matches. In the experiments, different types of satellite datasets, such as ZY3, GF1, IKONOS, and Google Earth images, with complex background variations are used to evaluate the performance of the proposed method. The experimental results demonstrate that the proposed method, which can significantly improve the matching performance of multi-temporal remote sensing images with complex background variations, is better than the state-of-the-art matching methods. In our experiments, the proposed method obtained a large number of evenly distributed matches (at least 10 times more than other methods) and achieved a high accuracy (less than 1 pixel in terms of root mean square error). Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Graphical abstract

Review

Jump to: Research

28 pages, 35856 KiB

Open AccessReview

True Orthophoto Generation from Aerial Frame Images and LiDAR Data: An Update

by Hamid Gharibi and Ayman Habib

Remote Sens. 2018, 10(4), 581; https://doi.org/10.3390/rs10040581 - 9 Apr 2018

Cited by 21 | Viewed by 14548

Abstract

Image spectral and Light Detection and Ranging (LiDAR) positional information can be related through the orthophoto generation process. Orthophotos have a uniform scale and represent all objects in their correct planimetric locations. However, orthophotos generated using conventional methods suffer from an artifact known [...] Read more.

Image spectral and Light Detection and Ranging (LiDAR) positional information can be related through the orthophoto generation process. Orthophotos have a uniform scale and represent all objects in their correct planimetric locations. However, orthophotos generated using conventional methods suffer from an artifact known as the double-mapping effect that occurs in areas occluded by tall objects. The double-mapping problem can be resolved through the commonly known true orthophoto generation procedure, in which an occlusion detection process is incorporated. This paper presents a review of occlusion detection methods, from which three techniques are compared and analyzed using experimental results. The paper also describes a framework for true orthophoto production based on an angle-based occlusion detection method. To improve the performance of the angle-based technique, two modifications to this method are introduced. These modifications, which aim at resolving false visibilities reported within the angle-based occlusion detection process, are referred to as occlusion extension and radial section overlap. A weighted averaging approach is also proposed to mitigate the seamline effect and spectral dissimilarity that may appear in true orthophoto mosaics. Moreover, true orthophotos generated from high-resolution aerial images and high-density LiDAR data using the updated version of angle-based methodology are illustrated for two urban study areas. To investigate the potential of image matching techniques in producing true orthophotos and point clouds, a comparison between the LiDAR-based and image-matching-based true orthophotos and digital surface models (DSMs) for an urban study area is also presented in this paper. Among the investigated occlusion detection methods, the angle-based technique demonstrated a better performance in terms of output and running time. The LiDAR-based true orthophotos and DSMs showed higher qualities compared to their image-matching-based counterparts which contain artifacts/noise along building edges. Full article

(This article belongs to the Special Issue Multisensor Data Fusion in Remote Sensing)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Multisensor Data Fusion in Remote Sensing

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (16 papers)

Research

Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI