Remote Sensing

Research

Jump to: Other

19 pages, 14785 KiB

Open AccessArticle

SRODNet: Object Detection Network Based on Super Resolution for Autonomous Vehicles

by Yogendra Rao Musunuri, Oh-Seol Kwon and Sun-Yuan Kung

Remote Sens. 2022, 14(24), 6270; https://doi.org/10.3390/rs14246270 - 10 Dec 2022

Cited by 7 | Viewed by 2055

Abstract

Object detection methods have been applied in several aerial and traffic surveillance applications. However, object detection accuracy decreases in low-resolution (LR) images owing to feature loss. To address this problem, we propose a single network, SRODNet, that incorporates both super-resolution (SR) and object [...] Read more.

Object detection methods have been applied in several aerial and traffic surveillance applications. However, object detection accuracy decreases in low-resolution (LR) images owing to feature loss. To address this problem, we propose a single network, SRODNet, that incorporates both super-resolution (SR) and object detection (OD). First, a modified residual block (MRB) is proposed in the SR to recover the feature information of LR images, and this network was jointly optimized with YOLOv5 to benefit from hierarchical features for small object detection. Moreover, the proposed model focuses on minimizing the computational cost of network optimization. We evaluated the proposed model using standard datasets such as VEDAI-VISIBLE, VEDAI-IR, DOTA, and Korean highway traffic (KoHT), both quantitatively and qualitatively. The experimental results show that the proposed method improves the accuracy of vehicular detection better than other conventional methods. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Figure 1

18 pages, 18782 KiB

Open AccessArticle

Hyperspectral Imaging Zero-Shot Learning for Remote Marine Litter Detection and Classification

by Sara Freitas, Hugo Silva and Eduardo Silva

Remote Sens. 2022, 14(21), 5516; https://doi.org/10.3390/rs14215516 - 2 Nov 2022

Cited by 13 | Viewed by 3263

Abstract

This paper addresses the development of a novel zero-shot learning method for remote marine litter hyperspectral imaging data classification. The work consisted of using an airborne acquired marine litter hyperspectral imaging dataset that contains data about different plastic targets and other materials and [...] Read more.

This paper addresses the development of a novel zero-shot learning method for remote marine litter hyperspectral imaging data classification. The work consisted of using an airborne acquired marine litter hyperspectral imaging dataset that contains data about different plastic targets and other materials and assessing the viability of detecting and classifying plastic materials without knowing their exact spectral response in an unsupervised manner. The classification of the marine litter samples was divided into known and unknown classes, i.e., classes that were hidden from the dataset during the training phase. The obtained results show a marine litter automated detection for all the classes, including (in the worst case of an unknown class) a precision rate over 56% and an overall accuracy of 98.71%. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Figure 1

20 pages, 22357 KiB

Open AccessFeature PaperArticle

TRQ3DNet: A 3D Quasi-Recurrent and Transformer Based Network for Hyperspectral Image Denoising

by Li Pang, Weizhen Gu and Xiangyong Cao

Remote Sens. 2022, 14(18), 4598; https://doi.org/10.3390/rs14184598 - 14 Sep 2022

Cited by 32 | Viewed by 2932

Abstract

We propose a new deep neural network termed TRQ3DNet which combines convolutional neural network (CNN) and transformer for hyperspectral image (HSI) denoising. The network consists of two branches. One is built by 3D quasi-recurrent blocks, including convolution and quasi-recurrent pooling operation. Specifically, the [...] Read more.

We propose a new deep neural network termed TRQ3DNet which combines convolutional neural network (CNN) and transformer for hyperspectral image (HSI) denoising. The network consists of two branches. One is built by 3D quasi-recurrent blocks, including convolution and quasi-recurrent pooling operation. Specifically, the 3D convolution can extract the spatial correlation within a band, and spectral correlation between different bands, while the quasi-recurrent pooling operation is able to exploit global correlation along the spectrum. The other branch is composed of a series of Uformer blocks. The Uformer block uses window-based multi-head self-attention (W-MSA) mechanism and the locally enhanced feed-forward network (LeFF) to exploit the global and local spatial features. To fuse the features extracted by the two branches, we develop a bidirectional integration bridge (BI bridge) for better preserving the image feature information. Experimental results on synthetic and real HSI data show the superiority of our proposed network. For example, in the case of Gaussian noise with sigma 70, the PSNR value of our method significantly increases about 0.8 compared with other state-of-the-art methods. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Figure 1

18 pages, 4607 KiB

Open AccessArticle

Rapid Target Detection of Fruit Trees Using UAV Imaging and Improved Light YOLOv4 Algorithm

by Yuchao Zhu, Jun Zhou, Yinhui Yang, Lijuan Liu, Fei Liu and Wenwen Kong

Remote Sens. 2022, 14(17), 4324; https://doi.org/10.3390/rs14174324 - 1 Sep 2022

Cited by 21 | Viewed by 3276

Abstract

The detection and counting of fruit tree canopies are important for orchard management, yield estimation, and phenotypic analysis. Previous research has shown that most fruit tree canopy detection methods are based on the use of traditional computer vision algorithms or machine learning methods [...] Read more.

The detection and counting of fruit tree canopies are important for orchard management, yield estimation, and phenotypic analysis. Previous research has shown that most fruit tree canopy detection methods are based on the use of traditional computer vision algorithms or machine learning methods to extract shallow features such as color and contour, with good results. However, due to the lack of robustness of these features, most methods are hardly adequate for the recognition and counting of fruit tree canopies in natural scenes. Other studies have shown that deep learning methods can be used to perform canopy detection. However, the adhesion and occlusion of fruit tree canopies, as well as background noise, limit the accuracy of detection. Therefore, to improve the accuracy of fruit tree canopy recognition and counting in real-world scenarios, an improved YOLOv4 (you only look once v4) is proposed, using a dataset produced from fruit tree canopy UAV imagery, combined with the Mobilenetv3 network, which can lighten the model and increase the detection speed, combined with the CBAM (convolutional block attention module), which can increase the feature extraction capability of the network, and combined with ASFF (adaptively spatial feature fusion), which enhances the multi-scale feature fusion capability of the network. In addition, the K-means algorithm and linear scale scaling are used to optimize the generation of pre-selected boxes, and the learning strategy of cosine annealing is combined to train the model, thus accelerating the training speed of the model and improving the detection accuracy. The results show that the improved YOLOv4 model can effectively overcome the noise in an orchard environment and achieve fast and accurate recognition and counting of fruit tree crowns while lightweight the model. The mAP reached 98.21%, FPS reached 96.25 and F1-score reached 93.60% for canopy detection, with a significant reduction in model size; the average overall accuracy (AOA) reached 96.73% for counting. In conclusion, the YOLOv4-Mobilenetv3-CBAM-ASFF-P model meets the practical requirements of orchard fruit tree canopy detection and counting in this study, providing optional technical support for the digitalization, refinement, and smart development of smart orchards. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Graphical abstract

18 pages, 6020 KiB

Open AccessArticle

Dictionary Learning- and Total Variation-Based High-Light-Efficiency Snapshot Multi-Aperture Spectral Imaging

by Feng Huang, Peng Lin, Rongjin Cao, Bin Zhou and Xianyu Wu

Remote Sens. 2022, 14(16), 4115; https://doi.org/10.3390/rs14164115 - 22 Aug 2022

Cited by 4 | Viewed by 1911

Abstract

Conventional multispectral imaging systems based on bandpass filters struggle to record multispectral videos with high spatial resolutions because of their limited light efficiencies. This paper proposes a multi-aperture multispectral imaging system based on notch filters that overcomes this limitation by allowing light from [...] Read more.

Conventional multispectral imaging systems based on bandpass filters struggle to record multispectral videos with high spatial resolutions because of their limited light efficiencies. This paper proposes a multi-aperture multispectral imaging system based on notch filters that overcomes this limitation by allowing light from most of the spectrum to pass through. Based on this imaging principle, a prototype multi-aperture multispectral imaging system comprising notch filters was built and demonstrated. Further, a dictionary learning- and total variation-based spectral super-resolution algorithm was developed to reconstruct spectral images. The simulation results obtained using public multispectral datasets showed that, compared to the dictionary learning-based spectral super-resolution algorithm, the proposed algorithm reconstructed the spectral information with a higher accuracy and removed noise, and the verification experiments confirmed the performance efficiency of the prototype system. The experimental results showed that the proposed imaging system can capture images with high spatial and spectral resolutions under low illumination conditions. The proposed algorithm improved the spectral resolution of the acquired data from 9 to 31 bands, and the average peak signal-to-noise ratio remained above 43 dB, which is 13 dB higher than those of the state-of-the-art coded aperture snapshot spectral imaging methods. Simultaneously, the frame rate of the imaging system was up to 5000 frames/s under natural daylight. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Graphical abstract

20 pages, 12315 KiB

Open AccessArticle

A Novel Multispectral Line Segment Matching Method Based on Phase Congruency and Multiple Local Homographies

by Haochen Hu, Boyang Li, Wenyu Yang and Chih-Yung Wen

Remote Sens. 2022, 14(16), 3857; https://doi.org/10.3390/rs14163857 - 9 Aug 2022

Cited by 3 | Viewed by 1820

Abstract

Feature matching is a fundamental procedure in several image processing methods applied in remote sensing. Multispectral sensors with different wavelengths can provide complementary information. In this work, we propose a multispectral line segment matching algorithm based on phase congruency and multiple local homographies [...] Read more.

Feature matching is a fundamental procedure in several image processing methods applied in remote sensing. Multispectral sensors with different wavelengths can provide complementary information. In this work, we propose a multispectral line segment matching algorithm based on phase congruency and multiple local homographies (PC-MLH) for image pairs captured by the cross-spectrum sensors (visible spectrum and infrared spectrum) in man-made scenarios. The feature points are first extracted and matched according to phase congruency. Next, multi-layer local homographies are derived from clustered feature points via random sample consensus (RANSAC) to guide line segment matching. Moreover, three geometric constraints (line position encoding, overlap ratio, and point-to-line distance) are introduced in cascade to reduce the computational complexity. The two main contributions of our work are as follows: First, compared with the conventional line matching methods designed for single-spectrum images, PC-MLH is robust against nonlinear radiation distortion (NRD) and can handle the unknown multiple local mapping, two common challenges associated with multispectral feature matching. Second, fusion of line extraction results and line position encoding for neighbouring matching increase the number of matched line segments and speed up the matching process, respectively. The method is validated using two public datasets, CVC-multimodal and VIS-IR. The results show that the percentage of correct matches (PCM) using PC-MLH can reach 94%, which significantly outperforms other single-spectral and multispectral line segment matching methods. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Graphical abstract

28 pages, 55640 KiB

Open AccessArticle

Site Selection via Learning Graph Convolutional Neural Networks: A Case Study of Singapore

by Tian Lan, Hao Cheng, Yi Wang and Bihan Wen

Remote Sens. 2022, 14(15), 3579; https://doi.org/10.3390/rs14153579 - 26 Jul 2022

Cited by 7 | Viewed by 4626

Abstract

Selection of store sites is a common but challenging task in business practices. Picking the most desirable location for a future store is crucial for attracting customers and becoming profitable. The classic multi-criteria decision-making framework for store site selection oversimplifies the local characteristics [...] Read more.

Selection of store sites is a common but challenging task in business practices. Picking the most desirable location for a future store is crucial for attracting customers and becoming profitable. The classic multi-criteria decision-making framework for store site selection oversimplifies the local characteristics that are both high dimensional and unstructured. Recent advances in deep learning enable more powerful data-driven approaches for site selection, many of which, however, overlook the interaction between different locations on the map. To better incorporate the spatial interaction patterns in understanding neighborhood characteristics and their impact on store placement, we propose to learn a graph convolutional network (GCN) for highly effective site selection tasks. Furthermore, we present a novel dataset that encompasses land use information as well as public transport networks in Singapore as a case study to benchmark site selection algorithms. It allows us to construct a geospatial GCN based on the public transport system to predict the attractiveness of different store sites within neighborhoods. We show that the proposed GCN model outperforms the competing methods that are learning from local geographical characteristics only. The proposed case study corroborates the geospatial interactions and offers new insights for solving various geographic and transport problems using graph neural networks. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Graphical abstract

19 pages, 22600 KiB

Open AccessArticle

Spatial and Spectral-Channel Attention Network for Denoising on Hyperspectral Remote Sensing Image

by Hong-Xia Dou, Xiao-Miao Pan, Chao Wang, Hao-Zhen Shen and Liang-Jian Deng

Remote Sens. 2022, 14(14), 3338; https://doi.org/10.3390/rs14143338 - 11 Jul 2022

Cited by 10 | Viewed by 2634

Abstract

Hyperspectral images (HSIs) are frequently contaminated by different noises (Gaussian noise, stripe noise, deadline noise, impulse noise) in the acquisition process as a result of the observation environment and imaging system limitations, which makes image information lost and difficult to recover. In this [...] Read more.

Hyperspectral images (HSIs) are frequently contaminated by different noises (Gaussian noise, stripe noise, deadline noise, impulse noise) in the acquisition process as a result of the observation environment and imaging system limitations, which makes image information lost and difficult to recover. In this paper, we adopt a 3D-based SSCA block neural network of U-Net architecture for remote sensing HSI denoising, named SSCANet (Spatial and Spectral-Channel Attention Network), which is mainly constructed by a so-called SSCA block. By fully considering the characteristics of spatial-domain and spectral-domain of remote sensing HSIs, the SSCA block consists of a spatial attention (SA) block and a spectral-channel attention (SCA) block, in which the SA block is to extract spatial information and enhance spatial representation ability, as well as the SCA block to explore the band-wise relationship within HSIs for preserving spectral information. Compared to earlier 2D convolution, 3D convolution has a powerful spectrum preservation ability, allowing for improved extraction of HSIs characteristics. Experimental results demonstrate that our method holds better-restored results than other compared approaches, both visually and quantitatively. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Figure 1

19 pages, 6523 KiB

Open AccessArticle

Mismatching Removal for Feature-Point Matching Based on Triangular Topology Probability Sampling Consensus

by Zaixing He, Chentao Shen, Quanyou Wang, Xinyue Zhao and Huilong Jiang

Remote Sens. 2022, 14(3), 706; https://doi.org/10.3390/rs14030706 - 2 Feb 2022

Cited by 17 | Viewed by 2404

Abstract

Feature-point matching between two images is a fundamental process in remote-sensing applications, such as image registration. However, mismatching is inevitable, and it needs to be removed. It is difficult for existing methods to remove a high ratio of mismatches. To address this issue, [...] Read more.

Feature-point matching between two images is a fundamental process in remote-sensing applications, such as image registration. However, mismatching is inevitable, and it needs to be removed. It is difficult for existing methods to remove a high ratio of mismatches. To address this issue, a robust method, called triangular topology probability sampling consensus (TSAC), is proposed, which combines the topology network and resampling methods. The proposed method constructs the triangular topology of the feature points of two images, quantifies the mismatching probability for each point pair, and then weights the probability into the random process of RANSAC by calculating the optimal homography matrix between the two images so that the mismatches can be detected and removed. Compared with the state-of-the-art methods, TSAC has superior performances in accuracy and robustness. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Graphical abstract

22 pages, 11576 KiB

Open AccessArticle

Efficient Instance Segmentation Paradigm for Interpreting SAR and Optical Images

by Fan Fan, Xiangfeng Zeng, Shunjun Wei, Hao Zhang, Dianhua Tang, Jun Shi and Xiaoling Zhang

Remote Sens. 2022, 14(3), 531; https://doi.org/10.3390/rs14030531 - 23 Jan 2022

Cited by 24 | Viewed by 3831

Abstract

Instance segmentation in remote sensing images is challenging due to the object-level discrimination and pixel-level segmentation for the objects. In remote sensing applications, instance segmentation adopts the instance-aware mask, rather than horizontal bounding box and oriented bounding box in object detection, or category-aware [...] Read more.

Instance segmentation in remote sensing images is challenging due to the object-level discrimination and pixel-level segmentation for the objects. In remote sensing applications, instance segmentation adopts the instance-aware mask, rather than horizontal bounding box and oriented bounding box in object detection, or category-aware mask in semantic segmentation, to interpret the objects with the boundaries. Despite these distinct advantages, versatile instance segmentation methods are still to be discovered for remote sensing images. In this paper, an efficient instance segmentation paradigm (EISP) for interpreting the synthetic aperture radar (SAR) and optical images is proposed. EISP mainly consists of the Swin Transformer to construct the hierarchical features of SAR and optical images, the context information flow (CIF) for interweaving the semantic features from the bounding box branch to mask branch, and the confluent loss function for refining the predicted masks. Experimental conclusions can be drawn on the PSeg-SSDD (Polygon Segmentation—SAR Ship Detection Dataset) and NWPU VHR-10 instance segmentation dataset (optical dataset): (1) Swin-L, CIF, and confluent loss function in EISP acts on the whole instance segmentation utility; (2) EISP* exceeds vanilla mask R-CNN 4.2% AP value on PSeg-SSDD and 11.2% AP on NWPU VHR-10 instance segmentation dataset; (3) The poorly segmented masks, false alarms, missing segmentations, and aliasing masks can be avoided to a great extent for EISP* in segmenting the SAR and optical images; (4) EISP* achieves the highest instance segmentation AP value compared to the state-of-the-art instance segmentation methods. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Figure 1

24 pages, 10273 KiB

Open AccessArticle

Remote Sensing Pansharpening by Full-Depth Feature Fusion

by Zi-Rong Jin, Yu-Wei Zhuo, Tian-Jing Zhang, Xiao-Xu Jin, Shuaiqi Jing and Liang-Jian Deng

Remote Sens. 2022, 14(3), 466; https://doi.org/10.3390/rs14030466 - 19 Jan 2022

Cited by 8 | Viewed by 3026

Abstract

Pansharpening is an important yet challenging remote sensing image processing task, which aims to reconstruct a high-resolution (HR) multispectral (MS) image by fusing a HR panchromatic (PAN) image and a low-resolution (LR) MS image. Though deep learning (DL)-based pansharpening methods have achieved encouraging [...] Read more.

Pansharpening is an important yet challenging remote sensing image processing task, which aims to reconstruct a high-resolution (HR) multispectral (MS) image by fusing a HR panchromatic (PAN) image and a low-resolution (LR) MS image. Though deep learning (DL)-based pansharpening methods have achieved encouraging performance, they are infeasible to fully utilize the deep semantic features and shallow contextual features in the process of feature fusion for a HR-PAN image and LR-MS image. In this paper, we propose an efficient full-depth feature fusion network (FDFNet) for remote sensing pansharpening. Specifically, we design three distinctive branches called PAN-branch, MS-branch, and fusion-branch, respectively. The features extracted from the PAN and MS branches will be progressively injected into the fusion branch at every different depth to make the information fusion more broad and comprehensive. With this structure, the low-level contextual features and high-level semantic features can be characterized and integrated adequately. Extensive experiments on reduced- and full-resolution datasets acquired from WorldView-3, QuickBird, and GaoFen-2 sensors demonstrate that the proposed FDFNet only with less than 100,000 parameters performs better than other detail injection-based proposals and several state-of-the-art approaches, both visually and quantitatively. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Figure 1

Other

Jump to: Research

14 pages, 1100 KiB

Open AccessTechnical Note

Self-Supervised Encoders Are Better Transfer Learners in Remote Sensing Applications

by Zachary D. Calhoun, Saad Lahrichi, Simiao Ren, Jordan M. Malof and Kyle Bradbury

Remote Sens. 2022, 14(21), 5500; https://doi.org/10.3390/rs14215500 - 1 Nov 2022

Cited by 6 | Viewed by 2975

Abstract

Transfer learning has been shown to be an effective method for achieving high-performance models when applying deep learning to remote sensing data. Recent research has demonstrated that representations learned through self-supervision transfer better than representations learned on supervised classification tasks. However, little research [...] Read more.

Transfer learning has been shown to be an effective method for achieving high-performance models when applying deep learning to remote sensing data. Recent research has demonstrated that representations learned through self-supervision transfer better than representations learned on supervised classification tasks. However, little research has focused explicitly on applying self-supervised encoders to remote sensing tasks. Using three diverse remote sensing datasets, we compared the performance of encoders pre-trained through both supervision and self-supervision on ImageNet, then fine-tuned on a final remote sensing task. Furthermore, we explored whether performance benefited from further pre-training on remote sensing data. Our experiments used SwAV due to its comparably lower computational requirements, as this method would prove most easily replicable by practitioners. We show that an encoder pre-trained on ImageNet using self-supervision transfers better than one pre-trained using supervision on three diverse remote sensing applications. Moreover, self-supervision on the target data alone as a pre-training step seldom boosts performance beyond this transferred encoder. We attribute this inefficacy to the lower diversity and size of remote sensing datasets, compared to ImageNet. In conclusion, we recommend that researchers use self-supervised representations for transfer learning on remote sensing data and that future research should focus on ways to increase performance further using self-supervision. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Graphical abstract

16 pages, 9456 KiB

Open AccessTechnical Note

A Novel and Effective Cooperative RANSAC Image Matching Method Using Geometry Histogram-Based Constructed Reduced Correspondence Set

by Kuo-Liang Chung, Ya-Chi Tseng and Hsuan-Ying Chen

Remote Sens. 2022, 14(14), 3256; https://doi.org/10.3390/rs14143256 - 6 Jul 2022

Cited by 5 | Viewed by 1786

Abstract

The success of many computer vision and pattern recognition applications depends on matching local features on two or more images. Because the initial correspondence set—i.e., the set of the initial feature pairs—is often contaminated by mismatches, removing mismatches is a necessary task prior [...] Read more.

The success of many computer vision and pattern recognition applications depends on matching local features on two or more images. Because the initial correspondence set—i.e., the set of the initial feature pairs—is often contaminated by mismatches, removing mismatches is a necessary task prior to image matching. In this paper, we first propose a fast geometry histogram-based (GH-based) mismatch removal strategy to construct a reduced correspondence set

C^{r e d u c e d, G H}

from the initial correspondence set

C^{i n i}

. Next, we propose an effective cooperative random sample consensus (COOSAC) method for remote sensing image matching. COOSAC consists of a RANSAC, called

R A N S A C^{i n i}

working on

C^{i n i}

, and a tiny RANSAC, called

R A N S A C^{t i n y, G H}

working on a randomly selected subset of

C^{r e d u c e d, G H}

. In

R A N S A C^{t i n y, G H}

, an iterative area constraint-based sampling strategy is proposed to estimate the model solution of

C^{t i n y, G H}

until the specified confidence level is reached, and then

R A N S A C^{i n i}

utilizes the estimated model solution of

C^{t i n y, G H}

to calculate the inlier rate of

C^{i n i}

. COOSAC repeats the above cooperation between

R A N S A C^{t i n y, G H}

and

R A N S A C^{i n i}

until the specified confidence level is reached, reporting the resultant model solution of

C^{i n i}

. For convenience, our image matching method is called the GH-COOSAC method. Based on several testing datasets, thorough experimental results demonstrate that the proposed GH-COOSAC method achieves lower computational cost and higher matching accuracy benefits when compared with the state-of-the-art image matching methods. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Figure 1

15 pages, 7168 KiB

Open AccessTechnical Note

UAV Remote Sensing Image Automatic Registration Based on Deep Residual Features

by Xin Luo, Guangling Lai, Xiao Wang, Yuwei Jin, Xixu He, Wenbo Xu and Weimin Hou

Remote Sens. 2021, 13(18), 3605; https://doi.org/10.3390/rs13183605 - 10 Sep 2021

Cited by 5 | Viewed by 2932

Abstract

With the rapid development of unmanned aerial vehicle (UAV) technology, UAV remote sensing images are increasing sharply. However, due to the limitation of the perspective of UAV remote sensing, the UAV images obtained from different viewpoints of a same scene need to be [...] Read more.

With the rapid development of unmanned aerial vehicle (UAV) technology, UAV remote sensing images are increasing sharply. However, due to the limitation of the perspective of UAV remote sensing, the UAV images obtained from different viewpoints of a same scene need to be stitched together for further applications. Therefore, an automatic registration method of UAV remote sensing images based on deep residual features is proposed in this work. It needs no additional training and does not depend on image features, such as points, lines and shapes, or on specific image contents. This registration framework is built as follows: Aimed at the problem that most of traditional registration methods only use low-level features for registration, we adopted deep residual neural network features extracted by an excellent deep neural network, ResNet-50. Then, a tensor product was employed to construct feature description vectors through exacted high-level abstract features. At last, the progressive consistency algorithm (PROSAC) was exploited to remove false matches and fit a geometric transform model so as to enhance registration accuracy. The experimental results for different typical scene images with different resolutions acquired by different UAV image sensors indicate that the improved algorithm can achieve higher registration accuracy than a state-of-the-art deep learning registration algorithm and other popular registration algorithms. Full article

(This article belongs to the Special Issue Machine Vision and Advanced Image Processing in Remote Sensing)

► Show Figures

Graphical abstract

Journal Menu

Journal Browser

Machine Vision and Advanced Image Processing in Remote Sensing

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (14 papers)

Research

Other

Further Information

Guidelines

MDPI Initiatives

Follow MDPI