A Hierarchical Low-Rank Denoising Model for Remote Sensing Images Based on Deep Unfolding
Abstract
:1. Introduction
- We think the residuals of traditional low-rank models contain not only noise but also high-frequency edge information that has been filtered out. Therefore, we tried to re-extract edge information from the residuals to construct a hierarchical low-rank model for denoising remote sensing images. To better achieve edge extraction, we introduced manifold learning into the model, ensuring that useful structural information, rather than noise, is extracted in the edge subspace. Additionally, a new prior knowledge regulation was designed to distinguish between clean pixels and noisy pixels, guiding the model to learn useful information during the denoising process;
- Due to the number of iterations required, traditional LRR models usually take a long time to process large-sized remote sensing images. To tackle this problem, we designed a hierarchical denoising model based on deep unfolding by combining shallow models with deep autoencoders. By mapping traditional iterative optimization algorithms onto the structure of neural networks, each iterative step was transformed into a layer of the network. This technique enhances the interpretability of neural networks, making each layer no longer a black box model but rather corresponding to a specific optimization step;
- Experiments conducted on three types of images—optical remote sensing images, hyperspectral images, and SAR images—demonstrated the effectiveness of our proposed HLR-DUR model. HLR-DUR not only enhances denoising performance but also better preserves sufficient edge detail compared to existing methods, achieving better edge-preserving denoising.
2. Related Work
3. Proposed Methodology
3.1. Hierarchical Low-Rank Model
3.1.1. Model Formulation
3.1.2. Prior Knowledge Regulation Construction
- (1)
- , which means that this pixel is a point target and needs to be preserved. So, we set the corresponding value in to 0.9 (a very big value).
- (2)
- , update as according to (12).
- (3)
- , which represents that this pixel is a noisy pixel and needs to be smoothed. So, we set the corresponding value in to 0.1 (a respectively small value).
3.2. Deep Unfolding Model HLR-DUR Based on Autoencoder
- low-rank module
- edge module
- other variables
- Lagrange multipliers
4. Results
4.1. Experiments on Optical Images
4.1.1. Dataset Description
4.1.2. Experiments and Analysis
4.2. Experiments on Hyperspectral Images
4.2.1. Dataset Description
4.2.2. Experiments and Analysis
4.3. Experiments on SAR Images
4.3.1. Dataset Description
4.3.2. Experiments and Analysis
4.4. Processing Speed and Ablation Experiments
4.4.1. Ablation Experiment and Accuracy and Loss Curves
4.4.2. Results of the Processing Speed and Parameters Experiment
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Sabins, F.F., Jr.; Ellis, J.M. Remote Sensing: Principles, Interpretation, and Applications; Waveland Press: Long Grove, IL, USA, 2020. [Google Scholar]
- Rasti, B.; Chang, Y.; Dalsasso, E.; Denis, L.; Ghamisi, P. Image restoration for remote sensing: Overview and toolbox. IEEE Geosci. Remote Sens. Mag. 2021, 10, 201–230. [Google Scholar] [CrossRef]
- Li, G.; Gu, X.; Chen, C.; Zhou, C.; Xiao, D.; Wan, W.; Cai, H. Low-Frequency Magnetotelluric Data Denoising Using Improved Denoising Convolutional Neural Network and Gated Recurrent Unit. IEEE Trans. Geosci. Remote Sens. 2024, 62, 5909216. [Google Scholar] [CrossRef]
- Li, G.; Wu, S.; Cai, H.; He, Z.; Liu, X.; Zhou, C.; Tang, J. IncepTCN: A new deep temporal convolutional network combined with dictionary learning for strong cultural noise elimination of controlled-source electromagnetic data. Geophysics 2023, 88, E107–E122. [Google Scholar] [CrossRef]
- Han, L.; Zhao, Y.; Lv, H.; Zhang, Y.; Liu, H.; Bi, G. Remote sensing image denoising based on deep and shallow feature fusion and attention mechanism. Remote Sens. 2022, 14, 1243. [Google Scholar] [CrossRef]
- Huang, Z.; Zhu, Z.; Wang, Z.; Shi, Y.; Fang, H.; Zhang, Y. DGDNet: Deep gradient descent network for remotely sensed image denoising. IEEE Geosci. Remote Sens. Lett. 2023, 20, 6002405. [Google Scholar] [CrossRef]
- Tian, C.; Fei, L.; Zheng, W.; Xu, Y.; Zuo, W.; Lin, C.-W. Deep learning on image denoising: An overview. Neural Netw. 2020, 131, 251–275. [Google Scholar] [CrossRef] [PubMed]
- Jiao, L.; Zhao, J. A survey on the new generation of deep learning in image processing. IEEE Access 2019, 7, 172231–172263. [Google Scholar] [CrossRef]
- Feng, X.; Zhang, W.; Su, X.; Xu, Z. Optical remote sensing image denoising and super-resolution reconstructing using optimized generative network in wavelet transform domain. Remote Sens. 2021, 13, 1858. [Google Scholar] [CrossRef]
- Huang, Z.; Zhang, Y.; Li, Q.; Li, Z.; Zhang, T.; Sang, N.; Xiong, S. Unidirectional variation and deep CNN denoiser priors for simultaneously destriping and denoising optical remote sensing images. Int. J. Remote Sens. 2019, 40, 5737–5748. [Google Scholar] [CrossRef]
- Xie, Y.; Qu, Y.; Tao, D.; Wu, W.; Yuan, Q.; Zhang, W. Hyperspectral image restoration via iteratively regularized weighted schatten $ p $-norm minimization. IEEE Trans. Geosci. Remote Sens. 2016, 54, 4642–4659. [Google Scholar] [CrossRef]
- Dong, W.; Wang, H.; Wu, F.; Shi, G.; Li, X. Deep spatial–spectral representation learning for hyperspectral image denoising. IEEE Trans. Comput. Imaging 2019, 5, 635–648. [Google Scholar] [CrossRef]
- Gu, F.; Zhang, H.; Wang, C. A two-component deep learning network for SAR image denoising. IEEE Access 2020, 8, 17792–17803. [Google Scholar] [CrossRef]
- Feng, X.; Tian, S.; Abhadiomhen, S.E.; Xu, Z.; Shen, X.; Wang, J.; Zhang, X.; Gao, W.; Zhang, H.; Wang, C. Edge-Preserved Low-Rank Representation via Multi-Level Knowledge Incorporation for Remote Sensing Image Denoising. Remote Sens. 2023, 15, 2318. [Google Scholar] [CrossRef]
- Chen, Y.; Huang, T.-Z.; He, W.; Zhao, X.-L.; Zhang, H.; Zeng, J. Hyperspectral image denoising using factor group sparsity-regularized nonconvex low-rank approximation. IEEE Trans. Geosci. Remote Sens. 2021, 60, 5515916. [Google Scholar] [CrossRef]
- He, W.; Zhang, H.; Zhang, L.; Shen, H. Hyperspectral image denoising via noise-adjusted iterative low-rank matrix approximation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 3050–3061. [Google Scholar] [CrossRef]
- Ma, T.-H.; Xu, Z.; Meng, D. Remote sensing image denoising via low-rank tensor approximation and robust noise modeling. Remote Sens. 2020, 12, 1278. [Google Scholar] [CrossRef]
- Zhang, H.; Liu, L.; He, W.; Zhang, L. Hyperspectral image denoising with total variation regularization and nonlocal low-rank tensor decomposition. IEEE Trans. Geosci. Remote Sens. 2019, 58, 3071–3084. [Google Scholar] [CrossRef]
- Li, C.; Ma, Y.; Huang, J.; Mei, X.; Ma, J. Hyperspectral image denoising using the robust low-rank tensor recovery. J. Opt. Soc. Am. A 2015, 32, 1604–1612. [Google Scholar] [CrossRef] [PubMed]
- Guan, D.; Xiang, D.; Tang, X.; Kuang, G. SAR image despeckling based on nonlocal low-rank regularization. IEEE Trans. Geosci. Remote Sens. 2019, 57, 3472–3489. [Google Scholar] [CrossRef]
- Chen, G.; Li, G.; Liu, Y.; Zhang, X.-P.; Zhang, L. SAR image despeckling based on combination of fractional-order total variation and nonlocal low rank regularization. IEEE Trans. Geosci. Remote Sens. 2020, 58, 2056–2070. [Google Scholar] [CrossRef]
- Geng, J.; Fan, J.; Ma, X.; Wang, H.; Cao, K. An iterative low-rank representation for SAR image despeckling. In Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, 10–15 July 2016; pp. 72–75. [Google Scholar]
- Baier, G.; He, W.; Yokoya, N. Robust nonlocal low-rank SAR time series despeckling considering speckle correlation by total variation regularization. IEEE Trans. Geosci. Remote Sens. 2020, 58, 7942–7954. [Google Scholar] [CrossRef]
- Zhang, H.; Chen, H.; Yang, G.; Zhang, L. LR-Net: Low-rank spatial-spectral network for hyperspectral image denoising. IEEE Trans. Image Process. 2021, 30, 8743–8758. [Google Scholar] [CrossRef]
- Nguyen, H.V.; Ulfarsson, M.O.; Sigurdsson, J.; Sveinsson, J.R. Deep sparse and low-rank prior for hyperspectral image denoising. In Proceedings of the IGARSS 2022–2022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, 17–22 July 2022; pp. 1217–1220. [Google Scholar]
- Sun, H.; Liu, M.; Zheng, K.; Yang, D.; Li, J.; Gao, L. Hyperspectral image denoising via low-rank representation and CNN denoiser. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2021, 15, 716–728. [Google Scholar] [CrossRef]
- Zhang, Q.; Yuan, Q.; Song, M.; Yu, H.; Zhang, L. Cooperated spectral low-rankness prior and deep spatial prior for HSI unsupervised denoising. IEEE Trans. Image Process. 2022, 31, 6356–6368. [Google Scholar] [CrossRef]
- Xiong, K.; Zhao, G.; Wang, Y.; Shi, G. SAR imaging and despeckling based on sparse, low-rank, and deep CNN priors. IEEE Geosci. Remote Sens. Lett. 2021, 19, 4501205. [Google Scholar] [CrossRef]
- Monga, V.; Li, Y.; Eldar, Y.C. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Process. Mag. 2021, 38, 18–44. [Google Scholar] [CrossRef]
- Passah, A.; Amitab, K.; Kandar, D. SAR image despeckling using deep CNN. IET Image Process. 2021, 15, 1285–1297. [Google Scholar] [CrossRef]
- Wang, S.; Wang, X.; Zhang, L.; Zhong, Y. Deep low-rank prior for hyperspectral anomaly detection. IEEE Trans. Geosci. Remote Sens. 2022, 60, 5527017. [Google Scholar] [CrossRef]
- Candès, E.J.; Li, X.; Ma, Y.; Wright, J. Robust principal component analysis? J. ACM 2011, 58, 11. [Google Scholar] [CrossRef]
- Dong, Y.; Chan, R.H.; Xu, S. A detection statistic for random-valued impulse noise. IEEE Trans. Image Process. 2007, 16, 1112–1120. [Google Scholar] [CrossRef]
- Lopes, A.; Touzi, R.; Nezry, E. Adaptive speckle filters and scene heterogeneity. IEEE Trans. Geosci. Remote Sens. 1990, 28, 992–1000. [Google Scholar] [CrossRef]
- Hong, M.; Luo, Z.-Q. On the linear convergence of the alternating direction method of multipliers. Math. Program. 2017, 162, 165–199. [Google Scholar] [CrossRef]
- Huang, Z.; Wang, Z.; Zhu, Z.; Zhang, Y.; Fang, H.; Shi, Y.; Zhang, T. DLRP: Learning deep low-rank prior for remotely sensed image denoising. IEEE Geosci. Remote Sens. Lett. 2022, 19, 6508905. [Google Scholar] [CrossRef]
- Zhang, K.; Zuo, W.; Zhang, L. FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 2018, 27, 4608–4622. [Google Scholar] [CrossRef] [PubMed]
- Yuan, Q.; Zhang, Q.; Li, J.; Shen, H.; Zhang, L. Hyperspectral image denoising employing a spatial–spectral deep residual convolutional neural network. IEEE Trans. Geosci. Remote Sens. 2018, 57, 1205–1218. [Google Scholar] [CrossRef]
- Cao, X.; Fu, X.; Xu, C.; Meng, D. Deep spatial-spectral global reasoning network for hyperspectral image denoising. IEEE Trans. Geosci. Remote Sens. 2021, 60, 5504714. [Google Scholar] [CrossRef]
- Shan, W.; Liu, P.; Mu, L.; Cao, C.; He, G. Hyperspectral image denoising with dual deep CNN. IEEE Access 2019, 7, 171297–171312. [Google Scholar] [CrossRef]
- Lin, C.; Qiu, C.; Jiang, H.; Zou, L. A Deep Neural Network Based on Prior-Driven and Structural Preserving for SAR Image Despeckling. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2023, 16, 6372–6392. [Google Scholar] [CrossRef]
Noise Variance | Evaluation Metrics | DnCNN | FFDNet | HSID-CNN | GRN | DD-CNN | PDSNet | HLR-DUR |
---|---|---|---|---|---|---|---|---|
0.01 | MPSNR | 33.8562 | 29.1558 | 25.1696 | 23.4655 | 24.7268 | 28.4567 | 36.8927 |
MSSIM | 0.7627 | 0.7924 | 0.7248 | 0.7085 | 0.7376 | 0.7073 | 0.7706 | |
0.02 | MPSNR | 31.9557 | 27.5778 | 25.0029 | 22.9981 | 22.9527 | 25.4539 | 36.4328 |
MSSIM | 0.7250 | 0.7176 | 0.7222 | 0.6749 | 0.6778 | 0.7018 | 0.7625 | |
0.04 | MPSNR | 29.7556 | 26.0261 | 24.4276 | 21.1774 | 19.5923 | 23.5846 | 35.6956 |
MSSIM | 0.6772 | 0.6445 | 0.7106 | 0.4552 | 0.3993 | 0.6936 | 0.7433 | |
0.06 | MPSNR | 28.3637 | 25.1928 | 23.8439 | 17.4169 | 16.8556 | 20.3977 | 34.9296 |
MSSIM | 0.6562 | 0.5838 | 0.6651 | 0.2329 | 0.1986 | 0.5725 | 0.6706 | |
0.08 | MPSNR | 27.7164 | 24.1979 | 23.1341 | 17.3257 | 14.8928 | 18.2633 | 32.9710 |
MSSIM | 0.6366 | 0.5186 | 0.5973 | 0.2312 | 0.1250 | 0.5591 | 0.6290 | |
0.1 | MPSNR | 27.0592 | 23.4049 | 22.3332 | 14.5862 | 14.2963 | 18.0361 | 30.8986 |
MSSIM | 0.5575 | 0.4681 | 0.5214 | 0.1491 | 0.1099 | 0.4826 | 0.5928 |
Noisy Image | DnCNN | FFDNet | HSID- CNN | GRN | DD-CNN | PDSNet | HLR-DUR | ||
---|---|---|---|---|---|---|---|---|---|
Image 1 | ENL | 5.5820 | 103.5885 | 118.0177 | 114.4488 | 125.6164 | 98.0918 | 116.3475 | 125.9335 |
EPI | - | 0.5839 | 0.6674 | 0.6937 | 0.7022 | 0.5803 | 0.6309 | 0.7356 | |
Image 2 | ENL | 9.3213 | 96.2285 | 95.7900 | 101.6122 | 104.6173 | 84.0133 | 107.3329 | 115.3813 |
EPI | - | 0.5236 | 0.5937 | 0.6796 | 0.6725 | 0.6116 | 0.5682 | 0.6823 | |
Image 3 | ENL | 7.6195 | 107.8969 | 116.3891 | 113.5986 | 116.4556 | 100.3012 | 114.2744 | 124.5229 |
EPI | - | 0.6365 | 0.8779 | 0.8727 | 0.8991 | 0.6013 | 0.8917 | 0.9164 | |
Image 4 | ENL | 7.7927 | 76.6310 | 73.9612 | 80.8328 | 80.9381 | 60.4646 | 83.6534 | 81.9254 |
EPI | - | 0.5929 | 0.7398 | 0.8634 | 0.8929 | 0.7013 | 0.7972 | 0.9147 | |
Image 5 | ENL | 8.8592 | 103.7392 | 109.3430 | 116.1278 | 110.9981 | 101.4649 | 117.5947 | 125.6757 |
EPI | - | 0.7030 | 0.8562 | 0.8519 | 0.8244 | 0.7778 | 0.8139 | 0.8662 |
Noisy Image | DnCNN | FFDNet | HSID- CNN | GRN | DD-CNN | PDSNet | HLR-DUR | ||
---|---|---|---|---|---|---|---|---|---|
Image 1 | ENL | 10.7727 | 67.4415 | 55.9390 | 54.1302 | 57.7159 | 58.7127 | 65.9892 | 72.2878 |
EPI | - | 0.4201 | 0.5413 | 0.5265 | 0.5159 | 0.5337 | 0.5712 | 0.6175 | |
Image 2 | ENL | 12.6964 | 62.5110 | 67.8803 | 63. 4276 | 63.5245 | 69.7125 | 77. 0636 | 92.3050 |
EPI | - | 0.6521 | 0.7019 | 0.7060 | 0.8226 | 0.6715 | 0.8064 | 0.8626 | |
Image 3 | ENL | 10.6408 | 56.4970 | 63.1859 | 70.1377 | 66.9889 | 54.0735 | 82.4010 | 78.4357 |
EPI | - | 0.6361 | 0.71845 | 0.6423 | 0.7223 | 0.6531 | 0.7312 | 0.7907 |
Evaluation Metrics | 0.01 | 0.02 | 0.04 | 0.06 | 0.08 | 0.1 | |
---|---|---|---|---|---|---|---|
Without Edge Module | MPSNR | 35.3927 | 35.0233 | 34.6201 | 33.2930 | 31.9223 | 29.1146 |
MSSIM | 0.6111 | 0.6032 | 0.5725 | 0.5549 | 0.7022 | 0.5803 | |
HLR-DUR | MPNSR | 36.8927 | 36.4328 | 35.6956 | 34.9296 | 32.9710 | 30.8986 |
MSSIM | 0.7023 | 0.6882 | 0.6620 | 0.6321 | 0.6290 | 0.6116 |
Methods | DnCNN | FFDNet | HSID- CNN | GRN | DD-CNN | PDSNet | HLR-DUR |
---|---|---|---|---|---|---|---|
Time(ms) | 22.51 | 50.13 | 47.13 | 67.78 | 63.16 | 76.26 | 1.2 |
Parameters | 667,008 | 485,316 | 556,097 | 1,322,251 | 7,955,552 | 1,366,111 | 28,311 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Shao, F.; Feng, X.; Tian, S.; Zhang, T. A Hierarchical Low-Rank Denoising Model for Remote Sensing Images Based on Deep Unfolding. Sensors 2024, 24, 4574. https://doi.org/10.3390/s24144574
Shao F, Feng X, Tian S, Zhang T. A Hierarchical Low-Rank Denoising Model for Remote Sensing Images Based on Deep Unfolding. Sensors. 2024; 24(14):4574. https://doi.org/10.3390/s24144574
Chicago/Turabian StyleShao, Fanqi, Xiaolin Feng, Sirui Tian, and Tianyi Zhang. 2024. "A Hierarchical Low-Rank Denoising Model for Remote Sensing Images Based on Deep Unfolding" Sensors 24, no. 14: 4574. https://doi.org/10.3390/s24144574
APA StyleShao, F., Feng, X., Tian, S., & Zhang, T. (2024). A Hierarchical Low-Rank Denoising Model for Remote Sensing Images Based on Deep Unfolding. Sensors, 24(14), 4574. https://doi.org/10.3390/s24144574