SwinDenoising: A Local and Global Feature Fusion Algorithm for Infrared Image Denoising
Abstract
:1. Introduction
- Hierarchical Feature Extraction: SwinDenoising introduces a hierarchical approach to feature extraction, which enables the model to capture image features at multiple scales. This is particularly beneficial for infrared images, where both small-scale details and broader structures need to be preserved.
- Global–Local Feature Fusion: The method integrates both local and global features through the Swin Transformer’s multi-head self-attention mechanism. This fusion is critical for enhancing the model’s ability to handle diverse noise patterns, ensuring that noise is effectively reduced without compromising the integrity of the image’s structural details.
- Improved Robustness in Denoising: SwinDenoising demonstrates significant improvements in robustness, particularly under high levels of Gaussian and Poisson noise. The method’s ability to maintain high PSNR and SSIM values across various noise levels underscores its effectiveness in real-world infrared image denoising scenarios.
2. Related Work
2.1. Traditional and Hybrid Denoising Techniques
2.2. Deep-Learning-Based Methods
3. Method
3.1. The Local Feature Extraction Module
3.2. Global Feature Extraction Module
3.3. Image Recovery Module
4. Experiment
4.1. Experiment Setup
4.2. Experimental Results for Infrared Image Denoising under Complex Noise
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Appendix A
References
- Tan, C.L.; Mohseni, H. Emerging technologies for high performance infrared detectors. Nanophotonics 2018, 7, 169–197. [Google Scholar] [CrossRef]
- Ng, W.; Minasny, B.; McBratney, A. Convolutional neural network for soil microplastic contamination screening using infrared spectroscopy. Sci. Total. Environ. 2020, 702, 134723. [Google Scholar] [CrossRef]
- Wu, S.; Zhang, K.; Li, S.; Yan, J. Learning to track aircraft in infrared imagery. Remote Sens. 2020, 12, 3995. [Google Scholar] [CrossRef]
- Li, Y.; Yu, L.; Zheng, C.; Ma, Z.; Yang, S.; Song, F.; Zheng, K.; Ye, W.; Zhang, Y.; Wang, Y.; et al. Development and field deployment of a mid-infrared CO and CO2 dual-gas sensor system for early fire detection and location. Spectrochim. Acta Part A Mol. Biomol. Spectrosc. 2022, 270, 120834. [Google Scholar] [CrossRef]
- Chen, Q.; Li, Y.; Liu, Z.; Gao, J.; Li, X.; Zhang, S. Fault detection and analysis of voltage transformer secondary terminal based on infrared temperature measurement technology. In Proceedings of the 2019 IEEE 4th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chengdu, China, 20–22 December 2019; IEEE: New York, NY, USA, 2019; Volume 1, pp. 1283–1286. [Google Scholar]
- Mambou, S.J.; Maresova, P.; Krejcar, O.; Selamat, A.; Kuca, K. Breast cancer detection using infrared thermal imaging and a deep learning model. Sensors 2018, 18, 2799. [Google Scholar] [CrossRef]
- Arun, M.; Baraneetharan, E.; Kanchana, A.; Prabu, S. Detection and monitoring of the asymptotic COVID-19 patients using IoT devices and sensors. Int. J. Pervasive Comput. Commun. 2020, 18, 407–418. [Google Scholar]
- Zhang, R.; Li, H.; Duan, K.; You, S.; Liu, K.; Wang, F.; Hu, Y. Automatic detection of earthquake-damaged buildings by integrating UAV oblique photography and infrared thermal imaging. Remote Sens. 2020, 12, 2621. [Google Scholar] [CrossRef]
- Shen, Y.; Chen, Y.; Liu, Q.; Lou, S.; Yu, W.; Wang, X.; Chen, H. Improved Anscombe transformation and total variation for denoising of lowlight infrared images. Infrared Phys. Technol. 2018, 93, 192–198. [Google Scholar] [CrossRef]
- Chen, X.; Liu, L.; Zhang, J.; Shao, W. Infrared image denoising based on the variance-stabilizing transform and the dual-domain filter. Digit. Signal Process. Rev. J. 2021, 113, 103012. [Google Scholar] [CrossRef]
- Jiang, M. Edge enhancement and noise suppression for infrared image based on feature analysis. Infrared Phys. Technol. 2018, 91, 142–152. [Google Scholar] [CrossRef]
- Liu, C.; Sui, X.; Kuang, X.; Liu, Y.; Gu, G.; Chen, Q. Adaptive contrast enhancement for infrared images based on the neighborhood conditional histogram. Remote Sens. 2019, 11, 1381. [Google Scholar] [CrossRef]
- Li, Z.; Luo, S.; Chen, M.; Wu, H.; Wang, T.; Cheng, L. Infrared thermal imaging denoising method based on second-order channel attention mechanism. Infrared Phys. Technol. 2021, 116, 103789. [Google Scholar] [CrossRef]
- Zhang, Z.; Zheng, W.; Ma, Z.; Yin, L.; Xie, M.; Wu, Y. Infrared star image denoising using regions with deep reinforcement learning. Infrared Phys. Technol. 2021, 117, 103819. [Google Scholar] [CrossRef]
- Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021; pp. 10012–10022. [Google Scholar]
- Binbin, Y. An improved infrared image processing method based on adaptive threshold denoising. EURASIP J. Image Video Process. 2019, 2019, 1–12. [Google Scholar] [CrossRef]
- Song, L.; Huang, H. Simultaneous Destriping and Image Denoising Using a Nonparametric Model With the EM Algorithm. IEEE Trans. Image Process. 2023, 32, 1065–1077. [Google Scholar] [CrossRef]
- He, Y.; Zhang, C.; Zhang, B.; Zhi, C. FSPnP: Plug-and-Play Frequency–Spatial-Domain Hybrid Denoiser for Thermal Infrared Image. IEEE Trans. Geosci. Remote Sens. 2024, 62, 5000416. [Google Scholar] [CrossRef]
- Tang, Z.; Jian, X. Thermal Fault Diagnosis of Complex Electrical Equipment Based on Infrared Image Recognition. Sci. Rep. 2024, 14, 5547. [Google Scholar] [CrossRef]
- Dong, G.; Ma, Y.; Basu, A. Feature-guided CNN for denoising images from portable ultrasound devices. IEEE Access 2021, 9, 28272–28281. [Google Scholar] [CrossRef]
- Li, N.; Wang, B.; Goudail, F.; Zhao, Y.; Pan, Q. Joint Denoising-Demosaicking Network for Long-Wave Infrared Division-of-Focal-Plane Polarization Images with Mixed Noise Level Estimation. IEEE Trans. Image Process. 2023, 32, 5961–5976. [Google Scholar] [CrossRef]
- Shi, M.; Wang, H. Infrared dim and small target detection based on denoising autoencoder network. Mob. Netw. Appl. 2020, 25, 1469–1483. [Google Scholar] [CrossRef]
- Li, X.; Zhang, J.; Shi, J. Quantitative nondestructive testing of broken wires for wire rope based on magnetic and infrared information. J. Sensors 2020, 2020, 1–14. [Google Scholar] [CrossRef]
- Liu, T.; Zhang, Y.; Yang, F. Fractional order total variational infrared image denoising with fused flower pollination particle swarm optimization. Opt. Eng. 2023, 62, 033105. [Google Scholar] [CrossRef]
- Cao, Z.; Zhong, Y.; Gaotang, L.; Zhao, H.; Pang, L. A successive approach to enhancement of infrared facial images. In Proceedings of the Infrared Technology and Applications XLVIII, Orlando, FL, USA, 3–7 April 2022; SPIE: Philadelphia, PA, USA, 2022; Volume 12107, pp. 407–416. [Google Scholar]
- Budhiraja, S.; Agrawal, S.; Sohi, B.S.; Sharma, N. Effect of pre-processing on MST based infrared and visible image fusion. In Proceedings of the 2021 3rd International Conference on Signal Processing and Communication (ICPSC), Coimbatore, India, 13–14 May 2021; IEEE: New York, NY, USA, 2021; pp. 522–526. [Google Scholar]
- Yang, S.; Qin, H.; Yan, X.; Yuan, S.; Zeng, Q. Mid-Wave Infrared Snapshot Compressive Spectral Imager with Deep Infrared Denoising Prior. Remote Sens. 2023, 15, 280. [Google Scholar] [CrossRef]
- Zamir, S.W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F.S.; Yang, M.H. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 5728–5739. [Google Scholar]
- Chen, L.; Chu, X.; Zhang, X.; Sun, J. Simple baselines for image restoration. In Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel, 23–27 October 2022; Springer: Berlin/Heidelberg, Germany, 2022; pp. 17–33. [Google Scholar]
- Jing, J.; Deng, X.; Xu, M.; Wang, J.; Guan, Z. Hinet: Deep image hiding by invertible network. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021; pp. 4733–4742. [Google Scholar]
15 dB | 25 dB | 50 dB | ||||
---|---|---|---|---|---|---|
Method | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR |
HINet0.5x | 0.850 | 33.50 | 0.708 | 27.73 | 0.704 | 29.21 |
HINet1x | 0.848 | 33.38 | 0.789 | 31.58 | 0.708 | 29.31 |
NAFNet32 | 0.841 | 33.30 | 0.786 | 31.54 | 0.691 | 28.90 |
NAFNet64 | 0.826 | 32.52 | 0.792 | 31.64 | 0.690 | 28.89 |
Restormer | 0.835 | 33.14 | 0.681 | 29.02 | 0.714 | 29.39 |
SwinDenoising (Ours) | 0.869 | 33.74 | 0.801 | 31.92 | 0.726 | 29.42 |
= 50 | = 100 | |||
---|---|---|---|---|
Method | SSIM | PSNR | SSIM | PSNR |
HINet0.5x | 0.916 | 36.12 | 0.885 | 34.64 |
HINet1x | 0.919 | 36.32 | 0.888 | 34.74 |
NAFNet32 | 0.921 | 36.57 | 0.890 | 35.10 |
NAFNet64 | 0.918 | 36.39 | 0.891 | 35.14 |
Restormer | 0.914 | 36.09 | 0.871 | 34.18 |
SwinDenoising (Ours) | 0.927 | 37.17 | 0.892 | 36.06 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wu, W.; Dong, X.; Li, R.; Chen, H.; Cheng, L. SwinDenoising: A Local and Global Feature Fusion Algorithm for Infrared Image Denoising. Mathematics 2024, 12, 2968. https://doi.org/10.3390/math12192968
Wu W, Dong X, Li R, Chen H, Cheng L. SwinDenoising: A Local and Global Feature Fusion Algorithm for Infrared Image Denoising. Mathematics. 2024; 12(19):2968. https://doi.org/10.3390/math12192968
Chicago/Turabian StyleWu, Wenhao, Xiaoqing Dong, Ruihao Li, Hongcai Chen, and Lianglun Cheng. 2024. "SwinDenoising: A Local and Global Feature Fusion Algorithm for Infrared Image Denoising" Mathematics 12, no. 19: 2968. https://doi.org/10.3390/math12192968
APA StyleWu, W., Dong, X., Li, R., Chen, H., & Cheng, L. (2024). SwinDenoising: A Local and Global Feature Fusion Algorithm for Infrared Image Denoising. Mathematics, 12(19), 2968. https://doi.org/10.3390/math12192968