A Two-Stage SAR Image Generation Algorithm Based on GAN with Reinforced Constraint Filtering and Compensation Techniques
Abstract
:1. Introduction
2. Related Works
2.1. ACGAN
2.2. Top-k Training
3. The Proposed RCFCA-GAN Algorithm
3.1. Problems with Traditional GAN Models
3.2. Stage 1 of RCFCA-GAN Algorithm: Reinforced Constraint Filtering Based on Top-k
3.3. Stage 2 of the RCFCA-GAN Algorithm: Compensation Afterwards for Less Trained Categories in Bottom-n
3.4. Summary of the Proposed RCFCA-GAN Algorithm
4. Experiments
4.1. Network Architecture and Parameter Settings
4.2. FID of Generated Images
4.3. Recognition Accuracy
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Wu, X.; Zhang, Z.; Xiong, S.; Zhang, W.; Tang, J.; Li, Z.; An, B.; Li, R. A Near-Real-Time Flood Detection Method Based on Deep Learning and SAR Images. Remote Sens. 2023, 15, 2046. [Google Scholar] [CrossRef]
- Sun, Z.; Leng, X.; Lei, Y.; Xiong, B.; Ji, K.; Kuang, G. BiFA-YOLO: A Novel YOLO-Based Method for Arbitrary-Oriented Ship Detection in High-Resolution SAR Images. Remote Sens. 2021, 13, 4209. [Google Scholar] [CrossRef]
- Ding, B.; Wen, G.; Huang, X.; Ma, C.; Yang, X. Target Recognition in Synthetic Aperture Radar Images via Matching of Attributed Scattering Centers. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2017, 10, 3334–3347. [Google Scholar] [CrossRef]
- Zhou, X.; Wang, J.; Shan, B.; He, Y. Early-Season Crop Classification Based on Local Window Attention Transformer with Time-Series RCM and Sentinel-1. Remote Sens. 2024, 16, 1376. [Google Scholar] [CrossRef]
- Sajjad, M.; Wang, J.; Afzal, Z.; Hussain, S.; Siddique, A.; Khan, R.; Iqbal, J. Assessing the Impacts of Groundwater Depletion and Aquifer Degradation on Land Subsidence in Lahore, Pakistan: A PS-InSAR Approach for Sustainable Urban Development. Remote Sens. 2023, 15, 5418. [Google Scholar] [CrossRef]
- Shakeel, M.S.; Lam, K.M. Deep-Feature Encoding-Based Discriminative Model for Age-Invariant Face Recognition. Pattern Recognit. 2019, 93, 442–457. [Google Scholar] [CrossRef]
- He, X.; Yan, S.; Hu, Y.; Niyogi, P.; Zhang, H.J. Face Recognition Using Laplacianfaces. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27, 328–340. [Google Scholar]
- Alabi, T.R.; Adewopo, J.; Duke, O.P.; Kumar, P.L. Banana Mapping in Heterogenous Smallholder Farming Systems Using High-Resolution Remote Sensing Imagery and Machine Learning Models with Implications for Banana Bunchy Top Disease Surveillance. Remote Sens. 2022, 14, 5206. [Google Scholar] [CrossRef]
- Tetila, E.C.; Machado, B.B.; Menezes, G.K.; Oliveira, A.S. Automatic Recognition of Soybean Leaf Diseases Using UAV Images and Deep Convolutional Neural Networks. IEEE Geosci. Remote Sens. Lett. 2020, 17, 903–907. [Google Scholar] [CrossRef]
- Muhammad, K.; Ullah, A.; Lloret, J.; Del, J.; Abuquerque, V.H.C. Deep Learning for Safe Autonomous Driving: Current Challenges and Future Directions. IEEE Trans. Intell. Transp. Syst. 2020, 22, 4316–4336. [Google Scholar] [CrossRef]
- Du, P.; Li, E.; Xia, J.; Samat, A.; Bai, X. Feature and Model Level Fusion of Pretrained CNN for Remote Sensing Scene Classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2019, 12, 2600–2611. [Google Scholar] [CrossRef]
- Zhang, X.; Wang, L.; Su, Y. Visual place recognition: A Survey From Deep Learning Perspective. Pattern Recognit. 2021, 113, 107760–107802. [Google Scholar] [CrossRef]
- Rouast, P.V.; Adam, M.T.; Chiong, R. Deep Learning for Human Affect Recognition: Insights and New Developments. IEEE Trans. Affect. Comput. 2019, 12, 524–543. [Google Scholar] [CrossRef]
- Yu, Z.; Yu, L.; Cheng, P.; Chen, J.; Chi, C. A Comprehensive Survey on SAR ATR in Deep-Learning Era. Remote Sens. 2023, 15, 1454. [Google Scholar] [CrossRef]
- Zhou, J.; Shi, Z.; Xiao, C.; Qiang, F. Automatic Target Recognition of SAR Images Based on Global Scattering Center Model. IEEE Trans. Geosci. Remote Sens. 2011, 49, 3713–3729. [Google Scholar]
- Wen, X.; Zhang, S.; Wang, J.; Yao, T.; Tang, Y. A CFAR-Enhanced Ship Detector for SAR Images Based on YOLOv5s. Remote Sens. 2024, 16, 733. [Google Scholar] [CrossRef]
- Huang, Z.; Pan, Z.; Lei, B. Transfer Learning with Deep Convolutional Neural Network for SAR Target Classification with Limited Labeled Data. Remote Sens. 2017, 9, 907. [Google Scholar] [CrossRef]
- Wang, L.; Qi, Y.; Mathiopoulos, P.T.; Zhao, C.; Mazhar, S. An Improved SAR Ship Classification Method Using Text-to-Image Generation-Based Data Augmentation and Squeeze and Excitation. Remote Sens. 2024, 16, 1299. [Google Scholar] [CrossRef]
- Yang, Y.; Chen, J.; Sun, L.; Zhou, Z.; Huang, Z.; Wu, B. Unsupervised Domain-Adaptive SAR Ship Detection Based on Cross-Domain Feature Interaction and Data Contribution Balance. Remote Sens. 2024, 16, 420. [Google Scholar] [CrossRef]
- Balz, T.; Stilla, U. Hybrid GPU-Based Single-and Double-Bounce SAR Simulation. IEEE Trans. Geosci. Remote Sens. 2009, 47, 3519–3529. [Google Scholar] [CrossRef]
- Yu, Q.; Hu, H.; Geng, X.; Jiang, Y.; An, J. High-Performance SAR Automatic Target Recognition under Limited Data Condition Based on a Deep Feature Fusion Network. IEEE Access 2019, 7, 165646–165658. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep Learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
- Hao, X.; Liu, L.; Yang, R.; Yin, L.; Zhang, L.; Li, X. A Review of Data Augmentation Methods of Remote Sensing Image Target Recognition. Remote Sens. 2023, 15, 827. [Google Scholar] [CrossRef]
- Yang, R.; Wang, R.; Deng, Y.; Jia, X.; Zhang, H. Rethinking the Random Cropping Data Augmentation Method Used in the Training of CNN-Based SAR Image Ship Detector. Remote Sens. 2020, 13, 34. [Google Scholar] [CrossRef]
- Khalifa, N.E.; Loey, M.; Mirjalili, S. A Comprehensive Survey of Recent Trends in Deep Learning for Digital Images Augmentation. Artif. Intell. Rev. 2022, 55, 2351–2377. [Google Scholar] [CrossRef] [PubMed]
- Tao, J.; Auer, S.; Palubinskas, G.; Reinartz, P.; Bamler, R. Automatic SAR Simulation Technique for Object Identification in Complex Urban Scenarios. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 994–1003. [Google Scholar] [CrossRef]
- Hammer, H.; Schulz, K. Coherent Simulation of SAR Images. SPIE 2009, 7477, 74771G-1–74771G-9. [Google Scholar]
- Wang, C.; Xu, C.; Yao, X.; Tao, D. Evolutionary Generative Adversarial Networks. IEEE Trans. Evol. Comput. 2019, 23, 921–934. [Google Scholar] [CrossRef]
- Gao, R.; Hou, X.; Qin, J. Zero-VAE-GAN: Generating Unseen Features for Generalized and Transductive Zero-Shot Learning. IEEE Trans. Image Process. 2020, 29, 3665–3680. [Google Scholar] [CrossRef]
- Liu, Y.; Liu, Z.; Li, S.; Yu, Z.; Guo, Y.; Liu, Q.; Wang, G. Cloud-VAE: Variational Autoencoder with Concepts Embedded. Pattern Recognit. 2023, 140, 109530–109572. [Google Scholar] [CrossRef]
- He, J.; Ouyang, M.; Chen, Z.; Chen, D.; Liu, S. A Deep Transfer Learning Fault Diagnosis Method Based on WGAN and Minimum Singular Value for Non-Homologous Bearing. IEEE Trans. Instrum. Meas. 2022, 71, 3509109. [Google Scholar] [CrossRef]
- Yang, W.; Song, H.; Huang, X.; Xu, X.; Liao, M. Change Detection in High-Resolution SAR Images Based on Jensen–Shannon Divergence and Hierarchical Markov Model. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2014, 7, 3318–3327. [Google Scholar] [CrossRef]
- Shi, Y.; Han, L.; Chang, S.; Hu, T.; Dancey, D. A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-Resolution. IEEE Trans. Geosci. Remote Sens. 2022, 60, 5534819. [Google Scholar] [CrossRef]
- Li, W.; Fan, L.; Wang, Z.; Ma, C.; Cui, X. Tackling Mode Collapse in Multi-Generator GANs with Orthogonal Vectors. Pattern Recognit. 2021, 110, 107646–107690. [Google Scholar] [CrossRef]
- Yang, X.; Zhao, J.; Wei, Z.; Wang, N.; Gao, X. SAR-to-Optical Image Translation Based on Improved CGAN. Pattern Recognit. 2022, 121, 108208–108217. [Google Scholar] [CrossRef]
- Zhang, X.; Wang, Z.; Lu, K.; Pan, Q.; Li, Y. Data Augmentation and Classification of Sea–Land Clutter for Over-the-Horizon Radar Using AC-VAEGAN. IEEE Trans. Geosci. Remote Sens. 2023, 61, 5104416. [Google Scholar] [CrossRef]
- Cao, C.; Cao, Z.; Cui, Z. LDGAN: A Synthetic Aperture Radar Image Generation Method for Automatic Target Recognition. IEEE Trans. Geosci. Remote Sens. 2020, 58, 3495–3508. [Google Scholar] [CrossRef]
- Du, S.; Wang, J.; Qi, Y. A High-quality Multicategory SAR Images Generation Method with Multi-Constraint GAN for ATR. IEEE Geosci. Remote Sens. Lett. 2021, 19, 1–5. [Google Scholar]
- Shi, X.; Xing, M.; Zhang, J.; Sun, G. ISAGAN: A High-Fidelity Full-Azimuth SAR Image Generation Method. In Proceedings of the 2022 3rd China International SAR Symposium (CISS), Shanghai, China, 2–4 November 2022; pp. 1–41. [Google Scholar]
- Oghim, S.; Kim, Y.; Bang, H. SAR Image Generation Method Using DH-GAN for Automatic Target Recognition. Sensors 2024, 24, 670. [Google Scholar] [CrossRef]
- Sun, Y.; Wang, Y.; Hu, L. Attribute-Guided Generative Adversarial Network with Improved Episode Training Strategy for Few-Shot SAR Image Generation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2023, 16, 1785–1801. [Google Scholar] [CrossRef]
- Wang, C.; Pei, J.; Liu, X. SAR Target Image Generation Method Using Azimuth-Controllable Generative Adversarial Network. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2022, 15, 9381–9397. [Google Scholar] [CrossRef]
- Zeng, Z.; Tan, X.; Zhang, X. ATGAN: A SAR Target Image Generation Method for Automatic Target Recognition. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2024, 17, 2986–3001. [Google Scholar] [CrossRef]
- Sinha, S.; Zhao, Z.; Goyal, A.C.; Odena, A. Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples. Proc. Adv. Neural Inf. Process. Syst. 2020, 33, 14638–14649. [Google Scholar]
- Keydel, E.R.; Lee, S.W.; Moore, J.T. MSTAR Extended Operating Conditions: A Tutorial. Algorithms SAR Imagery 1996, 2757, 228–242. [Google Scholar]
- Szegedy, C.; Vanhoucke, V.; Ioffe, S. Rethinking the Inception Architecture for Computer Vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; Volume 85, pp. 2818–2826. [Google Scholar]
- Borji, A. Pros and Cons of GAN Evaluation Measures. Comput. Vis. Image Underst. 2019, 179, 41–65. [Google Scholar] [CrossRef]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet Classification with Deep Convolutional Neural Networks. Proc Adv. Neural Inf. Process. Syst. 2012, 25, 84–90. [Google Scholar] [CrossRef]
- Chen, S.; Wang, H.; Xu, F.; Jin, Y.Q. Target Classification Using the Deep Convolutional Networks for SAR Images. IEEE Trans. Geosci. Remote Sens. 2016, 54, 4806–4817. [Google Scholar] [CrossRef]
- Ye, M.; Ni, R.; Zhang, C.; Gong, H.; Hu, T.; Li, S.; Sun, Y.; Zhang, T.; Guo, Y. A Lightweight Model of VGG-16 for Remote Sensing Image Classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2021, 14, 6916–6922. [Google Scholar] [CrossRef]
- Li, L.; Wang, C.; Zhang, H.; Zhang, B. SAR Image Ship Object Generation and Classification with Improved Residual Conditional Generative Adversarial Network. IEEE Geosci. Remote Sens. Lett. 2020, 19, 4000105. [Google Scholar] [CrossRef]
Target Category | Depression Angle | Number of Samples |
---|---|---|
BMP2 | 17° | 233 |
15° | 587 | |
BTR70 | 17° | 233 |
15° | 196 | |
T72 | 17° | 232 |
15° | 582 | |
BTR60 | 17° | 256 |
15° | 195 | |
2S1 | 17° | 299 |
15° | 274 | |
BRDM2 | 17° | 298 |
15° | 274 | |
D7 | 17° | 299 |
15° | 274 | |
T62 | 17° | 299 |
15° | 273 | |
ZIL131 | 17° | 299 |
15° | 274 | |
ZSU23/4 | 17° | 299 |
15° | 274 |
BS × 90% | BS × 70% | BS × 50% | BS × 30% | |
---|---|---|---|---|
FID | 39.1 | 38.3 | 38.9 | 45.3 |
Generation Model | FID Scores |
---|---|
ACGAN | 57.2 |
WACGAN-GP | 41.5 |
Single top-k | 39.6 |
RCFCA-GAN (Stage 1) | 38.3 |
Original | ACGAN | WACGAN-GP | RCFCA-GAN (Stage 1) | RCFCA-GAN (Stage 1 + Stage 2) | ||
---|---|---|---|---|---|---|
AlexNet | 90.5 | 91.6 | 92.1 | 92.3 | 92.8 | 95.1 |
AconvNet | 95.4 | 95.8 | 96.4 | 96.9 | 97.4 | 97.8 |
VGG | 94.9 | 95.6 | 96.5 | 97.0 | 97.3 | 97.9 |
ResNet | 94.8 | 95.5 | 96.6 | 96.8 | 97.2 | 97.5 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, M.; Wang, H.; Chen, S.; Tao, M.; Wei, J. A Two-Stage SAR Image Generation Algorithm Based on GAN with Reinforced Constraint Filtering and Compensation Techniques. Remote Sens. 2024, 16, 1963. https://doi.org/10.3390/rs16111963
Liu M, Wang H, Chen S, Tao M, Wei J. A Two-Stage SAR Image Generation Algorithm Based on GAN with Reinforced Constraint Filtering and Compensation Techniques. Remote Sensing. 2024; 16(11):1963. https://doi.org/10.3390/rs16111963
Chicago/Turabian StyleLiu, Ming, Hongchen Wang, Shichao Chen, Mingliang Tao, and Jingbiao Wei. 2024. "A Two-Stage SAR Image Generation Algorithm Based on GAN with Reinforced Constraint Filtering and Compensation Techniques" Remote Sensing 16, no. 11: 1963. https://doi.org/10.3390/rs16111963
APA StyleLiu, M., Wang, H., Chen, S., Tao, M., & Wei, J. (2024). A Two-Stage SAR Image Generation Algorithm Based on GAN with Reinforced Constraint Filtering and Compensation Techniques. Remote Sensing, 16(11), 1963. https://doi.org/10.3390/rs16111963