An Algorithm for Ship Detection in Complex Observation Scenarios Based on Mooring Buoys
Abstract
:1. Introduction
- Based on its special application environment, the swaying and shaking of the buoy itself will cause the ship to appear shaky and blurred in the camera. This leads to the easy loss of key details of the ship during the extraction of edges and textures, thereby affecting the detection effectiveness;
- The marine environment is complex and variable, and cameras are often disturbed by weather such as rain and fog, which can lead to ship targets being easily occluded, posing significant challenges to accurate positioning and recognition of ships;
- Small ships that frequently damage buoys are more likely to exhibit smaller scales in the image, which may lead to missed detection of small targets. Therefore, ship detection algorithms need to have stronger small object detection capabilities to effectively identify and detect these small ships.
- Aiming at the problem of blurring ship images and a reduction in target spatial resolution due to camera shake, this study redesigned the bottleneck structure of the C2f module, combined with the polarized self-attention mechanism, to reduce information loss due to blurring through dual enhancement of spatial and channel dimensions, and to ensure a more comprehensive extraction and fusion of ship features;
- For the feature of rain and fog obscuring ship targets in bad sea conditions, this study introduces the multi-head self-attention module in the neck network. This approach captures the relative position relationship between features by introducing relative position coding, which fully exploits the correlation between features, effectively weakens the interference of rain and fog, and significantly improves the accuracy of ship detection;
- In response to the common problem of low accuracy in small vessel detection, this study combines an independently designed small target detection head and a larger size feature map to enhance the detection accuracy of small ships by extracting richer feature information from the shallow feature map.
2. Materials and Methods
2.1. Materials
2.1.1. Dataset Description
2.1.2. Data Preprocessing
- For the motion blur caused by camera shake, this paper uses a blur kernel of size for simulation and normalizes the blur kernel to ensure that the total brightness of the processed image remains unchanged;
- For the effect of rainy days, this paper first generates a noise layer on the input image to simulate raindrops and then rotates and stretches the noise layer to increase the real sensation and dynamic effect of the simulated raindrops. Finally, we use the transparency blending technique to blend the simulated raindrop layer with the original image, which finally enhances the visual impact of the rainy day;
- In order to simulate the effect of a real foggy day, this paper adopts the method of synthesizing fog at the center point. The specific operation is to set the fog center at the center of the image, then calculate the Euclidean distance of each pixel point relative to the fog center, then use the attenuation coefficient to adjust the pixel brightness accordingly in order to simulate the natural attenuation of the light when it passes through the fog, and finally multiply the luminance value of each pixel by the attenuation coefficient, so as to achieve the simulation of the fog effect.
2.2. Methods
2.2.1. Overall Network Introduction
2.2.2. C2f_PSA Module
2.2.3. MHSA Module
2.2.4. Small Ship Target Detection Head
3. Experiments
3.1. Evaluation Metrics
3.2. Experimental Platform
3.3. Ablation Experiment
3.4. Performance Comparison of Multiple Models
4. Conclusions
- The YOLOv8-PMH algorithm improved with PSA attention maintains C/2 in the channel dimension and [H, W] in the spatial dimension, a mechanism that effectively preserves the high-resolution information in the original deep convolutional neural network and significantly reduces the effect of blurring due to camera shake on information loss. Especially in the ship target recognition task, the C2f module with integrated PSA attention leads to a 0.9% improvement in the mAP50 of the algorithm and a 1.2% improvement in mAP50:95;
- The MHSA attention mechanism based on the Transformer architecture effectively reduces rain and fog background interference and enhances the ability of feature fusion. After introducing this attention into the neck module, the algorithm achieved a 0.7% improvement in mAP50 and a 0.9% improvement in mAP50:95;
- Based on the original algorithm, the newly designed small ship target detection head significantly improves the feature extraction capability for small ships without affecting the detection performance for large ship targets. With this improvement, the mAP50 of the algorithm realizes a 0.8% improvement, and the mAP50:95 obtains a 0.9% improvement;
- After integrating the above improvements, compared with the original YOLOv8 algorithm, the mAP50 of the YOLOv8-PMH algorithm has increased by 2%, and the mAP50:95 has increased by 2.8%.
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Xu, R.; Wang, H.; Xi, Z.; Wang, W.; Xu, M. Recent Progress on Wave Energy Marine Buoys. J. Mar. Sci. Eng. 2022, 10, 566. [Google Scholar] [CrossRef]
- Canepa, E.; Pensieri, S.; Bozzano, R.; Faimali, M.; Traverso, P.; Cavaleri, L. The ODAS Italia 1 buoy: More than forty years of activity in the Ligurian Sea. Prog. Ocean. 2015, 135, 48–63. [Google Scholar] [CrossRef]
- Park, Y.W.; Kim, T.W.; Kwak, J.S.; Kim, I.K.; Park, J.E.; Ha, K.H. Design of Korean Standard Modular Buoy Body Using Polyethylene Polymer Material for Ship Safety. J. Mater. Sci. Chem. Eng. 2016, 4, 65–73. [Google Scholar] [CrossRef]
- Li, X.; Bian, Y. Modeling and prediction for the Buoy motion characteristics. Ocean Eng. 2021, 239, 109880. [Google Scholar] [CrossRef]
- Teng, C.; Cucullu, S.; Mcarthur, S.; Kohler, C.; Burnett, B. Buoy Vandalism Experienced by NOAA National Data Buoy Center. In Proceedings of the OCEANS 2009, Biloxi, MS, USA, 26–29 October 2009; pp. 1–8. [Google Scholar]
- Hwang, H.G.; Kim, B.S.; Kim, H.W.; Gang, Y.S.; Kim, D.H. A development of active monitoring and approach alarm system for marine buoy protection and ship accident prevention based on trail cameras and AIS. J. Korea Inst. Inf. Commun. Eng. 2018, 22, 1021–1029. [Google Scholar]
- Zheng, Y. Application of CCTV monitoring equipment in navigation mark management. China Water Transp. 2019, 19, 101–102. [Google Scholar]
- Zhao, T.; Qi, J.; Ruan, D.; Shan, R. Design of Marine buoy early warning system based on geomagnetic and infrared dual mode detection. J. Mar. Technol. 2017, 36, 15–21. [Google Scholar]
- Chen, S.; Xiang, H.; Gao, S. Intelligent anti-collision warning method of fairway buoy based on passive underwater acoustic positioning. In Proceedings of the IOP Conference Series: Earth and Environmental Science, Hulun Buir, China, 28–30 August 2020. [Google Scholar]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar]
- Girshick, R. Fast r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7–13 December 2015; pp. 1440–1448. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef] [PubMed]
- Gu, J.; Li, B.; Liu, K. Infrared ship target detection Algorithm based on Improved Faster R-CNN. Infrared Technol. 2021, 43, 170–178. [Google Scholar]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhad, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788. [Google Scholar]
- Redmon, J.; Farhadi, A. YOLOv3: An Incremental Improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Bochkovskiy, A.; Wang, C.; Liao, H. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv 2020, arXiv:2004.10934. [Google Scholar]
- Ultralytics. YOLOv5: Object Detection. Available online: https://github.com/ultralytics/yolov5 (accessed on 18 May 2020).
- Wang, C.; Bochkovskiy, A.; Liao, H. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv 2022, arXiv:2207.02696. [Google Scholar]
- Ultralytics/Ultralytics. Available online: https://github.com/ultralytics/ultralytics (accessed on 20 November 2023).
- Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.Y.; Berg, A.C. SSD: Single shot multibox detector. In Computer Vision–ECCV 2016, Proceedings of the 14th European Conference, Amsterdam, The Netherlands, 11–14 October 2016; Proceedings, Part I 14; Lecture Notes in Computer Science; Springer International Publishing: Berlin/Heidelberg, Germany, 2016; pp. 21–37. [Google Scholar]
- Zhang, Q.; Li, Y.; Zhang, Z.; Yin, S.; Ma, L. Marine target detection for PPI images based on YOLO-SWFormer. Alex. Eng. J. 2023, 82, 396–403. [Google Scholar] [CrossRef]
- Zheng, Y.; Zhang, Y.; Qian, L.; Zhang, X.; Diao, S.; Liu, X.; Huang, H. A lightweight ship target detection model based on improved YOLOv5s algorithm. PLoS ONE 2023, 18, e0283932. [Google Scholar] [CrossRef]
- Shang, Y.; Yu, W.; Zeng, G.; Li, H.; Wu, Y. StereoYOLO: A Stereo Vision-Based Method for Maritime Object Recognition and Localization. J. Mar. Sci. Eng. 2024, 12, 197. [Google Scholar] [CrossRef]
- Wang, J.; Pan, Q.; Lu, D.; Zhang, Y. An Efficient Ship-Detection Algorithm Based on the Improved YOLOv5. Electronics 2023, 12, 3600. [Google Scholar] [CrossRef]
- Zhao, Q.; Wu, Y.; Yuan, Y. Ship Target Detection in Optical Remote Sensing Images Based on E2YOLOX-VFL. Remote Sens. 2024, 16, 340. [Google Scholar] [CrossRef]
- Si, J.; Song, B.; Wu, J.; Lin, W.; Huang, W.; Chen, S. Maritime Ship Detection Method for Satellite Images Based on Multiscale Feature Fusion. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2023, 16, 6642–6655. [Google Scholar] [CrossRef]
- Wang, Y.; Wang, B.; Huo, L.; Fan, Y. GT-YOLO: Nearshore Infrared Ship Detection Based on Infrared Images. J. Mar. Sci. Eng. 2024, 12, 213. [Google Scholar] [CrossRef]
- Prasad, D.K.; Rajan, D.; Rachmawati, L.; Rajabally, E.; Quek, C. Video processing from electro-optical sensors for object detection and tracking in a maritime environment: A survey. IEEE Trans. Intell. Transp. Syst. 2017, 18, 1993–2016. [Google Scholar] [CrossRef]
- Shao, Z.; Wu, W.; Wang, Z.; Du, W.; Li, C. SeaShips: A large-scale precisely annotated dataset for ship detection. IEEE Trans. Multimed. 2018, 20, 2593–2604. [Google Scholar] [CrossRef]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; Hu, Q. ECA-Net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 11534–11542. [Google Scholar]
- Cao, Y.; Xu, J.; Lin, S.; Wei, F.; Hu, H. Gcnet: Non-local networks meet squeeze-excitation networks and beyond. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea, 27–28 October 2019; pp. 1971–1980. [Google Scholar]
- Fu, J.; Liu, J.; Tian, H.; Li, Y.; Bao, Y.; Fang, Z.; Lu, H. Dual attention network for scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 16–20 June 2019; pp. 3146–3154. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. Cbam: Convolutional Block Attention Module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Liu, H.; Liu, F.; Fan, X.; Huang, D. Polarized self-attention: Towards high-quality pixel-wise regression. arXiv 2021, arXiv:2107.00782. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Polosukhin, I. Attention is all you need. Adv. Neural Inf. Process. Syst. 2017, 30, 1–11. [Google Scholar]
- Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Houlsby, N. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv 2020, arXiv:2010.11929. [Google Scholar]
- Touvron, H.; Cord, M.; Douze, M.; Massa, F.; Sablayrolles, A.; Jégou, H. Training data-efficient image transformers & distillation through attention. arXiv 2021, arXiv:2012.12887. [Google Scholar]
- Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Guo, B. Swin transformer: Hierarchical vision transformer using shifted windows. arXiv 2021, arXiv:2103.14030. [Google Scholar]
- Srinivas, A.; Lin, T.Y.; Parmar, N.; Shlens, J.; Abbeel, P.; Vaswani, A. Bottleneck transformers for visual recognition. arXiv 2021, arXiv:2101.11605. [Google Scholar]
Name | Number (SMD) | Number (SeaShips) | Number (All) |
---|---|---|---|
container ship | 98 | 179 | 1066 |
general cargo ship | 183 | 0 | 1289 |
fishing boat | 0 | 24 | 845 |
passenger ship | 86 | 158 | 1639 |
small wooden boat | 0 | 0 | 806 |
sailboat | 17 | 0 | 655 |
Name | Number |
---|---|
resolution | 640 640 |
optimizer | Adam |
learning rate | 0.01 |
momentum | 0.937 |
batch size | 16 |
epochs | 200 |
Model | AP(%) | mAP50 (%) | mAP50:95 (%) | |||||
---|---|---|---|---|---|---|---|---|
CS | GCS | FB | PS | SWB | SB | |||
YOLOv8n | 93.7 | 94.7 | 86.5 | 91.4 | 93.3 | 90.1 | 92.2 | 70.4 |
YOLOv8n + PSA | 97.5 | 94.7 | 88.4 | 92.5 | 93.5 | 92 | 93.1 | 71.6 |
YOLOv8n + MHSA | 97.4 | 94.6 | 88.1 | 92.2 | 93.4 | 91.7 | 92.9 | 71.3 |
YOLOv8n + 4-Head | 97.2 | 94.5 | 88.9 | 92.3 | 93.5 | 91.8 | 93 | 71.3 |
YOLOv8n + PSA + MHSA | 97.6 | 94.7 | 89.1 | 92.8 | 93.4 | 92.1 | 93.3 | 72.5 |
YOLOv8n + PSA + 4-Head | 97.4 | 94.5 | 89.5 | 93.8 | 93.5 | 92.4 | 93.5 | 72.4 |
YOLOv8n + MHSA + 4-Head | 97.5 | 94.6 | 90.5 | 93.4 | 93.5 | 92.2 | 93.6 | 72.8 |
YOLOv8n + PSA + MHSA + 4-Head | 97.8 | 94.7 | 91.3 | 94.9 | 93.7 | 92.7 | 94.2 | 73.2 |
Model | AP(%) | mAP50 (%) | mAP50:95 (%) | |||||
---|---|---|---|---|---|---|---|---|
CS | GCS | FB | PS | SWB | SB | |||
Faster R-CNN | 97.4 | 94.5 | 87.4 | 90.9 | 86.5 | 89.4 | 92.2 | - |
SSD | 96.1 | 93.9 | 82.8 | 90.3 | 84.2 | 85.5 | 88.8 | - |
YOLOv3-tiny | 96.3 | 94.3 | 82.1 | 91.6 | 89.6 | 89.2 | 90.5 | 65.5 |
YOLOv4-tiny | 97.3 | 94.6 | 88.2 | 91.4 | 91.8 | 79.5 | 90.5 | 64.4 |
YOLOv5n | 97.2 | 93.6 | 86.3 | 92.6 | 91.6 | 85.3 | 91.1 | 68.2 |
YOLOv6n | 97.5 | 94.6 | 85.6 | 92.8 | 90.8 | 88.2 | 91.6 | 70.5 |
YOLOv8n | 97.3 | 94.7 | 86.5 | 91.4 | 93.3 | 90.1 | 92.2 | 70.4 |
YOLOv8-PMH | 97.8 | 94.7 | 91.3 | 94.9 | 93.7 | 92.7 | 94.2 | 73.2 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, W.; Ning, C.; Fang, Y.; Yuan, G.; Zhou, P.; Li, C. An Algorithm for Ship Detection in Complex Observation Scenarios Based on Mooring Buoys. J. Mar. Sci. Eng. 2024, 12, 1226. https://doi.org/10.3390/jmse12071226
Li W, Ning C, Fang Y, Yuan G, Zhou P, Li C. An Algorithm for Ship Detection in Complex Observation Scenarios Based on Mooring Buoys. Journal of Marine Science and Engineering. 2024; 12(7):1226. https://doi.org/10.3390/jmse12071226
Chicago/Turabian StyleLi, Wenbo, Chunlin Ning, Yue Fang, Guozheng Yuan, Peng Zhou, and Chao Li. 2024. "An Algorithm for Ship Detection in Complex Observation Scenarios Based on Mooring Buoys" Journal of Marine Science and Engineering 12, no. 7: 1226. https://doi.org/10.3390/jmse12071226
APA StyleLi, W., Ning, C., Fang, Y., Yuan, G., Zhou, P., & Li, C. (2024). An Algorithm for Ship Detection in Complex Observation Scenarios Based on Mooring Buoys. Journal of Marine Science and Engineering, 12(7), 1226. https://doi.org/10.3390/jmse12071226