An Improved Deep Learning Model for Underwater Species Recognition in Aquaculture
Abstract
:1. Introduction
2. Related Work
3. Background
3.1. YOLO v5 Algorithm
3.2. Image Segmentation with U-Net
3.3. Transfer Learning
3.4. Roboflow Platform
3.5. Keras Library
4. Methodology
4.1. Data Collection
4.1.1. CarMask Dataset
4.1.2. FishSpecies Dataset
4.1.3. FishMask Dataset
4.2. Data Preprocessing
4.3. A Deep Learning-Based New Architecture for Underwater Species Recognition
4.3.1. CarMask Model
4.3.2. Fine-Tuning Process
4.3.3. FishMask Model
4.3.4. Our Improved Model FishDETECT
4.3.5. Model Integration into an Embedded Device
4.4. The YOLO v5 FishDETECT Model Architecture
5. Results and Discussion
5.1. Pre-Trained Models
5.2. FishDETECT Model Performances
5.3. Detection and Recognition Results
5.4. FishDETECT Model Integration into the Raspberry Pi-4
6. Further Discussion
7. Conclusions and Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Bhari, B.; Visvanathan, C.T. Sustainable aquaculture: Socio-economic and environmental assessment. In Sustainable Aquaculture; Springer: Cham, Switzerland, 2018; pp. 63–93. [Google Scholar]
- Stankus, A. State of world aquaculture 2020 and regional reviews: FAO webinar series. FAO Aquac. Newsl. 2021, 63, 17–18. [Google Scholar]
- Arechavala-Lopez, P.; Cabrera-Álvarez, M.J.; Maia, C.M.; Saraiva, J.L. Environmental enrichment in fish aquaculture: A review of fundamental and practical aspects. Rev. Aquac. 2022, 14, 704–728. [Google Scholar] [CrossRef]
- Conte, F.S. Stress and the welfare of cultured fish. Appl. Anim. Behav. Sci. 2004, 86, 205–223. [Google Scholar] [CrossRef]
- Li, D.; Li, X.; Wang, Q.; Hao, Y. Advanced Techniques for the Intelligent Diagnosis of Fish Diseases: A Review. Animals 2022, 12, 2938. [Google Scholar] [CrossRef]
- An, D.; Hao, J.; Wei, Y.; Wang, Y.; Yu, X. Application of computer vision in fish intelligent feeding system—A review. Aquac. Res. 2021, 52, 423–437. [Google Scholar] [CrossRef]
- Li, D.; Du, L. Recent advances of deep learning algorithms for aquacultural machine vision systems with emphasis on fish. Artif. Intell. Rev. 2022, 1, 1–40. [Google Scholar] [CrossRef]
- Olesen, I.; Myhr, A.I.; Rosendal, G.K. Sustainable aquaculture: Are we getting there? Ethical perspectives on salmon farming. J. Agric. Environ. Ethics 2011, 24, 381–408. [Google Scholar] [CrossRef]
- Lafont, M.; Dupont, S.; Cousin, P.; Vallauri, A.; Dupont, C. Back to the future: IoT to improve aquaculture: Real-time monitoring and algorithmic prediction of water parameters for aquaculture needs. In Proceedings of the Global IoT Summit (GIoTS), Aarhus, Denmark, 17–21 June 2019; pp. 1–6. [Google Scholar]
- Yang, L.; Liu, Y.; Yu, H.; Fang, X.; Song, L.; Li, D.; Chen, Y. Computer vision models in intelligent aquaculture with emphasis on fish detection and behavior analysis. Arch. Comput. Methods Eng. 2021, 28, 2785–2816. [Google Scholar] [CrossRef]
- Jalal, A.; Salman, A.; Mian, A.; Shortis, M.; Shafait, F. Fish detection and species classification in underwater environments using deep learning with temporal information. Ecol. Inform. 2020, 57, 101088. [Google Scholar] [CrossRef]
- Deep, B.V.; Dash, R. Underwater fish species recognition using deep learning techniques. In Proceedings of the 6th International Conference on Signal Processing and Integrated Networks (SPIN), Noida, India, 7–8 March 2019; pp. 665–669. [Google Scholar]
- Chakravorty, H.; Paul, R.; Das, P. Image processing technique to detect fish disease. Int. J. Comput. Sci. Secur. (IJCSS) 2015, 9, 121–131. [Google Scholar]
- Saberioon, M.; Gholizadeh, A.; Cisar, P.; Pautsina, A.; Urban, J. Application of machine vision systems in aquaculture with emphasis on fish: State-of-the-art and key issues. Rev. Aquac. 2017, 9, 369–387. [Google Scholar] [CrossRef]
- Storbeck, F.; Daan, B. Fish species recognition using computer vision and a neural network. Fish. Res. 2001, 51, 11–15. [Google Scholar] [CrossRef]
- Hu, J.; Li, D.; Duan, Q.; Han, Y.; Chen, G.; Si, X. Fish species classification by color, texture and multi-class support vector machine using computer vision. Comput. Electron. Agric. 2012, 88, 133–140. [Google Scholar] [CrossRef]
- Hridayami, P.; Putra, I.K.; Wibawa, K.S. Fish species recognition using VGG16 deep convolutional neural network. J. Comput. Sci. Eng. 2019, 13, 124–130. [Google Scholar] [CrossRef]
- Ju, Z.; Xue, Y. Fish species recognition using an improved AlexNet model. Optik 2020, 223, 165499. [Google Scholar] [CrossRef]
- Dos Santos, A.A.; Gonçalves, W.N. Improving Pantanal fish species recognition through taxonomic ranks in convolutional neural networks. Ecol. Inform. 2019, 53, 100977. [Google Scholar] [CrossRef]
- Alaba, S.Y.; Nabi, M.M.; Shah, C.; Prior, J.; Campbell, M.D.; Wallace, F.; Ball, J.E.; Moorhead, R. Class-aware fish species recognition using deep learning for an imbalanced dataset. Sensors 2022, 22, 8268. [Google Scholar] [CrossRef]
- Ovalle, J.C.; Vilas, C.; Antelo, L.T. On the use of deep learning for fish species recognition and quantification on board fishing vessels. Mar. Policy 2022, 139–105015. [Google Scholar] [CrossRef]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 779–788. [Google Scholar]
- Du, J. Understanding of object detection based on CNN family and YOLO. J. Phys. 2018, 1004–012029. [Google Scholar] [CrossRef]
- Diwan, T.; Anirudh, G.; Tembhurne, J.V. Object detection using YOLO: Challenges, architectural successors, datasets and applications. Multimed. Tools Appl. 2023, 82, 9243–9275. [Google Scholar] [CrossRef]
- Wageeh, Y.; Mohamed, H.E.; Fadl, A.; Anas, O.; ElMasry, N.; Nabil, A.; Atia, A. YOLO fish detection with Euclidean tracking in fish farms. J. Ambient. Intell. Humaniz. Comput. 2021, 12, 5–12. [Google Scholar] [CrossRef]
- Buric, M.; Pobar, M.; Ivasic-Kos, M. Ball detection using YOLO and Mask R-CNN. In Proceedings of the International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA, 13–15 December 2018; pp. 319–323. [Google Scholar]
- Hu, H.; Tang, C.; Shi, C.; Qian, Y. Detection of residual feed in aquaculture using YOLO and Mask RCNN. Aquac. Eng. 2023, 100, 102304. [Google Scholar] [CrossRef]
- Park, J.H.; Kang, C. A study on enhancement of fish recognition using cumulative mean of YOLO network in underwater video images. J. Mar. Sci. Eng. 2020, 8, 952. [Google Scholar] [CrossRef]
- Zhang, Y.; Guo, Z.; Wu, J.; Tian, Y.; Tang, H.; Guo, X. Real-time vehicle detection based on improved yolo v5. Sustainability 2022, 14, 12274. [Google Scholar] [CrossRef]
- Szeliski, R. Computer Vision: Algorithms and Applications; Springer Nature: Berlin/Heidelberg, Germany, 2022. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI, Munich, Germany, 5–9 October 2015; Volume 18, pp. 234–241. [Google Scholar]
- Villa, M.; Dardenne, G.; Nasan, M.; Letissier, H.; Hamitouche, C.; Stindel, E. FCN-based approach for the automatic segmentation of bone surfaces in ultrasound images. Int. J. Comput. Assist. Radiol. Surg. 2018, 13, 1707–1716. [Google Scholar] [CrossRef]
- He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask r-cnn. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2961–2969. [Google Scholar]
- kaggle. Carvana Image Masking (PNG). Available online: https://www.kaggle.com/datasets/ipythonx/carvana-image-masking-png (accessed on 31 August 2023).
- Raza, K.; Song, H. Fast and accurate fish detection design with improved YOLO-v3 model and transfer learning. Int. J. Adv. Comput. Sci. Appl. 2020, 11, 7–16. [Google Scholar] [CrossRef]
- Kuswantori, A.; Suesut, T.; Tangsrirat, W.; Schleining, G.; Nunak, N. Fish Detection and Classification for Automatic Sorting System with an Optimized YOLO Algorithm. Appl. Sci. 2023, 13, 3812. [Google Scholar] [CrossRef]
- Liu, Y.; Chu, H.; Song, L.; Zhang, Z.; Wei, X.; Chen, M.; Shen, J. An improved tuna-YOLO model based on YOLO v3 for real-time tuna detection considering lightweight deployment. J. Mar. Sci. Eng. 2023, 11, 542. [Google Scholar] [CrossRef]
- Adiwinata, Y.; Sasaoka, A.; Bayupati, I.A.; Sudana, O. Fish species recognition with faster r-cnn inception-v2 using qut fish dataset. Lontar Komput. J. Ilm. Teknol. Inf. 2020, 11, 144–154. [Google Scholar] [CrossRef]
- Zhao, Z.; Liu, Y.; Sun, X.; Liu, J.; Yang, X.; Zhou, C. Composited FishNet: Fish detection and species recognition from low-quality underwater videos. IEEE Trans. Image Process. 2021, 30, 4719–4734. [Google Scholar] [CrossRef]
Fish Species | Count of Samples | Train | Validation | Test |
---|---|---|---|---|
Gilt Head Bream | 1200 | 840 | 180 | 180 |
Red Sea Bream | 900 | 630 | 135 | 135 |
Sea Bass | 1160 | 812 | 174 | 174 |
Red Mullet | 900 | 630 | 135 | 135 |
Horse Mackerel | 1000 | 700 | 150 | 150 |
Black Sea Sprat | 1150 | 805 | 173 | 172 |
Striped Red Mullet | 1030 | 721 | 155 | 154 |
Trout | 1290 | 903 | 194 | 193 |
Shrimp | 820 | 574 | 123 | 123 |
Epoch Number | Metrics | YOLO v5 Nano | COCO | YOLO v5 Large | Our Pre-Trained Model |
---|---|---|---|---|---|
1 | Precision | 0.113 | 0.137 | 0.164 | 0.245 |
Recall | 0.280 | 0.436 | 0.332 | 0.510 | |
mAP50 | 0.131 | 0.144 | 0.241 | 0.339 | |
10 | Precision | 0.742 | 0.832 | 0.895 | 0.912 |
Recall | 0.736 | 0.861 | 0.866 | 0.932 | |
mAP50 | 0.742 | 0.884 | 0.941 | 0.963 | |
20 | Precision | 0.832 | 0.943 | 0.948 | 0.962 |
Recall | 0.887 | 0.967 | 0.936 | 0.978 | |
mAP50 | 0.904 | 0.971 | 0.976 | 0.995 |
Reference | Method | Data Characteristics | Fish Species | Performance Indicators |
---|---|---|---|---|
[35] | The method used in this study consists of creating an improved YOLO v3 model that uses the idea of anchor boxes in the prediction phase. The detection scale is raised to 4 instead of 3. To obtain a suitable size of anchor boxes, the K-Means++ algorithm is run with the dataset. Transfer learning took advantage of pre-trained CNN architecture, which had been trained with nearly 1.2 million ImageNet dataset samples and 1000 classes. | The dataset samples were gathered from diverse sources. The dataset’s samples are all different sizes, such as , , and . | Anemone-fish. Jelly-fish. Star-fish. Shark. | mAP: |
[36] | This study provides a method based on the YOLO v4 recognition algorithm that has been optimized with a novel labeling technique. | 160 images extracted from videos captured underwater. | Yeesok. Nuanchan. Tapian. Nai. Jeen Ban. Jeen To. Nin. Sawai. | Precision: F-Score: |
[37] | An improved real-time detection network was proposed for tuna detection based on the YOLO v3 network, which used lightweight design on the backbone and combined the CBAM attention mechanism module on the basis of the MobileNet v3 network structure to build an efficient tuna detection network, Tuna-YOLO. Following annotation of the dataset, the K-means algorithm was used to obtain nine better anchor boxes based on label information, which was then used to improve detection precision. | All of the image data came from Liancheng Overseas Fishery (Shenzhen) Co., Ltd., and all of the fish were shot on the boat to create catch statistics. | Xiphias gladius. Thunnus obesus. Thunnus albacares. Makaira mazara. | Precision: 95.83% mAP50: 85.74% |
[38] | This study experimented with object detection method based on deep learning, such as Faster R-CNN, which can distinguish the species of fish within an image without additional image preprocessing. | The dataset is obtained from the QUT FISH Dataset. It contains 500 images of 50 classes of fish, with 10 images per class. | Anyperodon leucogrammicus. Bodianus diana. Cephalopholis sexmaculata. Pseudocheilinus hexataenia. | Accuracy: 80.4% |
[39] | This research introduces Composited FishNet, a unique composite fish detection framework based on a composite backbone and an upgraded path aggregation network. A new composite backbone network (CBresnet) is designed to learn scene change information, which is caused by differences in image brightness, fish orientation, seabed structure, aquatic plant movement, fish species shape, and texture differences. | For training, the SeaCLEF 2017 benchmark dataset For training, the SeaCLEF 2017 benchmark dataset is used. This benchmark dataset was created primarily to give resources for evaluating detection algorithms in image and video sequences. The dataset contains 20 low-resolution videos and over 20,000 sample photos of 15 different fish species in their natural coral reef habitat. There are five videos with 640 × 480 pixel resolution and 15 films with 320 × 240 pixel resolution. | 15 different fish species. | Average Precision 0.5:0.95: 75.2% Average Precision 0.5: 92.8% Average Recall: 81.1% |
This work | The main issue in the field of underwater computer vision is the quality of the image, which is influenced by several factors. Our work consists in developing an improved YOLO v5 model. To detect and classify fish species objects, the model is based on another pre-trained fish masks model instead of using other classical transfer learning sources, such as coco. | The first dataset is called Mask dataset. It contains capture objects of different positions and their appropriate masks. The second is called Fish-Species dataset; it contains images of nine species of fish. The third is FishMask dataset; it consists of the fish masks generated from the second dataset. | Gilt Head Bream. Red Sea Bream. Sea Bass. Red Mullet. Horse Mackerel. Black Sea Sprat. Striped Red Mullet. Trout. Shrimp. | Precision: 0.962 Recall: 0.978 mAP50: 0.995 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Hamzaoui, M.; Ould-Elhassen Aoueileyine, M.; Romdhani, L.; Bouallegue, R. An Improved Deep Learning Model for Underwater Species Recognition in Aquaculture. Fishes 2023, 8, 514. https://doi.org/10.3390/fishes8100514
Hamzaoui M, Ould-Elhassen Aoueileyine M, Romdhani L, Bouallegue R. An Improved Deep Learning Model for Underwater Species Recognition in Aquaculture. Fishes. 2023; 8(10):514. https://doi.org/10.3390/fishes8100514
Chicago/Turabian StyleHamzaoui, Mahdi, Mohamed Ould-Elhassen Aoueileyine, Lamia Romdhani, and Ridha Bouallegue. 2023. "An Improved Deep Learning Model for Underwater Species Recognition in Aquaculture" Fishes 8, no. 10: 514. https://doi.org/10.3390/fishes8100514
APA StyleHamzaoui, M., Ould-Elhassen Aoueileyine, M., Romdhani, L., & Bouallegue, R. (2023). An Improved Deep Learning Model for Underwater Species Recognition in Aquaculture. Fishes, 8(10), 514. https://doi.org/10.3390/fishes8100514