Quantitative and Qualitative Analysis of Agricultural Fields Based on Aerial Multispectral Images Using Neural Networks
Abstract
:1. Introduction
2. Related Work
3. Multispectral Image Processing
4. Neural Network Training
5. Implementation
- Image Container: Stores an object containing the loaded image and allows interaction with it through mouse cursor movement and scaling. It aggregates the Image Element.
- Image Element: A crucial element responsible for storing, displaying, and creating maps on the graphical interface. It reloads the image and displays it after analysis when the appropriate signal is received from the backend.
- Polygon Canvas: Enables drawing polygons on the image.
- Polygon List: Stores a list of drawn polygons, allowing the user to select areas of the map for analysis.
- Dialogs: Manages dialog windows, pop-ups, and feedback messages. It handles project creation, selection of feedback information files related to the analysis, and other summary windows.
- Menu Top: Contains tools that users can use during interaction with the application.
- Connections: Responsible for managing signals and connections between the frontend and backend as well as signals within classes and elements of the user interface.
- OpencvImageProvider: Responsible for loading images, storing them in the engine’s memory, and appropriately managing all the results of the conducted analyses. Additionally, it facilitates sending images to the user interface.
- ProjectManager: Used in the initial phase to create a project and later to save it and generate reports from the conducted analyses.
- PolygonManager: Stores data related to specific analyses. For qualitative analysis, it provides the averaged index value for the area, while for quantitative analysis, it provides the number of detected object instances. Its data are used by ProjectManager to generate the appropriate report. It also contains information about the list of polygons and points, which is used to visualize them on the user interface.
- Processing: Implements the entire process of both qualitative and quantitative analyses, including generating index maps and using artificial intelligence for quantitative analysis.
6. Results
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Chakhvashvili, E.; Bendig, J.; Siegmann, B.; Muller, O.; Verrelst, J.; Roscher, U. LAI and Leaf Chlorophyll Content Retrieval Under Changing Spatial Scale Using a UAV-Mounted Multispectral Camera. In Proceedings of the IGARSS 2022—2022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, 17–22 July 2022; pp. 7891–7894. [Google Scholar] [CrossRef]
- Osco, L.P.; Arruda, M.D.S.D.; Marcato Junior, J.; Da Silva, N.B.; Ramos, A.P.M.; Moryia, É.A.S.; Imai, N.N.; Pereira, D.R.; Creste, J.E.; Matsubara, E.T.; et al. A convolutional neural network approach for counting and geolocating citrus-trees in UAV multispectral imagery. ISPRS J. Photogramm. Remote Sens. 2020, 160, 97–106. [Google Scholar] [CrossRef]
- Guo, Y.; Chen, S.; Li, X.; Cunha, M.; Jayavelu, S.; Cammarano, D.; Fu, Y. Machine Learning-Based Approaches for Predicting SPAD Values of Maize Using Multi-Spectral Images. Remote Sens. 2022, 14, 1337. [Google Scholar] [CrossRef]
- Landsat 1. Available online: https://landsat.gsfc.nasa.gov/satellites/landsat-1/ (accessed on 30 August 2023).
- Jose, B.; Nicolas, M.; Danilo, C.; Eduardo, A. Multispectral NDVI aerial image system for vegetation analysis by using a consumer camera. In Proceedings of the 2014 IEEE International Autumn Meeting on Power, Electronics and Computing (ROPEC), Ixtapa, Mexico, 5–7 November 2014; pp. 1–6. [Google Scholar] [CrossRef]
- Kharchenko, V.P.; Kukush, A.G.; Kuzmenko, N.S.; Ostroumov, I.V. Probability density estimation for object recognition in unmanned aerial vehicle application. In Proceedings of the 2017 IEEE 4th International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), Kyiv, Ukraine, 17–19 October 2017; pp. 233–236. [Google Scholar] [CrossRef]
- Qu, Y.; Jiang, L.; Guo, X. Moving vehicle detection with convolutional networks in UAV videos. In Proceedings of the 2016 2nd International Conference on Control, Automation and Robotics (ICCAR), Hong Kong, China, 28–30 April 2016; pp. 225–229. [Google Scholar] [CrossRef]
- Yu, H.; Yang, W.; Zhang, H.; He, W. A UAV-based crack inspection system for concrete bridge monitoring. In Proceedings of the 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Fort Worth, TX, USA, 23–28 July 2017; pp. 3305–3308. [Google Scholar] [CrossRef]
- Jeyaraj, S.; Ramakrishnan, B.; Ramsankaran, R. Application of Unmanned Aerial Vehicle (UAV) in the assessment of beach volume change—A case study of Malgund beach. In Proceedings of the OCEANS 2022, Chennai, India, 21–24 February 2022; pp. 1–4. [Google Scholar] [CrossRef]
- Jiménez-Torres, M.; Silva, C.P.; Riquelme, C.; Estay, S.A.; Soto-Gamboa, M. Automatic Recognition of Black-Necked Swan (Cygnus melancoryphus) from Drone Imagery. Drones 2023, 7, 71. [Google Scholar] [CrossRef]
- Nugroho, G.; Satrio, M.; Rafsanjani, A.A.; Trio Sadewo, R.R. Avionic system design Unmanned Aerial Vehicle for disaster area monitoring. In Proceedings of the 2015 International Conference on Advanced Mechatronics, Intelligent Manufacture, and Industrial Automation (ICAMIMIA), Surabaya, Indonesia, 15–17 October 2015; pp. 198–201. [Google Scholar] [CrossRef]
- Dang Nguyen, K.; Nguyen, T.-T. Vision-Based Software-in-the-Loop-Simulation for Unmanned Aerial Vehicles Using Gazebo and PX4 Open Source. In Proceedings of the 2019 International Conference on System Science and Engineering (ICSSE), Dong Hoi, Vietnam, 20–21 July 2019; pp. 429–432. [Google Scholar] [CrossRef]
- Buyukkelek, A.F.; Erturan, A.M.; Dagadasi, M.; Celik, S.; Aslanbas, M.; Durdu, A. Image Processing Based Transportation in Unmanned Aerial Vehicles. In Proceedings of the 2020 28th Signal Processing and Communications Applications Conference (SIU), Gaziantep, Turkey, 5–7 October 2020; pp. 1–4. [Google Scholar] [CrossRef]
- Martinez Rocamora, B.; Lima, R.R.; Samarakoon, K.; Rathjen, J.; Gross, J.N.; Pereira, G.A.S. Oxpecker: A Tethered UAV for Inspection of Stone-Mine Pillars. Drones 2023, 7, 73. [Google Scholar] [CrossRef]
- La Salandra, M.; Colacicco, R.; Dellino, P.; Capolongo, D. An Effective Approach for Automatic River Features Extraction Using High-Resolution UAV Imagery. Drones 2023, 7, 70. [Google Scholar] [CrossRef]
- Zhang, W.; Peng, X.; Cui, G.; Wang, H.; Takata, D.; Guo, W. Tree Branch Skeleton Extraction from Drone-Based Photogrammetric Point Cloud. Drones 2023, 7, 65. [Google Scholar] [CrossRef]
- Rezinkina, M.; Rezinkin, O.; Zaporozhets, A. UAVs Application in Power Engineering. In Proceedings of the 2021 IEEE 6th International Conference on Actual Problems of Unmanned Aerial Vehicles Development (APUAVD), Kyiv, Ukraine, 19–21 October 2021; pp. 161–164. [Google Scholar] [CrossRef]
- Rezinkina, M.; Rezinkin, O.; Lytvynenko, S.; Tomashevskyi, R. Electromagnetic Compatibility at UAVs Usage for Power Transmission Lines Monitoring. In Proceedings of the 2019 IEEE 5th International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), Kyiv, Ukraine, 22–24 October 2019; pp. 157–160. [Google Scholar] [CrossRef]
- Mochammad, F.; Putra, A.R.; Trilaksono, B.R. Implementation of hazardous chemical gas monitoring system using unmanned aerial vehicle (UAV). In Proceedings of the 2016 6th International Conference on System Engineering and Technology (ICSET), Bandung, Indonesia, 3–4 October 2016; pp. 163–168. [Google Scholar] [CrossRef]
- Paredes, J.A.; Gonzalez, J.; Saito, C.; Flores, A. Multispectral imaging system with UAV integration capabilities for crop analysis. In Proceedings of the 2017 First IEEE International Symposium of Geoscience and Remote Sensing (GRSS-CHILE), Valdivia, Chile, 15–16 June 2017; pp. 1–4. [Google Scholar] [CrossRef]
- Liu, B.; Zhang, J.; Li, Z. An Improved APAP Algorithm via Line Segment Correction for UAV Multispectral Image Stitching. In Proceedings of the IGARSS 2022—2022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, 17–22 July 2022; pp. 6057–6060. [Google Scholar] [CrossRef]
- Wu, M.Y.; Cheng, S.; Qin, L.; Wu, G. Ground Object Classification Based on UAV Multispectral Images. In Proceedings of the 2019 Chinese Automation Congress (CAC), Hangzhou, China, 22–24 November 2019; pp. 4629–4634. [Google Scholar] [CrossRef]
- Jacobsen, R.H.; Matlekovic, L.; Shi, L.; Malle, N.; Ayoub, N.; Hageman, K.; Hansen, S.; Nyboe, F.F.; Ebeid, E. Design of an Autonomous Cooperative Drone Swarm for Inspections of Safety Critical Infrastructure. Appl. Sci. 2023, 13, 1256. [Google Scholar] [CrossRef]
- Zhang, Y.; Hu, Q.; Li, H.; Li, J.; Liu, T.; Chen, Y.; Ai, M.; Dong, J. A Back Propagation Neural Network-Based Radiometric Correction Method (BPNNRCM) for UAV Multispectral Image. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2023, 16, 112–125. [Google Scholar] [CrossRef]
- Boyuk, M.; Duvar, R.; Urhan, O. Deep Learning Based Vehicle Detection with Images Taken from Unmanned Air Vehicle. In Proceedings of the 2020 Innovations in Intelligent Systems and Applications Conference (ASYU), Istanbul, Turkey, 15–17 October 2020; pp. 1–4. [Google Scholar] [CrossRef]
- Selbesoğlu, M.O.; Bakirman, T.; Vassilev, O.; Ozsoy, B. Mapping of Glaciers on Horseshoe Island, Antarctic Peninsula, with Deep Learning Based on High-Resolution Orthophoto. Drones 2023, 7, 72. [Google Scholar] [CrossRef]
- Xu, W.; Yang, W.; Chen, P.; Zhan, Y.; Zhang, L.; Lan, Y. Cotton Fiber Quality Estimation Based on Machine Learning Using Time Series UAV Remote Sensing Data. Remote Sens. 2023, 15, 586. [Google Scholar] [CrossRef]
- Haavardsholm, T.V.; Skauli, T.; Stahl, A. Multimodal Multispectral Imaging System for Small UAVs. IEEE Robot. Autom. Lett. 2020, 5, 1039–1046. [Google Scholar] [CrossRef]
- Huaman, S.; Castro, A.; Morales, G.; Telles, J. Regression Models between Active Sensor-Measured NDVI and UAV-Acquired Multispectral Images with Positioning Uncertainty. IEEE Lat. Am. Trans. 2019, 17, 1055–1067. [Google Scholar] [CrossRef]
- Chabalala, Y.; Adam, E.; Ali, K.A. Exploring the Effect of Balanced and Imbalanced Multi-Class Distribution Data and Sampling Techniques on Fruit-Tree Crop Classification Using Different Machine Learning Classifiers. Geomatics 2023, 3, 70–92. [Google Scholar] [CrossRef]
- Lee, K.; Wang, B.; Lee, S. Analysis of YOLOv5 and DeepLabv3+ Algorithms for Detecting Illegal Cultivation on Public Land: A Case Study of a Riverside in Korea. Int. J. Environ. Res. Public Health 2023, 20, 1770. [Google Scholar] [CrossRef] [PubMed]
- Ren, Y.; Li, Q.; Du, X.; Zhang, Y.; Wang, H.; Shi, G.; Wei, M. Analysis of Corn Yield Prediction Potential at Various Growth Phases Using a Process-Based Model and Deep Learning. Plants 2023, 12, 446. [Google Scholar] [CrossRef] [PubMed]
- Chen, C.; Li, Y.; Cao, G.; Zhang, J. Research on Dynamic Scheduling Model of Plant Protection UAV Based on Levy Simulated Annealing Algorithm. Sustainability 2023, 15, 1772. [Google Scholar] [CrossRef]
- Valdez, A.E.; Castaneda, M.A.P.; Gomez-Rojas, J.; Ramos, R.L. Canopy Extraction in a Banana Crop from UAV Captured Multispectral Images. In Proceedings of the 2022 IEEE 40th Central America and Panama Convention (CONCAPAN), Panama, Panama, 9–12 November 2022; pp. 1–6. [Google Scholar] [CrossRef]
- Lin, J.; Zuo, H.; Ye, Y.; Liao, X. Histogram-Based Autoadaptive Filter for Destriping NDVI Imagery Acquired by UAV-Loaded Multispectral Camera. IEEE Geosci. Remote Sens. Lett. 2019, 16, 648–652. [Google Scholar] [CrossRef]
- Pederi, Y.A.; Cheporniuk, H.S. Unmanned Aerial Vehicles and new technological methods of monitoring and crop protection in precision agriculture. In Proceedings of the 2015 IEEE International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), Kyiv, Ukraine, 13–15 October 2015; pp. 298–301. [Google Scholar] [CrossRef]
- Gunchenko Yu, A.; Shvorov, S.A.; Zagrebnyuk, V.I.; Kumysh, V.U.; Lenkov, E.S. Using UAV for unmanned agricultural harvesting equipment route planning and harvest volume measuring. In Proceedings of the 2017 IEEE 4th International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), Kyiv, Ukraine, 17–19 October 2017; pp. 262–265. [Google Scholar] [CrossRef]
- Gergel, I.A.; Kortunov, V.I. Analysis of impact of external and internal aerial photosystem parameters on quality of aerial photos shooting polygon objects with mini-UAV. In Proceedings of the 2015 IEEE International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), Kyiv, Ukraine, 13–15 October 2015; pp. 221–223. [Google Scholar] [CrossRef]
- Ajith, G.; Narasimha Bharadwaj, C.; Naveen Kumar, T.S.; Sriharsha Nag, T.S.; Gururaj, C. UAV Aided Irrigation Using Object Detection Through Wireless Communication Technology. In Proceedings of the 2018 2nd International Conference on Trends in Electronics and Informatics (ICOEI), Tirunelveli, India, 11–12 May 2018; pp. 714–719. [Google Scholar] [CrossRef]
- Navia, J.; Mondragon, I.; Patino, D.; Colorado, J. Multispectral mapping in agriculture: Terrain mosaic using an autonomous quadcopter UAV. In Proceedings of the 2016 International Conference on Unmanned Aircraft Systems (ICUAS), Arlington, VA, USA, 7–10 June 2016; pp. 1351–1358. [Google Scholar] [CrossRef]
- Morales, V.A.; Rodriguez Galvis, J.; Garcia, E.G.; Lizarazo Salcedo, I. Estimation of Individual Potato Plants Area and Volume from Uav-Based Multispectral Images. In Proceedings of the IGARSS 2019—2019 IEEE International Geoscience and Remote Sensing Symposium, Yokohama, Japan, 28 July–2 August 2019; pp. 6259–6262. [Google Scholar] [CrossRef]
- Zaman, B.; Jensen, A.M.; McKee, M. Use of high-resolution multispectral imagery acquired with an autonomous unmanned aerial vehicle to quantify the spread of an invasive wetlands species. In Proceedings of the 2011 IEEE International Geoscience and Remote Sensing Symposium, Vancouver, BC, Canada, 24–29 July 2011; pp. 803–806. [Google Scholar] [CrossRef]
- Raeva, P.L.; Šedina, J.; Dlesk, A. Monitoring of crop fields using multispectral and thermal imagery from UAV. Eur. J. Remote Sens. 2019, 52, 192–201. [Google Scholar] [CrossRef]
- Common Objects in Context. Available online: https://cocodataset.org/ (accessed on 30 August 2023).
- Understanding PASCAL VOC Dataset. Available online: https://www.section.io/engineering-education/understanding-pascal-voc-dataset/ (accessed on 30 August 2023).
- Detectron2. Available online: https://ai.meta.com/tools/detectron2/ (accessed on 30 August 2023).
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. In Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks; Ren, S., He, K., Girshick, R., Sun, J., Eds.; NIPS: Grenada, Spain, 2015. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Lin, T.Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; Zitnick, C.L. Microsoft COCO: Common Objects in Context. In Proceedings of the 13th European Conference Computer Vision (ECCV 2014), Zurich, Switzerland, 6–12 September 2014; Available online: https://arxiv.org/abs/1405.0312 (accessed on 16 November 2023).
Prediction | AP | AP50 | AP75 | AP50:95 |
---|---|---|---|---|
Bounding boxes | 56.096 | 89.069 | 66.683 | 56.096 |
Segmentation masks | 54.253 | 89.069 | 58.230 | 54.253 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Strzępek, K.; Salach, M.; Trybus, B.; Siwiec, K.; Pawłowicz, B.; Paszkiewicz, A. Quantitative and Qualitative Analysis of Agricultural Fields Based on Aerial Multispectral Images Using Neural Networks. Sensors 2023, 23, 9251. https://doi.org/10.3390/s23229251
Strzępek K, Salach M, Trybus B, Siwiec K, Pawłowicz B, Paszkiewicz A. Quantitative and Qualitative Analysis of Agricultural Fields Based on Aerial Multispectral Images Using Neural Networks. Sensors. 2023; 23(22):9251. https://doi.org/10.3390/s23229251
Chicago/Turabian StyleStrzępek, Krzysztof, Mateusz Salach, Bartosz Trybus, Karol Siwiec, Bartosz Pawłowicz, and Andrzej Paszkiewicz. 2023. "Quantitative and Qualitative Analysis of Agricultural Fields Based on Aerial Multispectral Images Using Neural Networks" Sensors 23, no. 22: 9251. https://doi.org/10.3390/s23229251
APA StyleStrzępek, K., Salach, M., Trybus, B., Siwiec, K., Pawłowicz, B., & Paszkiewicz, A. (2023). Quantitative and Qualitative Analysis of Agricultural Fields Based on Aerial Multispectral Images Using Neural Networks. Sensors, 23(22), 9251. https://doi.org/10.3390/s23229251