Reshaping Leaf-Level Reflectance Data for Plant Species Discrimination: Exploring Image Shape’s Impact on Deep Learning Results
Abstract
:1. Introduction
2. Data and Methods
2.1. Data Source
2.2. Data Preprocessing
2.2.1. Selection of Plant Species
2.2.2. Reflectance Data Preprocessing
2.3. CNN Model Architectures
2.4. Model, Image Shape Comparison and Evaluation
2.5. Flowchart of the Process
3. Results
3.1. The Performance of Models
3.2. The Best Performance of Each Image Shape (Regardless of the CNN Models)
3.3. Identification Results
4. Discussion and Future Work
4.1. Comparative Analysis with Prior Research
4.2. Image Shape’s Impact on Species Discrimination Results
4.3. Uncertainty of Approach and Future Studies
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Farmonov, N.; Amankulova, K.; Szatmari, J.; Sharifi, A.; Abbasi-Moghadam, D.; Mirhoseini Nejad, S.M.; Mucsi, L. Crop Type Classification by DESIS Hyperspectral Imagery and Machine Learning Algorithms. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2023, 16, 1576–1588. [Google Scholar] [CrossRef]
- Liu, K.H.; Yang, M.H.; Huang, S.T.; Lin, C. Plant Species Classification Based on Hyperspectral Imaging via a Lightweight Convolutional Neural Network Model. Front. Plant Sci. 2022, 13, 763. [Google Scholar] [CrossRef] [PubMed]
- Mäyrä, J.; Keski-Saari, S.; Kivinen, S.; Tanhuanpää, T.; Hurskainen, P.; Kullberg, P.; Poikolainen, L.; Viinikka, A.; Tuominen, S.; Kumpula, T.; et al. Tree species classification from airborne hyperspectral and LiDAR data using 3D convolutional neural networks. Remote Sens. Environ. 2021, 256, 112322. [Google Scholar] [CrossRef]
- Hassoon, I.M.; Kassir, S.A.; Altaie, S.M. A Review of Plant Species Identification Techniques. Int. J. Sci. Res. 2018, 7, 2016–2019. [Google Scholar] [CrossRef]
- Cope, J.S.; Corney, D.; Clark, J.Y.; Remagnino, P.; Wilkin, P. Plant species identification using digital morphometrics: A review. Expert Syst. Appl. 2012, 39, 7562–7573. [Google Scholar] [CrossRef]
- Song, G.; Wang, Q. Species classification from hyperspectral leaf information using machine learning approaches. Ecol. Inform. 2023, 76, 102141. [Google Scholar] [CrossRef]
- Bahrami, M.; Mobasheri, M.R. Plant species determination by coding leaf reflectance spectrum and its derivatives. Eur. J. Remote Sens. 2020, 53, 258–273. [Google Scholar] [CrossRef]
- Meireles, J.E.; Cavender-Bares, J.; Townsend, P.A.; Ustin, S.; Gamon, J.A.; Schweiger, A.K.; Schaepman, M.E.; Asner, G.P.; Martin, R.E.; Singh, A.; et al. Leaf reflectance spectra capture the evolutionary history of seed plants. New Phytol. 2020, 228, 485–493. [Google Scholar] [CrossRef]
- Jantzen, J.R.; Laliberté, E.; Carteron, A.; Beauchamp-Rioux, R.; Blanchard, F.; Crofts, A.L.; Girard, A.; Hacker, P.W.; Pardo, J.; Schweiger, A.K.; et al. Evolutionary history explains foliar spectral differences between arbuscular and ectomycorrhizal plant species. New Phytol. 2023, 238, 2651–2667. [Google Scholar] [CrossRef]
- Hycza, T.; Stereńczak, K.; Bałazy, R. Potential use of hyperspectral data to classify forest tree species. N. Z. J. For. Sci. 2018, 48, 18. [Google Scholar] [CrossRef]
- Badola, A.; Padalia, H.; Belgiu, M.; Verma, P.A. Tree Species Mapping in Tropical Forests Using Hyperspectral Remote Sensing and Machine Learning. In Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium, 11–16 July 2021; pp. 5421–5424. [Google Scholar] [CrossRef]
- Hu, Y.; Xu, L.; Huang, P.; Luo, X.; Wang, P.; Kang, Z. Reliable identification of oolong tea species: Nondestructive testing classification based on fluorescence hyperspectral technology and machine learning. Agriculture 2021, 11, 1106. [Google Scholar] [CrossRef]
- Cao, J.; Liu, K.; Liu, L.; Zhu, Y.; Li, J.; He, Z. Identifying mangrove species using field close-range snapshot hyperspectral imaging and machine-learning techniques. Remote Sens. 2018, 10, 2047. [Google Scholar] [CrossRef]
- Marrs, J.; Ni-Meister, W. Machine learning techniques for tree species classification using co-registered LiDAR and hyperspectral data. Remote Sens. 2019, 11, 819. [Google Scholar] [CrossRef]
- Al-Awadhi, M.A.; Deshmukh, R.R. Honey Classification using Hyperspectral Imaging and Machine Learning. In Proceedings of the 2021 Smart Technologies, Communication and Robotics (STCR), Tamil Nadu, India, 9–10 October 2021. [Google Scholar] [CrossRef]
- Shenming, Q.; Xiang, L.; Zhihua, G. A new hyperspectral image classification method based on spatial-spectral features. Sci. Rep. 2022, 12, 1541. [Google Scholar] [CrossRef] [PubMed]
- Nezami, S.; Khoramshahi, E.; Nevalainen, O.; Pölönen, I.; Honkavaara, E. Tree species classification of drone hyperspectral and RGB imagery with deep learning convolutional neural networks. Remote Sens. 2020, 12, 1070. [Google Scholar] [CrossRef]
- Fricker, G.A.; Ventura, J.D.; Wolf, J.A.; North, M.P.; Davis, F.W.; Franklin, J. A convolutional neural network classifier identifies tree species in mixed-conifer forest from hyperspectral imagery. Remote Sens. 2019, 11, 2326. [Google Scholar] [CrossRef]
- Khokhar, A.A.; Yadav, S.; Khan, F.; Gindi, S. Plant Species Classification with CNN. Int. J. Emerg. Technol. Innov. Res. 2021, 8, 236–240. [Google Scholar]
- Kiran, S.G.; Ashwini, S.G. Deep Learning for Plant Species Classification. Int. J. Emerg. Technol. Innov. Res. 2020, 7, 99–105. [Google Scholar]
- Sobha, P.G.M.; Thomas, P.A. Deep Learning for Plant Species Classification Survey. In Proceedings of the 2019 International Conference on Advances in Computing, Communication and Control (ICAC3), Mumbai, India, 20–21 December 2019. [Google Scholar] [CrossRef]
- Kiss, N.; Czuni, L. Mushroom image classification with CNNs: A case-study of different learning strategies. In Proceedings of the 2021 12th International Symposium on Image and Signal Processing and Analysis (ISPA), Zagreb, Croatia, 13–15 September 2021; pp. 165–170. [Google Scholar] [CrossRef]
- Liu, Q. The Development of Image Classification Algorithms Based on CNNs. Highlights Sci. Eng. Technol. 2023, 34, 275–280. [Google Scholar] [CrossRef]
- Tropea, M.; Fedele, G. Classifiers Comparison for Convolutional Neural Networks (CNNs) in Image Classification. In Proceedings of the 2019 IEEE/ACM 23rd International Symposium on Distributed Simulation and Real Time Applications (DS-RT), Cosenza, Italy, 7–9 October 2019. [Google Scholar] [CrossRef]
- Kan, M.; Aliev, R.; Rudenko, A.; Drobyshev, N.; Petrashen, N.; Kondrateva, E.; Sharaev, M.; Bernstein, A.; Burnaev, E. Interpretation of 3D CNNs for Brain MRI Data Classification. In Proceedings of the Communications in Computer and Information Science, Virtual Event, 27–30 September 2021; Volume 1357 CCIS, pp. 229–241, ISBN 9783030712136. [Google Scholar] [CrossRef]
- Zeng, F.; Peng, W.; Kang, G.; Feng, Z.; Yue, X. Spectral Data Classification by One-Dimensional Convolutional Neural Networks. In Proceedings of the 2021 IEEE International Performance, Computing, and Communications Conference (IPCCC), Austin, TX, USA, 28–30 October 2021. [Google Scholar]
- Chen, J.; Pisonero, J.; Chen, S.; Wang, X.; Fan, Q.; Duan, Y. Convolutional neural network as a novel classification approach for laser-induced breakdown spectroscopy applications in lithological recognition. Spectrochim. Acta-Part B At. Spectrosc. 2020, 166, 105801. [Google Scholar] [CrossRef]
- Gao, H.; Yang, Y.; Li, C.; Zhou, H.; Qu, X. Joint alternate small convolution and feature reuse for hyperspectral image classification. Can. Hist. Rev. 2018, 7, 349. [Google Scholar] [CrossRef]
- Yuan, S.; Song, G.; Huang, G.; Wang, Q. Reshaping Hyperspectral Data into a Two-Dimensional Image for a CNN Model to Classify Plant Species from Reflectance. Remote Sens. 2022, 14, 3972. [Google Scholar] [CrossRef]
- Shahid, S.M.; Ko, S.; Kwon, S. Performance Comparison of 1D and 2D Convolutional Neural Networks for Real-Time Classification of Time Series Sensor Data. In Proceedings of the 2022 International Conference on Information Networking (ICOIN), Jeju-si, Republic of Korea, 12–15 January 2022; pp. 507–511. [Google Scholar] [CrossRef]
- Zhu, Y.; Brettin, T.; Xia, F.; Partin, A.; Shukla, M.; Yoo, H.; Evrard, Y.A.; Doroshow, J.H.; Stevens, R.L. Converting tabular data into images for deep learning with convolutional neural networks. Sci. Rep. 2021, 11, 11325. [Google Scholar] [CrossRef] [PubMed]
- Olson, J.M. Cartography. In International Encyclopedia of the Social & Behavioral Sciences; Smelser, N.J., Baltes, P.B., Eds.; Pergamon: Oxford, UK, 2001; pp. 1495–1501. ISBN 978-0-08-043076-8. [Google Scholar] [CrossRef]
- Opiyo, J. How Does the Size of Input Affect the Performance of a Convolutional NEURAL Network (CNN)? Available online: https://www.quora.com/How-does-the-size-of-input-affect-the-performance-of-a-convolutional-neural-network-CNN (accessed on 24 August 2023).
- Aravind, R. How to Pick the Optimal Image Size for Training Convolution Neural Network? Available online: https://medium.com/analytics-vidhya/how-to-pick-the-optimal-image-size-for-training-convolution-neural-network-65702b880f05 (accessed on 24 August 2023).
- Jacquemound, S.; Bidel, L.; Francois, C.; Pavan, G. ANGERS Leaf Optical Properties Database. 2003. Available online: https://ecosis.org/package/angers-leaf-optical-properties-database--2003- (accessed on 5 February 2021).
- Kattenborn, T.; Schiefer, F.; Schmidtlein, S. Leaf Reflectance Plant Functional Gradient IFGG/KIT. Available online: https://ecosis.org/package/leaf-reflectance-plant-functional-gradient-ifgg-kit (accessed on 14 May 2022).
- Kothari, S.; Beauchamp-Rioux, R.; Blanchard, F.; Crofts, A.L.; Girard, A.; Guilbeault-Mayers, X.; Hacker, P.W.; Pardo, U.; Schweiger, A.K.; Demers-Thibeault, S.; et al. CABO 2018–2019 Leaf-Level Spectra v2. Available online: https://ecosis.org/package/cabo-2018-2019-leaf-level-spectra-v2 (accessed on 14 May 2022).
- Wang, Z. Fresh Leaf Spectra to Estimate LMA over NEON Domains in Eastern United States. Available online: https://ecosis.org/package/fresh-leaf-spectra-to-estimate-lma-over-neon-domains-in-eastern-united-states (accessed on 5 February 2021).
- Serbin, S.P.; Townsend, P.A. NASA FFT Project Leaf Reflectance Morphology and Biochemistry for Northern Temperate Forests. Available online: https://ecosis.org/package/nasa-fft-project-leaf-reflectance-morphology-and-biochemistry-for-northern-temperate-forests (accessed on 14 May 2022).
- Burnett, A.C.; Serbin, S.P.; Davidson, K.J.; Ely, K.S.; Rogers, A. Hyperspectral Leaf Reflectance, Biochemistry, and Physiology of Droughted and Watered Crops. Available online: https://ecosis.org/package/hyperspectral-leaf-reflectance--biochemistry--and-physiology-of-droughted-and-watered-crops (accessed on 14 May 2022).
- Beamlab You Can Probably Use Deep Learning Even If Your Data Isn’t that Big. Available online: https://beamandrew.github.io/deeplearning/2017/06/04/deep_learning_works.html (accessed on 30 June 2021).
- Gavrikov, P. Visualkeras. Available online: https://github.com/paulgavrikov/visualkeras (accessed on 25 November 2023).
- Buitinck, L.; Louppe, G.; Blondel, M.; Pedregosa, F.; Mueller, A.; Grisel, O.; Niculae, V.; Prettenhofer, P.; Gramfort, A.; Grobler, J.; et al. {API} design for machine learning software: Experiences from the scikit-learn project. In Proceedings of the ECML PKDD Workshop: Languages for Data Mining and Machine Learning, Prague, Czech Republic, 23–27 September 2013; pp. 108–122. [Google Scholar]
- Song, G.; Wang, Q. Including leaf traits improves a deep neural network model for predicting photosynthetic capacity from reflectance. Remote Sens. 2021, 13, 4467. [Google Scholar] [CrossRef]
- Wang, Z.; Skidmore, A.K.; Wang, T.; Darvishzadeh, R.; Hearne, J. Applicability of the PROSPECT model for estimating protein and cellulose + lignin in fresh leaves Remote Sensing of Environment Applicability of the PROSPECT model for estimating protein and cellulose + lignin in fresh leaves. Remote Sens. Environ. 2015, 168, 205–218. [Google Scholar] [CrossRef]
- Castro-Esau, K.L.; Sánchez-Azofeifa, G.A.; Caelli, T. Discrimination of lianas and trees with leaf-level hyperspectral data. Remote Sens. Environ. 2004, 90, 353–372. [Google Scholar] [CrossRef]
Latin Name | Symbol | Code | Group | Source | Total | |||||
---|---|---|---|---|---|---|---|---|---|---|
CA | KA | AN | UP | NE | NC | |||||
Betula papyrifera Marshall | BEPA | 0 | Tree | 31 | 51 | 10 | 92 | |||
Quercus rubra L. | QURU | 1 | Tree | 21 | 90 | 84 | 195 | |||
Raphanus sativus L. | RASA | 2 | Herb | 195 | 195 | |||||
Acer saccharum Marshall | ACSA | 3 | Tree | 81 | 18 | 99 | ||||
Betula populifolia Marshall | BEPO | 4 | Tree, Shrub | 104 | 104 | |||||
Phalaris arundinacea Linnaeus | PHAR | 5 | Herb | 75 | 22 | 97 | ||||
Andropogon gerardii Vitman | ANGE | 6 | Herb | 89 | 2 | 91 | ||||
Acer rubrum L. | ACRU | 7 | Tree | 57 | 96 | 42 | 195 | |||
Tsuga canadensis (L.) Carrière | TSCA | 8 | Tree | 112 | 112 | |||||
Acacia smallii Isely | ACSM | 9 | Tree, Shrub | 119 | 119 | |||||
Capsicum annuum L. | CAAN | 10 | Herb | 195 | 195 | |||||
Populus ×canadensis Moench | POCA | 11 | Tree | 195 | 195 | |||||
Helianthus annuus L. | HEAN | 12 | Herb | 172 | 172 | |||||
Populus tremuloides Michx. | POTR | 13 | Tree | 120 | 120 | |||||
Quercus alba L. | QUAL | 14 | Tree | 10 | 78 | 60 | 148 | |||
Acer pseudoplatanus L. | ACPS | 15 | Tree | 181 | 181 | |||||
Pinus strobus L. | PIST | 16 | Tree | 12 | 82 | 94 | ||||
Cucurbita pepo L. | CUPE | 17 | Vine, Herb | 195 | 195 | |||||
Abies balsamea (L.) Mill. | ABBA | 18 | Tree | 19 | 133 | 152 | ||||
Setaria italica (L.) P. Beauv. | SEIT | 19 | Herb | 96 | 96 | |||||
Sorghum bicolor (L.) Moench | SOBI | 20 | Herb | 151 | 151 | |||||
Fagus grandifolia Ehrh. | FAGR | 21 | Tree | 47 | 39 | 18 | 104 | |||
Total | 577 | 22 | 181 | 1199 | 461 | 662 | 3102 |
CNN1A | CNN1B | CNN2A | CNN2B | CNN2C | CNN3A | CNN3B | CNN3C | CNN3D | ||
---|---|---|---|---|---|---|---|---|---|---|
Input | L × W × 3 | |||||||||
Rescaling | L × W × 3 | |||||||||
Conv1 | Kernel | 3 × 3 | ||||||||
Stride | 1 × 1 | |||||||||
Output | L × W × 32 | |||||||||
Pooling | Output | - | L1 × W1 × 32 | - | L1 × W1 × 32 | - | L1 × W1 × 32 | |||
Conv2 | Kernel | - | - | 3 × 3 | ||||||
Stride | - | - | 1 × 1 | |||||||
Output | - | - | L × W × 32 | L1 × W1 × 32 | L × W × 32 | L1 × W1 × 32 | ||||
Pooling | Output | - | - | - | - | L2 × W2 × 32 | - | - | L2 × W2 × 32 | |
Conv3 | Kernel | - | - | - | - | - | 3 × 3 | |||
Stride | - | - | - | - | - | 1 × 1 | ||||
Output | - | - | - | - | - | L × W × 32 | L1 × W1 × 32 | L2 × W2 × 32 | ||
Pooling | Output | - | - | - | - | - | - | - | - | L3 × W3 × 64 |
Dropout | Rate (%) | 0.2 | ||||||||
Flatten | Last Output Calculated Value | |||||||||
Dense | Flatten × 128 | |||||||||
Output | 1 × 22 |
Models | Image Shape | Accuracy (%) | Precision (%) | F1-Score |
---|---|---|---|---|
cnn1A | k | 89.77 ± 4.69 | 91.03 ± 3.94 | 0.90 ± 0.05 |
cnn1B | j | 91.15 ± 1.50 | 92.07 ± 1.21 | 0.91 ± 0.02 |
cnn2A | l | 93.82 ± 1.32 | 94.45 ± 1.09 | 0.94 ± 0.01 |
cnn2B | l | 93.63 ± 1.16 | 94.23 ± 0.98 | 0.94 ± 0.01 |
cnn2C | k | 92.55 ± 1.60 | 93.23 ± 1.33 | 0.93 ± 0.02 |
cnn3A | l | 93.84 ± 1.65 | 94.41 ± 1.30 | 0.94 ± 0.02 |
cnn3B | l | 93.95 ± 1.30 | 94.55 ± 1.03 | 0.94 ± 0.01 |
cnn3C | l | 92.67 ± 1.61 | 93.45 ± 1.17 | 0.93 ± 0.02 |
cnn3D | l | 92.76 ± 1.36 | 93.31 ± 1.15 | 0.93 ± 0.01 |
Image Shape | Accuracy (%) | Precision (%) | F1-Score |
---|---|---|---|
a | 79.27 ± 4.75 | 81.09 ± 4.94 | 0.79 ± 0.05 |
b | 88.17 ± 2.53 | 89.62 ± 2.02 | 0.88 ± 0.03 |
c | 87.69 ± 2.32 | 89.15 ± 1.86 | 0.88 ± 0.02 |
d | 89.78 ± 2.17 | 90.88 ± 1.69 | 0.90 ± 0.02 |
e | 90.51 ± 2.19 | 91.44 ± 1.87 | 0.90 ± 0.02 |
f | 91.30 ± 2.00 | 92.43 ± 1.47 | 0.91 ± 0.02 |
g | 91.40 ± 2.06 | 92.64 ± 1.58 | 0.91 ± 0.02 |
h | 93.07 ± 1.87 | 94.00 ± 1.26 | 0.93 ± 0.02 |
i | 92.62 ± 1.52 | 93.38 ± 1.21 | 0.93 ± 0.02 |
j | 92.82 ± 2.19 | 93.61 ± 1.72 | 0.93 ± 0.02 |
k | 92.82 ± 1.64 | 93.58 ± 1.31 | 0.93 ± 0.02 |
l | 93.95 ± 1.30 | 94.55 ± 1.03 | 0.94 ± 0.01 |
m | 91.08 ± 2.94 | 92.09 ± 2.58 | 0.91 ± 0.03 |
n | 88.54 ± 3.09 | 89.92 ± 2.68 | 0.88 ± 0.03 |
o | 77.99 ± 6.72 | 79.82 ± 7.51 | 0.77 ± 0.07 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yuan, S.; Song, G.; Gong, Q.; Wang, Q.; Wang, J.; Chen, J. Reshaping Leaf-Level Reflectance Data for Plant Species Discrimination: Exploring Image Shape’s Impact on Deep Learning Results. Remote Sens. 2023, 15, 5628. https://doi.org/10.3390/rs15245628
Yuan S, Song G, Gong Q, Wang Q, Wang J, Chen J. Reshaping Leaf-Level Reflectance Data for Plant Species Discrimination: Exploring Image Shape’s Impact on Deep Learning Results. Remote Sensing. 2023; 15(24):5628. https://doi.org/10.3390/rs15245628
Chicago/Turabian StyleYuan, Shaoxiong, Guangman Song, Qinghua Gong, Quan Wang, Jun Wang, and Jun Chen. 2023. "Reshaping Leaf-Level Reflectance Data for Plant Species Discrimination: Exploring Image Shape’s Impact on Deep Learning Results" Remote Sensing 15, no. 24: 5628. https://doi.org/10.3390/rs15245628
APA StyleYuan, S., Song, G., Gong, Q., Wang, Q., Wang, J., & Chen, J. (2023). Reshaping Leaf-Level Reflectance Data for Plant Species Discrimination: Exploring Image Shape’s Impact on Deep Learning Results. Remote Sensing, 15(24), 5628. https://doi.org/10.3390/rs15245628