Application of Attention-Enhanced 1D-CNN Algorithm in Hyperspectral Image and Spectral Fusion Detection of Moisture Content in Orah Mandarin (Citrus reticulata Blanco)
Abstract
:1. Introduction
- Performing a linear dimensionality reduction of the hyperspectral image and obtaining its color and textural features.
- Proposing a 1D-CNN model with fusion information of spectral and image information for the detection of moisture content in Orah mandarin and utilizing the powerful information extraction function of the convolutional neural network to reduce information loss.
- Attention mechanisms are integrated into the 1D-CNN model to improve performance, and the addition of the self-attention layer makes the model more focused on the key information of extracted features, which improves accuracy and generalization.
- In particular, visualizing the spatial distribution of moisture content in the sample and dynamically monitoring changing trends at different times, providing a new perspective for the measurement and analysis of the moisture content of Orah mandarin.
2. Related Work
3. Materials and Methods
3.1. Sample Preparation
3.2. Acquisition of Hyperspectral Images
3.3. Data Acquisition Method
3.3.1. Acquisition and Correction of Hyperspectral Images
3.3.2. The Extraction of the Spectrum, Color, and Texture of Orah mandarin
3.3.3. Data Fusion of Spectral and Image Data
3.4. Reference Value Measurement of Moisture Content
3.5. Data Preprocessing
3.6. Model Establishment Methods of 1D-CNN
3.7. Model Evaluation Metrics
4. Results and Discussion
4.1. Spectral Analysis and Preprocessing of Orah Mandarin
4.2. Reference Measurements
4.3. Establishment and Analysis of Quantitative Model
4.3.1. Establishment and Analysis of PLSR and Machine Learning Model
4.3.2. Establishment and Analysis of Attention-Enhanced 1D-CNN Model
4.4. Visual Distribution of Moisture Content in Orah Mandarin
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
ML | machine learning |
DL | deep learning |
1D-CNN | one-dimensional convolutional neural network |
ECAM | squeeze-and-excitation attention module |
ECAM | efficient channel attention module |
CBAM | convolutional block attention module |
PLSR | partial least squares regression |
SVR | support vector regression |
ANN | artificial neural network |
DT | decision Tree |
RF | random forest |
LightGBM | light gradient boosting machine |
ROI | region of interest |
References
- Huang, Q.; Liu, J.; Hu, C.; Wang, N.; Zhang, L.; Mo, X.; Li, G.; Liao, H.; Huang, H.; Ji, S. Integrative analyses of transcriptome and carotenoids profiling revealed molecular insight into variations in fruits color of Citrus reticulata Blanco induced by transplantation. Genomics 2022, 114, 110291. [Google Scholar] [CrossRef] [PubMed]
- He, Y.; Li, W.; Zhu, P.; Wang, M.; Qiu, J.; Sun, H.; Zhang, R.; Liu, P.; Ling, L.; Fu, X. Comparison between the vegetative and fruit characteristics of ‘Orah’ (Citrus reticulata Blanco) mandarin under different climatic conditions. Sci. Hortic. 2022, 300, 111064. [Google Scholar] [CrossRef]
- Maurizzi, E.; Bigi, F.; Volpelli, L.A.; Pulvirenti, A. Improving the post-harvest quality of fruits during storage through edible packaging based on guar gum and hydroxypropyl methylcellulose. Food Packag. Shelf Life 2023, 40, 101178. [Google Scholar] [CrossRef]
- Dhami, K.; Asrey, R.; Awasthi, O.; Bhowmik, A. Pre and postharvest treatments of methyl jasmonate: Maintain quality and shelf-life of Kinnow mandarin fruit during cold storage. South Afr. J. Bot. 2022, 151, 808–815. [Google Scholar] [CrossRef]
- Yuan, Y.; Chen, X. Vegetable and fruit freshness detection based on deep features and principal component analysis. Curr. Res. Food Sci. 2024, 8, 100656. [Google Scholar] [CrossRef]
- Wang, H.; Zhou, Z.; Yang, Z.; Cao, Y.; Zhang, C. Constraints on the high-quality development of Chinese fruit industry. China Fruits 2023, 65, 1–9. [Google Scholar] [CrossRef]
- Chen, H.; Wu, Q.; Xu, B.; Wang, B.; Jiang, H. Exploration and practice of citrus information system. China Fruits 2024, 66, 122–127. [Google Scholar] [CrossRef]
- Mei, M.; Li, J. An overview on optical non-destructive detection of bruises in fruit: Technology, method, application, challenge and trend. Comput. Electron. Agric. 2023, 213, 108195. [Google Scholar] [CrossRef]
- Wang, M.; Xu, Y.; Yang, Y.; Mu, B.; Nikitina, M.A.; Xiao, X. Vis/NIR optical biosensors applications for fruit monitoring. Biosens. Bioelectron. X 2022, 11, 100197. [Google Scholar] [CrossRef]
- Min, D.; Zhao, J.; Bodner, G.; Ali, M.; Li, F.; Zhang, X.; Rewald, B. Early decay detection in fruit by hyperspectral imaging–Principles and application potential. Food Control 2023, 152, 109830. [Google Scholar] [CrossRef]
- Wakchaure, G.; Nikam, S.B.; Barge, K.R.; Kumar, S.; Meena, K.K.; Nagalkar, V.J.; Choudhari, J.; Kad, V.; Reddy, K.S. Maturity stages detection prototype device for classifying custard apple (Annona squamosa L) fruit using image processing approach. Smart Agric. Technol. 2024, 7, 100394. [Google Scholar] [CrossRef]
- Rodríguez-Ortega, A.; Aleixos, N.; Blasco, J.; Albert, F.; Munera, S. Study of light penetration depth of a Vis-NIR hyperspectral imaging system for the assessment of fruit quality. A case study in persimmon fruit. J. Food Eng. 2023, 358, 111673. [Google Scholar] [CrossRef]
- Sanchez, P.D.C.; Hashim, N.; Shamsudin, R.; Nor, M.Z.M. Applications of imaging and spectroscopy techniques for non-destructive quality evaluation of potatoes and sweet potatoes: A review. Trends Food Sci. Technol. 2020, 96, 208–221. [Google Scholar] [CrossRef]
- Jiang, H.; Jiang, X.; Ru, Y.; Chen, Q.; Li, X.; Xu, L.; Zhou, H.; Shi, M. Rapid and non-destructive detection of natural mildew degree of postharvest Camellia oleifera fruit based on hyperspectral imaging. Infrared Phys. Technol. 2022, 123, 104169. [Google Scholar] [CrossRef]
- Xuan, G.; Gao, C.; Shao, Y. Spectral and image analysis of hyperspectral data for internal and external quality assessment of peach fruit. Spectrochim. Acta Part A Mol. Biomol. Spectrosc. 2022, 272, 121016. [Google Scholar] [CrossRef] [PubMed]
- Cevoli, C.; Iaccheri, E.; Fabbri, A.; Ragni, L. Data fusion of FT-NIR spectroscopy and Vis/NIR hyperspectral imaging to predict quality parameters of yellow flesh “Jintao” kiwifruit. Biosyst. Eng. 2024, 237, 157–169. [Google Scholar] [CrossRef]
- Rajkumar, P.; Wang, N.; EImasry, G.; Raghavan, G.; Gariepy, Y. Studies on banana fruit quality and maturity stages using hyperspectral imaging. J. Food Eng. 2012, 108, 194–200. [Google Scholar] [CrossRef]
- Gao, S.; Wang, Q. Non-destructive testing of red globe grape sugar content and moisture content based on visible/near infrared spectroscopy transmission technology. Chin. Opt. 2021, 14, 566–577. [Google Scholar]
- Liu, Y.; Wang, S. Research on Non-Destructive Testing of Navel Orange Shelf-Life Imaging Based on Hyperspectral Image and Spectrum Fusion. Spectrosc. Spectr. Anal. 2022, 42, 1792–1797. [Google Scholar]
- Jiang, F.; Shen, Z.; Yang, L.; Chen, Y.; Sun, B. Hyperspectral imaging for quality detection of Agaricus bisporus under vibration stress. Food Sci. 2022, 43, 353–360. [Google Scholar] [CrossRef]
- Wieme, J.; Mollazade, K.; Malounas, I.; Zude-Sasse, M.; Zhao, M.; Gowen, A.; Argyropoulos, D.; Fountas, S.; Van Beek, J. Application of hyperspectral imaging systems and artificial intelligence for quality assessment of fruit, vegetables and mushrooms: A review. Biosyst. Eng. 2022, 222, 156–176. [Google Scholar] [CrossRef]
- Tian, S.; Wang, S.; Xu, H. Early detection of freezing damage in oranges by online Vis/NIR transmission coupled with diameter correction method and deep 1D-CNN. Comput. Electron. Agric. 2022, 193, 106638. [Google Scholar] [CrossRef]
- Olisah, C.C.; Trewhella, B.; Li, B.; Smith, M.L.; Winstone, B.; Whitfield, E.C.; Fernández, F.F.; Duncalfe, H. Convolutional neural network ensemble learning for hyperspectral imaging-based blackberry fruit ripeness detection in uncontrolled farm environment. Eng. Appl. Artif. Intell. 2024, 132, 107945. [Google Scholar] [CrossRef]
- Chen, D.; Zhang, H.; Lin, L.; Zhang, Z.; Zeng, J.; Chen, L.; Chen, X. Auto-encoder design based on the 1D-VD-CNN model for the detection of honeysuckle from unknown origin. J. Pharm. Biomed. Anal. 2023, 234, 115572. [Google Scholar] [CrossRef] [PubMed]
- Teng, Y.; Chen, Y.; Chen, X.; Zuo, S.; Li, X.; Pan, Z.; Shao, K.; Du, J.; Li, Z. Revealing the adulteration of sesame oil products by portable Raman spectrometer and 1D CNN vector regression: A comparative study with chemometrics and colorimetry. Food Chem. 2024, 436, 137694. [Google Scholar] [CrossRef]
- Li, J.; Luo, W.; Wang, Z.; Fan, S. Early detection of decay on apples using hyperspectral reflectance imaging combining both principal component analysis and improved watershed segmentation method. Postharvest Biol. Technol. 2019, 149, 235–246. [Google Scholar] [CrossRef]
- Jiang, X.; Tian, J.; Huang, H.; Hu, X.; Han, L.; Huang, D.; Luo, H. Nondestructive visualization and quantification of total acid and reducing sugar contents in fermented grains by combining spectral and color data through hyperspectral imaging. Food Chem. 2022, 386, 132779. [Google Scholar] [CrossRef]
- Wang, Z.; Fan, S.; An, T.; Zhang, C.; Chen, L.; Huang, W. Detection of Insect-Damaged maize seeds using hyperspectral imaging and hybrid 1D-CNN-BiLSTM models. Infrared Phys. Technol. 2024, 137, 105208. [Google Scholar] [CrossRef]
- Qiao, M.; Cui, T.; Xia, G.; Xu, Y.; Li, Y.; Fan, C.; Han, S.; Dong, J. Integration of spectral and image features of hyperspectral imaging for quantitative determination of protein and starch contents in maize kernels. Comput. Electron. Agric. 2024, 218, 108718. [Google Scholar] [CrossRef]
- GB/T 5009.3-2016; National Standard for Food Safety Determination of Moisture in Foods. National Health and Family Planning Commission of the People’s Republic of China: Beijing, China, 2016.
- Hu, R.; Yu, Y.; Ni, M.; Yu, J.; Zhou, J. Identification of Lotus Seed Flour Adulteration Based on Near-Infrared Spectroscopy Combined with Deep Belief Network. Food Sci. 2020, 41, 298–303. [Google Scholar] [CrossRef]
- Luo, N.; Xu, D.; Xing, B.; Yang, X.; Sun, C. Principles and applications of convolutional neural network for spectral analysis in food quality evaluation: A review. J. Food Compos. Anal. 2024, 128, 105996. [Google Scholar] [CrossRef]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; Hu, Q. ECA-Net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 11534–11542. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.-Y.; Kweon, I.S. CBAM: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Liu, Y.; Li, M.; Hu, J.; Xu, Z.; Cui, H. Detection of Citrus Granulation Based on Near-Infrared Hyperspectral Data. Spectrosc. Spectr. Anal. 2022, 42, 1366–1371. [Google Scholar] [CrossRef]
- Zhang, Y.; Lee, W.S.; Li, M.; Zheng, L.; Ritenour, M.A. Non-destructive recognition and classification of citrus fruit blemishes based on ant colony optimized spectral information. Postharvest Biol. Technol. 2018, 143, 119–128. [Google Scholar] [CrossRef]
- Andries, J.P.; Vander Heyden, Y. Calibration set reduction by the selection of a subset containing the best fitting samples showing optimally predictive ability. Talanta 2024, 266, 124943. [Google Scholar] [CrossRef]
Quality Index | Sample Set | Samples | Maximum Value | Minimum Value | Average Value | Standard Deviation |
---|---|---|---|---|---|---|
Moisture content (%) | All samples | 195 | 91.82% | 75.07% | 84.06% | 3.67% |
Correction set | 146 | 91.82% | 75.07% | 84.07% | 3.61% | |
Prediction set | 49 | 89.27% | 77.08% | 84.01% | 3.40% |
Modeling Data | Input Number | Model | Correction Set | Prediction Set | ||
---|---|---|---|---|---|---|
RC | RMSEC | RP | RMSEP | |||
Spectral data | 520 | PLSR | 0.8572 | 0.0185 | 0.8535 | 0.0180 |
SVR | 0.8747 | 0.0174 | 0.8547 | 0.0179 | ||
ANN | 0.6757 | 0.0248 | 0.5696 | 0.0205 | ||
DT | 0.8757 | 0.0173 | 0.8410 | 0.0187 | ||
RF | 0.9446 | 0.0122 | 0.8227 | 0.0196 | ||
LightGBM | 0.8992 | 0.0162 | 0.8535 | 0.0182 | ||
Fused data | 543 | PLSR | 0.8781 | 0.0169 | 0.8718 | 0.0178 |
SVR | 0.8681 | 0.0175 | 0.8752 | 0.0198 | ||
ANN | 0.7712 | 0.0227 | 0.7557 | 0.0246 | ||
DT | 0.8713 | 0.0173 | 0.8629 | 0.0184 | ||
RF | 0.9267 | 0.0137 | 0.8770 | 0.0188 | ||
LightGBM | 0.8987 | 0.0174 | 0.7524 | 0.0252 |
Modeling Data | Input Number | Model | Correction Set | Prediction Set | ||
---|---|---|---|---|---|---|
RC | RMSEC | RP | RMSEP | |||
Spectral data | 520 | SEAM + 1D-CNN | 0.8573 | 0.0201 | 0.8614 | 0.0170 |
ECAM + 1D-CNN | 0.8545 | 0.0232 | 0.8831 | 0.0192 | ||
CBAM + 1D-CNN | 0.8255 | 0.0241 | 0.8324 | 0.0216 | ||
Fused data | 543 | SEAM + 1D-CNN | 0.8403 | 0.0202 | 0.9005 | 0.0150 |
ECAM + 1D-CNN | 0.8369 | 0.0212 | 0.8847 | 0.0165 | ||
CBAM + 1D-CNN | 0.8609 | 0.0192 | 0.9172 | 0.0149 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, W.; Wang, Y.; Yu, Y.; Liu, J. Application of Attention-Enhanced 1D-CNN Algorithm in Hyperspectral Image and Spectral Fusion Detection of Moisture Content in Orah Mandarin (Citrus reticulata Blanco). Information 2024, 15, 408. https://doi.org/10.3390/info15070408
Li W, Wang Y, Yu Y, Liu J. Application of Attention-Enhanced 1D-CNN Algorithm in Hyperspectral Image and Spectral Fusion Detection of Moisture Content in Orah Mandarin (Citrus reticulata Blanco). Information. 2024; 15(7):408. https://doi.org/10.3390/info15070408
Chicago/Turabian StyleLi, Weiqi, Yifan Wang, Yue Yu, and Jie Liu. 2024. "Application of Attention-Enhanced 1D-CNN Algorithm in Hyperspectral Image and Spectral Fusion Detection of Moisture Content in Orah Mandarin (Citrus reticulata Blanco)" Information 15, no. 7: 408. https://doi.org/10.3390/info15070408
APA StyleLi, W., Wang, Y., Yu, Y., & Liu, J. (2024). Application of Attention-Enhanced 1D-CNN Algorithm in Hyperspectral Image and Spectral Fusion Detection of Moisture Content in Orah Mandarin (Citrus reticulata Blanco). Information, 15(7), 408. https://doi.org/10.3390/info15070408