Next Article in Journal
Integrative Transcriptomic and Metabolomic Analysis Reveals Quinoa Leaf Response Mechanisms to Different Phosphorus Concentrations During Filling Stage
Previous Article in Journal
Effects of Foliar Application of Magnesium Fertilizer on Photosynthesis and Growth in Grapes
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Detection of Rice Leaf Folder in Paddy Fields Based on Unmanned Aerial Vehicle-Based Hyperspectral Images

1
Institute of Agricultural Economics and Information, Guangdong Academy of Agricultural Sciences, Guangzhou 510640, China
2
Key Laboratory of Urban Agriculture in South China, Ministry of Agriculture and Rural Affairs, Guangzhou 510640, China
3
Institute of Agricultural Information Technology, Henan Academy of Agricultural Sciences, Zhengzhou 450002, China
4
Key Laboratory of Huang-Huai-Hai Smart Agricultural Technology, Ministry of Agriculture and Rural Affairs, Zhengzhou 450002, China
*
Authors to whom correspondence should be addressed.
Agronomy 2024, 14(11), 2660; https://doi.org/10.3390/agronomy14112660
Submission received: 11 October 2024 / Revised: 6 November 2024 / Accepted: 8 November 2024 / Published: 12 November 2024
(This article belongs to the Section Pest and Disease Management)

Abstract

:
Pest infestations significantly impact rice production and threaten food security. Remote sensing offers a vital tool for the non-destructive, rapid detection of rice pests. Existing studies often focus on laboratory conditions at the leaf level, limiting their applicability for precise pesticide application. Therefore, this study aimed to develop a method for detecting rice pests (rice leaf folders) in paddy fields based on unmanned aerial vehicle (UAV) hyperspectral data. Firstly, a UAV imaging system collected hyperspectral images of rice plants in both the jointing and heading stages. A total of 222 field plots for investigating rice leaf folders was established during these two periods. Secondly, 23 vegetation indices were calculated as candidates for identifying rice pests. Then, hyperspectral data and field investigation data from the jointing stage were used to construct a machine learning (extreme gradient boosting, XGBoost) algorithm for detecting rice pests. The results showed that the XGBoost model exhibited the best performance when eight vegetation indices were considered as the selected input features for model construction: the Red-edge Normalized Difference Vegetation Index (red-edge NDVI), Structure Insensitive Pigment Index (SIPI), Enhanced Vegetation Index (EVI), Atmospherically Resistant Vegetation Index (ARVI), Soil-Adjusted Vegetation Index (SAVI), Red-edge Chlorophyll Index (CIred-edge), Pigment-Specific Simple Ratio680 (PSSR680), and Carotenoid Reflectance Index700 (CPI700). The training and testing accuracies reached 87.46% and 86%, respectively. Furthermore, the heading stage application confirmed the model’s feasibility. Thus, the XGBoost model with input features of eight vegetation indices provides an effective and reliable method for detecting rice leaf folders, supporting real-time, precise pesticide use in rice cultivation.

1. Introduction

Pests and diseases are main threats to crop growth, causing negative impacts on yield and quality, and can even lead to widespread crop failures [1,2,3]. Data provided by Food and Agriculture Organization (FAO) reveal approximately 30% of the grain output is lost globally every year due to pests and diseases [4]. In the context of global climate changes, the frequent occurrence of abnormal weather conditions and disasters has exacerbated the scope and intensity of agricultural diseases, thus making the prevention and monitoring of crop pests and diseases increasingly arduous [5,6]. Therefore, the establishment of a system that enables the timely and accurate monitoring and early detection of crop pests and diseases has become one of the most fundamental and critical challenges confronting agricultural production and management.
Rice is one of the main staple foods worldwide, contributing an estimated 35 percent of global production [7]. Despite this significant contribution, pest infestations (such as those of rice leaf folders and plant-hoppers) have emerged as one of the principal impediments to rice yield enhancement [8,9]. Statistical records indicate that China sustains substantial annual crop losses, ranging from 4 to 5 million metric tons, due to the ravages of rice pests [1]. Pest infestations significantly reduce the water content, chlorophyll concentration, leaf area, and dry weight in rice plants, leading to a decrease in the rice yield. Severe pest damage can result in the death of infected leaves, leading ultimately to defoliation and, consequently, the demise of the entire rice plant, resulting in a substantial decline in yield. To implement timely and accurate pesticide application, minimize pest control costs, benefit the environment, and reduce yield losses, it is essential to accurately quantify, detect, and identify the specific locations and extent of damage inflicted by pests on rice plants [10].
Field investigations and estimations by professional plant protection personnel remain the primary methods of collecting and monitoring information about pest infestations, given the fact that such investigations are labor-intensive, time-consuming, and prone to be subjective. More importantly, it is difficult to meet the demands for the real-time and dynamic monitoring of rice pests across extensive areas; thus, investigations suffer from limitations of the geographical coverage and spatial–temporal scale. With the rapid development of remote sensing technology with high spatial and spectral resolutions, remote sensing has been recognized as an effective method for monitoring and identifying rice pest infestations in large regions [11,12,13,14]. Rice stress caused by pests mainly manifests as physiological changes inside the crop and morphological changes outside the crop, such as the water content, pigment content, protein content, and leaf shape (i.e., shriveling, blotches, decay, etc.), being changed [15]. Such symptoms can cause clear changes in the spectral reflectance of rice [16,17]. In this way, identifying spectral changes caused by pests has been considered an effective method for monitoring rice pest infestations through remote sensing [18,19,20,21].
Remote sensing technology is usually influenced by the platform type (satellite, aerial, or leaf level), sensors (multispectral or hyperspectral remote sensing). For the remote sensing platforms, compared with satellites (e.g., Landsat, MODIS, SPOT) or leaf-level instruments (e.g., handheld spectrometer), unmanned aerial vehicles (UAVs) have advantages in several aspects: UAVs can cover larger regions than leaf-level handheld devices, and collect data at lower altitudes with a higher spatial resolution and higher monitoring frequencies than satellite platforms [22]. Currently, UAV platforms are primarily equipped with multispectral and hyperspectral sensors for identifying and monitoring crop pests [23,24,25]. In comparison to multispectral sensors, hyperspectral sensors obtain continuous data for both the spatial and spectral domains to create a three-dimensional dataset (hypercube), and this is significant for complex infection scenarios involved in identifying and monitoring crop pests [26]. In addition, hyperspectral sensors can provide hundreds of contiguous bands covering the visible and near-infrared (NIR) spectra, which are highly sensitive to subtle changes in crop leaf pigments (such as chlorophyll, carotenoids, and anthocyanins) and canopy shape [27]. However, owing to their high cost and complex operational processes, hyperspectral sensors are primarily applied in indoor laboratory conditions or near-ground canopy platforms for research on estimating phenotypic traits such as biomass, chlorophyll content, and nitrogen content, as well as identifying the occurrence of crop diseases [28,29,30]. Currently, research on using hyperspectral sensors installed on UAV platforms to monitor and identify crop pests remains relatively limited.
In practical applications, hyperspectral remote sensing data with a high number of spectral channels can lead to dimensionality challenges [31]. Therefore, key spectrum selection is often used as a preprocessing method before experiments. By mathematically combining key spectral bands, vegetation indices enable the capture of the important physiological and morphological characteristics of crops, which are widely used in identifying and monitoring crop pests. Studies indicate that the vegetation indices have advantages in monitoring crop pests that affect pigments, such as chlorophyll, carotenoids, and anthocyanins [32,33,34,35,36,37]. Currently, there is no common vegetation index to detect crop pests, since each pest influences spectral features in a specific way [12]. Therefore, finding or developing effective spectral indices is crucial for the remote sensing monitoring of crop pests.
Rice is vulnerable to pest infestations throughout its entire growth stages. Therefore, it is essential to develop methods for monitoring and detecting the spatial distribution of pest stress to achieve precise and targeted pesticide application, minimize pest control expenses, and reduce yield losses. Although there are numerous studies on identifying rice pests, most of them are conducted in laboratory conditions at the leaf-level scale [38,39]. This is due to the challenge of obtaining specific low-altitude and stable UAV remote sensing data for pest analysis in practical research. Conducting quantitative comparative experiments on rice pest infestation is quite difficult, and the lack of such data is the primary reason for the low accuracy of rice pest detection.
Infestation by rice pests, such as rice leaf folders (cnaphalocrocis medinalis), is one of the most common causes of yield losses in tropical areas, especially in Asia. Given the importance and challenge of detecting and monitoring the spatial distribution of pests, this study focuses on utilizing UAV-based hyperspectral data to detect rice leaf folders in the paddy field, thereby potentially providing guidance for precise pesticide application. The three primary objectives were as follows: (1) to evaluate the difference in spectral reference between healthy rice and rice infested by pests, (2) to develop an effective method for detecting the occurrence areas of rice pest infestations using a deep learning method with UAV-based hyperspectral data, and (3) to map the spatial distribution of rice pest infestations in the paddy field.

2. Study Area and Data Sources

2.1. Experimental Site

This study was conducted in an 8600 m2 paddy field, located in Huangpu District, Guangdong Province, China (23.33° N, 113.48° E), a typical site of rice cultivation in the Pearl River Delta region (Figure 1). Huangpu District is located in a subtropical monsoon climate region, with an annual average temperature of 21 °C, approximately 1694 mm of annual precipitation, and an average sunshine duration of 1906 h. The planting system in this area mostly implements a rice–rice double cropping system. The early rice generally grows from early April to mid-July, and the late rice usually grows from the end of August to the end of October. During the rice growth period, high-temperature and high-humidity conditions are conducive to the growth and reproduction of the rice leaf folder.
To develop the method for detecting the areas infested with rice leaf folder, two types of treatments (experimental area and control area) with different pest management strategies were designed in the paddy fields. Normal rice cultivation was carried out in the control area (Figure 1 (A)), with regular spraying of pesticides and appropriate implementation of water and fertilizer management. For the experimental area (Figure 1 (B)), pesticides of the rice leaf folder were not sprayed for pest control, and the rest of the rice cultivation process remained consistent with that of the control area. The treatment aimed to increase the areas of occurrence of rice leaf folder in the experimental area for developing a method for detecting rice pests.

2.2. Collection of Hyperspectral Images

A UAV platform (DJI M300, DJI Technology Co., Ltd., Shenzhen, China) was equipped with an airborne hyperspectral system (300TC, from Yiruisi Remote Sensing Technology Co., Ltd., Beijing, China) for spatial spectral data acquisition. The spectral data were geo-referenced by the UAV’s onboard GPS. The spectral range was 400~1000 nm, and the spectral resolution was 3~4 nm. The flight altitude of the UAV was 100 m, and the corresponding spatial resolution of hyperspectral image was approximately 4.5 cm. The lateral and heading spatial overlap ratio was set as 70%. Canopy spectral data of the paddy field were collected on 28th September 2023 (rice in jointing stages) and 25th October 2023 (rice in heading stages), and all data were collected at around noon with a clear and cloudless sky.

2.3. Data Preprocessing

Hyperspectral images of study area were initially exported from MegaCube software (Version 2.14.0, from Yiruisi Remote Sensing Technology Co., Ltd., Beijing, China), and subjected to geometric correction based on radiation calibration file and digital elevation data. Subsequently, ArcGIS software (Version 10.0, from Environmental Systems Research Institute, Inc., Redlands, CA, USA) was utilized for geographical registration of hyperspectral images based on orthographic images and ground control points of study area. ENVI software (Version 5.6, from Environmental Systems Research Institute, Inc., Redlands, CA, USA) was employed for hyperspectral image stitching, and then the hypercube data were generated based on MegaCube software (Version 2.14.0). Ultimately, the hypercube data converted into reflectance data based on the ground whiteboard reflectance data and reflectance gray cloth data to obtain hyperspectral images that can be used for spectral analysis.

2.4. Field Investigation of Rice Pests

A total of 222 observation plots were deployed in the control area and experimental area for field investigation of rice leaf folders (Figure 1). Each plot contained 5 clusters of rice. A cluster of rice contained 15 to 20 plants, and a rice plant contained 4 to 5 leaves. During the sample survey, we referred to the national standard of pest control of China (GB/T 15793-2011) to classify rice pest infestations into two categories: healthy (leaf-roll rate < 5%) and infested (leaf-roll rate ≥ 5%). The specific execution process is as follows. Firstly, observation plots were randomly arranged within the paddy field, and marked with ropes to indicate their location on-site. Then, for each plot, the number of leaf rolls in all the rice plants was counted separately by three agronomy surveyors. The average of the counts from each surveyor was calculated to determine the number of leaf rolls for each plot. Finally, the types of rice pest infestation for each plot were determined by the leaf-roll rate: if the leaf-roll rate was less than 5%, the plot was recorded as healthy, otherwise, it is determined as infested. Field investigation of rice pests was conducted in two periods. Table 1 provides the numbers of healthy plots and infested plots in control area and experimental area. In total, 140 healthy plots and 82 infested plots were observed on the 28th of September, and 80 healthy plots and 142 infested plots were observed on the 25th of October.
The leaf-roll rate can be calculated as follows:
R = l p × 100 % ,
where R is leaf-roll rate; l is the number of leaf rolls in each plot; p is the number of the rice leaves in each plot.

3. Method

This study aimed to construct an effective method for identifying and detecting the spatial distribution map of rice leaf folders. Figure 2 illustrates the flowchart of this study. Firstly, hyperspectral data were acquired using a UAV platform with an airborne hyperspectral system. Then, a rice pest identification model was developed based on the data from the 28th of September. A series of vegetation indices were calculated for identifying rice pests. For each field plot (healthy or infested), the values of the vegetation indices were calculated as plot-level averages based on the hyperspectral images. The recursive feature elimination (RFE) method was utilized to select sensitive features (vegetation indices) for rice pest identification. Extreme gradient boosting (XGBoost) was utilized to map the spatial distribution of rice pests. Further, with the model for identifying rice pests and the corresponding hyperspectral data, the spatial distribution map of the rice pests from the 25th of October was generated. Finally, the model performance was evaluated by the field investigation of the rice pests from the 25th of October.

3.1. Model Construction

3.1.1. Vegetation Index Calculation

Previous studies have shown that pest infestations can lead to a decrease in the nitrogen and chlorophyll content of rice leaves [18]. Vegetation indices have advantages in monitoring crop pests that affect pigments, such as chlorophyll, carotenoids, and anthocyanins. Therefore, 23 vegetation indices were calculated as candidates for identifying rice pests (Table 2). All the vegetation indices were normalized between 0 and 1.

3.1.2. Feature Selection

Among the 23 candidate vegetation indices, the method of RFE was used to select the sensitive features for identifying rice pests. The principle of RFE is to iteratively eliminate candidate features and compare their contributions to the model based on the performance of each variable (candidate vegetation index) during the classification process. Specifically, during the initial training stages of the RFE process, the overall accuracy (OA) improves as the number of features increases, until the primary features are selected, after which the accuracy begins to stabilize. We established a rule that when the improvement in OA is less than 1%, we consider the current feature combination to be stable. At this point, adding more features to achieve minor improvements in accuracy is unnecessary and could lead to computational redundancy. In this study, the XGBoost model (see Section 3.1.3 for detailed description) was utilized as the basic evaluator of RFE. After evaluating each variable, those that contributed relatively more to the classification results were selected as input features for model construction.

3.1.3. Identification of Rice Pests

The rice pest detection model was constructed based on the XGBoost model, which is a representative ensemble algorithm of machine learning, offering accurate and fast training via a gradient boosting framework [40]. XGBoost, a decision tree-based model, initializes an initial predicted value at the root of the tree. With this initial prediction, the residuals (the differences between the predicted and the actual values) of the dataset are computed, and these residuals serve as the basis for further calculations. The trees in the XGBoost model are built sequentially, with each new tree aiming to correct the mistakes made by its predecessor. The core principle of the XGBoost model is to accumulate multiple base learners and to form a more robust classifier through iterations [41,42]. Moreover, the XGBoost model is not affected by the highly correlated features, effectively addressing the issue of feature multicollinearity [43]. It has implemented parallelization design in computing feature gain, depth-first tree pruning, and algorithmic enhancements to ensure both a high accuracy and faster training times [44]. Previous studies have indicated that, following suitable hyperparameter optimization, the XGBoost model outperforms other machine learning algorithms, such as the random forest model and support vector machines [45,46,47].
Table 3 shows the main optimized hyperparameters of the XGBoost model for the optimization algorithm. In this study, these hyperparameters were optimized using the grid search method combined with 10-fold cross-validation. Based on the XGBoost model with input features of vegetation indices and optimized hyperparameters, all pixels in the hyperspectral images were classified into two types: healthy and infested, and the spatial distribution map of the rice pests (infested pixels) was generated.

3.1.4. Model Performance Assessment

For rice pest identification by the XGBoost model, 70% of the data (training set) was applied to train the classification model, and the remaining data (test set) were used to assess the model performance. In the process of training the model, after optimizing the hyperparameters for each model, a 10-fold cross-validation was utilized to assess its training performance. The generality of the classification model was further assessed using the test set. The model performance was evaluated through three metrics: the user accuracy (UA), producer accuracy (PA), and OA. These three metrics are calculated as follows:
OA = T P + T N T P + F P + F N + T N ,
UA = T P T P + F P ,
PA = T P T P + F N ,

3.2. Model Application and Assessment

The model of rice pest identification was constructed using both hyperspectral images (input features of vegetation indices) and field data on rice pests from the 28th of September. On the basis of the constructed model, the hyperspectral data on the 25th of October were input into the model for rice pest identification, and the spatial distribution map of rice pests from the 25th of October was generated. In the application process, input features of the vegetation indices and optimized hyperparameters in the XGBoost model were consistent with the inputs from the 28th of September. To evaluate the feasibility and performance of the model in the application of rice pest identification, the total field data on rice pests from the 25th of October were used for accuracy verification. Three metrics were used to verify the classification of rice pests: UA, PA, and OA.

4. Results

4.1. Difference in Spectral Characteristics Between Healthy Rice and Infested Rice

The canopy spectra of healthy rice and infested rice are shown in Figure 3. Different types of spectra were collected from the UAV hyperspectral images corresponding to the field investigation plots. The reflection peak (near 550 nm) in the visible light range is related to the content of chlorophyll, while the absorption peak around 480 nm is due to the absorption of pigments. The spectral reflectance sharply rises around 680–750 nm, which is called the red-edge position. The red-edge position shifted left with a decrease in the chlorophyll content under stresses, and the spectral reflectance between the healthy rice and infested rice in this study showed a similar trend. Moreover, it was also found that the spectral reflectance of the infested rice in the range of the near-infrared spectral region (around 760–800 nm) was generally lower than that of the healthy rice.

4.2. Model Performance

The features that contributed relatively more to the performance improvement were selected using the method of RFE. Figure 4 depicts the feature importance based on the OA values. The results showed that the XGBoost model exhibited the best prediction performance when eight features (red-edge NDVI, SIPI, EVI, ARVI, SAVI, CIred-edge, PSSR680, and CPI700) were selected as the input features, achieving an OA of 86.32%. When the improvement in the OA is less than 1%, we consider the current feature combination to be stable, and adding more features to achieve minor improvements in accuracy is unnecessary. Figure 5 shows the outputs of the 10-fold cross-validation in the model training process using the above eight features. The average training accuracy was 87.46%, with a variance of 0.25%, indicating that the model had a high level of accuracy and stability. Furthermore, the performance of this model was further evaluated based on the test set. The assessment results indicated that the model performed well for prediction, achieving UA = 84.24%, PA = 89.68%, and OA = 86% on the test set. Therefore, the XGBoost model with input features of eight vegetation indices and optimized hyperparameters can be used as an effective method for identifying the spatial distribution of rice leaf folders.

4.3. Distribution of Rice Pests

In this study, all the pixels in the hyperspectral image were classified into two types, healthy and infested, and the spatial distribution map of the rice pests (infested pixels) from the 28th of September was generated (Figure 6A). The constructed model was employed to generate a spatial distribution map of the rice pests on the 25th of October (Figure 6B). To evaluate the performance in the application of rice pest identification, the field data of the rice pest types (healthy plots or infested) from the 25th of October was used for accuracy verification. The accuracy results of the rice pests were as follows: UA = 83.10%, PA = 88.06%, and OA = 81.98%. It was inferred that misclassification occurred more frequently in the prediction of rice pests, and there were relatively few missing samples in the classification. Misclassification was more dominant in the following scenario: the pixels of healthy rice surrounded by infested rice were easily misclassified as pixels of infested rice.
From the distribution map of the rice pests, the areas of paddy infested by rice leaf folder showed an expanding trend. The area proportions of the rice pest infestations increased from 64.28% on the 28th of September to 90.53% on the 25th of October. The spatial distribution of rice pests can be used as a basis for targeted management basis for precise pesticide utilization, in order to further minimize control expenses, benefit the environment, and reduce yield losses.

5. Discussion

The timely, reliable, and efficient monitoring of crops over large areas is crucial for effective crop protection assessment and management [18]. In recent decades, remote sensing techniques have demonstrated their ability to identify and monitor the range, location, and damage degree of crop diseases and pests [11,28,48,49]. The effectiveness of agricultural remote sensing is influenced by various factors such as the platform, sensor, resolution. Research indicates that UAV remote sensing can effectively detect plot-level crop pests and support precision pesticide application [22,50]. Hyperspectral sensors can sensitively detect changes in leaf pigments, critical for pest detection and monitoring [27]. This has advanced the theoretical foundation for regional-scale remote sensing detection of crop pests. In this study, UAV-based hyperspectral data were used to monitor rice pests, and the results indicated that this approach was effective in identifying and detecting the spatial distribution of rice pests in paddy fields.
Hyperspectral data capture a continuous dataset in both the spectral and spatial domains, which may exhibit spectral correlation and spatial correlation. The spectral characteristics of plant chemistry and physiology in visible and near-infrared channels are identifiable. The visible region of 400–760 nm is primarily influenced by pigments, while the leaf water content and structure dominate the near-infrared region of 760–1100 nm. Thus, it is necessary to select sensitive spectral bands or construct vegetation indices for further analysis. Previous studies have indicated that spectral bands near 410, 445, 470, 490, 570, 625, 665, 720, and 757 nm are sensitive bands for detecting rice leaf folders [11,19,20]. While hyperspectral reflectance from the rice canopy can reveal different growth statuses, external factors such as the surrounding environment and soil background can affect the spectral characteristics related to rice pigments [51]. Vegetation indices are commonly used to mitigate these effects, but no single index is universally suitable for pest identification, as each pest influences spectral characteristics uniquely.
In this study, 23 vegetation indices were evaluated as potential inputs for a classification model aimed at identifying crop pests. During feature selection, we established a criterion that when the improvement in model accuracy is less than 1%, the current feature combination is considered stable. At this point, adding more features to achieve minor accuracy improvements is unnecessary and can lead to computational redundancy. This approach ensures that the model remains efficient while balancing complexity and performance. It also facilitates a more interpretable model, as it reduces the number of features to a manageable and meaningful subset, enhancing the practical applicability of the model in real-world scenarios. After thorough testing and validation, the model performed optimally when using eight specific vegetation indices: red-edge NDVI, SIPI, EVI, ARVI, SAVI, CIred-edge, PSSR680, and CPI700. These vegetation indices cover the visible light (400–760 nm) and near-infrared (760–800 nm) spectral ranges, which align with the sensitive bands for crop pest monitoring identified by previous researchers [11,19,20]. These indices are particularly effective in capturing spectral features related to crop pests: the red-edge NDVI and CIred-edge are sensitive to changes in the chlorophyll content and leaf structure, which are often altered by pest infestations. The SIPI is effective in detecting changes in the carotenoid and chlorophyll ratios, which can indicate stress from pests. The EVI and SAVI account for soil background effects, improving the accuracy of pest detection in varying environments. The ARVI reduces atmospheric interference, enhancing the reliability of spectral measurements. The PSSR680 and CPI700 are sensitive to changes in the photosynthetic activity and pigment content, which are indicative of pest damage. The combination of these vegetation indices not only improved the model’s ability to distinguish between healthy and infested crops but also provided a robust framework for monitoring pest infestations over large areas. Moreover, these indices might aid in the channel design of pest monitoring sensors.
This study developed a method for identifying and detecting the spatial distribution of rice leaf folders. The results of the model performance evaluation and application demonstrated that this method was feasible for the detection of rice leaf folders using a remote sensing method. This method provides timely and precise data on the location of pests, ensuring that pest management strategies are both highly effective and resource-efficient. By enabling accurate interventions, our method significantly reduces the risk of infestation spread while minimizing unnecessary pesticide use. This not only enhances the sustainability of rice production but also supports environmental conservation efforts by reducing chemical inputs. The method outlined in this study can be adapted and applied to other crops and pest species, offering a versatile tool for remote sensing applications in agricultural management. By facilitating timely and targeted pest management, this approach contributes to sustainable agriculture and food security.
Nonetheless, certain limitations should be considered in this study. In terms of the experimental design, although two types of treatments (experimental area and control area) with different pest management strategies were set up in the paddy fields, they did not affect the experimental results. This is because the rice leaf folder moves within the paddy fields, which makes it difficult to distinguish the pest infestations between the experiment area and control area. Therefore, the sample methods for field investigation and image processing were applied across the entire study area, treating it as a single unit for detecting the spatial distribution of rice leaf folders. In addition, while the spatial information of rice pests can be used as a precise guide to implement timely and accurate pesticide applications, it is equally important to obtain the damage degree and different stages of rice pest infestation. In further studies, the precise monitoring of the leaf-roll rate and different infestation stages of rice pests needs to be conducted. Moreover, the rice species and growing stage would affect the model performance in the detection of rice leaf folders, which should be considered during model construction. Additionally, identifying rice pests before severe crop damage occurs also faces challenges. Therefore, it is necessary to develop early monitoring and early warning methods for rice pests, in order to achieve the timely, rapid, and accurate monitoring and early warning of rice leaf rollers before the pest characteristics can be visually observed.

6. Conclusions

This study developed a method for identifying and detecting the spatial distribution of rice leaf folders using a deep learning model with UAV-based hyperspectral data. The principal conclusions are as follows. The XGBoost model exhibited the best performance when eight vegetation indices (the red-edge NDVI, SIPI, EVI, ARVI, SAVI, CIred-edge, PSSR680, and CPI700) were selected as the input features for the model construction. To evaluate the performance in the application of rice pest identification, the model was applied to generate a spatial distribution map of the rice pests in the next rice growing stage. The accuracy results demonstrated that the developed method had significant potential in detecting rice leaf folders. Thus, the XGBoost model with the selected input features of eight vegetation indices can be used as an effective and reliable method for identifying and detecting the spatial distribution of rice leaf folders. In conclusion, the integration of UAV-based hyperspectral data with the XGBoost model provides a robust and efficient method for detecting rice leaf folders. This approach has the potential to significantly provide real-time and accurate guidance for pesticide use in rice cultivation, ensuring timely pest management interventions and sustainable rice production.

Author Contributions

Conceptualization, S.F.; data curation, S.F., S.J., X.H., L.Z. and Y.G.; funding acquisition, L.W. and C.Z.; investigation, S.J., X.H., L.Z. and Y.G.; methodology, S.F., S.J., X.H., L.Z. and Y.G.; supervision, L.W. and C.Z.; validation, X.H., L.Z. and Y.G.; visualization, S.J.; writing—original draft, S.F.; writing—review and editing, S.J. and X.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Open Project of Key Laboratory of Huang-Huai-Hai Smart Agricultural Technology, Ministry of Agriculture and Rural Affairs (202407), Special Funds for Science and Technology Talent Introduction of Guangdong Academy of Agricultural Sciences (R2022YJ-YB1002), National Natural Science Foundation of China (42401221), Rural Revitalization Strategy Special Project of the Guangdong Provincial Department of Agriculture and Rural Affairs in 2024 (143).

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available since they also form part of an ongoing study.

Acknowledgments

The authors are grateful to the handling editor and the anonymous reviewers for helping improve the manuscript.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Carvajal, Y.M.; Cardwell, K.; Nelson, A.; Garrett, K.A.; Giovani, B.; Saunders, D.G.O.; Kamoun, S.; Legg, J.P.; Verdier, V.; Lessel, J. A global surveillance system for crop diseases. Science 2019, 364, 1237–1239. [Google Scholar] [CrossRef] [PubMed]
  2. Huang, Y.; Li, Z.; Zhu, H. The use of UAV remote sensing technology to identify crop stress: A review. J. Geo-Inf. Sci. 2019, 21, 512–523. [Google Scholar]
  3. Tian, L.; Xue, B.; Wang, Z.; Li, D.; Cheng, T. Spectroscopic detection of rice leaf blast infection from asymptomatic to mild stages with integrated machine learning and feature selection. Remote Sens. Environ. 2021, 257, 112350. [Google Scholar] [CrossRef]
  4. Becker-Ritt, A.B.; Carlini, C.R. Fungitoxic and insecticidal plant polypeptides. Pept. Sci. 2012, 98, 367–384. [Google Scholar] [CrossRef] [PubMed]
  5. Mondal, K.; Mani, V. Emergence of bacterial panicle blight caused by Burkholderia glumae in North India. Plant Dis. 2015, 15, 150311115126000. [Google Scholar] [CrossRef]
  6. Karthikeyan, I. A review of remote sensing applications in agriculture for food security: Crop growth and yield, irrigation, and crop losses. J. Hydrol. 2020, 586, 124905. [Google Scholar] [CrossRef]
  7. Government Network of Ministry of Agriculture and Rural Affairs of the People’s Republic of China. The Seed of a Better Life. China 2021. Available online: http://www.zys.moa.gov.cn/mhsh/202104/t20210420_6366193.htm (accessed on 20 April 2021). (In Chinese)
  8. Huang, S.; Wang, L.; Liu, L.; Fu, Q.; Zhu, D. Nonchemical pest control in China rice: A review. Agron. Sustain. Dev. 2014, 34, 275–291. [Google Scholar] [CrossRef]
  9. Kim, Y.; Roh, J.H.; Kim, H. Early Forecasting of Rice Blast Disease Using Long Short-Term Memory Recurrent Neural Networks. Sustainability 2018, 10, 34. [Google Scholar] [CrossRef]
  10. Zheng, Q.; Huang, W.; Xia, Q.; Dong, Y.; Ye, H.; Jiang, H.; Chen, S.; Huang, S. Remote Sensing Monitoring of Rice Diseases and Pests from Different Data Sources: A Review. Agronomy 2023, 13, 1851. [Google Scholar] [CrossRef]
  11. Huang, J.; Liao, H.; Zhu, Y.; Sun, J.; Sun, Q.; Liu, X. Hyperspectral detection of rice damaged by rice leaf folder (Cnaphalocrocis medinalis). Comput. Electron. Agric. 2012, 82, 100–107. [Google Scholar] [CrossRef]
  12. Mahlein, A.K.; Rumpf, T.; Welke, P.; Dehne, H.W.; Plumer, L.; Steiner, U.; Oerke, E.C. Development of spectral indices for detecting and identifying plant diseases. Remote Sens. Environ. 2013, 128, 21–30. [Google Scholar] [CrossRef]
  13. Mahlein, A.K. Plant disease detection by imaging sensors–parallels and specific demands for precision agriculture and plant phenotyping. Plant Dis. 2016, 100, 241–251. [Google Scholar] [CrossRef] [PubMed]
  14. Xu, T.; Wang, F.; Yi, Q.; Xie, L.; Yao, X. A bibliometric and visualized analysis of research progress and trends in rice remote sensing over the past 42 years (1980–2021). Remote Sens. 2022, 14, 3607. [Google Scholar] [CrossRef]
  15. Shi, Y.; Huang, W.; González-Moreno, P.; Luke, B.; Dong, Y.; Zheng, Q.; Ma, H.; Liu, L. Wavelet-based rust spectral feature set (WRSFS): A novel spectral feature set based on continuous wavelet transformation for tracking progressive host–pathogen interaction of yellow rust on wheat. Remote Sens. 2018, 10, 525. [Google Scholar] [CrossRef]
  16. Zheng, Q.; Huang, W.; Cui, X.; Shi, Y.; Liu, L. New spectral index for detecting wheat yellow rust using Sentinel-2 multispectral imagery. Sensors 2018, 18, 868. [Google Scholar] [CrossRef]
  17. Cao, Y.; Xu, H.; Song, J.; Yang, Y.; Hu, X.; Wiyao, K.T.; Zhai, Z. Applying spectral fractal dimension index to predict the SPAD value of rice leaves under bacterial blight disease stress. Plant Methods 2022, 18, 67. [Google Scholar] [CrossRef]
  18. Zhang, J.; Huang, Y.; Pu, R.; Gonzalez-Moreno, P.; Yuan, L.; Wu, K.; Huang, W. Monitoring plant diseases and pests through remote sensing technology: A review. Comput. Electron. Agric. 2019, 165, 104943. [Google Scholar] [CrossRef]
  19. Liu, T.; Shi, T.; Zhang, H.; Wu, C. Detection of Rise Damage by Leaf Folder (Cnaphalocrocis medinalis) Using Unmanned Aerial Vehicle Based Hyperspectral Data. Sustainability 2020, 12, 9343. [Google Scholar] [CrossRef]
  20. Yang, C.; Cheng, C.; Chen, R. Changes in spectral characteristics of rice canopy infested with brown planthopper and leaffolder. Crop Sci. 2007, 47, 329–335. [Google Scholar] [CrossRef]
  21. Das, P.K.; Laxman, B.; Kameswara Rao, S.V.C.; Seshasai, M.V.R.; Dadhwal, V.K. Monitoring of bacterial leaf blight in rice using ground-based hyperspectral and LISS IV satellite data in Kurnool, Andhra Pradesh, India. Int. J. Pest. Manag. 2015, 61, 359–368. [Google Scholar] [CrossRef]
  22. Arnal Barbedo, J.G. A Review on the Use of Unmanned Aerial Vehicles and Imaging Sensors for Monitoring and Assessing Plant Stresses. Drones 2019, 3, 40. [Google Scholar] [CrossRef]
  23. Dammer, K.H.; Garz, A.; Hobart, M.; Schirrmann, M. Combined UAV-and Tractor-based Stripe Rust Monitoring in Winter Wheat under Field Conditions. Agron. J. 2022, 114, 651–661. [Google Scholar] [CrossRef]
  24. Yang, C.; Yan, G.; Du, S.; Li, X. Application Review of Unmanned Aerial Vehicle Remote Sensing Technology Wheat Production. Henan Sci. 2021, 39, 1598–1602. [Google Scholar]
  25. Yang, G.; He, Y.; Feng, X.; Li, X.; Zhang, J.; Yu, Z. Methods and New Research Progress of Remote Sensing Monitoring of Crop Disease and Pest Stress Using Unmanned Aerial Vehicle. Smart Agric. 2022, 4, 1–16. [Google Scholar]
  26. Thomas, S.; Wahabzada, M.; Kuska, M.T.; Rascher, U.; Mahlein, A.K. Observation of plant-pathogen interaction by simultaneous hyperspectral imaging reflection and transmission measurements. Funct. Plant Biol. 2017, 44, 23–34. [Google Scholar] [CrossRef]
  27. Zhao, G.; Zhang, Y.; Lan, Y.; Deng, J.; Zhang, Q.; Zhang, Z.; Li, Z.; Liu, L.; Huang, X.; Ma, J. Application Progress of UAV-LARS in Identification of Crop Diseases and Pests. Agronomy 2023, 13, 2232. [Google Scholar] [CrossRef]
  28. Deng, J.; Zhang, X.; Yang, Z.; Zhou, C.; Wang, R.; Zhang, K.; Lv, X.; Yang, L.; Wang, Z.; Li, P.; et al. Pixel-level regression for UAV hyperspectral images: Deep learning-based quantitative inverse of wheat stripe rust disease index. Comput. Electron. Agric. 2023, 215, 108434. [Google Scholar] [CrossRef]
  29. Bohnenkamp, D.; Behmann, J.; Mahlein, A. In-Field Detection of Yellow Rust in Wheat on the Ground Canopy and UAV Scale. Remote Sens. 2019, 11, 2495. [Google Scholar] [CrossRef]
  30. Yang, L.; Wang, Z.; Wu, C. Research on Large-scale Monitoring of Spider Mite Infestation in Xinjiang Cotton Field Based on Multi-source Data. Spectrosc. Spectr. Anal. 2021, 41, 3949–3956. [Google Scholar]
  31. Bruce, L.M.; Koger, C.H.; Li, J. Dimensionality reduction of hyperspectral data using discrete wavelet transform feature extraction. IEEE Trans. Geosci. Remote Sens. 2002, 40, 2331–2338. [Google Scholar] [CrossRef]
  32. Coops, N.C.; Wulder, M.A.; White, J.C. Integrating remotely sensed and ancillary data sources to characterize a mountain pine beetle infestation. Remote Sens. Environ. 2006, 105, 83–97. [Google Scholar] [CrossRef]
  33. Huang, W.; Lamb, D.W.; Niu, Z.; Zhang, Y.; Liu, L.; Wang, J. Identification of yellow rust in wheat using in-situ spectral reflectance measurements and airborne hyperspectral imaging. Precis. Agric. 2007, 8, 187–197. [Google Scholar] [CrossRef]
  34. Naidu, R.A.; Perry, E.M.; Pierce, F.J.; Mekuria, T. The potential of spectral reflectance technique for the detection of Grapevine leafroll-associated virus-3 in two red-berried wine grape cultivars. Comput. Electron. Agric. 2009, 66, 38–45. [Google Scholar] [CrossRef]
  35. Adelabu, S.; Mutanga, O.; Adam, E. Evaluating the impact of red-edge band from Rapideye image for classifying insect defoliation levels. ISPRS J. Photogramm. Remote Sens. 2014, 95, 34–41. [Google Scholar] [CrossRef]
  36. Hillnhütter, C.; Mahlein, A.; Sikora, R.A.; Oerke, E. Remote sensing to detect plant stress induced by Heterodera schachtii and Rhizoctonia solani in sugar beet fields. Field Crop Res. 2011, 122, 70–77. [Google Scholar] [CrossRef]
  37. Yuan, L.; Huang, Y.; Loraamm, R.W.; Nie, C.; Wang, J.; Zhang, J. Spectral analysis of winter wheat leaves for detection and differentiation of diseases and insects. Field Crop Res. 2014, 156, 199–207. [Google Scholar] [CrossRef]
  38. Harshadkumar, B.P.; Jitesh, P.S.; Vipul, K.D. Detection and Classification of Rice Plant Diseases. Intell. Decis. Technol. 2018, 11, 357–373. [Google Scholar]
  39. Rahman, C.R.; Arko, P.S.; Ali, M.E.; Khan, M.A.I.; Apon, S.H.; Nowrin, F.; Abu, W. Identification and recognition of rice diseases and pests using convolutional neural networks. Biosyst. Eng. 2020, 194, 112–120. [Google Scholar] [CrossRef]
  40. Chen, T.; Guestrin, C. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), San Francisco, CA, USA, 13–17 August 2016; pp. 785–794. [Google Scholar]
  41. Song, Y.; Zhang, C.; Jin, X.; Zhao, X.; Huang, W.; Sun, X.; Yang, Z.; Wang, S. Spatial prediction of PM2.5 concentration using hyper-parameter optimization XGBoost model in China. Environ. Technol. Inno. 2023, 32, 103272. [Google Scholar] [CrossRef]
  42. Joshi, A.; Pradhan, B.; Chakraborty, S.; Behera, M. Winter wheat yield prediction in the conterminous United States using solar-induced chlorophyll fluorescence data and XGBoost and random forest algorithm. Ecol. Inform. 2023, 77, 102194. [Google Scholar] [CrossRef]
  43. Parsa, A.B.; Movahedi, A.; Taghipour, H.; Derrible, S.; Mohammadian, A.K. Toward safer highways, application of XGBoost and SHAP for real-time accident detection and feature analysis. Accident. Anal. Prev. 2020, 136, 105405. [Google Scholar] [CrossRef] [PubMed]
  44. Li, Y.; Zeng, H.; Zhang, M.; Wu, B.; Zhao, Y.; Yao, X.; Cheng, T.; Qin, X.; Wu, F. A county-level soybean yield prediction framework coupled with XGBoost and multidimensional feature engineering. Int. J. Appl. Earth Obs. Geoinf. 2023, 118, 103269. [Google Scholar] [CrossRef]
  45. Cao, Z.; Ma, R.; Melack, J.; Duan, H.; Liu, M.; Kutser, T.; Xue, K.; Shen, M.; Qi, T.; Yuan, H. Landsat observations of chlorophyll-a variations in Lake Taihu from 1984 to 2019. Int. J. Appl. Earth Obs. 2022, 106, 102642. [Google Scholar] [CrossRef]
  46. Huang, X.; Zhang, Q.; Hu, L.; Zhu, T.; Zhou, X.; Zhang, Y.; Xu, Z.; Ju, W. Monitoring Damage Caused by Pantana phyllostachysae Chao to Moso Bamboo Forests Using Sentinel-1 and Sentinel-2 Images. Remote Sens. 2022, 14, 5012. [Google Scholar] [CrossRef]
  47. Xie, B.; Ding, J.; Ge, X.; Li, X.; Han, L.; Wang, Z. Estimation of soil organic carbon content in the Ebinur Lake wetland, Xinjiang, China, based on multisource remote sensing data and ensemble learning algorithms. Sensors 2022, 22, 2685. [Google Scholar] [CrossRef] [PubMed]
  48. Grisham, M.P.; Johnson, R.M.; Zimba, P.V. Detecting sugarcane yellow leaf virus infection in asymptomatic leaves with hyperspectral remote sensing and associated leaf pigment changes. J. Virol. Method 2010, 167, 140–145. [Google Scholar] [CrossRef]
  49. Prabhakar, M.; Prasad, Y.G.; Thirupathi, M.; Sreedevi, G.; Dharajothi, B.; Venkateswarlu, B. Use of ground based hyperspectral remote sensing for detection of stress in cotton caused by leafhopper (Hemiptera: Cicadellidae). Comput. Electron. Agric. 2011, 79, 189–198. [Google Scholar] [CrossRef]
  50. Shakoor, N.; Lee, S.; Mockler, T.C. High throughput phenotyping to accelerate crop breeding and monitoring of diseases in the field. Curr. Opin. Plant. Biol. 2017, 38, 184–192. [Google Scholar] [CrossRef]
  51. Darvishzadeh, R.A.K.; Skidmore, A.; Atzberger, C.; Van Wieren, S.E. Estimation of vegetation LAI from hyperspectral reflectance data: Effects of soil type and plant architecture. Int. J. Appl. Earth Obs. Geoinf. 2008, 10, 358–373. [Google Scholar] [CrossRef]
Figure 1. Experimental site of the paddy field.
Figure 1. Experimental site of the paddy field.
Agronomy 14 02660 g001
Figure 2. Technical flow of this study.
Figure 2. Technical flow of this study.
Agronomy 14 02660 g002
Figure 3. Spectral reflectance of healthy rice and infested rice: (A) 28th Sept.; (B) 25th Oct.
Figure 3. Spectral reflectance of healthy rice and infested rice: (A) 28th Sept.; (B) 25th Oct.
Agronomy 14 02660 g003
Figure 4. The contribution ranking of different features.
Figure 4. The contribution ranking of different features.
Agronomy 14 02660 g004
Figure 5. The accuracy for 10-fold cross-validation in the model training process.
Figure 5. The accuracy for 10-fold cross-validation in the model training process.
Agronomy 14 02660 g005
Figure 6. Spatial distribution of rice pest infestations: (A) 28th Sept.; (B) 25th Oct.
Figure 6. Spatial distribution of rice pest infestations: (A) 28th Sept.; (B) 25th Oct.
Agronomy 14 02660 g006
Table 1. Field investigation of rice pests for control area and experimental area in two periods.
Table 1. Field investigation of rice pests for control area and experimental area in two periods.
28th Sept.25th Oct.
Number of Healthy PlotsNumber of Infested PlotsNumber of Healthy PlotsNumber of Infested Plots
Control area (A)91277345
Experimental area (B)4955797
Total number1408280142
Table 2. Candidate vegetation indices used for identifying rice pests.
Table 2. Candidate vegetation indices used for identifying rice pests.
Vegetation IndicesAbbreviationFormula
Normalized Difference Vegetation IndexNDVI s 810 s 670 s 810 + s 670 ,(2)
Red Normalized Difference Vegetation IndexRed NDVI s 760 s 670 s 760 + s 670 ,(3)
Red-edge Normalized Difference Vegetation IndexRed-edge NDVI s 760 s 730 s 760 + s 730 ,(4)
Green Ratio Vegetation IndexGRVI s 800 s 550 (5)
Structure Insensitive Pigment IndexSIPI s 800 s 445 s 800 + s 680 ,(6)
Enhanced Vegetation IndexEVI 2.5 × s 800 s 680 s 800 + s 680 7.5 × s 450 + 1 ,(7)
Renormalized Difference Vegetation IndexRDVI s 800 s 680 s 800 + s 680 ,(8)
Atmospherically Resistant Vegetation IndexARVI s 800 2 × s 680 + s 450 s 800 + 2 × s 680 + s 450 ,(9)
Visible Atmospherically Resistant IndexVARI s 550 s 680 s 550 + s 680 s 450 ,(10)
Soil-Adjusted Vegetation IndexSAVI 1.5 × s 800 s 680 s 800 + 2 × s 680 + 0.5 ,(11)
Modified Triangular Vegetation Index (Improved)MTVII 1.5 × 1.2 × s 740 s 550 2.5 × s 680 s 550 2 × s 740 + 1 2 6 × s 740 5 × s 680 0.5 ,(12)
Red-edge Chlorophyll IndexCIred-edge s 800 s 720 1 ,(13)
Green Chlorophyll IndexCIgreen s 800 s 550 1 ,(14)
Chlorophyll Absorption Reflectance IndexCARI s 700 s 670 0.2 × s 700 s 550 ,(15)
Modified Chlorophyll Absorption Reflectance IndexMCARI s 700 s 670 0.2 × ( s 700 s 550 ) × s 700 s 670 ,(16)
Transformed Chlorophyll Absorption Reflectance IndexTCARI 3 × s 700 s 670 0.2 × ( s 700 s 550 ) × s 700 s 670 ,(17)
Pigment-Specific Simple Ratio680PSSR680 s 800 s 680 ,(18)
Pigment-Specific Simple Ratio635PSSR635 s 800 s 635 ,(19)
Pigment-Specific Simple Ratio470PSSR470 s 800 s 470 ,(20)
Carotenoid Reflectance Index550CRI550 1 s 515 1 s 550 ,(21)
Carotenoid Reflectance Index700CRI700 1 s 515 1 s 700 ,(22)
Plant Senescence Reflectance IndexPSRI s 670 s 450 s 740 ,(23)
Nitrogen Reflectance IndexNRI s 550 s 670 s 550 + s 670 ,(24)
Note: s represents the spectral band, and the value following it represents the reflectance of the corresponding spectrum.
Table 3. Main hyperparameters of the XGBoost model.
Table 3. Main hyperparameters of the XGBoost model.
HyperparametersOptimized Value
Boosting learning rate (LR)0.27
Number of the base learner (NE)73
Maximum tree depth of the base learner (MD)6
Minimum sum of the instance weight needed in a child (MCW)1
Minimum value of the loss function required for leaf node branching (GAM)0
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Feng, S.; Jiang, S.; Huang, X.; Zhang, L.; Gan, Y.; Wang, L.; Zhou, C. Detection of Rice Leaf Folder in Paddy Fields Based on Unmanned Aerial Vehicle-Based Hyperspectral Images. Agronomy 2024, 14, 2660. https://doi.org/10.3390/agronomy14112660

AMA Style

Feng S, Jiang S, Huang X, Zhang L, Gan Y, Wang L, Zhou C. Detection of Rice Leaf Folder in Paddy Fields Based on Unmanned Aerial Vehicle-Based Hyperspectral Images. Agronomy. 2024; 14(11):2660. https://doi.org/10.3390/agronomy14112660

Chicago/Turabian Style

Feng, Shanshan, Shun Jiang, Xuying Huang, Lei Zhang, Yangying Gan, Laigang Wang, and Canfang Zhou. 2024. "Detection of Rice Leaf Folder in Paddy Fields Based on Unmanned Aerial Vehicle-Based Hyperspectral Images" Agronomy 14, no. 11: 2660. https://doi.org/10.3390/agronomy14112660

APA Style

Feng, S., Jiang, S., Huang, X., Zhang, L., Gan, Y., Wang, L., & Zhou, C. (2024). Detection of Rice Leaf Folder in Paddy Fields Based on Unmanned Aerial Vehicle-Based Hyperspectral Images. Agronomy, 14(11), 2660. https://doi.org/10.3390/agronomy14112660

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop