Determination of Dry-Matter Content of Kiwifruit before Harvest Based on Hyperspectral Imaging

Abstract

Determining pre-harvest fruit maturity is vital to ensure the quality of kiwifruit, and dry-matter content is an important indicator of kiwifruit ripeness. To predict the pre-harvest dry-matter content of kiwifruit continuously in real-time with high accuracy, this study uses hyperspectral data of pre-harvest Jintao kiwifruit obtained by using a hyperspectral image acquisition device. The raw data underwent whiteboard correction, spectral data extraction, spectral pre-processing, and feature-band extraction, following which the dry-matter content of the fruit was predicted by using partial least squares (PLS) regression. The feature bands extracted by the random frog method were 538.93, 671.14, 693.41, 770.61, 796.98, 813.24, 841.21, 843.29, and 856.80 nm, which improve the accuracy of the PLS method for predicting dry-matter content, with R² = 0.92 and a root mean square error (RMSE) of 0.41% for the training set, and R² = 0.85 and a RMSE of 0.50% for the test set. These results show that the proposed method reduces the number of required bands while maintaining the prediction accuracy, thereby demonstrating the reliability of using hyperspectral data to predict the pre-harvest dry-matter content of kiwifruit. This method can effectively guide the management of kiwifruit harvesting period, establishing a theoretical foundation for precise unmanned harvesting.

1. Introduction

Kiwifruit is popular because it is nutritionally rich, including vitamin C, amino acids, and various minerals [1], and has a pleasing taste and consistency. However, the inconsistent quality of kiwifruit on the market results in a significant price disparity and degrades the earnings of fruit growers and the development of the kiwifruit industry. Consumer demand for high-quality kiwifruit has increased rapidly in recent years. High-quality kiwifruit is characterized not only by a better visual appearance but also by a superior internal quality, such as being rich in juice and having an appropriate hardness and soluble solids content [2]. The internal quality of kiwifruit is closely related to the fruit maturity at harvest [3]. Harvesting too early makes the fruit growth period too short, which causes insufficient accumulation of organic matter and degrades the taste of the fruit. Due to the high maturity near respiratory climacteric, harvesting too late leads to rapid fruit decay in the post-ripening process, degrades the storage and transportation of the fruit, and increases loss [4]. Therefore, it is crucial to determine kiwifruit maturity before harvesting to improve fruit quality and increase the profits of fruit growers.

Dry-matter content is a meaningful indicator of fruit maturity [5,6,7]. Santagapita et al. [8], Kim et al. [9], and Ciccoritti et al. [10] have previously used dry-matter content to determine the maturity of kiwifruit. Dry matter is the organic matter remaining after the organism has been completely dried at a constant temperature of 60–90 °C, mainly including elements such as C, O, H, N, and ash. The current commonly used method to measure the dry matter includes two steps: first, measuring separately the sample mass before and after drying; second, the ratio of these two measurements is determined as the dry-matter content. In recent years, an increasing number of studies have used handheld near-infrared (NIR) spectroscopy devices for non-destructive pre-harvest detection of fruit quality [11].

In previous work along these lines, Zhou et al. [12] examined the soluble solids and hardness of kiwifruit using a detector and achieved superior detection results. Fan et al. [11] non-destructively evaluated the soluble-solid content of apples by using a portable visible-NIR (Vis-NIR) device, and Zhu et al. [13] used a portable NIR spectrometer to determine the sugar content and firmness of Fuji apples. Although these devices can detect the pre-harvest dry-matter content of fruit relatively accurately and conveniently [14,15], they still require manual measurement of individual fruit, which constitutes a large workload. In large-scale orchard production, NIR spectroscopy also requires sampling and measurements, similar to that carried out in traditional destructive methods to estimate the dry-matter content of fruits that are representative of the entire orchard. However, fruits grown in an orchard may vary due to factors such as weather, soil composition, available sunlight, and the availability of certain nutrients. Therefore, it is necessary to develop a non-destructive, continuous, and large-scale method to determine the dry-matter content of pre-harvest fruits.

Hyperspectral technology is superior to the aforementioned methods for determining the dry-matter content of pre-harvest fruit because it allows for real-time, high-throughput non-destructive testing. The feasibility of using hyperspectral techniques for predicting the dry-matter content of pre-harvest kiwifruit in an open outdoor environment has been demonstrated [16], and related research has also been conducted on mango, apple, and other fruit [17,18]. Wendel et al. [18] used hyperspectral data to predict the pre-harvest dry-matter content of mangos. Wang et al. [17] determined the pre-harvest dry-matter content of apples by using hyperspectral data. However, kiwifruit differs from mangos, apples, and other macrophanerophytes: as a vine, kiwifruit grow under the vine, which makes data acquisition more challenging.

In this paper, we further investigate the use of hyperspectral data to determine the pre-harvest dry-matter content of kiwifruit. This work extends our previous study, which demonstrated the feasibility of this approach. The specific objectives of this study are to (1) establish a hyperspectral image acquisition system suitable for kiwifruit orchards; (2) extract feature bands from spectral data of pre-harvest kiwifruit using four different feature-band-extraction methods and compare the accuracy of their predictions by using partial least squares (PLS) models; and (3) determine the feature bands most informative of the dry-matter content of kiwifruit and use these feature bands in a PLS model to predict the dry-matter content of kiwifruit. This method will effectively achieve non-destructive estimation of dry matter in pre-harvest kiwifruit, thus guiding management of the kiwifruit harvesting period and conducting precise unmanned harvesting.

2. Materials and Methods

This study used a hyperspectral camera (Rikola, SENOP, Finland, 500–900 nm) to acquire spectral data from pre-harvest kiwifruit and the PLS method to model prediction and analysis after the processing steps of whiteboard correction, spectral data extraction, spectral data pre-processing, sample delineation, and feature-band extraction. The technical strategy is shown in Figure 1.

2.1. Material

The experiment used Jintao kiwifruit (Actinidia chinensis var. chinensis cultivar Jintao) grown in Xixia County, Henan Province. The test area is a demonstration base for kiwifruit production (Figure 2A) and has level topography. The soil type is mainly yellow soil, with an annual sunshine of 2019 h, an average annual temperature of 15.1 °C, an average annual rainfall of about 830 mm, and an average frost-free period of 236.2 days. It is generally harvested from August to October in China. Due to the dry weather in the test year, there may have been some heterogeneity in kiwifruit maturity.

2.2. Data Acquisition

This study used a hyperspectral image-acquisition device consisting of a Rikola portable hyperspectral imager (SENOP, Finland) and an accompanying relative radiation sensor, a computer with hyperspectral camera-manipulation software (Rikola Hyper Spectral Imager 2.0, Rikola HSI 2.0), a tripod, and a standard whiteboard which has a spectral reflectance of over 99% in all wavelengths for the acquisition of spectral data (Figure 2C). Due to the growth characteristics of kiwifruit, its fruit hangs below the vine, so the camera’s shooting angle can only be from bottom to top. The height of the camera can be determined based on the growth density of the fruit in the orchard, ensuring that a certain number of fruits can be included in the image and each fruit is clearly visible. In this study, the distance between the camera and the fruit was 40–50 cm.

To obtain data for different growth periods of kiwifruit, data were acquired at the ripening of kiwifruit from October 19 to 20 in the kiwifruit orchard. The data were acquired under good lighting conditions and in places where the fruit was dense and at a uniform height. The hyperspectral image-acquisition device was assembled and placed under the kiwifruit canopy, with the camera lens adjusted so that it was perpendicular to the kiwifruit canopy, and the relative radiation sensor was placed where it could be directly illuminated by the sun so that it could be used to calibrate radiation brightness values (Figure 2B). We used Rikola HSI 2.0 to set the acquisition spectral range to 500–900 nm with a wavelength interval of 2 nm, for a total of 195 bands. First, a black opaque cover was used to completely obscure the camera lens and blackboard hyperspectral data were collected. Next, data from kiwifruits and from the standard whiteboard were acquired. In the sampling, seven hyperspectral images and 66 kiwifruit samples were collected, respectively. Figure 2D shows one original hyperspectral image of the pre-harvest kiwifruit.

To realize the whole process of non-destructive data acquisition, after acquiring the spectral image data, the dry-matter content for each pre-harvest fruit was measured, as the true value, by using an H100-C internal fruit quality non-destructive measuring instrument (Beijing Yang Tech, detection band range from 650 to 950 nm, detection error ±0.5%), with the resulting data used to validate accuracy of the prediction models.

2.3. Data-Processing Methods

2.3.1. Whiteboard Calibration

When the Rikola hyperspectral imager acquires hyperspectral data, it automatically corrects for dark current based on the blackboard data to generate hyperspectral image data containing irradiance information. The acquired hyperspectral image data are corrected by using the whiteboard calibration, resulting in hyperspectral image data containing spectral reflectance information for subsequent analysis. The whiteboard correction was performed by using ENVI 5.3 software (Environment for Visualizing Images software, Research Systems Inc., Boulder, CO, USA) and involved selecting 10 or more pixels in the whiteboard section of a hyperspectral image containing a standard whiteboard. Each spectral band of these whiteboard pixels was matched to the known reflectance of the corresponding band, following which the reflectance of other pixels in the image in each band was corrected according to their irradiance values:

$$R=R_w/L_w\times L$$

(1)

where $R$ is the reflectance of the pixel point to be calculated, $R_w$ is the whiteboard reflectance, $L_w$ is the whiteboard pixel point irradiance, and $L$ is the corresponding pixel point irradiance of the hyperspectral image.

2.3.2. Extraction of Spectral Data

In the hyperspectral image data after whiteboard correction, the kiwifruit portion was selected as the region of interest through ENVI 5.3, and the mean spectral reflectance of each region of interest for each band was used as the raw spectral reflectance data of the corresponding fruit. From the acquired hyperspectral images, spectral data were extracted from a total of 66 fruits.

2.3.3. Pre-Processing of Spectral Data

Hyperspectral images are affected by light scattering that occurs on the sample, which produces baseline drift in the reflectance spectra, so the pre-calibrated reflectance spectra should be pre-processed afterward [19]. Multiplicative scatter correction (MSC) is a multivariate scatter-correction technique that eliminates baseline drift by using ideal spectra to increase the signal-to-noise ratio of the original spectral absorbance so that they contain more spectral information related to the component or content to be measured. MSC processing was finished with Matlab R2021b.

The specific pre-processing steps of MSC follow:

(1)

Obtain the ideal spectrum. In practical applications, the ideal spectrum is often not available, so the average of the spectra from all samples is usually taken as the ideal spectrum. This is calculated as follows:

$${\overline{A}}_{i,j}=1/n{\displaystyle \sum }_{i=1}^n{A}_{i,j}$$

(2)

where $i$ is the sample-fruit number, $j$ is the spectral band, $A_{i,j}$ is the spectral reflectance of the sample fruit in spectral band $j$, n is the total number of fruits sampled, and ${\overline{A}}_{i,j}$ is the mean value of all fruit samples in spectral band j.

(2)

One-dimensional linear regression of the raw hyperspectral data. The tilt offset and linear translation are obtained as follows:

$$A_i=m_i{\overline{A}}_{i,j}+b_i$$

(3)

where $A_i$ is the spectral reflectance value of each sample to be measured, and $m_i$ and $b_i$ are the tilt offset and linear translation, respectively.

(3)

Subtract the original hyperspectral data. The original hyperspectral data are subtracted from the linear translation data and divided by the tilt offset to obtain the MSC-corrected hyperspectral data:

$${\overline{A}}_i=\left(\left(A_i-b_i \right)\right)/m_i$$

(4)

where $A_i$ is the hyperspectral data after MSC processing.

The MSC-processed hyperspectral data were used as sample-fruit spectral reflectance data for subsequent feature-band extraction and to build and validate models that predict the dry-matter content of kiwifruit.

2.3.4. Division of Sample Data

Forty-six of the sixty-six sample fruits were grouped in the training set and the remaining twenty in the test set by using the random number sorting method. Fruits from the training set were used to extract feature bands and build a prediction model, following which the test set was used to validate the accuracy of the models.

2.3.5. Feature-Band Extraction

Feature bands were extracted mainly to solve the problem of the large amount and high redundancy of hyperspectral data. In this study, we used competitive adaptive reweighted sampling (CARS), moving window PLS (MWPLS), Monte Carlo uninformative variables elimination (MCUVE), and random frog (RF) to extract feature bands from the MSC-processed spectral data through Matlab R2021b.

The CARS method is a variable selection method that uses adaptive resampling and is based on the magnitude of the absolute value of the regression coefficients. Xing et al. [20] used CARS to extract feature bands of soil spectra, which significantly improved PLSR prediction models for soil organic matter. The MWPLS is a method for finding wavelength intervals with a low residual sum of squares of the correction set by continuously moving the window and building PLS submodels on the move. Chen et al. [21] used MWPLS for model optimization and band selection for NIR spectral analysis of soil organic matter, and the predictions of the resulting model were much more accurate than those obtained by using all the spectral bands. The MCUVE method combines Monte Carlo sampling with the uninformative variable elimination method to determine the importance of variables based on the stability of regression coefficients and by selecting a certain threshold and retaining all bands with stability values above this threshold as characteristic bands [22]. The RF method is a new feature-band selection method, which obtains different models based on the sequential method, calculates the selection frequency of each band variable, so as to judge the importance of the band, and realizes the variable selection of high-dimensional data with high efficiency. Hu et al. [23] showed that the model has considerable promise for predicting the mechanical properties of blueberries based on hyperspectral data after screening the spectral bands using RF.

2.4. Prediction Model Building and Testing Methods

The spectral data of pre-harvest kiwifruit were modeled by using the PLS method in Matlab R2021b after whiteboard correction, MSC processing, data segmentation, and feature-band extraction. PLS is a method to find the optimal function that matches a set of data by finding the least error sum of squares. This method enables regression modeling in the presence of high multiple correlations of sample variables and it is also commonly used in spectral analysis [24]. The coefficient of determination $R^2$ and the root mean square error (RMSE) are used as test criteria to analyze the prediction accuracy obtained when using different feature-band-extraction methods with the PLS model. The extraction methods with higher $R^2$ and lower RMSE are selected to construct the prediction model. $R^2$ and RMSE are given by:

$$R^2=1-\left[{{\displaystyle \sum }}_{\mathrm{i}=1}^n{\left(P{V}_i-R{V}_i\right)}^2\left]/\right[{{\displaystyle \sum }}_{\mathrm{i}=1}^n{\left(P{V}_i-\overline{RV}\right)}^2\left]=\right[{{\displaystyle \sum }}_{i=1}^n{\left(P{V}_i-\overline{RV}\right)}^2\left]/\right[{{\displaystyle \sum }}_{i=1}^n{\left(R{V}_i-\overline{RV}\right)}^2\right]$$

(5)

$$\mathrm{RMSE}=\sqrt{\left[{{\displaystyle \sum }}_{i=1}^n{\left({PV}_i-{RV}_i\right)}^2\right]/n}$$

(6)

where $P{V}_i$ is the predicted dry-matter content of fruit $i$, $R{V}_i$ is the measured dry-matter content of fruit $i$, $\overline{RV}$ is the average of the measured dry-matter content of the sample fruits, and $n$ is the number of the fruit being sampled.

3. Results and Discussion

3.1. Analysis of Spectral Curve

Figure 3a,b show the original spectral curves of the sample fruits before and after MSC pre-treatment. The patterns of the spectral curves from the 66 sample fruits are similar. The spectral reflectance curves tighten after MSC treatment, which shows that the baseline drift is eliminated and the signal-to-noise ratio increases. Due to the various capacities to absorb light at different wavelengths (due to varying chlorophyll and water content inside the fruit) [25], the spectral reflectance of the sample fruits differ significantly. Between 504.78 and 684.75 nm, the spectral reflectance remains at a low level and within the absorption bands of chlorophyll, which are related to the C–H spectrally sensitive group in chlorophyll. The spectral reflectance increases strongly at 684.75–740.87 nm and peaks at 740.87 nm. In the range 770.61–905.38 nm, the spectral reflectance gradually decreases, which is related to the gradual increase in the absorption rate of the O–H spectrally sensitive groups in water within this band [25]. Another relevant phenomenon is that fruits with greater dry-matter content generally have a lower spectral reflectance in the overall distribution of the spectral curve, which is consistent with the pattern found by Li et al. [26] in their study of cherry ripeness using hyperspectral techniques.

3.2. Extraction of Feature Bands

The four methods CARS, MWPLS, MCUVE, and RF were used to extract the feature bands from the sample fruits in the pre-processed training set, which consists of 46 fruits from six hyperspectral images of the second sampling. Figure 4 shows the extraction results.

The graph of the results of the CARS method (Figure 4) shows that the number of remaining bands decreases monotonically from 195 as the number of iterations increases. The root mean square error of cross-validation (RMSECV) reaches a minimum at the 28th iteration (at the vertical line consisting of blue stars). At this point, 16 bands remain, and the CARS method extracts these 16 bands as its feature bands. The feature bands in the remaining three methods are then extracted to obtain, to the extent possible, the same number of feature bands as obtained by the CARS method. The graph of the result for the MWPLS method (Figure 4) shows the curve that gives the root mean square error fit (RMSEF) as a function of window position for window width 15, which is set according to the total number of spectral bands. With the principle of trying to select a similar number of characteristic bands, the 12 bands corresponding to RMSEF < 0.55 are extracted as the feature bands. In the graph of the results from the MCUVE method, using the same principle as MWPLS method, 13 bands corresponding to Reliability > 4 are extracted as the feature bands. For the RF method, the bands with a selection probability greater than 1.7 are extracted, for a total of 9 feature bands extracted according to the same principle mentioned before.

Table 1. Specific feature bands extracted.

Method	Result (nm)
CARS	537.03	538.93	540.83	631.09	669.09	671.14
	672.57	693.41	740.87	759.22	781.27	796.98
	805.14	815.31	817.32	856.80
MWPLS	551.05	678.79	680.78	682.85	684.75	686.54
	689.43	691.38	693.41	695.39	697.38	699.45
MCUVE	661.00	663.08	665.09	667.08	669.09	671.14
	672.57	674.56	676.67	678.79	796.98	813.24
	856.80
RF	538.93	671.14	693.41	770.61	796.98	813.24
	841.21	843.29	856.80

lists the specific feature bands extracted.

The bands extracted by each of the four methods are labeled on the average spectral curve of all sample fruits, as shown in Figure 5. In terms of feature-band distribution, most feature bands are distributed between 600 and 700 nm, with the next largest number of bands distributed between 700 and 800 nm and 800 and 905.38 nm. The fewest feature bands are distributed between 504.78 and 600 nm. In addition, some methods extract too many feature bands in specific wavelength regions, such as the MWPLS method, which selects 678.79–699.45 nm as a feature band, and the MCUVE method, which selects all the feature bands within 661.00 to 678.79 nm as feature bands. This may result in redundancy in the feature bands extracted by these two methods. In contrast, the feature bands extracted by the CARS and RF methods are relatively evenly distributed over the entire spectral curve, so the probability of redundancy is lower.

3.3. Analysis of PLS prediction

The dry-matter content of the sample fruits and the MSC-processed spectral reflectance corresponding to the feature bands extracted by the four methods serve as inputs and are modeled using the PLS method to predict the dry-matter content of the sample fruits. We compare the predictions and select the method with the most accurate prediction and then test the predictions by using the test set.

3.3.1. Results for Training Set

The full spectral bands and feature bands of the spectral data from the 46 sample fruits in the training set were input into the PLS calibration models, and the results are shown in

Table 2. PLS calibration results based on training set.

Method	R2	RMSE (%)
All spectral bands	0.89	0.47
CARS	0.91	0.43
MWPLS	0.86	0.53
MCUVE	0.86	0.53
RF	0.92	0.41

.

Using all spectral bands, we obtain R² = 0.89 and RMSE = 0.47%; thus, the model produces accurate calibration results. However, in actual orchard production, collecting all 195 spectral bands from fruits is extremely time-consuming and laborious, and the computational costs for making predictions with models also greatly increase, which may prevent the prediction system from achieving its goal of real-time and rapid dry-matter content prediction. Extracting the feature bands reduces the number of sampled bands, which accelerates the computation of the model and improves the calculation efficiency.

Among the calibration results of the four methods to extract feature bands, those extracted by the CARS and RF methods produce more accurate calibration results than when using the full-spectrum bands in the PLS model. The most accurate calibration results are obtained when using the feature bands extracted by the RF method, with R² = 0.92 and RMSE = 0.41%. Calibration models using the feature bands extracted by the CARS method produced R² = 0.91 and RMSE = 0.43%. In contrast, the feature bands extracted by the MWPLS and MCUVE model do not reflect the dry-matter content of kiwifruit because of their significant redundancy, which leads to inaccurate calibration results.

3.3.2. Prediction Results for Test Set

Based on this analysis of the calibration results, we select the nine feature bands extracted by the RF method: 538.93, 671.14, 693.41, 770.61, 796.98, 813.24, 841.21, 843.29, and 856.80 nm, to reflect the dry-matter content of kiwifruit. Once the feature bands are extracted from the spectral data of the 20 sample fruits in the test set, the PLS model is used to make predictions. The PLS prediction model produces R² = 0.85 and RMSE = 0.50%. These results show that the feature bands extracted by the RF model are small in number (which improves the calculation efficiency) and ensure accurate predictions. Note that Wendel et al. [18] used hyperspectral data to study the dry-matter content of pre-harvest mango fruits using a prediction model they developed and obtained R² = 0.64 and RMSE = 1.08% for the prediction results, which demonstrate the accuracy of the hyperspectral method used herein.

Figure 6 shows the difference between the predicted and measured dry-matter content of some sample fruits (including sample fruits from the training set and sample fruits from the test set, and using the RF method to extract feature bands). These results show more intuitively that the PLS model makes relatively accurate predictions of the dry-matter content of pre-harvest kiwifruit.

3.4. Discussion

Although the use of hyperspectral technology to detect fruit quality in controlled indoor environments has been studied numerous times [27], the use of hyperspectral technology for predicting pre-harvest fruit dry-matter content in outdoor environments remains uncommon. Wendel et al. [18] used hyperspectral imaging equipment installed on an unmanned ground vehicle to predict the pre-harvest dry-matter content of mangos and obtained accurate predictions. Mangos are a type of macrophanerophyte and its fruit grows on both sides of the tree, so a hyperspectral camera benefits from good acquisition angles and lighting conditions for image acquisition. However, kiwifruit is a vine plant and its fruit grows below the vine, so a hyperspectral camera must acquire pictures from below at an elevated angle with respect to the ground, which limits the camera view and lighting conditions. However, the prediction results of this study are moderately accurate, which demonstrates that the pre-harvest dry-matter content of fruit acquired by using hyperspectral techniques can be accurate and reliable.

Weather conditions, especially lighting conditions, are an important factor affecting the quality of data acquisition in this study. If the lighting conditions are poor or change during data acquisition, the spectral data may be degraded, which would affect the reliability of the hyperspectral data and negatively impact the prediction of dry-matter content. Therefore, further research should consider ways to adapt to weather conditions in future experiments and other practical applications. For example, an adaptive light calibration model could be established to automatically correct the data acquired under different light intensities to eliminate the effects of light conditions.

The prediction model used herein was constructed based on spectral data from kiwifruit from the same year and location; therefore, its predictive effect for other regions, fruit varieties, and weather conditions is uncertain. In future research, the generalization of the model to different regions, fruit varieties, and weather conditions can be investigated to determine the generalization of the model or to try to establish a more universal prediction model.

4. Conclusions

Determining the dry-matter content is important to determine the maturity and improves the quality of kiwifruit. In this study, the pre-harvest dry-matter content of Jintao kiwifruit was predicted by using hyperspectral techniques. The results show that the spectral data of pre-harvest kiwifruits obtained by hyperspectral cameras and PLS-modeling prediction made from feature bands extracted by the RF method can produce accurate predictions of the pre-harvest dry-matter content of kiwifruit. The feature bands extracted by the RF method are 538.93, 671.14, 693.41, 770.61, 796.98, 813.24, 841.21, 843.29, and 856.80 nm, and the prediction produced by the PLS model for the test set has R² = 0.85 and RMSE = 0.50%, which shows that the use of hyperspectral data produced reliable predictions of the pre-harvest dry-matter content of kiwifruit. This method can achieve the real time, continuous, and accurate dry-matter estimation of pre-harvest kiwifruit, and effectively guide the management of the kiwifruit harvesting period and enable precise unmanned harvesting.

In future studies, we plan to combine the proposed method of predicting the pre-harvest dry-matter content of kiwifruit with multispectral data acquired from a mobile ground platform, which should enable real-time prediction of fruit maturity in standardized kiwifruit orchards, allowing farmers to accurately detect the fruit maturity in the orchard. This should provide objective data to support kiwifruit harvesting and improve the quality of kiwifruit and the economic efficiency of kiwifruit agriculture.