Innovative Hyperspectral Imaging-Based Techniques for Quality Evaluation of Fruits and Vegetables: A Review

Abstract

New, non-destructive sensing techniques for fast and more effective quality assessment of fruits and vegetables are needed to meet the ever-increasing consumer demand for better, more consistent and safer food products. Over the past 15 years, hyperspectral imaging has emerged as a new generation of sensing technology for non-destructive food quality and safety evaluation, because it integrates the major features of imaging and spectroscopy, thus enabling the acquisition of both spectral and spatial information from an object simultaneously. This paper first provides a brief overview of hyperspectral imaging configurations and common sensing modes used for food quality and safety evaluation. The paper is, however, focused on the three innovative hyperspectral imaging-based techniques or sensing platforms, i.e., spectral scattering, integrated reflectance and transmittance, and spatially-resolved spectroscopy, which have been developed in our laboratory for property and quality evaluation of fruits, vegetables and other food products. The basic principle and instrumentation of each technique are described, followed by the mathematical methods for processing and extracting critical information from the acquired data. Applications of these techniques for property and quality evaluation of fruits and vegetables are then presented. Finally, concluding remarks are given on future research needs to move forward these hyperspectral imaging techniques.

1. Introduction

Food products are evaluated or inspected for their appearance attributes, such as color, size or shape and absence of surface defects, and internal properties and characteristics like defects and eating quality that is defined by texture and flavor attributes [1,2]. Traditionally, manual and/or destructive methods are commonly used for quality assessment of food products, and they are only limited to a small number of samples because the tests are destructive, slow, and costly or labor intensive. Food quality constitutes multiple attributes or characteristics, some of which are not well defined or can only be measured using empirical or destructive methods, such as the Magness-Taylor test for firmness measurement of horticultural products. While technologies for inspecting, sorting and grading surface characteristics of fruits and vegetables are now being widely used commercially, non-destructive evaluation of internal quality, such as texture and flavor, still presents great technical challenges.

Today, consumers have a wide range of choices for food products, and they are demanding higher quality, more consistent and safer products at a reasonable price. Hence the agricultural and food industries are being faced with more challenging tasks of inspecting food products to meet the ever-increasing consumer demand, while maintaining or enhancing profitability in the face of intensive market competition. Rapid, objective inspection, sorting and grading of each product item is a critical step for ensuring superior, consistent quality of the final products. The past two decades have seen significant progress in the development of non-destructive sensing technologies for automated quality evaluation and safety inspection of agricultural and food products. Among the most successful technologies are near-infrared spectroscopy (NIRS) and machine vision [3,4]. In NIRS, the near-infrared (NIR) light covering the wavelength range of about 750–2500 nm is incident on the food product in the form of diffuse reflectance, transmittance or interactance, and the acquired spectra are then quantitatively related to the chemical and physical properties of the product. NIRS is easy to use, fast and non-destructive (or requiring minimal sample preparation), and hence has been widely used for property characterization, composition analysis, and quality and safety evaluation of food and agricultural products. However, NIRS is point measurement and is incapable of, or inadequate for, evaluating the spatial features of the object being detected. Moreover, NIR measurements are considered empirical (or black-box measurements) and indirect, because they only provide aggregate amount of information on the light reflected from or transmitted through the sample and the spectral measurements are dependent on the specific setting or configuration of the lighting source and detection probe. Hence more recently, researchers in the food and agriculture field have explored or developed new spectroscopic measurement techniques that are grounded on the well-established physical principles (light scattering and absorption) and optical theories (e.g., radiation transfer or diffusion theory) [5]. Although still in an early developmental stage, these new techniques, including spatially-resolved, spatial frequency domain, time-resolved and frequency domain, have the potential for providing more effective quality evaluation of horticultural and food products.

Machine vision is conventionally referred to as a collection of the imaging methods that use broadband visible (VIS, 400–700 nm) light in monochromatic or color mode, and it simulates human vision to perform inspection tasks. Compared to NIRS, machine vision is capable of inspecting, grading and sorting horticultural and food products based on their spatial surface features such as shape, size and color [6,7], and the presence of surface defects [8,9]. The technology is now being widely used for quality inspection, sorting and grading of horticultural and food products. However, machine vision is not suitable for assessing internal quality attributes, whether chemical or physical, and it also is not very effective for defect detection, due to the lack of sufficient spectral information.

Hyperspectral imaging has emerged as a powerful inspection tool in the agricultural and food domain since the late 1990s [10,11]. A hyperspectral imaging system produces a two-dimensional (2-D) spatial matrix of vectors, each of which represents a spectrum, typically in the wavelengths ranging from VIS to NIR, thus forming a three-dimensional (3-D) image dataset, known as hypercube. The technology integrates or bridges spectroscopy and imaging to acquire both spectral and spatial information from a product simultaneously. It can thus provide a more effective means for detection of chemical compositions as well as external or internal quality attributes in the spatial dimensions. Consequently, research on hyperspectral imaging for food quality and safety inspection has expanded tremendously over the past decade [12,13,14,15]. With large volumes of image data, often covering over 100 or even 1000 wavebands, hyperspectral imaging is limited by the speed of image acquisition, processing and analysis. Numerous studies in recent years have thus been focused on determining a few most effective wavebands by using hyperspectral imaging, so that they can be implemented in a multispectral imaging configuration [16,17,18,19,20]. Researchers at the U.S. Department of Agriculture (USDA) Agricultural Research Service (ARS) successfully developed online hyperspectral imaging systems that are implemented in multispectral imaging mode for safety inspection of poultry carcasses [21,22,23]. With the rapid evolution in both hardware and software, hyperspectral imaging is expected to be increasingly used by the food and agricultural industries for quality and safety inspection of food and agricultural products in the coming years.

Over the last 15 years, our group has been actively engaged in the development and application of hyperspectral imaging technology for property and quality evaluation of horticultural and food products. In addition to conventional reflectance imaging mode, we have developed several innovative hyperspectral imaging-based techniques or platforms, including spectral scattering for evaluation of firmness and soluble solids content (SSC) [24,25], full transmittance for internal defect detection [26,27], integrated reflectance and transmittance for detecting surface and internal quality or defect [28,29], and spatially-resolved spectroscopy for measuring the optical absorption and scattering properties of fruits, vegetables and other food products [30,31].

In the following sections, an overview will be first provided of hyperspectral imaging configurations and common sensing modes, followed by a review on each of these hyperspectral imaging-based techniques. The basic principle, instrumentation and data processing methods for each technique are first introduced, followed with applications in food quality assessment.

2. Hyperspectral Imaging Configurations

Hyperspectral imaging, broadly speaking, is a type of spectral imaging technology that integrates imaging and spectroscopy to obtain 3-D data cubes, which contain 2-D spatial and one-dimensional (1-D) spectral information from products. It is closely related to another spectral imaging technology, called multispectral imaging, which also has been used for food quality and safety assessment. The main difference between the two technologies resides in the number of wavelengths, spectral resolution and the way of acquiring image data. It is generally acknowledged that imaging at fewer than 10 discrete wavelengths or wavebands (usually broad wavebands) is termed multispectral, while imaging at more than 10 contiguous wavelengths or narrow wavebands is termed hyperspectral.

2.1. Image Acquisition Modes

Generally, there are four approaches to the acquisition of hyperspectral image data: point scanning, line scanning, area scanning and single shot, as shown in Figure 1. Point scanning captures spectral data for each pixel each time, and scans the full spatial scene by moving either the detector or the object continuously in the two spatial dimensions to obtain 3-D image cubes. This approach is usually used in microscopic imaging, where double scanning is required; however, it is very time consuming, and also requires repositioning the hardware to ensure the repeatability of results. Hence point scanning is rarely used for food quality or safety evaluation. Line scanning is a push-broom method, in which a line of spatial information with a full spectrum for each pixel is captured, and a complete volume of spatial-spectral data is obtained by moving the sample line by line. This approach is well suited for conveyor belt systems in the production line for online inspection of food items. Unlike point scanning and line scanning, which are spatial scanning methods, area scanning is a spectral scanning method, in which the sample is kept stationary and the scan with full spatial information is performed sequentially over a full spectral range. This approach is suitable for applications where sample movement is not required. Last, the single shot approach uses a large area detector to acquire a 3-D image cube with one exposure, without scanning in either spectral or spatial domain. This approach is very attractive due to its fastest speed, and it is emerging recently as a viable method for rapid acquisition of hyperspectral images. However, it has limited spatial resolutions and narrow spectral ranges.

2.2. Instrumentation

Despite various image acquisition configurations, most hyperspectral imaging systems share three major components, i.e., light source, wavelength dispersive device and area array detector.

Light sources generate light to illuminate the sample, which basically fall into three types, i.e., halogen lamps, light emitting diodes (LEDs) and lasers. A halogen lamp, also known as quartz tungsten halogen (QTH) lamp, provides a smooth broadband spectrum covering the region from VIS to NIR, which is commonly used in hyperspectral imaging for food detection. However, QTHs have a relatively short lifetime, high heat generation, spectral shift due to temperature changes and unstable output due to operating voltage fluctuations. LEDs have evolved rapidly in recent years and become increasingly popular in machine vision applications. They are able to generate both narrowband and broadband light, and support flexible illumination arrangements such as spot, line, and ring lights, depending on applications. Compared to QTHs, LEDs have numerous advantages including small size, fast response, low cost, long lifetime, low heat generation, and low energy consumption. Lasers are powerful monochromatic light sources, distinguished from QTH and LED by the properties of monochromaticity, directionality, and coherence. Lasers are frequently used as excitation light sources in hyperspectral fluorescence and Raman imaging measurements for food quality and safety detection.

A wavelength dispersive device is to disperse the incident broadband light into different wavelengths and project the dispersed light to the detector, which is essential for a hyperspectral imaging system. The common wavelength dispersive devices are imaging spectrographs and electronically tunable filters (ETFs). Imaging spectrographs use diffraction gratings for wavelength dispersion based on the fact that the direction of light propagation is wavelength dependent. There are two types of imaging spectrographs, as illustrated in Figure 2, according to whether a reflection or transmission grating is used for wavelength dispersion, both of which have been used in line-scan hyperspectral imaging systems. Two major types of ETFs are liquid-crystal tunable filter (LCTF) and acousto-optic tunable filter (AOTF). The LCTF is a birefringent-based filter that uses the phase retardation between the ordinary and extraordinary rays passing through a liquid crystal to create constructive and destructive interferences, thus dispersing an incident broadband light into different wavelengths. The AOTF, on the other hand, is a solid-state device that isolates a single wavelength based on light-sound interactions in an anisotropic crystal that behaves like a transmission diffraction grating, and the tuned wavelength can be controlled by varying the frequency of an acoustic wave propagating inside the crystal. Unlike imaging spectrographs, both LCTF and AOTF are generally used in area-scanning hyperspectral imaging systems.

An area array detector, or also commonly called a digital camera, is to measure the intensity of the collected light in two spatial dimensions. Charge-coupled devices (CCDs) are the mainstream detectors used in hyperspectral imaging systems. The CCD sensor consists of many small photodiodes, known as pixels, which are made of photosensitive materials such as silicon. Different sensing materials have different spectral responses of the CCD sensor, which are quantified by quantum efficiency (QE). Regular silicon-based CCD cameras have a bell-shaped QE curve over the visible and short-wave near-infrared regions of 400–1000 nm, and they usually have lower QEs. For hyperspectral imaging applications, high performance CCD cameras, such as electron-multiplying, intensified and back thinned (or back illuminated), are often necessary, because they have better QE responses and higher signal-to-noise ratios. In recent years, complementary metal-oxide-semiconductor (CMOS) has also been increasingly used for hyperspectral imaging, due to its advantages of low cost, low power consumption, small size, and fast signal transfer. CMOS, however, has some drawbacks including high noise and dark current and low dynamic range and sensitivity. For applications over the NIR region, indium gallium arsenide (InGaAs) detectors are often used, which typically cover the wavelength range of about 900–1700 nm, and can be extended to 2.6 μm.

2.3. Commen Sensing Modes

There are three commonly used sensing modes in hyperspectral imaging, i.e., reflectance, transmittance and interactance, as illustrated in Figure 3. These sensing modes are mainly different in their lighting configuration for imaging the sample, and each has different implications in food quality and safety assessment. In reflectance mode (Figure 3a), the light source and the detector are placed above the sample and the measured light is diffusely reflected from the illuminated area of the sample. This sensing mode is easy to implement and has been widely used to detect quality attributes, both external and internal, of food products [16,32,33,34]. However, in reflectance mode, light has limited penetration in horticultural and food products, and hence this sensing mode may not be quite effective for detecting internal quality. In transmittance mode (Figure 3b), the detector and the light source are positioned in the opposite side of the sample, and the measured light is transmitted through the whole product, carrying information about the internal features of the product. This sensing mode is well suited for evaluating the internal condition of food products, e.g., internal defects [26,27], but it requires a high-intensity light source and a high-sensitivity detector to obtain sufficient detectable signals, and thus is relatively difficult to implement in practice. Moreover, transmittance measurements are influenced by product size and shape, and the long light pathlength in the product could complicate quality assessment. In addition, transmittance mode can be implemented in a semi-transmittance setup [35], by placing the lamp horizontally at one side of the sample, which is able to acquire the information inside the sample with less light power supply. In interactance mode (Figure 3c), the detector and light source are positioned in the same side of the sample with a certain source-detector separation, and a light barrier is placed between them to ensure that the detected light has gone through a minimum distance, which is conducive to assessing the properties of sublayer tissues. This sensing mode, like semi-transmittance, makes a compromise between reflectance and transmittance measurements [36,37], but it may be unsuitable for detection of small samples.

3. Spectral Scattering Imaging

Scattering occurs, as photons traveling within biological tissues are encountering cellular structures, which serve as scattering particles, or the interfaces of two tissue layers with different refractive indexes, thus resulting in the change in the direction of light propagation. Light scattering depends on the density, cell structures, extra- and intra-cellular matrices of tissues, and it can thus provide insights into the structural and textural properties of biological tissues [24,38,39]. Quantifying light scattering, however, poses a challenge, because the scattering event is inevitably interwoven with light absorption, another fundamental phenomenon taking place during the interaction of light with tissues, which relates to tissue chemical composition.

Birth and colleagues [38] reported on the use of a light transmission technique coupled with integrating sphere measurements to quantify light scattering for assessment of pork quality. However, the transmission technique is not suitable for detecting fresh horticultural produce. Since the mid-1990s, there has been growing interest in non-destructive assessment of fruit texture, i.e., firmness, which led to the development of a number of light scattering techniques. In earlier studies, a single laser was explored as a light source for acquiring scattering images or profiles at a specific wavelength [40,41,42]. Single laser-based techniques, although having shown correlation between scattering features and fruit firmness, have limitations as the selected wavelength may be not optimal or insufficient for predicting firmness and other quality attributes like SSC. Our laboratory first proposed the concept of using spectral scattering imaging, including multispectral and hyperspectral, to measure scattering characteristics to enhance fruit quality assessment [24,43]. Multispectral scattering imaging allows acquiring scattering images at several discrete key wavelengths, while hyperspectral imaging enables acquisition of scattering images over a series of contiguous wavelengths, thus enhancing the detection capability. In implementing spectral scattering imaging, a broadband light source is used to illuminate the sample and backscattered images are acquired by a camera coupled with a wavelength selection device. In recent years, researchers have explored laser-based imaging systems for fruit quality assessment [44,45,46,47], which use a laser module as a light source that consists of several lasers operating at different wavelengths to acquire multispectral scatting images.

In the following subsections, we give a detailed description of several scattering configurations developed for scattering measurement, analysis of spectral scattering images, and applications of spectral scattering imaging for quality evaluation of apple and other horticultural products.

3.1. Spectral Scattering Imaging (SSI) Configurations

As shown in Figure 4, a general SSI system mainly consists of an illumination unit, an imaging unit and a computer. The illumination unit consists of a light controller, a light source, an optic fiber and a focusing lens. A QTH lamp is commonly used as a light source for providing broadband illumination over a wavelength range of about 400–1000 nm in both multispectral and hyperspectral measurement. The imaging unit includes a CCD camera and a wavelength dispersive device attached with a zoom lens.

Light passes through an optic fiber and a focusing lens to form a sharp, highly focused beam (usually between 1–2 mm in size), which is incident on the fruit surface. As the light propagates into the fruit tissue, photons are either absorbed or scattered. The photons, which are backscattered and then exit from the fruit in an area (scattering area), illuminate a portion of the fruit tissue surrounding the incident area, as shown in Figure 5. In designing an illumination unit, attention should be paid to the size of light beam for generating good scattering images [24]. A larger beam gives a higher light throughput and a larger scattering area, but it may not be desirable for quantifying light scattering characteristics. A smaller beam, on the other hand, albeit desirable for scattering measurement, results in lower intensities of images acquired and a lower signal-to-noise ratio (SNR) as well. A beam size of 1.6 mm in diameter is often used for acquiring scattering images from apple fruit with the maximum scattering area of about 25 mm in diameter [43,48].

The wavelength dispersive device is used to disperse the incoming broadband light into separate wavelengths, and it can be a line-scan imaging spectrograph using a prism-grating-prism configuration for hyperspectral scattering imaging [43,48], or a common aperture multi-channel imaging spectrograph for multispectral measurement [25,49]. In addition, multispectral measurement can also be achieved using a LCTF [50,51,52], or a mechanical rotating filter wheel containing four to six bandpass filters [24]. Hence there are basically four sensing configurations, as shown in Figure 6, for measuring scattering images.

The line-scan spectrograph based configuration (Figure 6a) allows simultaneous recording of a line of spatial information with a full spectral range for each spatial pixel of the scanned line, thus creating a special 2-D image with one axis representing the spatial dimension and the other spectral. For acquiring a complete 3-D hypercube, a series of scans need to be captured sequentially, which is time consuming and generates a large volume of data as well. To expedite the acquisition of hyperspectral scattering images, a simplified scanning approach can be used, since light scattering in the fruit tissue could be assumed to be symmetric with respect to the incident point. Thus, the complete scattering information of the fruit may be well represented by one single line scan that passes through the center of the incident area. However, in practical implementation, the scanning line often is slightly away from the incident center (1 to 2 mm) to avoid signal saturation [43,48]. This simplified approach to acquiring hyperspectral scattering images is suitable for real-time, online implementation. Likewise, the common-aperture multispectral imaging configuration (Figure 6b), which records scattering images at multiple wavelengths simultaneously, also is suitable for fast, real-time online applications, but it suffers from reduced spatial resolution and slightly deteriorated image quality, i.e., lower SNR and image distortion. On the other hand, the filter wheel and LCTF based imaging configurations (Figure 6c,d) acquire scattering images for the chosen wavelengths in sequence with the camera synchronized with the filtering device. The two configurations are relatively slow in imaging speed, but they are able to obtain high quality scattering images.

In addition to broadband illumination, a laser module that is composed of multiple lasers at different wavelengths can also be used as a light source for multispectral scattering measurement [49]. Compared to the broadband light source, the laser source has a higher power and efficiency of light utilization, and thus is more suitable for fast image acquisition. Figure 7 shows a laser-based multispectral scattering imaging prototype built for measurement of fruit firmness. The prototype uses a specially designed laser lighting module, which emits four laser beams that are coupled to a single optic fiber, and its imaging unit uses a common-aperture imaging spectrograph with four bandpass filters matching the four lasers, to acquire four scattering images all at once.

3.2. Analysis of Scattering Images

Figure 8 shows two typical multispectral and hyperspectral scattering images acquired from apple fruit. The two types of scattering images differ in obtaining scattering profiles, which are one dimensional, but follow the same procedures for analysing these scattering profiles. Moreover, for hyperspectral images, additional procedures are needed for extracting and exploiting spectral features. Figure 9 summarizes the overall procedures for processing these scattering images to obtain useful information for fruit quality assessment. The following discussion details these procedures for image analysis of scattering images individually.

3.2.1. Extraction of Scattering Profiles

The raw scattering images usually contain some pixels of unusually high or low intensities, compared to their adjacent pixels, due to the presence of abnormal or defective tissue spots at the surface of fruit [53]. These pixels appear as dark or bright spots in the images, which should be removed in the pre-processing step. A simple spatial filter, including pixel binning, can be used for the purpose. Besides, removal of random noise, residing in both multispectral and hyperspectral scattering images, needs to be done by smoothing or applying a frequency-domain filter.

A multispectral scattering image can be reduced to a 1-D spatial scattering profile, or scattering profile for brevity, by using a radial averaging technique [24]. Since the multispectral scattering image, as shown in Figure 10a, is basically circular and symmetric with respect to the incident point, it can be divided into a series of concentric annuli of equal width, with an assumption that the pixel intensities are approximately constant within each annulus. To divide the scattering area, the incident center needs to be identified beforehand, which is often defined as the pixel position corresponding to the maximum intensity [24]. Then, the radial averaging is performed by averaging all the pixels within each concentric circular band, thus leading to a scattering profile as shown in Figure 10b. The averaging procedure not only simplifies the data analysis, but also greatly improves the SNR of the data.

For a hyperspectral scattering image, as shown in Figure 8b, extracting the scattering profile is rather straightforward, since each horizontal line from the image represents a scattering profile at a specific wavelength. Moreover, considering the symmetric feature of the obtained scattering profile, as shown in Figure 10b, one can average the scattering profiles from the left and right sides with respect to the incident center, thus reducing the data by half and further enhancing the SNR.

3.2.2. Correction of Scattering Profiles

Scattering profiles obtained using the procedures described above may suffer from a certain degree of distortion due to the uneven instrumental responses and the surface curvature of the fruit [54,55]. Hence, corrections are needed to address the distortion for accurately quantifying scattering profiles.

The uneven instrumental responses refer to the uneven distribution of the pixel intensities when a flat homogeneous object is imaged under ideal uniform illumination. A number of factors, e.g., a varied viewing angle due to the finite fruit-camera distance, and the imperfection of imaging optics, can contribute to the uneven instrumental responses. Corrections for this problem can be made by imaging a flat homogenous reference sample like a Teflon disk. As shown in Figure 11a, a Teflon reference is placed on a motor-controlled stage that permits precise adjustment of the Teflon position; moving the reference back and forth causes intensity changes of the scattering image acquired, which are attributed to the uneven uniform instrumental response. Then, the response curve versus the displacement of the reference is extracted by recording the peak intensity values of the scattering image for each wavelength at each displacement position. With the response curve being known, correction is done by multiplying the raw scattering profile by the reciprocal of the normalized response curve. A more detailed description of the correction method can be found in [55].

The reference-based correction, however, has a major limitation in that it does not take into account the effect of the surface curvature of a fruit sample. The curved shape of the fruit can bring about two types of distortion, i.e., scattering distance distortion and intensity distortion [49]. For a given point m on the fruit surface, as shown in Figure 11b, the actual scattering distance for the point should be the arc length z instead of x, and assuming the fruit is spherical with a radius r, z can be calculated as follows:

$$z=r\arctan \frac{x}{\sqrt{r^2-x^2}}$$

(1)

where r and x are the radius of the fruit and the horizontal distance of the point m from the light beam incident center. So, the distance distortion can be easily corrected according to Equation (1).

The intensity distortion is mainly due to the varied viewing angle of the lens for different points on the fruit surface. From Figure 11b, the point closer to the incident center, has a larger viewing angle and thus a larger detected scattering intensity. To tackle this issue, a model-based method was proposed with an assumption that the angular distribution of scattering intensity on the fruit surface follows the Lambertian Cosine Law [49,54,56]. Then the detected scattering intensity, $S_d$, for the point m can be expressed as follows:

$$S_d=S_{total}f(x,r)$$

(2)

where $S_{total}=\pi I_{\max }^2/4$ and $I_{\max }^2$ is the normal component of the scattering intensity at the point m; $f(x,r)$ is a scaling factor that is a function of x and r. $S_d$ can be also expressed as an integration of the scattering intensity over the viewing angle α, equal to the area of the fan-shape zone m₁-m-m₂. Given the geometric relations shown in Figure 11b, $f(x,r)$ can be derived as follows [54]:

$$\begin{array}{cc}\hfill f(x,r)& =\frac{1}{\pi }\left[\arctan \left(\frac{x+k}{h+r-\sqrt{r^2-x^2}}\right)-\arctan \left(\frac{x-k}{h+r-\sqrt{r^2-x^2}}\right)\right]\hfill \\ & +\frac{1}{2\pi }\sin 2\left[\arcsin \left(\frac{x}{r}\right)+\arctan \left(\frac{x+k}{h+r-\sqrt{r^2-x^2}}\right)\right]\hfill \\ & -\frac{1}{2\pi }\sin 2\left[\arcsin \left(\frac{x}{r}\right)+\arctan \left(\frac{x-k}{h+r-\sqrt{r^2-x^2}}\right)\right]\hfill \end{array}$$

(3)

where k and h are the lens radius and the distance between the lens and the sample surface. Then a correction factor is obtain as follows:

$$C=f(0,r)/f(x,r)$$

(4)

The correction factor dictates that the scattering intensities on the fruit surface are calculated with the same viewing angle as that at the incident point (x = 0). Finally the corrected scattering profile is obtained by multiplying the measured scattered profile with the correction factor.

3.2.3. Feature Extraction

Multispectral and hyperspectral images can be treated as a number of scattering profiles at different wavelengths or spectra at different spatial points. These scattering profiles or spectra can be directly used for model calibration for quality prediction. However, it is cumbersome and also prone to overfitting due to a large amount of spatial and spectral data involved for each sample, especially for hyperspectral scattering images. Effective extraction of the meaningful information is one of the major tasks in analysing scattering images to assess fruit quality attributes.

Several methods have been proposed to perform feature extraction from scattering images. One of the methods is to extract features from a scattering profile to replace the whole scattering profile for subsequent data analysis. Lu [48] calculated two statistic features, mean and standard deviation (SD) of scattering profiles, thus reducing each scattering profile to two parameters. The method is fast and simple to use, but it may be inadequate for effectively characterizing scattering profiles. Peng and Lu [50] noted that the scattering profile could be represented by an appropriate mathematical function. Several distribution functions including exponential, Gaussian, Lorentzian and Gompertz, as summarized in

Table 1. Mathematical functions used for fitting scattering profiles *.

Function	Equation Form **	No. of Parameters
Exponential	$R(x)=\alpha +\beta \times \exp (-\left|x\right|/\delta )$	3
Gaussian	$R(x)=\alpha +\beta \times \exp [-0.5{(x/\delta )}^2]$	3
Lorentzian	$R(x)=\alpha +\frac{\beta }{1+{(x/\delta )}^{\eta }}$	4
	$R(x)=\beta +\frac{1-\alpha }{1+{(x/\delta )}^{\eta }}$	4
	$R(x)=\frac{1}{1+({x/\delta )}^{\eta }}$	2
Gompertz	$R(x)=\alpha +\beta (1-e^{-\exp (\delta -\eta x)})$	4
	$R(x)=1-(1-\alpha )e^{-\exp (\delta -\eta x)}$	3
	$R(x)=1-e^{-\exp (\delta -\eta x)}$	2

* References [25,49,50,51,53]. ** R is the light intensity; x is the scattering distance; α, β, δ and η are function parameters.

, which are commonly used for describing light scattering in optics, can model scattering profiles [25,50,51,52,54]. Parameters in these functions can be used as scattering features to characterize the scattering profiles and each scattering profile is thus reduced to two to four features, resulting in great reductions in the raw data volume. In addition to these distribution functions, a diffusion approximation model that describes the radiation transfer of light within biological tissues, can also be used to extract optical properties from a scattering profile, i.e., the absorption and scattering coefficients [31,57]. Details about the diffusion model and optical properties are further discussed in Section 5. Another method is to apply image processing algorithms for feature extraction. Qing et al. [30,58] proposed a histogram representation technique which extracted the histogram of frequencies of pixels at different intensities from a scattering image for quality prediction.

Estimation of the parameters for the mathematical functions or model-based methods requires non-linear iteration procedures, which are usually computationally intensive and also can result in large estimation errors if the initial guesses for the iteration parameters are not well chosen. Zhu et al. [59] proposed a simpler and faster non-iterative method for analysing hyperspectral images, based on extracting several shape features, including zero-order moment (ZOM) and first-order moment (FOM), from a scattering profile, and confirmed its effectiveness through multivariate calibration for firmness prediction of apples.

For hyperspectral scattering images, in addition to extracting the features of scattering profiles, there is a need for further reduction of spectral information. Principal component analysis (PCA) and minimum noise fraction (MNF) are two commonly used methods to transform a high-dimension dataset into a few informative or effective vectors. Wavelength selection is another way to identify several key wavelengths relevant to prediction of quality attributes. Step-wise multiple linear regression (MLR), while time consuming in searching for the optimal wavelengths, has been utilized to select the best single wavelength and combinations of wavelengths for assessing firmness and SSC of apples [43,54]. Huang and Lu [60] proposed the use of a hierarchical evolutionary algorithm (HEA), a variant of genetic algorithm (GA), coupled with subspace decomposition and partial least squares (PLS) modelling, for fast selection of effective wavelengths. The HEA encoded multiple scattering parameters in a hierarchical manner. Each chromosome in the HEA was composed of an upper control layer and a lower parameter layer; the parameter layer was used to store the scattering profiling parameters and the control layer was used for wavelength selection. By calculating the mean value of each scattering profile, a 2-D hyperspectral image can be reduced to a 1-D mean spectrum, which then enables the implementation of commonly used methods in NIR spectral analysis for wavelength selection. For mean spectra extracted from hyperspectral scattering images, uninformative variable elimination (UVA), affinity propagation and interval partial least squares have been used for wavelength selection for quality prediction [61,62,63,64].

The aforementioned feature extraction techniques are mostly used to extract features from a 1-D scattering profile, without considering the pixel intensity distribution in a 2-D scattering image, which may provide additional information to improve quality prediction. Mendoza et al. [65,66] explored various image processing methods to extract features from hyperspectral scattering images, which were used, coupled with the features from 1-D scattering profiles, for prediction of apple firmness and SSC. The authors extracted image texture features based on grey-level co-occurrence matrix, first-order statistics and Fourier analysis, as well as multi-resolution image features obtained by wavelet transform.

3.2.4. Multivariate Calibration

Multivariate calibration is to build a quantitative model relating extracted scattering and spectral features with quality attributes of target, which is the final step in processing scattering images. The multivariate calibration techniques basically fall into two groups, i.e., linear and nonlinear. Linear techniques, such as MLR, principal component regression (PCR) and PLS, are suited to model a linear relationship and have been widely used for prediction of quality attributes in apples; while non-linear techniques such as neural network (NN) and support vector machine (SVM) hold great potential of fulfilling calibration purposes since they can handle both linear and nonlinear problems. Detailed descriptions of these methods are beyond the scope of this paper, and readers are referred to relevant literature [67,68,69].

3.3. Quality Evaluation for Horticultural Products

3.3.1. Apple

The apple is one of the most valuable fruits around the world. Firmness and SSC are the two important quality attributes that determine the quality of apples and directly affect consumer acceptance and satisfaction. Firmness describes a sensation of texture in the mouth, which is related to the mechanical and structural properties of fruit tissue, while SSC is a measure of an aggregate amount of sugars (i.e., sucrose, glucose, and fructose) and is related to fruit sweetness.

Spectral scattering imaging has been used for assessing firmness and SSC of apple and other fruits. In using multispectral scattering imaging with a mechanical filter wheel of five wavebands (Figure 6d), coupled with neural network modeling, Lu [24] reported the firmness prediction with r = 0.87 and a standard error for prediction (SEP) of 5.8 N for ‘Delicious’ apples. Peng and Lu [50] reported on using a multispectral imaging system with LCTFs (Figure 6c) to acquire scattering images from ‘Delicious’ apples. After comparing the three distribution functions (i.e., exponential, Gaussian, and Lorentzian functions, each with three parameters), they concluded that the Lorentzian function was the best for characterization of scattering profiles. In a further study, Peng and Lu [53] reported that by using a modified four-parameter Lorentzian distribution (MLD) function for characterizing scattering profiles, the LCTF-based multispectral scattering system achieved firmness predictions with r = 0.90 and SEP = 6.4 N for ‘Delicious’, and r = 0.90 and SEP = 6.1 N for ‘Golden Delicious’. Peng and Lu [25] further demonstrated that the modified Gompertz function with three or four parameters performed as well as or better than the modified Lorentzian function, in predicting the firmness of ‘Golden Delicious’. All the aforementioned studies focused on detection of stationary samples. To achieve online real-time detection, Lu and Peng [49] developed a multi-laser based multispectral scattering imaging prototype for firmness prediction of apples (Figure 6b). Four lasers centered at 680, 880, 905, and 940 nm were fused into a single optic fiber, and the backscattering images generated by the fused lasers were then captured from the apples moving on the belt conveyor. Experimental tests showed the firmness prediction of ‘Golden Delicious’ and ‘Delicious’ apples with r = 0.86, demonstrating the feasibility of spectral scattering imaging technique for online fruit quality evaluation. However, the system was still too slow to meet the online speed requirement and there exists a great challenge for proper orientation of each fruit for scattering imaging.

Lu [48] used hyperspectral scattering imaging to predict firmness for apples. Mean and SD spectra were extracted from hyperspectral scattering images, and neural network modelling combined with PCA were used to develop calibration models, which produced the best results with r² = 0.76 and 0.55 and SEP = 6.2 and 6.1 N, for ‘Golden Delicious’ and ‘Delicious’, respectively. Peng and Lu [54] used the MLD function for characterizing hyperspectral scattering profiles and achieved the firmness prediction of apples with r = 0.89 and SEP = 6.1 N. Mendoza et al. [65] investigated the textural features of hyperspectral scattering images for prediction of apple firmness. Integrating textural features and wavelet coefficients of mean spectra, coupled with PLS modelling, was more effective than when they were used alone, achieving the predictions with r = 0.84–0.95 and SEP = 5.9–8.7 N.

In predicting SSC of ‘Delicious’ apples, Lu [24] employed multispectral scattering imaging with three wavelengths at 880, 905, and 940 nm and achieved the prediction of r² = 0.59 and SEP = 0.78 °Brix. In a later study, Lu [48] reported on the use of hyperspectral scattering imaging for apple SSC prediction. Mean and SD spectra were extracted from hyperspectral scattering images for NN modelling, resulting in prediction accuracies with r² = 0.79 and SEP = 0.72 °Brix and r² = 0.64 and SEP = 0.81 °Brix for ‘Golden Delicious’ and ‘Delicious’ apples, respectively. Peng and Lu [54] investigated 10 MLD functions to characterize hyperspectral scattering profiles. The best prediction with r² = 0.78 and SEP = 0.73 °Brix for ‘Golden Delicious’ apples was obtained by using the three-parameter MLD function combined with step-wise MLR for model calibration. Compared to the mean and SD spectra method, the MLD method did not achieve better accuracy, but instead needed considerably more computation. Mendoza et al. [70] reported on using the fused data from hyperspectral scattering and NIRS for SSC prediction. Compared to the data from a single sensor, the fused data led to more accurate and robust results. However, it is technically difficult and more costly to implement the sensor fusion technique for practical use. Later, Mendoza et al. [66] built an online grading system by integrating hyperspectral scattering imaging and visible and near-infrared (VIS/NIR) spectroscopy, for inspecting apples at a speed of 0.5 fruit/s. Three varieties of apples harvested from two harvest seasons, were graded into two classes for SSC by linear discriminant analysis (LDA), with accuracies varying between 62.0% and 91.7%. The fruit grading system, however, is not ready for practical adoption because of lower grading accuracy, slow speed and costly instrumentation.

Multispectral and hyperspectral scattering imaging has shown good performance in predicting fruit firmness, but they generally do not achieve better SSC prediction than conventional NIRS. This is because scattering imaging is primarily designed to enhance the measurement of scattering related properties like firmness, which may not be optimal for SSC assessment. Since NIRS and spectral scattering provide different aspects of information about optical properties of fruit, integrating two techniques could help improve prediction accuracies for both SSC and firmness. Although the concept of using multispectral and hyperspectral scattering imaging for online apple firmness and SSC evaluation has been demonstrated experimentally, considerable improvements are still needed to meet the speed requirement for online sorting and grading.

3.3.2. Other Horticultural and Food Products

Research has also been carried out to evaluate quality attributes of other horticultural products by using hyperspectral scattering imaging. Lu and Peng [43] first reported on the use of hyperspectral scattering imaging to predict firmness of peach fruit. Scattering images were acquired from ‘Red Haven’ and ‘Coral Star’ peaches over the spectral region of 500–1000 nm, and a two-parameter Lorentzian function was used to extract scattering features for firmness prediction. The wavelength of 677 nm, corresponding to chlorophyll absorption, was the most correlated with firmness with r = 0.69 and the best predictions with r² = 0.58–0.77 were obtained by cascading the two function parameters at a combination of several wavelengths. Rady et al. [63] used hyperspectral scattering imaging to predict glucose and sucrose contents of potatoes. They explored three types of datasets including the mean spectra, scattering profile features and their combination, and a number of calibration techniques including PLS and NNs, for model calibration. It was found that the mean spectra provided a more efficient and effective means than the features of scattering profiles, and good predictions for glucose were obtained by PLS with the best residual predictive deviation (RPD) of about 3.6, while poor sucrose predictions were obtained with RPDs of less than 1.2. Using mean spectra extracted from hyperspectral scattering images, Pan et al. [64] built PLS models to predict the sucrose, soluble solids, and moisture content of sugar beet, with correlation coefficients of 0.88, 0.83 and 0.76, respectively.

Romano et al. [44,71,72] initiated a study on monitoring the changes of moisture content, color, SSC and hardness of food products during the drying process by using laser light backscattering imaging. In a preliminary study on monitoring the moisture content of banana slices [44], the authors observed a significant relationship between changes in moisture content and backscattering area that was produced with laser diode emitting at 670 nm. Baranyai and Zude [73,74] developed a laser-light backscattering image (LLBI) technique, operating with two laser modules at 660 nm and 785 nm, to investigate light propagation in fruit tissues. Later, Hashim et al. [75] explored the technique for detecting chilling injuries in bananas by analyzing scattering profiles that were fitted with the MLD function, with the resultant detection errors of 0.67%–8%. In two separate studies, Lorente et al. [46,76] further explored LLBI with five laser modules at different wavelengths (532, 660, 785, 830 and 1060 nm), for detecting early decay in citrus fruit. The authors used the Gaussian-Lorentz cross product function with five parameters to characterize scattering profiles, and LDA combined with different features selection methods to build classification models, yielding the best overall detection accuracy of 93.4% [76].

4. Integrated Reflectance and Transmittance Imaging

Light reflectance and transmittance are two common sensing modes in VIS/NIR spectroscopy and hyperspectral imaging. The former measures the light that is diffusely reflected primarily from the upper or superficial region of a sample [4,77], while the latter measures the light that passes through a sample, which allows for detecting internal quality or defects in foods [78,79,80,81]. Most research on hyperspectral imaging has focused on the reflectance measurement, which is appropriate for evaluating surface or subsurface quality characteristics but not for internal defects in food products. Our group proposed a new concept of integrating reflectance and transmittance measurements in one hyperspectral imaging system for detecting both external quality attributes (e.g., color and size) and internal defects [28,82,83,84,85,86,87,88]. The concept is based on the fact that the visible light in the region of 400–675 nm, which is relatively poor in penetrating biological tissues, is more suitable for assessing surface features of the product, while the red and NIR light in the region of 675–1000 nm has better penetration capabilities and can thus be used for internal quality assessment in transmittance mode. Since most CCD-based hyperspectral imaging systems cover the visible and NIR region of 400–1000 nm, it is technically feasible and also advantageous to acquire hyperspectral reflectance (400–675 nm) and transmittance (675–1000 nm) images simultaneously using one imaging system. The following sections give an overview of the instrumental configuration of the integrated hyperspectral reflectance and transmittance imaging system and its applications for quality assessment of pickling cucumbers.

4.1. Instrumentation

A laboratory online hyperspectral imaging prototype was developed originally for quality evaluation of pickling cucumbers. It can operate in three different imaging modes, including single reflectance (400–1000 nm) and transmittance (400–1000 nm) modes and the integrated reflectance (400–675 nm) and transmittance (675–1000 nm) mode. As illustrated schematically in Figure 12, the system consists of three basic units, i.e., illumination, imaging and conveying [82].

The hyperspectral imaging unit is composed of a high-performance 12-bit CCD camera, a line-scan imaging spectrograph attached to the camera and a focusing lens connected to the front side of the spectrograph. The integrated reflectance and transmittance imaging mode is realized through the specially designed illumination and conveying units, plus an additional in-line light correction unit. The illumination unit uses two light sources. One 150-W QTH lamp light source with a DC-regulated power supply connected to a dual fiber optic line light is used for reflectance measurement. When the system operates in the integrated mode, a low-pass blocking filter with the cutting wavelength of 675 nm is placed between the reflectance lamp and optic fibers (Figure 12b). The other light source is a 410-W QHT lamp used for transmittance measurement (a long-pass filter is used for blocking the light below a specified wavelength or 675 nm), and the lamp housed in a custom-made enclosure is placed beneath the convey belts (Figure 12). It is noted that a blocking filter at other wavelengths can be also used to lend measuring flexibility to the system. The transmittance light is focused onto the bottom part of the sample between the two belts, and the optical axis of the transmittance light is slightly angled with respect to the vertical axis (Figure 12a), to avoid direct light from entering the camera when no sample is present in the scene.

The conveying unit consists of two lanes, operating with the same imaging unit, and each of the lane is constructed with two round belts driven by pulleys. The distance between the two belts can be adjusted to accommodate different sizes of commodities being detected, and the two-belt configuration enables the transmittance measurement.

The prototype has a built-in unit for continuous in-line light correction. During imaging, the output of the light sources can fluctuate due to the lamp aging and the variation of environmental factors. Correction for the output change is thus needed to ensure robust, consistent performance of the imaging prototype. As shown in Figure 13, three reference discs, two of them outside the lanes for correction of the transmittance image and the other placed between the lanes for reflectance image correction, are mounted onto the frame of the conveying unit, aligned with the scanning line of the imaging spectrograph. The reflectance reference, made of PVC, is illuminated by the optic fiber line lights above; while the two transmittance references, made of Teflon, are illuminated by the corresponding transmittance light source below (Figure 12). The three references are imaged at the same time with the sample for each scan line. Flat-field corrections are then performed to eliminate the effect of light source fluctuation, which will be further discussed in the following section.

4.2. Image Acquistion and Pre-Processing

The hyperspectral imaging prototype performs line scanning for acquiring reflectance and transmittance images simultaneously. The prototype is set up to accommodate both the needs for scanning speed and spatial resolution. Higher speed is desirable in practical applications, but it will sacrifice spatial resolution, possibly leaving undetected some small defects; while higher spatial resolution, on the other hand, can increase the workload of image acquisition and also result in an unmanageable data volume. For instance, in experiments for quality inspection of pickling cucumbers, the conveyor belt speed was set to 110 mm/s, and the camera was operated at an exposure time of 50 ms and a delay time of 50 ms [82]. As such, the prototype ran at a speed of 1–2 product items per second per lane, scanning a portion of each sample (about half of the surface area of the sample with the given exposure and delay times). This is acceptable, in terms of spatial resolution, for evaluation of color and size, whose variations within the scanning width are relatively small. Besides, fewer images are acquired for each sample, thus expediting the image processing.

Figure 14a shows typical 2-D line scanning images acquired by the prototype with and without samples (i.e., cucumbers) present on the convey belts. The leftmost and rightmost parts of the images represent the two transmittance references and the middle the reflectance reference. Each horizontal and vertical lines in the images represent a spatial profile across the scan line at a specific wavelength and a full spectrum at a given location in the scan line respectively. Figure 14b show examples of the spatial profiles at the wavelength of 623 nm, corresponding to the two horizontal dashed lines in Figure 14a. The spatial profile without samples, compared to the one with samples, is characterized by high intensity peaks resulting from the conveyor belts.

The acquired raw hyperspectral image first needs to be split into two separate images corresponding to each lane. If the split images is ‘blank’, indicating no sample present in the scene, it will be discarded from analysis; otherwise, the image is ‘non-blank’, containing sample information, and is kept. Consecutive ‘non-blank’ images between ‘blank’ images are identifying as coming from the same sample [82]. The split images are then processed for in-line spectral correction by dividing the spectra of references. Figure 14c shows the integrated reflectance and transmittance spectrum of the references and a sample spectrum, which correspond to the vertical dashed lines in the left lanes in Figure 14a. One notes that the integrated reference spectrum and the sample spectrum are divided into two parts for reflectance and transmittance, respectively, at the wavelength of 675 nm, which is the cut-off wavelength of the blocking filter installed in front of the reflectance illumination source. The spectral correction is performed line by line according to the equation as follows [82]:

$$I_c=\frac{I_s-I_d}{I_r-I_d}$$

(5)

where $I_c$ is the corrected sample spectrum, $I_s$ is the raw sample spectrum, $I_r$ is the integrated spectrum of the two reflectance and transmittance references and $I_d$ is the dark-current spectrum.

After completion of the reference-based in-line light calibration, further corrections for the effect of the fruit size on the transmittance measurement are needed. Since light attenuation within tissues is positively correlated with the optical pathlength, hence transmittance is affected by the geometry or diameter of the sample being detected. The correction for transmittance can be made, assuming the sample has a circular cross-section shape in the direction of light transmitting, using the following equation:

$$T_c=T_r\times {\left(d/\overline{d}\right)}^s$$

(6)

where $T_c$ is the diameter-corrected transmittance, $T_r$ is the uncorrected transmittance obtained from the in-line light calibration, $d$ is the diameter of a sample, $\overline{d}$ is the mean diameter of all tested samples and s is adjustable scaling factor. A typical s value is 1.3 for cucumbers [86]. However, no diameter correction is needed for the reflectance measurement.

4.3. Quality Evaluation for Pickling Cucumbers and Pickles

In 2015, pickling cucumber production in the U.S. was valued at nearly $173 million; the state of Michigan accounted for 32% of the total pickling cucumber tonnage [89]. Pickling cucumbers are primarily used for processing or pickling, and they are short and blocky, with thin skins and usually a color gradient from dark green at the stem end to light green at the blossom end [90]. Internal defects often occur in pickling cucumbers in the form of soft or watery carpel tissue, loose seeds and split or hollow center, due to physiological disorders and mechanical injuries [91], which downgrade the quality of processed products and cause economic loss for the pickling industry. Inspection and sorting of pickling cucumbers and pickles is needed for internal defects, together with external quality such as size, color and surface defects. The hyperspectral reflectance and transmittance imaging prototype was tested and evaluated for detection of internal defects and prediction of firmness and color of ‘Journey’ pickling cucumbers [28]. Mean and SD spectra were calculated from the spectra of all the pixels in the scan line, and used for model calibration. As shown in Figure 15, the reflectance spectra of normal and defective cucumbers, in the wavelength region of 500–675 nm, were strongly overlapped; while the transmittance of the defective cucumbers was consistently higher than that of normal cucumbers in the region of 675–1000 nm, suggesting the possibility of segregating defective cucumbers from normal ones by using the transmittance spectra. The difference in transmittance between normal and defective cucumbers is mainly attributed to the formation of water-soaked lesion, the tissue separation, or even the presence of hollow center in the defective cucumbers, which decrease the light scattering ability and thus allow more light to pass through. The use of the transmittance spectra combined with PLS discriminant analysis, achieved accuracies up to 99% for the detection of internal defects. The reflectance in the visible region, modelled by PLS, gave better predictions for skin color, with r² up to 0.76 and 0.75 for skin chroma and hue, respectively.

Ariana and Lu [83] further investigated PCA for processing hyperspectral images for detection of internal defects in desalted whole pickles. The second principal component (PC) score images exhibited the strongest contrast between normal and defective pickles, which achieved an overall classification accuracy of 86% by a simple global thresholding method. The efficacy of the PC score images, however, remained to be validated through supervised classifiers. Identifying effective wavelengths is critical for implementation in a multispectral imaging mode, which greatly reduces the time for image acquisition and can also improves the detection performance. Ariana and Lu [84] acquired hyperspectral reflectance (500–740 nm) and transmittance (740–1000 nm) images from normal and defective cucumbers and pickles (Figure 16). A branch and bound algorithm combined with the k-nearest neighbor (K-NN) classifier was utilized for wavelength selection. Two four-wavelength sets of 745, 805, 965 and 985 nm, and 745, 765, 885 and 965 nm were obtained, both of which were located in the NIR region and achieved the best detection rates of 94.7% and 82.9% for cucumbers and pickles, respectively. The accuracies were comparable to those obtained by using the entire set of wavelengths combined with PLS discriminant analysis. The authors also observed that a spectral resolution between 20 and 40 nm was sufficient for defect detection of cucumber and pickles, which allowed using bandpass filters with a relatively large bandwidth for implementing these wavelengths in a multispectral imaging system.

In another study using LEDs replacing broadband QTH lamps as illumination sources, Cen et al. [87] acquired hyperspectral reflectance images (400–700 nm) by using two white LEDs, and transmittance images (700–1000 nm) by using four 830-nm and four 890-nm infrared LEDs, from ‘Journey’ pickling cucumbers at two conveyor speeds of 85 and 165 mm/s. Reflectance images below 700 nm carried surface or color features information, while transmittance images above 700 nm contained information about the internal features of the cucumbers (Figure 17). Minimum redundancy-maximum relevance (MRMR) was used to identify the best five two-band ratios in the spectral region of 700–950 nm. Discriminant analysis based on the Mahalanobis distance was used to develop classification models, and the two-band ratio of 887/837 nm gave the best accuracies of 95.1% and 94.2% at the conveyor speeds of 85 and 165 mm/s, respectively, for a two-class classification scheme. The classification performance, however, deteriorated when one more class was added, mainly due to the difficulty differentiating between slightly and severely defective classes.

Pest/insect infestation can pose a serious quality and safety problem for the pickle processor, as many countries have zero tolerance for importing pest-infested pickled products. The infestation is generally hidden in the product and cannot be easily detected from outside. Lu and Ariana [86] reported on the detection of fruit fly infestation in pickling cucumbers (Figure 18) using hyperspectral reflectance and transmittance imaging. Three sensing modes, reflectance (450–740 nm), transmittance (740–1000 nm) and their combination were compared for differentiating infested cucumbers from normal ones by PLS discriminant analysis. The transmittance mode achieved the overall accuracies of 88%–93%, which were superior to the reflectance mode, while integration of the two sensing modes did not result in better detection accuracy compared to transmittance alone. Nevertheless, all the three sensing modes performed better than manual inspection.

Chilling injury is a physiological disorder that occurs when fresh products are exposed to low or chilling temperatures during postharvest handling. Cucumbers are susceptible to chilling injury, which is characterized by surface lesions, pitting, internal discoloration and decay. Cen et al. [88] used the hyperspectral imaging prototype for chilling injury detection of ‘Excursion’ pickling cucumbers. Affected cucumbers exhibited increased transmittance in the spectral region of 750–930 nm as chilling injury progressed, due to the occurrence of tissue collapse, translucent tissue and water-soaked spots. The authors explored different waveband selection methods and supervised classifiers, based on either spectral or image texture features, for chilling injury detection, and achieved the best overall detection accuracies of more than 90%.

5. Spatially-Resolved Spectroscopic Technique for Optical Property Measurement

Optical properties, according to the radiation transfer theory, mainly refer to two fundamental parameters, i.e., absorption (${\mu }_{\mathrm{a}}$) and scattering (${\mu }_{\mathrm{s}}$) or reduced scattering (${\mu }_{\mathrm{s}}^{\prime }$, a lumped parameter incorporating the anisotropy (g) and the scattering coefficient). The two parameters, along with the refractive index, largely determine how light transfers or propagates in turbid or diffusive media. The absorption coefficient measures the amount of the light being converted into other forms of energy, which is related to the chemical composition of the product. The scattering or reduced scattering coefficient, on the other hand, represents the probability of photons being scattered during travelling, which is closely associated with the cellular structures and density [92]. Hence, measurement of the optical properties can help us better understand light interactions with agro-food products and also provide an effective means for better evaluation of the physiological condition and quality attributes.

The optical absorption and scattering properties can be determined by quantifying the diffuse light remitted from a turbid medium in the spatially- or time-resolved domain (or frequency domain, which may be viewed as a special case for the time-resolved domain, or vice versa), and then using inverse algorithm for an appropriate light transfer model (e.g., diffusion approximation model) to fit the measured diffuse reflectance data. Because absorption and scattering are interwoven during light transfer in turbid media, fast and accurate measurement of the absorption and scattering coefficients still remains a challenging task. Over the years, researchers in the field of biomedical optics have developed several techniques for measuring the optical properties of biological tissues, which, according to their measurement principles, can be categorized into: time-resolved [93,94], frequency domain [95,96], spatially-resolved [56,97], and spatial frequency domain [98,99]. While time-resolved and frequency domain techniques are able to interrogate the tissues at a greater depth, they generally require more expensive and sophisticated instrumentation and have fewer wavelength selection options. On the other hand, spatially-resolved and frequency domain techniques are simpler in instrumentation and easier to implement with a broader spectral range; however, the measurement is usually confined to the superficial region of tissues. While these techniques provide a new approach for optical characterization of food and agricultural products [93,100,101], they are still short of meeting the expectations for food quality evaluation, because of the difficulty or inconvenience in carrying out measurements, sophistication in instrumentation, slow measurement speed and lack of robustness or measurement accuracy.

In recent years, our group has been working on a new technique based on the spatially-resolved principle and hyperspectral imaging for measuring optical properties of horticultural products [55,102,103,104]. The use of hyperspectral imaging technique provides a fast and more effective means for estimating the absorption and reduced scattering coefficients of horticultural and food products over a wide spectral range. In the following, we provide an overview on the development and application of this hyperspectral imaging-based spatially-resolved spectroscopic technique for optical property measurement and quality evaluation of horticultural products.

5.1. Principle and Mathematical Theory

Light propagation in turbid media can be described by the Boltzmann transport equation [105], and solutions to the transport equation for the light fluence rate can be an intractable problem. However, by assuming that scattering is dominant in the medium (i.e., ${\mu }_{\mathrm{s}}^{\prime }\gg {\mu }_{\mathrm{a}}$) and that the light source is isotropic, we can reduce the Boltzmann transport equation to a second-order partial differential equation [106]:

$$D{\nabla }^2\varphi (r,t)-\frac{1}{c}\frac{\partial \varphi (r,t)}{\partial t}-{\mu }_{\mathrm{a}}\varphi (r,t)=-S(r,t)$$

(7)

where $D=\frac{1}{3({\mu }_{\mathrm{a}}+{\mu }_{\mathrm{s}}^{\prime })}$ is the diffusion coefficient, ${\nabla }^2$ is the Laplace operator, c is the speed of light travelling in the medium, r is the distance from the light source (Figure 19), t is the time, $\varphi$ is the fluence rate and S is the spherical harmonic representing the isotropic source. At the steady-state condition, Equation (7) can be simplified as follows:

$${\nabla }^2\varphi (r,t)-\frac{{\mu }_{\mathrm{a}}}{D}\varphi (r,t)=-\frac{S(r,t)}{D}$$

(8)

This steady-state diffusion approximation equation provides a theoretical basis for different spatially-resolved or spatial frequency domain techniques.

For the homogeneous semi-infinite turbid medium, an analytical solution can be obtained by using an extrapolated boundary condition (i.e., the fluence rate is forced to zero at a height $z_b$ above the medium surface) and a photon dipole source [97]:

$$\varphi (r,z,z_0)=\frac{1}{4\pi D}\left[\frac{\exp (-{\mu }_{\mathrm{eff}}{r}_1)}{r_1}-\frac{\exp (-{\mu }_{\mathrm{eff}}{r}_2)}{r_2}\right]$$

(9)

where,

$${\mu }_{\mathrm{eff}}={\left[3{\mu }_{\mathrm{a}}({\mu }_{\mathrm{a}}+{\mu }_{\mathrm{s}}^{\prime })\right]}^{1/2}$$

(10)

$$r_1={\left[{(z-z_0)}^2+r^2\right]}^{1/2}$$

(11)

$$r_2={\left[{(z+z_0+2z_b)}^2+r^2\right]}^{1/2}$$

(12)

and z is the depth (Figure 19), $z_b=2AD$ where A is a constant that can be determined by an empirical approach [107]. Then the Green’s function for the diffuse reflectance can be calculated as the gradient of the fluence rate (i.e., flux):

$$\begin{array}{cc}\hfill R_{\mathrm{flux}}(r,z_0)& =-D\nabla \varphi (r,z,z_0){|}_{z=0}\hfill \\ & =\frac{1}{4\pi }\left[z_0({\mu }_{\mathrm{eff}}+\frac{1}{r_1})\frac{\exp (-{\mu }_{\mathrm{eff}}{r}_1)}{r_1^2}+(z_0+2z_b)({\mu }_{\mathrm{eff}}+\frac{1}{r_2})\frac{\exp (-{\mu }_{\mathrm{eff}}{r}_2)}{r_2^2}\right]\hfill \end{array}$$

(13)

where r₁ and r₂ are given by Equations (11) and (12), respectively, with z set to 0 (i.e., at the tissue surface). From Equation (13), it can be seen that the flux depends only on the distance and the two parameters ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$. Finally, the radically dependent diffuse reflectance for a single scatter source is given by integrating the Green’s function as follows [97]:

$$\begin{array}{cc}\hfill R(r)& ={\displaystyle \underset{0}{\overset{\infty }{\int }}{R}_{\mathrm{flux}}}(r,z_0)a^{\prime }\delta (z_0-\frac{1}{{\mu }_t^{\prime }})d{z}_0\hfill \\ & =a^{\prime }{R}_{\mathrm{flux}}(r,z_0){|}_{z_0=1/{\mu }_t^{\prime }}\hfill \end{array}$$

(14)

where $a^{\prime }={\mu }_{\mathrm{s}}^{\prime }/({\mu }_{\mathrm{a}}+{\mu }_{\mathrm{s}}^{\prime })$ is the transport albedo, ${\mu }_t^{\prime }={\mu }_{\mathrm{a}}+{\mu }_{\mathrm{s}}^{\prime }$ is the total attenuation coefficient and $\delta$ is the Dirac Delta function.

Later, Kienle and Patterson [108] proposed an improved solution by calculating the reflectance from the integral of the radiance over the backward hemisphere rather than from the flux alone, which is expressed as follows:

$$\tilde{R}(r)=c_1\varphi (r,z_0=1/{\mu }_t^{\prime },z=0)+c_2{R}_{\mathrm{flux}}(r,z_0=1/{\mu }_t^{\prime })$$

(15)

where $\tilde{R}(r)$ is the improved solution for diffuse reflectance; $\varphi (r,z=0,z_0=1/{\mu }_t^{\prime })$ and $R_{\mathrm{flux}}(r,z_0=1/{\mu }_t^{\prime })$ are given by Equations (9) and (13), respectively; $c_1$ and $c_2$ are constants, which are given as follows:

$$c_1=\frac{1}{4\pi }{\displaystyle \underset{2\pi }{\int }[1-R_{\mathrm{fres}}(\theta )]\cos (\theta )d\mathsf{\Omega }}$$

(16)

$$c_2=\frac{3}{4\pi }{\displaystyle \underset{2\pi }{\int }[1-R_{\mathrm{fres}}(\theta )]{\cos }^2(\theta )d\mathsf{\Omega }}$$

(17)

In Equations (16) and (17), $R_{\mathrm{fres}}(\theta )$ is the Fresnel reflection coefficient for a photon with an incident angle $\theta$ relative to the normal to the medium boundary and $\mathsf{\Omega }$ is the solid angle. A detailed expression of $R_{\mathrm{fres}}(\theta )$ can be found in Haskell et al. [106]. For biological materials with a typical refractive index $n=1.35$, $c_1$ and $c_2$ are equal to 0.1277 and 0.3269, respectively [109].

Spatially-resolved technique is based on the solution for diffuse reflectance, and is conceptually illustrated in Figure 19. As a narrow collimated light beam perpendicularly impinges onto a semi-infinite turbid medium, the light will scatter in the medium and a portion of the scattered light will remit from the medium. By measuring the remitted light at different distances from the incident point, a spatially-resolved profile of diffuse reflectance is obtained, which is then fitted by the diffusion model in Equation (14) or (15) to obtain the estimates of the optical parameters ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$.

Two types of sensing configurations for spatially-resolved technique have been developed, i.e., contact and non-contact sensing. The first uses either a single detection fiber or an array of detecting fibers, connected to a single spectrometer or an imaging spectrometer, for measuring diffuse reflectance at different positions [100,110,111,112]. This configuration requires good contact between the fiber and the sample being detected, which can be problematic for measuring horticultural products due to speed and safety concerns as well as the difficulty of maintaining good contact between the probe and food sample whose surface is often not flat or of irregular shape. The second uses an imaging device such as a CCD camera to capture diffuse reflectance images [56,113], which is better suitable for food products because no contact is needed. The spatially-resolved spectroscopic technique develop in our lab belongs to the latter, and compared to previously reported non-contact techniques, it is capable of efficiently measuring the optical properties over a broader spectral range.

5.2. Algorithms and Instrumentation

5.2.1. Inverse Algorithms and Instrumental Design Parameters

Accurate measurement of optical properties depends on the implementation of inverse algorithms and the instrumental design. A proper inverse algorithm needs to be developed and optimized for parameter estimation for the diffusion model. The inverse algorithm here refers to estimating the absorption and scattering coefficients from the measured diffuse reflectance by means of the curve fitting of diffusion model, using a non-linear least squares optimization procedure. Non-linear least squares algorithms begin with initial guesses of model parameters, proceed iteratively by using searching methods for solution updating, and finally converge to the best estimates that minimize the sum-of-squares (SS) of the difference between the true and predicted values. Gauss-Newton and Levenberg-Marquardt algorithms are often used for solving non-linear least squares problems [114], which can be readily implemented in commercial software like Matlab (The Mathworks, Inc., Natick, MA, USA) using its built-in functions. In implementing the inverse algorithm for estimating ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, Qin and Lu [55] proposed a three-step curve fitting procedure. First, both ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ were treated as unknown parameters, and their estimates were obtained by fitting the diffusion model. Second, the ${\mu }_{\mathrm{s}}^{\prime }$ values were updated by fitting the ${\mu }_{\mathrm{s}}^{\prime }$ values obtained in the first step with the function${\mu }_{\mathrm{s}}^{\prime }=a{\mathsf{\lambda }}^{-b}$, where a and b are parameters and λ is the wavelength [115]. Last, the ${\mu }_{\mathrm{s}}^{\prime }$ values from the second step were inserted into the diffusion model, and the curve fitting procedure was repeated to obtain the estimates of ${\mu }_{\mathrm{a}}$ values with ${\mu }_{\mathrm{s}}^{\prime }$ being known. The three-step procedure, compared to one-step curve fitting, can reduce the fitting noise for the two parameters and thus improve the fitting accuracy and stability. Cen et al. [104] further explored the effects of the data transformation and weighting methods on parameter estimation. Several methods, including logarithm and integral transformation and relative weighting, were used to pre-process the diffuse reflectance profiles generated by Monte Carlo (MC) simulations based on a total of 29 combinations of ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ that are typical for food and horticultural products. It was found that these pre-processing methods greatly reduced the estimation errors of ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, compared to those obtained from the original diffuse reflectance, and that the logarithm transformation achieved the best results with the average estimation errors of 10.4% and 6.6% for ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, respectively. The authors also reported that ${\mu }_{\mathrm{s}}^{\prime }$ can be estimated more accurately than ${\mu }_{\mathrm{a}}$ based on sensitivity analysis for the two parameters.

The diffusion models are derived based on the use of an infinitely small pencil beam incident normally on the semi-infinite turbid medium and are only valid when the source-detector distance exceeds a certain limit [97]. Hence, optimization of the instrumental system should consider the effects of light beam including size and shape, and source-detector distance including minimum and maximum source-detector distances, so as to achieve accurate measurement of optical properties. Cen and Lu [104] examined these factors in the development of the hyperspectral imaging-based spatially-resolved technique for optical property measurement. Three types of light beams, i.e., infinitely small, Gaussian and circular flat, were investigated based on MC simulations. The results showed that the error produced by the finite beam with the beam size less than 0.5 mm, compared to the infinitely small beam, was less than 1%, but it increased linearly with the beam size larger than 0.5 mm. In general, a 1-mm light beam introduces about 5% error for estimating ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ compared to the infinitely small beam. Hence the size of a light beam in the system should be less than 1 mm in order to control the error within 5%. Laboratory tests were performed to examine the actual light beam.

Further, MC simulations showed that the optimal minimum source-detector distance depended on the values of ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, and it ranged from 0.5 to 2 mm, corresponding to 1 to 4 transport mean free paths [${\mathrm{mfp}}^{\prime }$, ${\text{1mpf}}^{\prime }=({\mu }_{\mathrm{a}}+{\mu }_{\mathrm{s}}^{\prime }{)}^{-1}$] [104]. After consideration of the different combinations of ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, the optimal minimum source-detector distance was recommended to be 1.5 mm and the optimal maximum source-detector distance to be equivalent to 10–20 ${\mathrm{mfp}}^{\prime }$ or determined by the minimum SNR.

5.2.2. Instrument Development

A bench-top prototype, named Optical Property Analyzer (OPA) (Figure 20), has been developed for optical property measurement by incorporating the inverse algorithms and optimal design parameters. The OPA has two light sources; one is used for optical property measurement and the other is the line light for general hyperspectral imaging applications. The following discussion is only focused on its optical property measurement functions.

The hardware of the OPA, as shown in Figure 20a, consist of three major parts: imaging, illumination, and sample positioning. The imaging unit is composed of a high performance electron-multiplying CCD camera (Luca R604, Andor Technology, South Windsor, CT, USA), a line-scan imaging spectrograph (ImSpector V10E, Spectral Imaging Ltd., Oulu, Finland) covering the spectral range of 400–1000 nm, and a prime lens. The illumination unit uses a 20 W broadband QTH lamp as a light source, which is connected to a DC-regulated controller. The broadband light is delivered through an optical fiber that is coupled with a specially-designed focusing lens to generate a 1-mm diameter beam incident upon the sample at 15° from the vertical axis, with 1.5 mm offset distance between the incident point and the scanning line. The sample positioning unit includes a motorized horizontal stage, a manually adjustable vertical stage, and a sample holder for positioning samples to a specified position. During measurement, a sample is first moved to the specified height via the vertical stage, and it then starts to move horizontally, in synchronization with the imaging system.

The software of the OPA is developed using Microsoft Visual C#, providing integrated functions for system control, image acquisition, real-time data processing and on-screen output display. Figure 21 shows the windows of parameter setup for image acquisition and optical property computation.

The software offers two options of diffusion models, i.e., the single-layer Farrell model (Equation (14)) and single-layer Kienle model (Equation (15)), to calculate the spectra of ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ (The two-layer model, shown on the screen of Figure 21b, has not been activated). Prior to the computation of optical properties, several options for pre-processing raw spatially-resolved diffuse reflectance profiles (spatial profiles for short) are available in the software. Because of the symmetry of the spatial profiles with respect to the light incident point (Figure 22c), the spatial profiles from the left and right parts are usually averaged to enhance the SNR of the data. Furthermore, smoothing and correction for fruit geometry and non-uniform instrument responses can also be implemented. As shown in Figure 21b, once the parameters for pre-processing of spatial profiles, diffusion model, input and output options are all selected, the program will calculate the absorption and scattering spectra automatically. To improve the repeatability of measurements, the system would take 19 scans (the default setting) from each sample and the final optical property spectra are the average over the 19 measurements.

Cen and Lu [104] evaluated the performance of the OPA in terms of accuracy, precision and sensitivity by using liquid model samples made up with different dyes and fat emulsion, whose optical properties were determined by three reference methods (i.e., transmittance, integrating sphere and empirical equation). The average estimated errors were 23% for ${\mu }_{\mathrm{a}}$ over the spectral range of 530–850 nm and 7% for ${\mu }_{\mathrm{s}}^{\prime }$ over the spectral range of 500–900 nm, when the OPA was evaluated against the transmittance and integrating sphere methods for ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, respectively. The precision of the OPA was assessed by the coefficient of variation (CV) in the absorption peak at 555 nm, which was less than 10% and 4% for ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, respectively; and the minimum detectable value of ${\mu }_{\mathrm{a}}$ was 0.0082 cm⁻¹.

5.3. Quality Evaluation of Horticultural Products

The OPA was used for measuring the absorption and reduced scattering coefficients for a variety of horticultural products and for quality evaluation.

Figure 23 shows the absorption and scattering coefficient spectra for five fruit and vegetable samples. The ${\mu }_{\mathrm{a}}$ spectra of all the samples except the tomato have absorption peaks at 675 nm, which is associated with chlorophyll absorption, and the ${\mu }_{\mathrm{a}}$ values at this wavelength range from 0.10 to 0.48 cm⁻¹. The ‘Granny Smith’ apple has the highest chlorophyll absorption due to its greenish skin and flesh, while the ripened tomato does not exhibit the absorption peak, because chlorophyll in fully ripened tomatoes decreases dramatically or even disappears completely. In fully ripened tomatoes, anthocyanin becomes the dominant pigment, thus causing an absorption peak at 535 nm. The ${\mu }_{\mathrm{a}}$ values of these samples are relatively small and consistent over the spectral range of 720–900 nm, and they increase dramatically above 900 nm and peak at 970 nm due to the water absorption. The ${\mu }_{\mathrm{s}}^{\prime }$ spectra of these samples are generally more than 10 times greater in magnitude than the ${\mu }_{\mathrm{a}}$ spectra due to the dominant scattering effect, and they exhibit a steady downward trend as the wavelength increases, which is in agreement with the Mie scattering theory and other reported studies [116,117]. Apple samples have higher ${\mu }_{\mathrm{s}}^{\prime }$ values (9.0–17.0 cm⁻¹) over the entire spectral region of 500–1000 nm, while the tomato has the lowest ${\mu }_{\mathrm{s}}^{\prime }$ values (4.5–6.0 cm⁻¹). The ability of a scattering particles to scatter light depends on their density and size, according to the equation ${\mu }_{\mathrm{s}}^{\prime }=a{\mathsf{\lambda }}^{-b}$ where a is proportional to the density of the scattering particles and b depends on the particle size [110,115]. Hence, knowledge of ${\mu }_{\mathrm{s}}^{\prime }$ and its pattern of change with wavelength can provide useful information about the structural and physical characteristics of these samples, such as firmness in fruit.

In measuring the ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ of normal and bruised apples, Lu et al. [118] reported the different effects of bruising on the two parameters. While no consistent trend of changes in the ${\mu }_{\mathrm{a}}$ spectra was observed between bruised and normal apples, the values of ${\mu }_{\mathrm{s}}^{\prime }$ for normal apples were considerably larger than those for bruised apples, and they decreased consistently over time for a period of up to 23 days after the apples had been bruised. In a later study, Lu et al. [91] investigated the effect of mechanical damage on pickling cucumbers. As shown in Figure 24, one day after the damage, ${\mu }_{\mathrm{a}}$ increased significantly at 700–920 nm, whereas an opposite pattern of changes was found for ${\mu }_{\mathrm{s}}^{\prime }$. Mechanical damage had a greater impact on ${\mu }_{\mathrm{s}}^{\prime }$ than ${\mu }_{\mathrm{a}}$ in terms of absolute value. These two studies suggest that the optical properties, especially ${\mu }_{\mathrm{s}}^{\prime }$, are useful for evaluating horticultural products for defects due to mechanical damage. Effective defect detection can be achieved by means of enhancing scattering feature measurements. The concept was used in laser scattering detection of internal defect of pickling cucumbers [29].

Qin et al. [57] reported on the use of ${\mu }_{\mathrm{s}}^{\prime }$ and ${\mu }_{\mathrm{a}}$ spectra for predicting firmness and SSC of ‘Golden Delicious’ apples. Calibration models were developed for ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ using PLS, and for their combination using stepwise MLR. The results showed that both ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ were correlated with firmness and SSC, but ${\mu }_{\mathrm{a}}$ achieved higher correlations than ${\mu }_{\mathrm{s}}^{\prime }$. While ${\mu }_{\mathrm{s}}^{\prime }$ was assumed to be more related to firmness, it was not as effective as ${\mu }_{\mathrm{a}}$ for firmness prediction, which may be explained by the fact that the physiological process of the fruit is often accompanied with changes in chemical components. The combined data were found to be the most effective for firmness and SSC predictions with r = 0.86 and 0.75, respectively. Cen et al. [31] further investigated other types of the combined data for predicting apple firmness and SSC. Three combined data were formed, including ${\mu }_{\mathrm{a}}\&{\mu }_{\mathrm{s}}^{\prime }$, ${\mu }_{\mathrm{a}}\times {\mu }_{\mathrm{s}}^{\prime }$ and ${\mu }_{\mathrm{eff}}$, where ${\mu }_{\mathrm{a}}\&{\mu }_{\mathrm{s}}^{\prime }$ refers to the simple cascading of the ${\mu }_{\mathrm{s}}^{\prime }$ and ${\mu }_{\mathrm{a}}$ spectra into one spectrum, ${\mu }_{\mathrm{a}}\times {\mu }_{\mathrm{s}}^{\prime }$ is the multiplication of the ${\mu }_{\mathrm{s}}^{\prime }$ and ${\mu }_{\mathrm{a}}$ wavelength by wavelength, and ${\mu }_{\mathrm{eff}}={[3{\mu }_{\mathrm{a}}({\mu }_{\mathrm{a}}+{\mu }_{\mathrm{s}})]}^{1/2}$ is the effective attenuation coefficient characterizing the light penetration ability. A pooled set of apples freshly harvested and after storage were collected, and PLS was used for model calibration. The authors also found that ${\mu }_{\mathrm{a}}$ was more effective for firmness and SSC predictions than ${\mu }_{\mathrm{s}}^{\prime }$, and that the combined data resulted in improved predictions in most cases, compared to individual ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$. Figure 25 shows the best prediction results for firmness and SSC using the combined data. The ${\mu }_{\mathrm{a}}\times {\mu }_{\mathrm{s}}^{\prime }$ gave the best predictions of firmness and SSC for ‘Golden Delicious’ apples with r = 0.89 and 0.78, respectively, while for ‘Delicious’, ${\mu }_{\mathrm{eff}}$ and ${\mu }_{\mathrm{a}}\&{\mu }_{\mathrm{s}}^{\prime }$ were the most effective with r = 0.86 and 0.83 for firmness and SSC, respectively. In another study on peaches, Cen et al. [109] utilized the combined data, i.e., ${\mu }_{\mathrm{a}}\times {\mu }_{\mathrm{s}}^{\prime }$ and ${\mu }_{\mathrm{eff}}$, combined with least squares support vector machine (LS-SVM) modelling, for firmness and SSC prediction and color assessment as well. The best predictions were achieved by ${\mu }_{\mathrm{eff}}$ for firmness, skin lightness and flesh lightness with r = 0.75, 0.90 and 0.74, respectively, and by ${\mu }_{\mathrm{a}}\times {\mu }_{\mathrm{s}}^{\prime }$ for SSC with r = 0.50.

Recently, Zhu et al. [119] reported on evaluation of tomato ripeness based on the ${\mu }_{\mathrm{s}}^{\prime }$ and ${\mu }_{\mathrm{a}}$ spectra. Six grades of ‘Sun Bright’ tomatoes, i.e., ‘Green’, ‘Breaker’, ‘Turning’, ‘Pink’, ‘Light-red’ and ‘Red’, were defined based on the degree of ripeness, and three grades were also formed by combining the two adjacent ripeness grades. It was found that the ${\mu }_{\mathrm{a}}$ value at the wavelength 675 nm decreased progressively with the increasing ripeness of tomatoes, until it was close to zero for the ‘Red’ tomatoes. The classification models based on PLS discriminant analysis achieved detection rates of 92.3% by ${\mu }_{\mathrm{a}}\&{\mu }_{\mathrm{s}}^{\prime }$ for the three-grade classification, and 88.4% by ${\mu }_{\mathrm{eff}}$ for the six-grade classification.

Overall, both ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ are useful for evaluation of apple quality, while ${\mu }_{\mathrm{a}}$ seems better correlated with quality attributes of horticultural products than ${\mu }_{\mathrm{s}}^{\prime }$. Combining ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ is generally more effective for firmness and SSC prediction than using them alone. This is presumably because both ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ are related to the maturity and physiological processes of fruit, and each parameter provides different aspects of information about the compositional and structural properties of the fruit. A study on the relationship between the optical properties and the microstructures and mechanical properties of apples during storage [120], revealed the decrease in ${\mu }_{\mathrm{s}}^{\prime }$ with fruit softening and the positive correlation of ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$ with tissue firmness, elasticity and the cell’s area and equivalent diameter. Hyperspectral imaging-based spatially-resolved technique provides a new, potentially useful means for assessing quality of apple and other agro-food products. However, the technique requires complicated mathematical modeling and is prone to error during the image acquisition and curve fitting. Moreover, the technique currently is only suitable for laboratory use due to slow imaging and data processing speed. Further work is needed to improve the accuracy and speed for optical property measurement.

6. Concluding Remarks

With an ever-growing demand for consistent and high quality food, there is an increasing need for continued R&D of new and more effective sensing technologies for food quality assessment. Over the past decade, our group has developed several innovative hyperspectral imaging-based sensing techniques or platforms, including spectral scattering imaging, integrated reflectance and transmittance imaging, and spatially-resolved spectroscopy. While the three platforms share some common features in hardware, they also provide their unique capabilities for different applications in property and quality evaluation of horticultural and food products.

Spectral scattering imaging, including both multispectral and hyperspectral, provides an effective means for assessing internal quality of fruit. This technique is particularly suited for firmness prediction because it enhances the measurement of scattering or structure-related properties. Scattering imaging operated in multispectral mode is relatively simple and easy to implement, and the laser-based multispectral scattering imaging prototype has shown potential for online sorting and grading of apples for firmness. Our integrated hyperspectral reflectance and transmittance imaging system can operate in single reflectance and transmittance modes or in the integrated mode. The integrated sensing mode provides unique capabilities for simultaneous inspection of external quality characters (i.e., size, color and blemishes) and internal defects, and hence can be advantageous in food quality inspection. The hyperspectral imaging-based spatially-resolved spectroscopic technique offers a new approach to optical characterization of food and biological tissues over a broad spectral region. The technique is faster and simpler to implement than time-resolved and frequency domain techniques, and it is also advantageous compared to the existing optic fiber-based spatially-resolved configurations, in terms of speed, ease of use and spatial resolution in data acquisition. The prototype OPA, developed based on hyperspectral imaging technique, provides a valuable tool for property and quality evaluation of horticultural and food products.

While these techniques have been demonstrated in laboratory for their performance and utilities for horticultural and food products, further research, however, is needed to transform these techniques for practical use. Spectral scattering imaging relies on effective extraction of meaningful scattering and/or spectral features and an appropriate calibration model for accurate and reliable quality predictions. Further research is needed to explore new image processing methods to obtain superior results. To implement the technique for online sorting and grading of fruit, a fruit orientation device must be developed so that each fruit can be oriented properly for scattering imaging. Improvements in scattering imaging for SSC prediction should be made through better selection of wavelengths and development of a more effective calibration model. Moreover, sensor fusion should also be considered by integrating scattering technique with other proven techniques such as NIRS, for more robust and accurate evaluation of multiple quality attributes including firmness and SSC. While hyperspectral scattering imaging and integrated reflectance and transmittance imaging offer a new, promising approach to food quality evaluation, they are still facing the cost and speed issues in practical adoption. One possible solution is to implement the techniques in multispectral mode with the use of a few optimum wavelengths. Moreover, further improvements in both hardware including illumination, conveying and detection units, and software involving image analysis and model calibration algorithms, are also critical for achieving desired inspection speed and accuracy. The prototype OPA is designed for laboratory use, and much work is needed for fast, real-time measurement of horticultural and food products. Food products commonly have heterogeneous or layered structures and curved or irregular surfaces, all of which can affect the measurement accuracy and need to be addressed in future research. Research is also needed to optimize the instrument hardware for faster, better acquisition of scattering images, and develop more effective inverse algorithms based on the existing or improved light transfer models.