A Methodological Review of Fluorescence Imaging for Quality Assessment of Agricultural Products

Abstract

Currently, optical imaging techniques are extensively employed to automatically sort agricultural products based on various quality parameters such as size, shape, color, ripeness, sugar content, and acidity. This methodological review article examined different machine vision techniques, with a specific focus on exploring the potential of fluorescence imaging for non-destructive assessment of agricultural product quality attributes. The article discussed the concepts and methodology of fluorescence, providing a comprehensive understanding of fluorescence spectroscopy and offering a logical approach to determine the optimal wavelength for constructing an optimized fluorescence imaging system. Furthermore, the article showcased the application of fluorescence imaging in detecting peel defects in a diverse range of citrus as an example of this imaging modality. Additionally, the article outlined potential areas for future investigation into fluorescence imaging applications for the quality assessment of agricultural products.

1. Introduction

Awareness about maintaining food safety has risen in the public’s mind, such that it is now of great concern. As a result, customers have become more discerning about the quality of the food they consume and more demanding of “just in time” delivery. Therefore, to ensure the reputation of a supplier’s brand and the supply of quality products to the consumer, agricultural products are now being automatically graded and sorted in commercial and cooperative facilities of many countries using a machine vision system with image processing capabilities. This system can inspect products according to different parameters such as size, shape, color, weight, ripeness, presence of defects, and disease [1,2,3]. Machine vision systems in the packing lines perform repetitive tasks quicker, more accurately, and with greater consistency over time than humans, reduce the labor associated with grading, and maintain the reputation of supplier brand names, enhancing the supply of high-quality food to the market. Energy, labor, and materials costs are reduced, and packing is continuously performed 24 h a day, ultimately increasing the productivity of the packing house [1,4,5]. Research on machine vision application has become more and more common place in postharvest product quality evaluation and safety control, field robotic guidance, automated harvesting, and packing and wrapping [1,6,7].

Optical imaging techniques, in particular machine vision systems, have been widely utilized in the design of non-destructive biological product inspection equipment. Considering the recent importance of product quality detection, several techniques were studied using machine vision systems equipped with different monochrome, color, multispectral, and hyperspectral imaging sensors. Monochrome or grayscale imaging-based machine vision systems are used for the detection of postharvest quality of fruits [8,9,10,11] and vegetables [12], and examining meat quality [13,14,15,16,17]. The application in color imaging, i.e., the composition of fundamental color components RGB for the detection and grading of agricultural product quality, found vast and extensive applications [18,19,20,21,22,23,24,25], and researchers mostly used this means of imaging for the image classification process [4].

Spectral imaging is defined as a combination of imaging and spectroscopy, where a complete spectrum is collected at every location of an image plane. This powerful technique is known as multispectral (i.e., number of bands) or hyperspectral (i.e., much narrower bands: 10~20 nm) imaging [26]. Nowadays, there are many applications in the fields of agriculture, biomedical, ecology, oil and gas, oceanography, and atmospheric studies where multispectral [27,28,29,30,31] and hyperspectral [16,17,32,33,34,35] sensing techniques are being widely used. Hyperspectral imaging technology is used to acquire both spatial and spectral information on an object over the ultraviolet (UV), visible (VIS), and near-infrared (NIR) spectral regions at the same time. Due to the limitations of regular machine vision and spectroscopic techniques, this technology attracted the interest of researchers as a powerful tool used in a variety of scientific areas, including agriculture, pharmaceutical, and material science [32,34,36]. The study and research work on the application of hyperspectral imaging in reflectance mode has become more and more popular recently in food quality and safety control, such as the detection of defects [32,37,38,39,40,41], identification of contaminations [32,42,43], and quantification of meat ingredients [44,45].

Alternatively, apart from categorizing machine vision systems based on the electromagnetic wavelength they operate on, we can also utilize imaging systems that employ different modes or principles of measurement. While reflectance is the most commonly used in machine vision, another principle called fluorescence can be employed. Fluorescence imaging is an optical imaging methodology that is widely accepted and has been proven to be one of the most powerful techniques used in a variety of scientific areas, including environmental, industrial, and biotechnology applications. In particular, fluorescence imaging emerged as a promising adjunctive nondestructive technique for postharvest quality research for agricultural and food products and has been demonstrated to be a very useful technique for building generic inspection systems that can meet the speed requirements of commercial production lines [46,47]. Since fluorescence imaging can provide spatial information about fluorescence emission from a product item, it may offer a new opportunity for assessing the postharvest quality of horticultural products [48]. However, the application of fluorescence imaging to this date has not been fully explored to assess food quality and safety.

Recent research involved fluorescence imaging for measuring the quality and safety of food and agricultural products. A study was conducted to identify the optimum wavelengths for fluorescence excitation and the resulting fluorescence wavelengths in the range of 300–700 nm with citrus peels, which was concerned with the detection of surface defects of citrus fruits [46]. The physical causes for the fluorescent signatures that caused the citrus fruits to be rejected by the commercial packing house, even though their external appearance was fresh or normal to the naked eye under visible light, were examined. The study showed that UV-induced fluorescence imaging could detect unseen injuries or defects in undamaged-appearing citrus fruits and cull them from the packing line [49]. UV-induced fluorescence imaging could also reveal different image features that could not be seen in reflectance-based imaging, including color and texture. Both techniques also could be combined to improve the accuracy of prediction or classification [50,51]. Hyperspectral fluorescence imaging techniques were used to detect early bruises on pears [52], inspect fungal infection of stored apples [53], and measure laser-induced fluorescence scattering to assess the postharvest quality of apples [48]. To measure the freshness of rice quickly and conveniently, a fluorescence imaging method with ultraviolet (UV) excitation was designed and developed [54]. The feasibility of using light-emitting-diode (LED)-induced fluorescence imaging was investigated for rapid inspection of common organic poultry residues on poultry processing (i.e., chicken fat, blood, and feces) and stainless-steel equipment surfaces [55]. A recent study suggested a sanitation monitoring and management system using fluorescence imaging techniques to detect fruit residues on the surface of food processing equipment [56].

In the remainder of the review paper, the fundamental aspects underlying fluorescence will be described to further understanding of this type of imaging, primarily to determine the best wavelength for designing and constructing an optimized fluorescence imaging system. Following this, examples of its use in the evaluation of citrus are provided. It is hoped that these examples will provide methodologies that would be pertinent for both citrus and other biological products. Finally, an overview of areas of future investigation of fluorescence imaging applications for quality assessment of agricultural products is given. The overall goal of this review is to provide a logical methodology that is commonly used to apply fluorescence imaging for the quality assessment of agricultural products. This information will be very useful for researchers, scientists, and commercial packing houses in understanding, designing, constructing, and demonstrating an optimized fluorescence imaging system.

2. Fluorescence Concepts and Methodology

Fluorescence is a very useful property of some chemical substances and it is increasingly used in biosciences. Fluorescence is the emission of light by a substance that has absorbed light or other electromagnetic radiation. It is the molecular absorption of light energy at one wavelength and its nearly instantaneous re-emission at a longer wavelength or lower frequency. Some molecules fluoresce naturally, and others can be modified to make fluorescent compounds.

Fluorescence occurs when a fluorophore (fluorescent molecule, atom, or nanostructure) relaxes to its ground state after absorbing a photon. Usually, the absorbed photon is in the UV range, and the emitted light is in the visible range, but this depends on the absorbance curve and Stokes’s shift of the particular fluorophore. Once a molecule has absorbed photon energy in its ground state, i.e., the statistically most common energy state for room-temperature chemical reactions, it will be excited to a higher singlet state and often to a higher vibrational level in the excited states. Since excited states are unstable, the molecule will lose its excess energy and return to its ground state via several routes. Figure 1, termed a Jablonski diagram [57], shows a few of these energy dissipation processes. Following the absorption of a photon, the molecule will relax to a lower energy level in various pathways. The most likely is the relaxation to the lowest vibrational energy level of the first excited singlet states, (e.g., Sn → S1), which is called internal conversion (IC). The molecule will finally return from the lowest excited energy state to the ground state, i.e., if the photon emission occurs between states of the same spin state (e.g., S1 → S0), this is termed fluorescence (line F). If it occurs between different spin states, i.e., from triplet state to the ground state (e.g., T1 → S0), the process is called phosphorescence (line P) [54,58].

Fluorescent compounds have two characteristic spectra in common: an excitation spectrum, i.e., the wavelength and amount of light absorbed, and an emission spectrum, i.e., the wavelength and amount of light emitted, as shown in Figure 2. These spectra are often referred to as a compound’s fluorescence signature or fingerprint. No two compounds have the same fluorescence signature. The fluorescence excitation spectrum of a single fluorophore species in dilute solution is usually identical to its absorption spectrum. The absorption spectrum can, therefore, be used as a surrogate for the excitation spectrum. Under the same conditions, the fluorescence emission spectrum is independent of the excitation wavelength due to rapid internal conversion from a higher initial excited state to the lowest vibrational energy level in the lowest (S1) excited state. For many common fluorophores, the vibrational energy level spacing is similar for both the ground and excited states, which results in a fluorescence spectrum that strongly mirrors the absorption spectrum. This is because the same transitions are the most favorable for both absorption and emission. However, in solution (where fluorophores are generally studied), the details of the vibrational structure are typically lost, and the emission spectrum is a broadband [59].

3. Fluorescence Basis and Imaging Configuration

Fluorescence may be excited by X-rays, UV, or visible radiation. The fluorescence’s spectral distribution may range from the UV to the infrared, and the emitted light is always at a longer wavelength than the absorbed light due to limited energy loss by the molecule before emission. The most striking examples of fluorescence occur when the absorbed radiation is in the UV region of the spectrum, thus invisible to the human eye, and then emitted in the visible region [61,62]. For instance, most citrus species accumulate substantial quantities of fluorescent substances, most of which are associated with peel oil [63,64,65,66]. When the oil glands or cells of citrus are damaged or ruptured, the peel oil is released and migrates closer to the peel surface, allowing it to fluoresce and become visible when excited by UV light. For example, tangeretin, a component of peel oil, is a polymethoxylated flavone that fluoresces under UV light and is the likely source of the yellow fluorescence visible from damaged or decayed oranges [67]. Another example is heptamethoxy flavone, one of the major fluorescent substances in rotten citrus fruit that exhibits greenish fluorescence under UV excitation [68]. Except in rare instances, however, fluorescence from intact or unruptured cells is not observed. The application of fluorescence to detect surface defects, such as rotten, damaged, decayed, or injured parts, in a wide variety of citrus is, therefore, made feasible by the presence of these fluorescent substances that are associated with the rupturing of surface cells.

Fluorescence is a spectrochemical phenomenon where the molecules of the analyte (component) are excited by irradiation at a certain wavelength and emit radiation of a different wavelength, usually at a longer wavelength. The excitation and fluorescence spectra represent the relative photon intensity per wavelength interval. The excitation spectrum is a plot of the luminescence versus excitation wavelength with a constant emission wavelength. It is used to determine the best excitation wavelength for analysis. A plot of the luminescence signal versus emission wavelength with a constant excitation wavelength is denoted an emission spectrum. Fluorescence spectral data are generally presented as emission spectra [69]. The measurement of optimal excitation wavelength for the best fluorescent signature of sixteen common and leading varieties of Japanese citrus was studied by [46] using a fluoro-spectrophotometer (F-4500, Hitachi, Ltd., Tokyo, Japan). The study found that, except for one variety, fifteen varieties had general or good trends of fluorescence spectra in the UV and VIS regions with high, medium, and weak level of fluorescent signatures. The performance of a fluorescence image processing-based inspection system is highly dependent on the accurate selection of hardware specifically configured to the fluorescent characteristics such as lighting sources, lighting arrangements and lighting geometry, and camera resolutions [1]. Figure 3 shows energy flow from a lighting device to a TV camera [70]. The image obtained has a final energy of ERTLS. By selecting suitable “E”, “T”, “L”, and “S”, but not “R” reflectance, since it is an optical property of the target object, we can manipulate the final energy of the image. When a color camera is used, only “E” can be manipulated, because the optical filters are located on the color image sensor. Since “E” is the most important parameter, the spectral energy of light, lighting method, number of lighting devices and other factors should be carefully optimized. Moreover, the degree of peel defect fluorescence emission differs by variety, the variation between varieties can be classified as none, slight, medium, strong, and intense. Hence, it is important to discriminate selectively between the fluorescence emission wavelengths and the excitation wavelengths for the different varieties in order to identify defects in as many varieties as possible using the same system.

Therefore, the proper selection of a lighting system and categorization of different citrus varieties based on fluorescence emission characteristics is the key and critical part for optimizing a fluorescence imaging technique capable of being applied to a wide variety of citrus. The fluorescence and excitation spectra information presented in [46] enables the selection of the optimum hardware configuration for designing the image acquisition device. Thus, in order to effectively detect the fluorescent components associated with peel defects on the surface of the citrus fruits, a fluorescence imaging-based machine vision system with a combination of a normal color camera and an incident light source emitting rays in between 330 and 380 nm is needed. With the information observed from spectral study results of the selected citrus varieties, excitation wavelengths of the greenish fluorescence group are around 365 nm [46]. To acquire the fluorescence images of these groups of citrus, 365 nm UV LED excitation source could be used for illuminations. In order to observe the excitation illumination of bluish group citrus, 320~340 nm is required, or more appropriately at 330 nm. Since greenish and bluish fluorescence are emitted from peel extracts, it is possible that a color TV camera used for color imaging can also be used for fluorescence imaging.

4. Fluorescence Imaging Application—An Example

The fluorescence images of intact greenish group citrus were acquired using an image acquisition system composed of three sets of 365 nm UV LEDs (Figure 4) to compare with the spectral information results. The VGA format CCD camera (Sony ICX424AQ, Sony Corporation, Tokyo, Japan) of 8-bit gray levels fitted with a C mount lens of 6 mm focal length was placed 150 mm above from the fruit surface. To examine fluorescence oil cells, 10 mm × 10 mm area on the fruit skins was intentionally injured. Finally, after adjusted camera operation parameters such as iris: 1.4, shutter speed: 1/60 s, gain: 12 db, and gamma correction: 0.6, the target objects were oriented manually towards the camera and the image was captured.

The observed fluorescence grade of the citrus is shown in Figure 5, and it is apparent that the extent of peel defect UV-fluorescence highlighted by the white square was found to vary substantially. Peel defects for amanatsu, buntan, and dekopon varieties appeared as strong green fluorescence (top row of Figure 5); hassaku, iyokan, kiyomi, navel, sanpoukan, sweet springs, and unshu fluoresced moderately (first to second row of Figure 5), while harumi and setoka fluoresced only slightly (second row of Figure 5). These fluorescence images of twelve citrus varieties are in good agreement with spectral information described in [46]. In each image, the fluorescent substance on the surface of the injury part of fruit was caught as greenish color pixels highlighted by the white square. The purple color pixels were the halation of light reflection from the glossy surface (cuticular layers). The machine vision test results revealed that the fluorescence excitation information to select the wavelength of the incident light for inspecting citrus peel defects worked properly. When such a system was trialed, greenish type fluorescence was observed from injured areas on the surface of the peel; consistent with the in vitro fluorescence spectra results obtained.

5. Comparisons of Different Imaging Options for Quality Assessment

Non-destructive inspection equipment for agricultural products postharvest quality assessment has extensively employed different optical imaging techniques, including monochrome, color, and spectral. In cases where color is not a significant factor, monochromatic imaging can be utilized, requiring less data storage. In most cases, color imaging is highly efficient and powerful in extracting visible information of products such as color, bruise, shape, size, etc. Color imaging, which involves analyzing various color components (e.g., RGB, HSI, CIELab) to detect and grade agricultural product quality, has widespread and extensive applications in the commercial packing house [71]. The main drawback of color imaging is that it is limited to three color bands, providing limited multi-constituent information and often missing the most relevant surface features completely [72,73], as shown in Figure 6a,c. Thus, spectral imaging was used to overcome the limitation and is used extensively for the quality analysis of agricultural products. However, spectral imaging has some drawbacks, such as requiring expensive hardware for image acquisition, large data size for the processing step, and a time-consuming algorithm for image post-processing [74,75]. On the other hand, fluorescence imaging technology has various advantages, such as being relatively cheap, simple to use, rapid non-destructive measurements, creating high contrast, etc. [49,50,51]. For example, the acquired color images (Figure 6a,c) did not show any presence of surface damage. In contrast, the damage was readily visible in fluorescence imaging with a clear contrast between damaged and undamaged surfaces (Figure 6b,d).

6. Areas for Future Investigation

The fluorescence principle-based machine vision has been used for various kinds of applications in the last couple of decades. There are numerous advantages of fluorescence machine vision, especially for some cases that could not be solved using normal color imaging. The implementation of the fluorescence machine vision system varies from defect detection, antioxidant and freshness evaluation, to authentication of bioproducts [76,77,78]. Cases which use the exposure of the fluorescence substances on the surface of the object or the presence of the fluorophore on the surface of an object are commonly solved with fluorescence machine vision [79,80]. On the other hand, there are also uses related to the structure of the objects, for example, the existence of a thin cuticle layer on citrus, green bell pepper, tomato, and tea leaves, or a bran layer of rice, which could also affect the results of the fluorescence image [81,82,83]. There are still a lot of bioproducts containing fluorophore which have not been investigated. The structure of the bioproducts also varies, which will gives different results of light–object interaction including fluorescence. The investigation of the fluorescence characteristics of various bioproducts and the making of a database related to this will be useful for the development of proper fluorescence-based machine vision design. For instance, synthesizing an extensive database of fluorescence properties for the unique fresh produce varieties growing worldwide will enable the development of rapid, simple, and uniform fluorescence imaging systems for the online screening of fresh produce quality. Such a database will promote and enhance fluorescence characteristics research, particularly comparative studies and construction of customized systems and strengthen networking among fresh produce researchers, producers, policy makers, grading facilities, company personnel, and practitioners globally by sharing information in the future agricultural cloud technologies. Most of the applications mentioned before are using autofluorescence of bioproducts. On the other hand, fluorescence dye development for bio-molecule markers may also be necessary [84,85].

In terms of the instrumentation devices, lower-cost devices are necessary to implement the fluorescence techniques widely. More sensitive sensors are required, especially in the case of fluorescence lifetime imaging to avoid photodamage due to high-intensity ultraviolet light [86]. Among various sensor materials, the sensor based on the quantum principle has more potential to be improved especially for detection in the ultraviolet region [87,88]. For the light source, an ultraviolet range spectrum light source is often needed. UV LED has already replaced UV lamp for many applications, and it still could be improved in terms of intensity and efficiency [89]. In addition to the development of sensors and light sources, other optical devices such as high-quality filters (e.g., visible cut-band filter) and lenses also need to be improved. Multispectral fluorescence imaging techniques are another area of research that could be pursued in the future as an aid in the quality assessment of agricultural products [4].

Moreover, the X-ray fluorescence technique (XRFT) was recently used in food and agricultural research, such as evaluating food safety inspection, plant diseases, and Ca-related disorders in fruits and vegetables. More exhaustive work is needed to increase the application of XRFT widely and successfully [90,91] for quality evaluation of fruits and vegetables. Some other potential future investigations of fluorescence imaging applications for product quality assessment could be machine-learning-based fluorescence lifetime microscopy imaging [92], low-cost-smartphone-based fluorescence system [93], chlorophyll fluorescence imaging [94], and fluorescence spectroscopy [95].

In parallel to the hardware, the software is equally important for developing a generic machine vision system for the quality inspection of agricultural products. Recently, image processing coupled with machine learning algorithms (e.g., ANN, DNN, CNN) was successfully applied for food quality assessment and contamination detection [96,97]. The software aspect mainly includes image preprocessing, segmentation, feature extraction, and classification [71]. The image preprocessing involves color space transformation using the color space conversion relationships and noise reduction using smoothing, opening, closing, de-bridging, hole-filling, filtering, etc. Image segmentation is a process of pixel classification, one of the most difficult initial tasks in image processing. Though there is no unique standard approach to image segmentation, considerable care should be taken to select the proper segmentation method based on the source image characteristics, as the degree of segmentation accuracy affects the eventual success or failure of the overall image processing results [98,99]. The image segmentation algorithms are mainly categorized into three techniques namely: thresholding, pattern recognition, and deformable models [100]. Thresholding is one of the most popular approaches for image segmentation, and thresholding segmentation aims to search for an optimum grey level threshold value for distinguishing target objects and image background based on their grey value distribution patterns. The Otsu global thresholding selection method (proposed by Scholar Nobuyuki Otsu in 1979) is usually applied because it is a simple and effective idea [101]. CNN-based, state-of-the-art semantic segmentation approach, such as U-Net, FPN, LinkNet, etc., was successfully deployed for image segmentation operation [97,102]. After segmenting operation, target objects’ various features (e.g., color, texture, morphological) are calculated for image perception, interpretation, and classification. Object classification is an essential feature for evaluating agricultural product quality. Various popular machine learning algorithms, such as SVM, KNN, ANN, DNN, CNN, decision tree, and random decision forest, were applied for object classification [71].

In today’s business world, execution speed is important. Setting up defect sorting lines optimized for individual varieties would entail high capital expenditure and would lack the flexibility to rapidly switch a sorting line from one variety to another. An alternative would be to develop a generic system that can handle a wide range of varieties on the limited floor space of the packing house, thus maximizing utilization of facilities. This is in line with one of the current challenges for companies in the fresh produce industry of how to achieve more flexibility in the line configurations so that the sorting line operations are not a critical path to new product introductions. Considering the need for flexibility in the packing line, as well as for economic viability, it is important to develop a detection system configuration that is capable of detecting surface defects of a wide variety of products with minimal or quick reconfiguration of hardware and software. It may also be able to sort out not only product damaged by biotic factors, but also another product that is apparently undamaged but has in fact sustained damage caused by abiotic reasons.

7. Conclusions

The performance of a fluorescence imaging system for quality assessment is highly dependent on the accurate selection of hardware specifically configured to the fluorescent characteristics of agricultural products, such as the spectral energy of light, lighting arrangement, and number of lighting devices, camera resolutions, etc. For instance, the appropriate fluorescence excitation wavelength for optimal fluorescence emission and the resulting peak fluorescence wavelength was observed in the range 330–380 nm and 400–550 nm, respectively, for citrus peels collected from sixteen varieties. It is apparent that to effectively detect the fluorescent components associated with citrus peel defects, a fluorescence imaging system with a combination of a UV light source emitting rays between 330 and 380 nm and a standard RGB camera is needed. We envision that this methodology for developing a generic and unique system can also be applied to other products with proper image acquisition adjustment based on the spectra information.