Spectral Line Reflectance and Fluorescence Imaging Device for Skin Diagnostics

Featured Application

Non-contact express assessment of in-vivo skin malformations.

Abstract

The multi-spectral-line imaging concept, which was recently implemented for the snapshot mapping of three main skin chromophores—melanin, oxy-hemoglobin, and deoxy-hemoglobin, is further explored for the snapshot capturing of four spectral line images at wavelengths of 450, 523, 638, and 850 nm, with the consecutive acquiring of a 405 nm excited fluorescence image. A corresponding laser-based prototype device was designed and assembled. Processing of the mentioned five images enables obtaining distribution maps of four skin chromophores within the malformation and comparing their mean fluorescence intensity with that of the surrounding healthy skin. This set of information is helpful for dermatologists, cosmetologists, oncologists, and other healthcare professionals to quantify the diagnosis of skin malformations (including cancers) and to follow up the recovery process after therapy. This paper describes the design of the developed proof-of-concept prototype device and initial test results.

1. Introduction

Human skin malformations exhibit abnormal distribution of the absorbing chromophores (e.g., epidermal melanin, dermal hemoglobin); this clinical feature is typical for burns, bruises, and a number of vascular and pigmented skin lesions, including tumors [1]. Skin chromophore distribution maps can provide useful qualitative and quantitative information on the severity of pathology and its development, as well as on the efficiency of skin recovery processes after therapy. Simple and fast skin chromophore mapping facilitates decision taking and further strategy development in dermatology, cosmetology, oncology, surgery, and forensics. Currently, the procedures with commercially available devices able to provide skin chromophore distribution maps (e.g., SIAscope [2]) are time-consuming, the equipment is expensive, and it is inconvenient for use (cable-connected to computer), which limits applications in personalized medicine and in the field conditions.

Fluorescence imaging provides additional clinical information on skin malformations in a patient-friendly non-contact way. For example, images of skin autofluorescence (i.e., natural fluorescence without any additives) are helpful for skin cancer assessment [3]; autofluorescence imaging in slow video mode reveals fragments of different pathologies within complex skin malformations [4]. A combined multi-spectral-autofluorescence imaging shows promising potential for the early screening of malignant skin melanoma [5].

Each chromophore has its specific absorption spectrum [6], so the skin-remitted light intensity depends on the detection wavelength and composition of the comprised chromophores within the examined skin area. A set of several spectral images of the same skin target, captured within different wavelength bands, can be converted into a set of chromophore distribution maps [7]. There are two main approaches for obtaining spectral images—by narrowband spectral filtering of the image sensor at spectrally broadband illumination of the target area, or by means of spectrally narrowband illumination. The first approach is widely used in hyperspectral imaging (HSI) systems that capture sequential target images at several tens of adjacent spectral bands using interference filters [8], liquid crystal filters [9], or other means of spectral filtering. Commercial and custom-designed HSI systems were used for skin chromophore mapping [8,10,11,12]; however, routine clinical applications of the HSI systems are limited due to their robustness, expensiveness, and the need for powerful computing resources. The set of processed data in the case of HSI may include a number of spectral images not containing any useful information. The HSI image acquisition process may last from several seconds [12] to a minute or longer [13], so movements of the skin target area during the procedure can cause image artefacts. Special image stabilization algorithms [14] could reduce or fully avoid the motion artefacts, but it takes extra time and computing resources. In addition, in many cases, calculations of complicated integrals comprising spectral features of the chromophore absorbance and the filtered HSI band shape are necessary.

The second approach—spectrally selective illumination—has been mainly used in skin multi-spectral imaging (MSI) systems where sequentially switchable non-overlapping spectral bands of illumination (e.g., by several light-emitting diodes (LEDs) with different peak wavelengths at typical spectral full width at half maximum FWHM ≈20–50 nm) are exploited [15,16]. Advanced MSI systems are portable and self-sustained, much cheaper than HSI systems, and better adapted for clinical use. The MSI processing procedure is also faster thanks to the lower number of the spectral images, typically between 3 and 8. Still, skin motion artefacts and the processing of relatively broadband spectral images may cause serious problems.

Obviously, both the acquisition time of the spectral image set and the spectral bandwidth of each image have to be minimized. Ultimate performance can be achieved if the set of monochromatic (single-wavelength) spectral images is obtained by a single snapshot; this technique is provisionally called “snapshot multi-spectral-line imaging” or SMSLI [17,18]. In this case, a set of spectral line images is obtained under uniform illumination of the target simultaneously by several spectral lines, e.g., emitted by lasers. To extract n spectral line images in a snapshot, the number of spectral sensitivity bands of the image sensor system also has to be n. For instance, standard RGB color cameras can record three spectral line images by a single snapshot if the illumination spectrum comprises three spectral lines, each of them positioned within one of the detection bands (R, G, or B) [19,20,21]. Triple spectral line imaging has been successfully applied for colored counterfeit detection [17,22] and for remote distribution mapping of the main skin chromophores (melanin, oxy-, and deoxy-hemoglobin) [23,24]. However, also mapping of other skin chromophores (e.g., bilirubin, lipids, or water) is of diagnostic interest; therefore, SMSLI technologies for higher n values have to be developed. Our previous study in this direction resulted with a battery-powered proof-of-concept device for the snapshot acquisition of four spectral line images to map four skin chromophores, also being able to capture skin autofluorescence (AF) images under the violet 405 nm laser excitation [25]. However, its double-camera design appeared too robust and inconvenient for clinical applications. As the next step, we developed a more compact and handy single-camera design, which is described in this paper.

2. Method of Skin Chromophore Mapping

The general concept of snapshot skin chromophore mapping at fixed wavelengths, used in this work, is explained at Figure 1 for the case n = 3 [24]. The three-chromophore skin model suggests that the dominant absorbers in skin at any of the fixed wavelengths λ_j (j = 1, 2, 3) are oxy-hemoglobin (further abbreviated by a), deoxy-hemoglobin (b), and melanin (c)—see the crossings of their absorption curves with the vertical lines at Figure 1. The RGB image of the skin pathology spot is captured under illumination that comprises discrete spectral lines at wavelengths λ₁, λ₂ and λ₃. With respect to the spectral sensitivity of the image sensor and the cross-talk between its detection bands at the particular wavelengths, three spectral line images are extracted from the image dataset as described in [19,20]. If the skin surface reflection is suppressed (e.g., by means of crossed polarizers), variations in chromophore composition will lead to changes of the diffusely reflected light intensities at each of the fixed wavelengths. Such variations in the pathology region relatively to the healthy skin can be estimated by measuring reflected light intensities from equally sized regions of interest in the pathology (I_j) and the adjacent healthy skin (I_0j). The ratios I_j/I_0j at each pixel or pixel’s group of three spectral line images contain information on the concentration increase or decrease of all three regarded chromophores Δc_i (i = a, b, c), which can be further mapped over the whole image area.

The intensity changes at three exploited wavelengths due to absorption can be represented in terms of the Beer–Lambert law:

$$\{\begin{array}{c}\ln \left(\frac{I_1}{I_{01}}\right)=-l_1\left(\Delta c_a\cdot {\epsilon }_a\left({\lambda }_1\right)+\Delta c_b\cdot {\epsilon }_b\left({\lambda }_1\right)+\Delta c_c\cdot {\epsilon }_c\left({\lambda }_1\right)\right)\\ \ln \left(\frac{I_2}{I_{02}}\right)=-l_2\left(\Delta c_a\cdot {\epsilon }_a\left({\lambda }_2\right)+\Delta c_b\cdot {\epsilon }_b\left({\lambda }_2\right)+\Delta c_c\cdot {\epsilon }_c\left({\lambda }_2\right)\right)\\ \ln \left(\frac{I_3}{I_{03}}\right)=-l_3\left(\Delta c_a\cdot {\epsilon }_a\left({\lambda }_3\right)+\Delta c_b\cdot {\epsilon }_b\left({\lambda }_3\right)+\Delta c_c\cdot {\epsilon }_c\left({\lambda }_3\right)\right)\end{array}$$

(1)

where ε_i(λ_j) —extinction coefficients of the three regarded chromophores at three exploited wavelengths, and l_j—absorption path length in skin at a particular wavelength. Chromophore concentration increase or decrease at each image pixel or selected group of pixels is found by solving the linear equation system (1). For an increased number of chromophores to be mapped (e.g., for n = 4), the number of working wavelengths and equations in (1) has to be correspondingly increased.

3. Design of the Prototype Device

A block-diagram of the device is presented in Figure 2, and its design scheme is presented in Figure 3. Two laser modules—an RGB fiber-coupled module (Elite Optoelectronics, CN) simultaneously emitting three spectral lines (450, 523, and 638 nm, ≈20 mW power each) and a module emitting at 850 nm (RLDH850-40-3, Roithner, AT with a 800 nm long-pass filter) — are used for 4-wavelengths illumination of skin via a visible – near infrared polarizer (#33084, Edmund Optics, UK). A spiral-shaped flat loop of side-emitting 400 micron silica core optical fiber (Light Guide Optics Ltd., LV) serves as the illumination source [28]. The fiber is SMA-terminated at both ends; one of them is used for the RGB laser input and the other one is used for the 850 nm laser input. In result, the examined skin area is uniformly illuminated by the four above-mentioned laser spectral lines. Four 405 nm, 40 mW laser diodes (DL-5146-101S, Roithner, AT) are square-placed inside the fiber spiral to enable the excitation of skin autofluorescence. The skin target area is round with a diameter of 10, 20, or 30 mm, depending on the used changeable conical nozzle, which is inside-covered by a black coating film (Spectral Black, Actar, IL). The device is fully self-sustained using four rechargeable Li-ion batteries (INR18650-35E, Samsung, KR) for power supply.

A four-band RGB-NIR camera (MSC-RGBN-1-A, Spectral Devices Inc., CA) covered by a 420 nm long-pass filter, equipped with an objective lens (25 mm #67-715, Edmund Optics, US) and top-mounted orthogonally oriented VIS-NIR polarizer, is capturing an image of the targeted skin area with subsequent extraction of the four spectral line images. They are further stored in the embedded single-board computer (Rock960, Vamrs, CN) and wi-fi transmitted to an external computer for the calculation of four chromophore distribution maps using the previously developed algorithm [9]. Within a second, the laser modules are switched off, violet lasers are switched on, and an autofluorescence image at the G-channel of the camera is captured. This image is also transmitted to the external computer in order to discriminate seborrheic keratosis from skin melanoma and other pigmented lesions [29]. The second imaging step can be skipped or used alone, if necessary.

The device is initiated by an operator pushing the START button and selecting the appropriate operation mode (SMSLI only, AF image only, combined mode) and exposure time for the RGB-NIR camera. Then, the camera and lasers are switched on, and the device is properly placed on the skin target area (monitored on the round display). After pressing the SHOT button, a micro-controller (STM32G071, STMicroelectronics, CH) synchronizes the proper illumination of the target to capture the image (or set of images), which is/are read by the single-board-computer. The recorded images can be seen on the display and/or are transmitted via the single-board computer’s wi-fi to the remote computer for the calculation of chromophore maps or performing other tasks.

The recorded RAW images (2048 × 2048 pixels) are stored on the computer hard disk. In order to get correct multi-spectral reflectance images, first, the RAW data are pre-processed. RAW images are converted into R-G-B-IR spectral line images (512×512 pix.) using the SpectralDevices debayering algorithm and the available spectral sensitivity curves of four detection bands (Figure 4). Then, the spectral un-mixing is done using linear operation:

I = A * I_s⁻¹

(2)

where I_s(x,y,λ) is a matrix consisting of intensity values of the spectral image at the xy pixel and the given wavelength λ, and I(x,y,λ) is spectrally-unmixed image; A(λ) is the correction matrix calculated from the camera photo-sensitivity curves.

Next, flat-field correction for each spectral image is performed, using a special image filtering technique. The idea is to compensate for possible uneven illumination caused by a deformed skin surface caused by the device’s cap pressure. To find the non-evenness function of surface illumination, a 2D median filter is applied (kernel size 128 × 128 pixels); then, the spectral image is divided by this function. The normalized reflectance image is calculated as:

R = (I - I_d)/(I_f - I_d)

(3)

where R(x,y) is a reflectance at each xy pixel of the spectral image, I_d is an image measured in dark conditions (the image values depends on the camera gain, exposure, and temperature), and If(x,y) is the filtered image. The I_d spectral images are acquired before or after each measurement. Finally, the image outside the region of interest is masked with dark pixels to highlight the circular image area on the monitor screen. The processing of MS data is done by custom designed Matlab software. The software performs off-line processing of RAW image data using the algorithms described above.

The user’s interface (Figure 5) is designed for live showing of skin target and the recorded/stored skin images. The device can be managed by means of the touchscreen display. The software allows the manual adjustment of device parameters such as the intensity of light sources, camera gain, and exposure. Before each measurement, specific patient data and identification of the examined skin malformation are stored. The two image sets (four spectral line images and the autofluorescence image) are captured within two seconds. All recorded images are transmitted to the monitor screen. The measurement data also can be transferred to the remote server computer for more detailed analysis.

Outlook of the prototype device is presented in Figure 6.

4. The Device Test Results

The device clinical validation was just started, and only two trials have been performed so far. Examples of some captured/extracted spectral line image/autofluorescence image sets are presented in Figure 7. Depending on wavelength, the contrast in the spectral line images changes due to the different penetration depth in skin. In the case of dermal nevus (upper row)—a pigmented lesion located below the epidermal–dermal junction—notably increased melanin content is observed with slight changes in the concentrations of three other chromophores. Hemangioma (middle row) is a vascular malformation where an increased content of oxy-hemoglobin is accompanied with a decreased content of deoxy-hemoglobin (relatively to the surrounding healthy skin). Seborrheic keratosis (lower row) is a pigmented lesion that frequently is misdiagnosed as malignant basalioma or melanoma; it also exhibits increased melanin content. In contrary to the other skin malformations, seborrheic keratosis has a unique feature of brighter autofluorescence (under the 405 nm excitation) than that of the surrounding healthy skin [29]—this is clearly seen if the three presented autofluorescence images are compared.

To conclude, the initial test data are in general agreement with the data obtained in previous multi-spectral skin imaging studies [5,12,23,24] and confirm the potential of the proposed device for non-contact assessment of various skin malformations.

5. Discussion

The above-described device has complemented the family of our previously developed proof-of-concept prototypes for non-contact assessment of in vivo human skin [21,30]. Its design is compact and ergonomic, and it is well-adapted for routine clinical applications. However, several technical problems to be solved in near future still exist. Camera read-out software has to be updated in order to analyze all pixel information—currently, the used manufacturer’s software provides only ¼ of the full image data. The long-pass filter mounted in front of the camera seems to be responsible for some interference artefacts in the spectral line images to be removed in the future. It appears that the cooling system of the device has to be updated, as both the camera and the processing unit are heated up during the operation more than initially expected. The conical nozzles are to be supplemented with soft rings on their tips to enhance the patient comfort and to avoid pressure-caused deformation of the skin area under examination. In addition, the adaptation of illumination intensity for different skin photo-types under Fitzpatrick classification seems to be a future issue, as well as further improvements of the software for the mapping of four chromophore concentration distributions over the malformation area.

6. Patents

Patents [19] and [28] were used in the presented design solution.