Next Article in Journal
Development of a New Evaporation Exposure Model: Chemical Product Evaporation Model (CPEM)
Next Article in Special Issue
Investigation of an Influence Function Model as a Self-Rotating Wheel Polishing Tool and Its Application in High-Precision Optical Fabrication
Previous Article in Journal
Factors Affecting the Quality of Bacterial Genomes Assemblies by Canu after Nanopore Sequencing
Previous Article in Special Issue
A Fiducial-Aided Reconfigurable Artefact for the Estimation of Volumetric Errors of Multi-Axis Ultra-Precision Machine Tools
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Autostereoscopic-Raman Spectrometry-Based Three-Dimensional Metrology System for Measurements, Tracking and Identification in a Volume

1
Institute of Modern Optics, Nankai University, Tianjin 300350, China
2
Tianjin Key Laboratory of Optoelectronic Sensor and Sensing Network Technology, Tianjin 300350, China
3
College of Artificial Intelligence, Nankai University, Tianjin 300350, China
4
Tianjin Key Laboratory of Optoelectronic Detection Technology and Systems, School of Electronic and Information Engineering, Tiangong University, Tianjin 300387, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(6), 3111; https://doi.org/10.3390/app12063111
Submission received: 31 January 2022 / Revised: 11 March 2022 / Accepted: 15 March 2022 / Published: 18 March 2022
(This article belongs to the Special Issue Advances in Intelligent Robots and Precision Machining)

Abstract

:

Featured Application

In the industrial manufacturing of transparent semi-transparent materials, such as glass, plastic, resin, crystal or liquid, have wide application use. The cleanliness and purity of these components have very strict requirements, such as smart phone cover plates, liquid crystal display panels and camera lenses. The environments for manufacture process of above-mentioned industrial fields usually have high-standard requirement of cleanliness. Impurities or contaminants, such as fiber and dust, are defects that have to be identified, analyzed and controlled during the process. Although defect detection based on machine vision and neural network has made great progress, performing tracking, measurement or identification of targets of interests in a volume is still very challenging. This demand has already become much more urgent and favored by industrial customers to control the manufacturing quality of the multi-layer structure of glass, plastic or other composite material. Meanwhile, in biomedical industries, transparent or translucid materials or medium are in common use. Various targets of interest in a solution or other medium needs to be tracked, identified and analyzed. In this sense, the proposed Autostereoscopy-Raman Spectrometry-based (ARS) measurement methodology and its developed system in this paper are able to be widely applied to but not limited to the production quality control of multilayer glass structure, the manufacture and assembly of precision optical components, the production process control of liquid crystal panel in a clean room or in a laboratory testing, or analyzing equipment for biomedical applications.

Abstract

Three-dimensional compound measurement within a volume of interest is of great importance in industrial manufacturing and the biomedical field. However, there is no current method that can simultaneously perform spatial localization and 3D measurement in a non-scanning manner as well as the identification of material in a volume. In this paper, an Autostereoscopic-Raman Spectrometry-based (ARS) three-dimensional measurement system is proposed. The target object in a large depth range is initially positioned by the autostereoscopic 3D measurement method, and then the accurate position information is cross-checked and obtained by combining the spectral signal. Meanwhile, the spectral signal at the precise excitation position guided by the autostereoscopic signal also carries the material composition information. In order to verify the proposed ARS method, an associated measurement system was developed, and experimental studies of detecting various fibers of different depths in multi-layer glass structure were conducted. The spatial locations and dimensional information of multiple different targets can be measured in a volume, and their material can also be identified at the same time. The average error between the calculated position processed by the ARS system and the actual spatial position is within sub-micron levels, and the success rate of spectrum acquisition reaches 98%.

1. Introduction

In the industrial manufacturing and biomedical industries, there are many situations in which three-dimensional (3D) spatial measurement, tracking and identification of the targets of interest in a volume are required. Among these scenarios, one or all of status information including the dimensions, the locations and the materials are required to be detected due to the working conditions, operation flow, process control, and other requirements. According to the numerous different applications in consumer electronics, car industries, plane industries, or biomedical research and industries, transparent objects must meet different quality requirements with compound measurements of the target of interests. Usually, visual observation or automated machine vision techniques can partially fulfill the abovementioned tasks on the premise of human intervention or complicated and inefficient measurement processes, which obviously cannot meet the actual production requirements in terms of multiple measurands, measuring speed and on-machine measurement.
With the continuous advancement of science and technology, various methodologies and techniques have been developed to perform measurements of various measurands, including dimensions [1], spatial locations [2,3] and materials [4,5,6]. Three-dimensional measurement methods, such as light scattering techniques [7], fiber interferometry [8] and optical slope sensors [9], usually measure the surface profile rather than targets of interests in a volume or a free 3D space. Stereovision is an effective way to detect 3D objects, but often requires multiple image sensors, complex calibration process and high requirements of rigidity to keep the baseline. Fringe projection profilometry is another potential method to measure various targets in a volume, but the projected pattern has high vulnerability when facing semi-transparent medium and targets in motion. Moreover, considering the measurements in transparent material or in a volume, optical coherent tomography (OCT) can be used in glass inspection [10]. OCT can obtain clear images and defect depth, but the inevitable scanning makes its detection speed greatly reduced.
Autostereoscopic metrology is an emerging technology that was originally developed for the on-machine measurement of micro-structured surfaces [11]. The compact system design can provide fast data acquisition of 3D raw data through the incorporation of a micro lens array (MLA) and high accuracy in three-dimensional (3D) reconstruction due to the reversibility of optical rays in the recording process and reconstruction process of this autostereoscopic measurement method. The autostereoscopic 3D metrology method has very strong advantages in measuring targets in free 3D space, such as in a volume or a transparent material, since it has good extendibility of measuring scale from the macroscopic to the microscopic level and has few limitations of the optical information receiving process since the image sensors can receive reflected or refracted light signals from different targets and medium.
While measuring the three-dimensional information or locating certain targets in a 3D space using 3D measurement technologies, identifying the material is another measurement task of great importance. Spectrometry is an effective way to analyze the composition of substances. Among various spectral analysis techniques, Raman spectroscopy is an important technical means to quickly and non-destructively characterize the structure and composition of materials [12]. Raman spectroscopy has a wide range of applications in the field of materials research. Using Raman spectroscopy to monitor average degree of smoothness and extent of structural defects during materials synthesis to study the evolution of growth kinetics to optimize growth conditions [13]. Raman spectroscopy also has great potential in microbial detection. The use of Raman spectroscopy can rapidly identify causative agent and its susceptibility to antimicrobials [14], and the combination of Raman tweezers and microfluidics enables real-time analysis and identification of single microbial cells [15]. Rapid microbial detection can shorten hospital stay, reduce mortality and reduce costs of medical. It is likely to become a reliable and fast diagnostic tool in the future. Confocal Raman spectroscopy combines confocal microcopy and Raman spectroscopy, and is widely used in the fields of physical chemistry and biomedicine for its non-destructive testing, tomography, and molecular fingerprint characteristics [16,17,18]. High-lateral resolution and well-resolved depth profiling with the confocal Raman spectroscopy allows it to identify tiny targets and investigate hidden phases underneath the surface without destruction of extraterrestrial materials [19]. By adopting the reflective illumination light path, the designed confocal micro-Raman system has the enhanced intensity of the Raman spectrum signal and the improved signal-to-noise ratio [20]. In the field of in vivo confocal Raman spectroscopy, detection methods can be divided into the following two types. For the first type of system, confocal Raman spectra were obtained without imaging guidance. During the measurement, the biological tissue of interest is pressed on the cover glass, and Raman spectra of different depths are obtained by vertical scanning [21]. The second type of system combines Raman microscopy with reflectance confocal microscopy (RCM), in which Raman microscopy and RCM work alternately. RCM imaging is used to identify points of interest (POIs). The RCM imaging channel was then closed to acquire micro-Raman from a predetermined POI [22]. In order to expand application scenarios, methods to improve resolution have been proposed. Combining the confocal Raman spectroscopy imaging with radially polarized light compact focusing, the spatial resolution of confocal Raman spectroscopy imaging can be improved by compressing the incident spot diameter and image restoration techniques [23]. As a new type of Raman spectroscopy detection method, spatially offset Raman spectroscopy (SORS) can effectively suppress the interference of surface component Raman spectroscopy and fluorescence background. Combined with appropriate data processing methods, it is especially suitable for translucent or deep spectral information extraction of substances with highly fluorescent background materials [24]. Its detection depth is usually from 20 μm to 5 mm, which exceeds the detection depth limit of confocal Raman spectroscopy [25]. Samples from which spectra were collected do not need to be prepared and can be measured under high-temperature, low-temperature and high-pressure environments. The instrument is stable, moderate in size, has a low maintenance cost and is simple to use [26]. However, locating the right excitation position in order to precisely identify the material has always been a challenge in the abovementioned spectroscopic methodologies.
When using the autostereoscopic 3D measurement method to measure the spatial position of the target, the robustness of judging the depth by the sharpness function of the focus position is poor. There are two main reasons for this shortcoming. One is the unapparent visual characteristics of the object hinder the judgment of the algorithm, and the other one is that the optical system has a depth of field, and the ability to distinguish different depths of focus is not strong. Furthermore, it is not possible to discriminate the substance composition only using a visual inspection system. Using Raman spectroscopy to discriminate material composition is a good choice. However, when the existing spectroscopic system, such as confocal Raman spectroscopy, detects sparsely distributed targets in a three-dimensional space, the imaging system needs to scan the entire depth to find the target because the microscopic imaging system has a small depth of field. This scanning process obviously consumes a lot of time, and may cause system drift. In addition, SORS cannot accurately obtain the spatial position of the target. Based on the analysis above, there is currently no compound measurement method that can simultaneously perform spatial localization and 3D measurement in a non-scanning manner as well as the identification in a volume. In this paper, we present an Autostereoscopic-Raman Spectrometry-based (ARS) three-dimensional compound metrology system for measurements, tracking and identification in a volume. The ARS method avoids the mapping process and detects much faster than confocal Raman spectroscopy, which give it great potential for industrial applications.

2. Autostereoscopy-Raman Spectrometry-Based Measurement Methodology

The autostereoscopic 3D metrology method [27] makes use of a micro lens array (MLA) that is incorporated into a single-lens imaging system to capture the raw 3D information of the measured surface in a single snapshot. As a machine vision-based measurement method which uses pixel information as the main medium, the diversity of pixel information distribution (PID) determines the resolution in regard to the matching process. Here, the diversity of PID can be interpreted as certain regional pixel information has pixel values within a large range.
In the recording process, the principle of autostereoscopic 3D metrology method system, they use EIs that contain multi-viewpoint stereo information to record spatial targets. As shown in Figure 1, the virtual MLA simulated in computer and a 3D reconstruction algorithm are applied to calculate and rebuild the target in object space. By applying this special digital process, the target 3D information can be obtained quickly and easily, and it is more convenient to handle following measurement data [27,28]. The 3D target reconstructed by computed depth tomography algorithm (CDTA) is overlapped by a series of sharp depth slices. This process consists of three steps. First, the depth slices are obtained by re-mapping of the pixels from each EI. Second, the clear parts of depth slices are extracted by calculating sharpness function of each pixel. Finally, the clear parts are spliced to realize the reconstruction of the 3D target.
Specifically, to obtain image sequence of depth slices is a process of re-mapping the pixels of EIs by virtual MLA. It is an inverse process of recording. It projects each 2D EI onto a special depth plane in object space through optical axis of its respective elemental lens. In the image of a certain depth slice, only the object points belonging to the corresponding depth are clear, while the foreground and background are blurred, because this certain depth is finely focused and other depth planes are defocused. Based on the description of the recording process, the disparity can only be determined by the depth information in the object space. In other words, the disparity of the same depth plane is invariable, and the disparity of different depth planes is variable.
In the reconstructed image sequence of depth slices, different points have different focus levels and show as discs of confusion or focus points on a certain slice. The optimum focus position of a point in depth can be determined by calculating the maximum focus level of each point.
The sharpness function F is used to evaluate the focus level of a point, and it depends on the gray difference contained in the point and its small local neighborhood, in which the size is generally 3 × 3 , 5 × 5 and 7 × 7 . As shown in Figure 1, using the 3 × 3 neighborhood for example, k K , K is the total number of image sequence, I k i , j is the gray value of pixel i , j in k th slice, i , j x , y , and x , y means the region of EI. The center distance of consecutive pixels in the horizontal and vertical directions is normalized to 1, and the center distance of diagonal direction is normalized to 2 .
F k i , j   =   I k i , j     I k i 1 , j   +   I k i , j     I k i + 1 , j   +   I k i , j   I k i , j 1   +   I k i , j I k i , j + 1   +   1 2 I k i , j I k i 1 , j 1   +   I k i , j I k i + 1 , j 1   +   I k i , j I k i 1 , j + 1   +   I k i , j I k i + 1 , j + 1 .
Figure 1 shows the sharpness evaluation curve T i , j of each image point i , j in the EI region x , y is generated by the vector,
T i , j   =   F i , j   =     F 1 i , j , F 2 i , j , F 3 i , j , , F k i , j .
The optimum focus position P i , j of image point i , j can be characterized by the number k of image sequence,
P i , j   =   a r g m a x k T i , j .
P i , j in region x , y can form a 2D matrix by calculating each image point. Finally, the height of the target can be reconstructed by mapping the focusing position to actual depth information.
Compared with other reconstruction methods, the CDTA does not require a complicated image matching process, but only perform re-projection of EIs by MLA. It avoids the matching error of corresponding image points that greatly impact reconstruction accuracy. At the same time, CDTA is free to select a specific depth region to reconstruct 3D information of object space, even only segmentation of the target to meet the inspection requirement.
However, this CDTA method has encountered an unavoidable practical problem especially when facing with an object with limited features but with a larger thickness. It usually brought challenges to precisely identify the right peak of sharpness among multiple peaks. Considering the low robustness of the of current focus measure function (FMF), additional information needs to be incorporated to jointly identify the peak of the sharpness of certain object points in a series of the reconstructed image sequence of depth slices.
Raman spectroscopy is an important technical means to quickly and non-destructively characterize the structure and composition of materials. The sample does not need to be prepared and can be measured under various external environments with high tolerance. Raman spectral signal is a unique and effective signal to differentiate and identify different targets in a volume, especially when the target and medium in the volume are different [29]. In this sense, additional information is needed to provide solid and robust features on sharpness detection of certain object points among multiple interference peaks in a series of the reconstructed image sequence of depth slices. However, high requirements for the excitation and collection are required by the Raman spectroscopy, which means it is necessary to accurately locate the excitation position in order to effectively collect the spectral signal. Usually, this requires additional repeated operations, including moving to multiple trial excitation positions, outputting Raman spectral signal, comparing these signals and confirming one position as the right excitation position.
In this sense, in order to solve the problem of evaluating the fine focused point in a series of the reconstructed image sequence of depth slices generated by the autostereoscopic measurement method, to solve the problem of locating the exact excitation position of Raman spectroscopy, and to trace, measure and identify the targets of interest in a volume, we propose this comprehensive Autostereoscopic-Raman Spectrometry-based (ARS) three-dimensional measurement method, showed in Figure 2. As a compound measurement method, the sharpness function signal of autostereoscopic method and Raman spectral signal are providing cross-validation to each other.
The distribution of sharpness of a certain object point in a series of the reconstructed image sequence of depth slices generated from autostereoscopic method is able to guide spectral detection for the excitation position in order to receive the most significant and accurate spectral distribution curve for material identification. Meanwhile, the intensity of spectral detection can in return guide the precise detection of the true peak of the sharpness.
Accordingly, this compound signal of ARS 3D measurement method is able to simultaneously locate the exact spatial position of a certain point in a volume, measure the spatial dimensional information and identify the material of targets of interest in a volume.

3. Autostereoscopy-Raman Spectrometry-Based Measurement System

3.1. System Design and Optical Layout

Based on the proposed Autostereoscopy-Raman Spectrometry-based measurement methodology, an ARS 3D measurement system is developed accordingly.
The presented micro Raman spectrometer based on light field imaging is built. Figure 3 shows our design of optical layout. The Kohler lighting module illuminates the object, and the object is imaged through the microscopic objective (MO, NA = 0.4, 20×), and spectral information is collected by Raman scattering. The tube lens (TL) imaged the object once, and the micro lens array (MLA) imaged the object twice. The 750 nm shortpass dichroic mirror (BS1) separates visible light and Raman signals. MLA forms a relay imaging system with the microscopic system. The position of each micro lens satisfies the imaging formula:
1 / u + 1 / v = 1 / f ,
where u , v and f are the distance from the micro lens to the image plane of microscopic system, the distance from the micro lens to the sensor and the focal length of the micro lens respectively. In our setup, v is greater than f . In order to prevent the light field image from overlapping, we need the F number of the objective to match the F number of the micro lens array. The F number F n of the objective can be calculated by this formula [30]:
F n M / 2 N A ,
where M is the magnification of the objective. In order to ensure that there would be no overlapping, we chose MLA with a smaller F number, with p i t c h = 300   μ m and f = 5   mm . The image sensor is a monochrome CCD with a pixel size of 3.45 microns. The 785 nm beam generated by the semiconductor laser is purified by the 785 nm laser clean-up filter (EX) and reflected by the ultra-steep 785 nm laser dichroic mirror (BS2) into the objective. The objective focuses the laser on the sample to stimulate Raman scattering. Meanwhile, the scattering signal is collected by the objective. After being filtered by a 792 nm longpass emission filter (EM), the signal is coupled to the fiber with NA = 0.22 and core diameter of 600 μm by coupling objective (CO) and finally transmitted to the Maya2000 Pro High-Sensitivity Spectrometer (Ocean Insight Co., Shanghai, China). Greater detection depth can be obtained by changing the small magnification objective.

3.2. Data Collection and Processing Procedure

The digital reconstruction of the autostereoscopic 3D measurement method is actually a reconstructed process of the pixels from EIs, and it is able to provide a set of focus stack images reconstructed along the output plane. These images contain clearly focused images and defocused images. Depth information of objects can be obtained by the extraction of clearly focused information of the focal stack.
In this proposed method and its configuration, accurate calibration of the autostereoscopic 3D system is required to obtain the absolute depth of the object. First, the resolution plate is placed at the working distance of the objective, and the elemental images are obtained. Use the above method to reconstruct the light field image. The clearest position is the position of the primary image plane of the microscope system, and its conjugate plane is the object plane at the working distance of the microscope objective. This position is determined to be the initial zero position. The autostereoscopic system calibration is completed. The positional relationship between the object plane and the image plane of the microscope system can be approximated by this formula [31]:
Δ X Δ V / M 2 ,
where Δ X is the moving distance of object surface, and Δ V is the moving distance of image plane.
The data collection and processing procedure of the proposed ARS system is shown in Figure 4. As the first step, the object to be detected needs to be placed within the working distance of the objective. The second step is to obtain the elemental images, and then obtain image sequences of depth slices. the clear parts of depth slices are extracted by calculating sharpness function of each pixel. Usually, there are multiple peaks in the sharpness evaluation value due to the influence of noise. When there are multiple peaks, it is necessary to traverse all the peak positions, collect multiple spectra and select the position with the strongest spectral signal as the target existence position. The optimal spectrum is then analyzed to obtain the substance composition. As shown in Figure 1, when there is only one peak, the spectrum of this position is directly collected for analysis.
The proposed ARS system has a fast and large-depth detection scheme without scanning.
Meanwhile, different from the Raman spectrometry system, there is no need to scan the whole detection depth to find out excitation position, which will be guided by the autostereoscopic signal of z-axis positioning. Accordingly, the material information of the foreign body can be quickly obtained, so as to predict the source of the substance. Different from the traditional autostereoscopic 3D measurement system, there is no need to use additional complex image process techniques to find out the fine-focused depth information, Raman spectrometry signal can provide effective and robust additional information to rule out the false peaks from noise.

4. Experimental Studies

In order to verify the proposed Autostereoscopy-Raman Spectrometry-based measurement methodology and its associated measurement system, which is shown in Figure 5. As for the sample preparation for the verification of the proposed method, we specifically construct a multi-layer-structure and put three different kinds of objects between the cover glass and the substrate glass slide as test samples, including non-woven fibric from shoe covers, non-woven fabric from masks, meltblown fabric and anti-static clothing fibers, as show in Figure 6. Figure 6a is the first sample, which contains anti-static clothing fibers, non-woven fibric from shoe covers, and non-woven fibric from masks. The depths of the three targets are 40 microns, 80 microns and 120 microns, respectively. Figure 6b is the second sample, we replaced the fiber of the second layer with meltblown fabric and changed the depth of the target. The new depths were 50 microns, 80 microns and 110 microns. The meltblown fabric very different from other fibers, as vividly shown in Figure 6b, which can further verify the robustness of our method. The design of the target sample is to simulate the pollutants attached between glass interlayers which is especially similar to the industrial manufactured sample.
We simulated the contaminants in different thicknesses of glass interlayers by artificially locating them at different depths away from focal plane of the objective in order to provide the true value of depth, which can be used as reference data of experimental studies. To imitate the structure of the first sample, we artificially located the three targets at 40 microns, 80 microns and 120 microns away from the focal plane by moving the z axis, which has repeatability of 0.1 microns. For the second sample, the related offsets are 50 microns, 80 microns and 110 microns, respectively. Figure 7a,d,g are 2D images of anti-static clothing fibers, non-woven fibers from masks and non-woven fibers from shoe covers captured by a Nikon DS-Fi3 microscope camera under a 20× microscope. Since some of the CCD pixels are invalid, we do a basic cropping of the image. Figure 7b,e,h are the elemental images of the above-mentioned fibers, and Figure 7c,f,i are the reconstructed images. Figure 8 shows the 2D images, EIs and reconstructed images of the second sample. By processing the data via the purposely developed reconstructed algorithm and image evaluation function, we will get the sharpness evaluation function graph as shown in Figure 2 to roughly locate the spatial position of the target. Usually, there will be multiple peaks regarding different depths. We traverse these peaks and collect the associated Raman spectrum. As shown in Figure 2, Raman spectrometry signals with high signal-to-noise ratio are often obtained at the noise peaks, but they are mistaken for focal depth. Raman spectrometry signal can provide additional information to rule out the false peaks as noise. Then, the proposed ARS system is able to get both accurate location information and material composition information simultaneously.
As shown in Figure 9, the spatial positions of the six different types of fibers from the two samples are reconstructed by the autostereoscopic module of the proposed ARS system and the sharpness function from 0 μm to 150 μm are plotted accordingly. It can be seen that due to the calibration error of the autostereoscopic module, the calculated position is different from the actual position. The robustness of the main peak as a condition to determine the depth position is not high, and there are some other peaks around the main peak, and the peak of the correct spatial positions may be located among them. In this sense, Raman spectra at these positions, which are recognized as the peaks of the sharpness function, are collected by the Raman spectrometry module to verify and calibrate the calculated spatial positions. In this experimental study, all peaks within ±15 µm from the main peak are collected with the corresponding Raman spectral signal. Among these positions, the one with the highest signal-to-noise ratio of the Raman spectrum can be determined as the confirmed spatial position. The time resolution of the proposed ARS system mainly depends on two aspects. Autostereoscopy measurement methodology can finish the data acquisition process through single snapshot and can calculate all image stacks within 1 s, as well as initially identify the target depth range simultaneously. On the other hand, the acquisition of the Raman spectrum can be completed in a few hundred milliseconds to a few seconds. In this sense, the time resolution of the proposed ARS system is within 1 s. To be noticed, the calculated depth positions cross-checked and confirmed by Raman spectral correction of three different types of fibers are compared with the true depth, which is obtained by the working distance and defocusing amount, the average error of 15 repeated measurement process is within 1 μm. The spectrum collection success rate of this equipment is 98%.
Figure 9 also shows the Raman spectra of the three types of fibers, the Savizkg-Golay filter is used for smoothing and the adaptive iteratively reweighted Penalized Least Squares algorithm proposed by Zhang [32] is used to remove the baseline. Materials from different sources have distinct characteristic peaks that can be clearly distinguished. By building a database of Raman spectra of possible contaminants, we can easily classify contaminants in practical detection.

5. Discussion & Conclusions

In this paper, an Autostereoscopic-Raman Spectrometry-based (ARS) measurement methodology and its associated measurement system is proposed and developed accordingly, which can perform 3D positioning, 3D measurement and identification of material composition structure of certain depth at the same time in volume, that is verified by the experimental studies in this paper. Moreover, the research of the proposed ARS system shows the potential abilities of detecting multilayer glass structures without scanning, accurately locate the spatial position of impurities, and conduct material analysis through the proposed ARS system. As a pilot study, the autostereoscopic module and Raman spectrum module are currently add-on modules for commercial microscopes without changing the original system structure. A 20× objective that fixedly mounted on this commercial microscope was used to verify the performance of the proposed ARS system by testing the 3D positioning and identification capabilities in a volume that is constructed by a multiple-layer glass structure and three types of fibers. After Raman spectral signal correction, the average positioning error of the ARS system was sub-micron level, and the success rate of spectrum collection has been enhanced to 98%.
The proposed ARS system has proved its high potentials for locating, tracing, measuring and identifying various sources of impurities to accurately control the industrial production process and improve the yield rate with low-cost and high-efficiency methods. It is interesting to note that this ARS system is able to have a larger field of view and depth of field which are obtained by quickly altering the objective magnification and matched micro lens array, enabling better measurement scale and efficiency for industrial scenarios. In this sense, this comprehensive measurement method will have a wider range of applications in the future.

Author Contributions

Conceptualization, J.W., X.Z. and D.L.; methodology, J.W., X.Z. and D.L.; software, J.W., Y.W. and W.W.; validation, J.W. and D.L.; formal analysis, J.W.; investigation, J.W., X.Z. and D.L.; resources, X.Z., B.W., X.X. and H.B.; data curation, J.W.; writing—original draft preparation, J.W. and D.L.; writing—review and editing, X.Z. and D.L.; visualization, J.W. and D.L.; supervision, X.Z. and D.L.; project administration, X.Z.; funding acquisition, X.Z. and W.L. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by grants from the National Key R&D Program of China (2018YFB0504400), Tianjin Natural Science Foundation (No. 19JCZDJC36600), and the Tianjin Key R&D Program (No. 19YFZCSY00250).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Barrett, A.; Reuel, S.; Mohammad, M. A multivariate model to assess the probability of detection and sizing of defects in aluminum panels using eddy current inspections. Eng. Fail Anal. 2018, 94, 182–194. [Google Scholar] [CrossRef]
  2. Xing, Q.; Dai, Z.D.; Wang, H. A Method of Rapid Position Estimation. Appl. Mech. Mater. 2014, 461, 781–785. [Google Scholar] [CrossRef]
  3. Wu, H.; Liu, J.; Lu, L.; Sun, X.; Atubga, D.; Rao, Y. Multi-point disturbance detection and high-precision positioning of polarization-sensitive optical time-domain reflectometry. J. Light. Technol. 2016, 34, 5371–5377. [Google Scholar] [CrossRef]
  4. Trofimov, V.A.; Varentsova, S.A. An effective method for substance detection using the broad spectrum THz signal: A “terahertz nose”. Sensors 2015, 15, 12103–12132. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Trofimov, V.A.; Varentsova, S.A. Essential limitations of the standard THz TDS method for substance detection and identification and a way of overcoming them. Sensors 2016, 16, 502. [Google Scholar] [CrossRef] [Green Version]
  6. Hosseini, M.H.; Heidar, H.; Shams, M.H. Wideband nondestructive measurement of complex permittivity and permeability using coupled coaxial probes. IEEE Trans. Instrum. Meas. 2016, 66, 148–157. [Google Scholar] [CrossRef]
  7. Taguchi, A.; Miyoshi, T.; Takaya, Y.; Takahashi, S. Optical 3D profilometer for in-process measurement of microsurface based on phase retrieval technique. Precis. Eng. 2004, 28, 152–163. [Google Scholar] [CrossRef] [Green Version]
  8. Jiang, X. In situ real-time measurement for micro-structured surfaces. CIRP Ann. 2011, 60, 563–566. [Google Scholar] [CrossRef]
  9. Gao, W.; Tano, M.; Sato, S.; Kiyono, S. On-machine Measurement of a Cylindrical Surface with Sinusoidal Micro-structures by an Optical Slope Sensor. Precis. Eng. 2006, 30, 274–279. [Google Scholar] [CrossRef]
  10. Zhiyan, C.; Yi, S.; Wen, B.; Peng, L.; Xiaoping, W.; Zhihua, D. Identification of surface defects on glass by parallel spectral domain optical coherence tomography. Opt. Express 2015, 23, 23634–23646. [Google Scholar]
  11. Li, D.; Cheung, C.F.; Ren, M.; Zhou, L.; Zhao, X. Autostereoscopy-based three-dimensional on-machine measuring system for micro-structured surfaces. Opt. Express 2014, 22, 25635–25650. [Google Scholar] [CrossRef] [PubMed]
  12. Izake, E.L.; Sundarajoo, S.; Olds, W.; Cletus, B.; Jaatinen, E.; Fredericks, P.M. Standoff Raman spectrometry for the non-invasive detection of explosives precursors in highly fluorescing packaging. Talanta 2013, 103, 20–27. [Google Scholar] [CrossRef] [PubMed]
  13. Donato, M.; Messina, G.; Milone, C. Experiments on C nanotubes synthesis by Fe-assisted ethane decomposition. Diam. Relat. Mater. 2008, 17, 318–324. [Google Scholar] [CrossRef]
  14. Rebrošová, K.; Bernatová, S.; Šiler, M.; Uhlirova, M.; Samek, O.; Ježek, J.; Zemanek, P. Raman spectroscopy—A tool for rapid differentiation among microbes causing urinary tract infections. Anal. Chim. Acta 2022, 1191, 339292. [Google Scholar] [CrossRef]
  15. Bernatová, S.; Rebrošová, K.; Pilát, Z.; Šerý, M.; Gjevik, A.; Samek, O.; Zemánek, P. Rapid detection of antibiotic sensitivity of Staphylococcus aureus by Raman tweezers. Eur. Phys. J. Plus 2021, 136, 233. [Google Scholar] [CrossRef]
  16. Jiang, S.; Zhang, Y.; Zhang, R.; Hu, C.; Liao, M.; Luo, Y.; Hou, J.G. Distinguishing adjacent molecules on a surface using plasmon-enhanced Raman scattering. Nat. Nanotechnol. 2015, 10, 865–869. [Google Scholar] [CrossRef] [PubMed]
  17. Azan, A.; Untereiner, V.; Gobinet, C.; Sockalingum, G.D.; Breton, M.; Piot, O.; Mir, L.M. Demonstration of the protein involvement in cell electropermeabilization using confocal Raman microspectroscopy. Sci. Rep. 2017, 7, 40448. [Google Scholar] [CrossRef] [Green Version]
  18. Kallepitis, C.; Bergholt, M.S.; Mazo, M.M.; Leonardo, V.; Skaalure, S.C.; Maynard, S.A.; Stevens, M.M. Quantitative volumetric Raman imaging of three dimensional cell cultures. Nat. Commun. 2017, 8, 14843. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Park, S.Y.; Park, C. Spatially-resolved mineral identification and depth profiling on chondrules from the primitive chondrite Elephant Moraine 14,017 with confocal Raman spectroscopy. Spectrochim. Acta A 2019, 207, 46–53. [Google Scholar] [CrossRef]
  20. Lu, J.; Xue, Q.; Bai, H.; Wang, N. Design of a confocal micro-Raman spectroscopy system and research on microplastics detection. Appl. Opt. 2021, 60, 8375–8383. [Google Scholar] [CrossRef]
  21. Caetano, L.D.V.N.; de Oliveira Mendes, T.; Bagatin, E.; Miot, H.A.; Soares, J.L.M.; Martin, A.A. In vivo confocal Raman spectroscopy for intrinsic aging and photoaging assessment. J. Dermatol. Sci. 2017, 88, 199–206. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Wang, H.; Lee, A.; Lui, H. A Method for accurate in vivo micro-Raman spectroscopic measurements under guidance of advanced microscopy imaging. Sci. Rep. 2013, 3, 1890. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Li, S.; Qiu, L.; Wang, Y.; Cui, H.; Zhao, W. Super-resolution radially polarized pupil-filtering confocal Raman spectroscopy technology. Meas. Sci. Technol. 2019, 31, 035903. [Google Scholar] [CrossRef]
  24. Conti, C.; Colombo, C.; Realini, M.; Zerbi, G.; Matousek, P. Subsurface Raman analysis of thin painted layers. Appl. Spectrosc. 2014, 68, 686–691. [Google Scholar] [CrossRef]
  25. Liao, Z.; Sinjab, F.; Nommeots-Nomm, A.; Jones, J.; Ruiz-Cantu, L.; Yang, J.; Notingher, I. Feasibility of spatially offset Raman spectroscopy for in vitro and in vivo monitoring mineralization of bone tissue engineering scaffolds. Anal. Chem. 2017, 89, 847–853. [Google Scholar] [CrossRef] [PubMed]
  26. Waldron, A.; Allen, A.; Colón, A.; Carter, J.C.; Angel, S.M. A monolithic spatial heterodyne Raman spectrometer: Initial tests. Appl. Spectrosc. 2021, 75, 57–69. [Google Scholar] [CrossRef] [PubMed]
  27. Tang, J.; Qiu, Z.; Li, T. A novel measurement method and application for grinding wheel surface topography based on shape from focus. Measurement 2019, 133, 495–507. [Google Scholar] [CrossRef]
  28. Long, L.; Zhiyan, P.; Haoyang, C.; Jiaorong, L.; Shenchen, Y.; Lilan, L.; Yingzhong, T.; Wenbin, W. Adaptive window iteration algorithm for enhancing 3D shape recovery from image focus. Chin. Opt. Lett. 2017, 17, 061001. [Google Scholar]
  29. Khan, K.M.; Ghosh, N.; Majumder, S.K. Off-confocal Raman spectroscopy (OCRS) for subsurface measurements in layered turbid samples. J. Opt. 2016, 18, 095301. [Google Scholar] [CrossRef]
  30. Kim, J.; Jeong, Y.; Kim, H.; Lee, C.K.; Lee, B.; Hong, J.; Lee, B. F-number matching method in light field microscopy using an elastic micro lens array. Opt. Lett. 2016, 41, 2751–2754. [Google Scholar] [CrossRef]
  31. Lizuka, K. Engineering Optics, 3rd ed.; Springer: New York, NY, USA, 2008; pp. 316–317. [Google Scholar]
  32. Zhang, Z.M.; Chen, S.; Liang, Y.Z. Baseline correction using adaptive iteratively reweighted penalized least squares. Analyst 2010, 135, 1138–1146. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Working principle of reconstruction process.
Figure 1. Working principle of reconstruction process.
Applsci 12 03111 g001
Figure 2. Schematic diagram of cross validation between the sharpness function signal of autostereoscopic method and Raman spectrum signal.
Figure 2. Schematic diagram of cross validation between the sharpness function signal of autostereoscopic method and Raman spectrum signal.
Applsci 12 03111 g002
Figure 3. Schematic of the Autostereoscopy-Raman Spectrometry-based three-dimensional metrology system.
Figure 3. Schematic of the Autostereoscopy-Raman Spectrometry-based three-dimensional metrology system.
Applsci 12 03111 g003
Figure 4. The data collection and processing procedure of the proposed ARS system.
Figure 4. The data collection and processing procedure of the proposed ARS system.
Applsci 12 03111 g004
Figure 5. The Autostereoscopy-Raman Spectrometry-based measurement system.
Figure 5. The Autostereoscopy-Raman Spectrometry-based measurement system.
Applsci 12 03111 g005
Figure 6. Structure of the sample, (a) the first sample and (b) the second sample.
Figure 6. Structure of the sample, (a) the first sample and (b) the second sample.
Applsci 12 03111 g006
Figure 7. 2D images (a,d,g), elemental images (EIs) (b,e,h) and reconstructed images (c,f,i) of three different targets on the first sample in the experimental studies.
Figure 7. 2D images (a,d,g), elemental images (EIs) (b,e,h) and reconstructed images (c,f,i) of three different targets on the first sample in the experimental studies.
Applsci 12 03111 g007
Figure 8. 2D images (a,d,g), elemental images (EIs) (b,e,h) and reconstructed image (c,f,i) of three different targets on the second sample in the experimental studies.
Figure 8. 2D images (a,d,g), elemental images (EIs) (b,e,h) and reconstructed image (c,f,i) of three different targets on the second sample in the experimental studies.
Applsci 12 03111 g008
Figure 9. Sharpness evaluation function of three different kinds of fibers at different positions and related Raman spectrum acquired at these different positions; (a) is the result of the first sample, and (b) is the result of the second sample.
Figure 9. Sharpness evaluation function of three different kinds of fibers at different positions and related Raman spectrum acquired at these different positions; (a) is the result of the first sample, and (b) is the result of the second sample.
Applsci 12 03111 g009
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Wang, J.; Zhao, X.; Li, D.; Wen, Y.; Wang, W.; Wang, B.; Xu, X.; Bai, H.; Liu, W. Autostereoscopic-Raman Spectrometry-Based Three-Dimensional Metrology System for Measurements, Tracking and Identification in a Volume. Appl. Sci. 2022, 12, 3111. https://doi.org/10.3390/app12063111

AMA Style

Wang J, Zhao X, Li D, Wen Y, Wang W, Wang B, Xu X, Bai H, Liu W. Autostereoscopic-Raman Spectrometry-Based Three-Dimensional Metrology System for Measurements, Tracking and Identification in a Volume. Applied Sciences. 2022; 12(6):3111. https://doi.org/10.3390/app12063111

Chicago/Turabian Style

Wang, Jingfan, Xing Zhao, Da Li, Ya Wen, Weihao Wang, Bin Wang, Xiaoxuan Xu, Hua Bai, and Weiwei Liu. 2022. "Autostereoscopic-Raman Spectrometry-Based Three-Dimensional Metrology System for Measurements, Tracking and Identification in a Volume" Applied Sciences 12, no. 6: 3111. https://doi.org/10.3390/app12063111

APA Style

Wang, J., Zhao, X., Li, D., Wen, Y., Wang, W., Wang, B., Xu, X., Bai, H., & Liu, W. (2022). Autostereoscopic-Raman Spectrometry-Based Three-Dimensional Metrology System for Measurements, Tracking and Identification in a Volume. Applied Sciences, 12(6), 3111. https://doi.org/10.3390/app12063111

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop