Research on Morphology Detection of Metal Additive Manufacturing Process Based on Fringe Projection and Binocular Vision

Featured Application

The proposed method and system in this paper have the potential to be used for online inspection and quality control of additive manufacturing.

Abstract

This paper considers the three-dimensional (3D) shape measurement of metal parts during an additive manufacturing process in a direct energy deposition (DED) printing system with high temperature and strong light; a binocular measurement system based on ultraviolet light source projection is built using fringe projection and Fourier analysis. Firstly, ultraviolet light projection and an optical filter are used to obtain high-quality fringe patterns in an environment with thermal radiation. Then, Fourier analysis is carried out by using a single deformed fringe, and a spatial phase unwrapping algorithm is applied to obtain an unambiguous unwrapping phase, which is used as the guiding basis for the binocular matching process and 3D shape reconstruction. Finally, the accuracy of the measuring system is evaluated using a standard ball-bar gauge and the measurement error of this system is within 0.05 mm @ 100 × 100 mm. The results show that the system can measure 3D shape changes of metal parts in the additive manufacturing process. The proposed method and system have the potential to be used for online inspection and quality control of additive manufacturing.

1. Introduction

The rapid development of computer vision technology increasingly affects and even changes people’s lives, and it is widely used in the field of manufacturing industries. As an important branch of computer vision, three-dimensional (3D) visual imaging and detection has become a research hotspot in recent years with its high efficiency and precision. Therefore, it can meet various application requirements of automated production lines [1,2]. In the field of additive manufacturing, additive process quality testing is a major scientific problem [3,4,5]. European and American countries have made progress in the research of nondestructive testing of the additive manufacturing process [6,7,8,9], such as QMmeltpool 3D [10] from GE in the United States, PrintRite3D [11] from Sigma Labs, EOSTATE ExposureOT [12] from EOS in Germany, and other products. Since domestic additive manufacturing mainly focuses on application research, research on the online detection method of additive manufacturing has obviously fallen behind [13]. The traditional detection methods are mostly offline with separated manufacturing and detection processes.

Optical 3D imaging has become an important and helpful detection tool in the field of monitoring additive manufacturing process because of its non-contact, high speed, and high efficiency [14,15,16,17]. In various methods, the structured light measurement method (also known as the fringe projection method) is suitable for the measurement of a diffuse reflection surface. It has obvious technical advantages for the measurement of additive manufacturing parts with a rough surface [18,19,20,21], and, therefore, it has been introduced to realize the online detection for the 3D printing process in this paper. Through the real-time monitoring of the additive parts’ changing morphology, the internal quality changes and defect information of each printing layer in the forming process can be obtained. This will provide some useful analysis and build an effective bridge between the process parameters of the additive manufacturing system and the quality of printed metal parts.

Although structured light 3D imaging technology has been widely used to date, there are still some technical difficulties that have not been addressed, particularly when it is applied in the additive manufacturing process with high temperature and strong emitted light [22,23,24]. Based on this, according to the working environment of metal parts’ additive manufacturing, a binocular measurement system based on UV light source projection was proposed and built, as outlined in this paper. Firstly, the feasibility of the selected system hardware was tested in the actual measurement environment, and then the accuracy of the developed system was assessed. In the actual experiment, the phase information guided the binocular vision system to match the corresponding points, and the additive manufacturing process of metal parts was reconstructed, which proves the feasibility of the built measurement system.

2. Principle

2.1. Background and Spectral Characteristics in Laser Additive Manufacturing Process

The technology used in this paper in the additive manufacturing for metal parts is direct energy deposition (DED), as shown in Figure 1a. During the printing process, the laser and metal powder converge at the appropriate position on the substrate and form a molten pool. With the continuous movement of the printer nozzle in the X and Y direction, the molten pool cools and solidifies rapidly. Through the in-plane scanning path and layer-by-layer superposition of each layer, the printer nozzle scans above the substrate and rapidly melts and solidifies the metal powder to finally realize the rapid prototyping of metal parts.

Therefore, the interaction between laser and materials is often accompanied by strong light and splashing of metal powder. For this paper, the output wavelength of the printing laser is 1064 nm, the printing power is 1.8~2 kW, and the maximum temperature in the printing is about 2000 °C. The image during printing is shown in Figure 1b. It can be seen that the image of metal parts acquired in visible light is overexposed due to the interference of strong light and the thermal effect of the printed parts. Consequently, the traditional structured light 3D shape measurement system cannot meet the measurement requirement in this environment.

The laser additive manufacturing process produces a strong glow, and it cannot be directly observed by the naked eye. The light radiation is relatively complex and mainly comes from three types of components:

Partial refection of the laser by the molten metal surface, and its wavelength is the same as that of laser; that is, 1064 nm.

Forming metal vapor plasma [23]. During the interaction between the laser and the metal powder, the metal material with high laser energy density will melt and evaporate rapidly, and the accumulation of high temperature in the vicinity will aggravate the metal gasification and metal vapor ionization around the molten pool. The spectrum produced by the ionizing radiation process is generally between 400–600 nm [24].

The thermal effect is considerable in the process of laser melting metal powder. The wavelength of metal melting and thermal radiation covers the range of visible light wavelength (380~780 nm) to infrared wavelength (~14 µm).

Therefore, spectral characteristics in the laser additive manufacturing process will be considered in the construction of a measurement system, and a UV light source below 380 nm is selected as the measurement light source in this paper to reduce the influence of light radiation.

2.2. Principle of Binocular 3D Measurement Based on Fringe Projection and Phase Matching

An ideal binocular vision system is composed of two cameras. As shown in Figure 2a, O_l and O_r are the optical centers of the left and right cameras, respectively, and their connecting line is called the baseline. Z_l and Z_r are the optical axes of the two cameras, respectively. Ideally, they are parallel to each other [25]. The point P(X_W, Y_W, Z_W) is imaged at the position of p_l in the left image pixel coordinate system and p_r in the right image pixel coordinate system, and these two points are called a pair of homonymous points.

Similar to the process of human eye imaging, the binocular stereo vision model uses the parallax to calculate the height of objects. Figure 2b shows the schematic diagram of the calculation process of z on the o-xz plane in the ideal system. This is assuming that the effective focal length of two cameras is f, and the row coordinate values of the point P in the image plane of two cameras are, respectively, u_l and u_r. According to the triangle similarity of PO_lO_r and Pp_lp_r, the proportional relationship between the baseline distance b and the height z in the left camera coordinate system can be listed as follows:

$$\frac{b}{z}=\frac{b-K_l-K_r}{z-f}$$

(1)

in which K_l and K_r are the distance between the principal point and the image point in two cameras, respectively. From the camera model, it is known that the image pixel coordinate system takes the upper left corner as the origin coordinate. Assuming that the number of columns in the left and right images is W, and the optical axis passes through the center of the imaging plane, the relationship between u_l and K_l can be expressed as K_l = u_l − W/2. Similarly, the relationship between u_r and K_r can be obtained as K_r = u_r − W/2, which can be substituted into Equation (1) for the following equation:

$$\frac{b}{z}=\frac{b-(u_l-\raisebox{1ex}{$W$}\!\left/ \!\raisebox{-1ex}{$2$}\right.)-(\raisebox{1ex}{$W$}\!\left/ \!\raisebox{-1ex}{$2$}\right.-u_r)}{z-f}=\frac{b-(u_l-u_r)}{z-f}$$

(2)

where b is the baseline length of the measurement system, and the difference of u_i − u_r = d, the disparity used for height reconstruction. The expression of height z with respect to disparity d can be obtained by:

$$z=\frac{b\cdot f}{u_l-u_r}=\frac{b\cdot f}{d}$$

(3)

Similarly, the expression for the space point P in the x and y directions in the left camera coordinate system can be obtained according to the triangular relationship:

$$x=\frac{b\cdot u_l}{d},y=\frac{b\cdot v_l}{d}$$

(4)

The baseline b and focal length f are the parameters of the binocular system, which can be obtained through the system calibration process with known size feature points. However, in the process of installing the left and right cameras, one cannot see the optical axis, so there is a certain angle between the optical axes of two cameras, and it is necessary to calculate 3D point coordinates in combination with the camera calibration and the stereo calibration [26,27].

The binocular measurement system, based on phase matching, takes the phase information as the bridge to guide the matching of homonymous points. On the basis of the original binocular vision system, a structured light projector is added as shown in Figure 3. By projecting encoded fringes onto the measured object, the deformed fringes images are obtained by the two cameras, respectively. In this paper, the Fourier fringe analysis method [28] is used to extract their phase information from the single deformed fringe obtained by the monocular camera, respectively. Due to the different shooting angle and field of view, the phase distributions in the two images are also different. According to the calibration parameters of the system, epipolar correction [29] can be performed on these two images. After epipolar correction, the unambiguous phase is used to guide the matching of homonymous points of two-phase distributions. Finally, the parallax map for height reconstruction is calculated. Combined with the system calibration parameters, the 3D shape data of the object is restored.

2.3. Construction of Binocular Measurement System for Projection of Ultraviolet Light Source

Based on the above-motioned background, a UV light source and a physical grating are selected as fringe projection equipment, and a binocular measurement system based on a UV light source projection is built, as shown in Figure 4.

The chosen light source wavelength is 365 nm and the power is 3W, shown in Figure 4a. The experimental device is shown in Figure 4a. An IDS industrial camera with the resolution of 1920 × 1200 pixels was used, and the field of view of the projection device is about 100 × 100 mm². The system components are shown in Figure 4b.

The aperture of the camera was adjusted to its maximum value and the system parameters, such as the exposure time of the camera, have also been fixed. The deformed fringe images of the formed metal parts under the illumination of the UV light source were obtained, as shown in Figure 5a. Figure 5b shows the center line distribution of the Fourier spectrum of a fringe pattern. After filtering out the fundamental frequency component, and calculating the corresponding wrapped phase after spatial phase unwrapping, the unwrapped phase distribution shown in Figure 5c can be used to reconstruct 3D morphology at the corresponding sampling time.

The measurement system is built with a filter with a central wavelength of 365 nm, a bandwidth of 20 nm, and a cutoff band transmittance of 10⁻⁴, which is placed in the front of the binocular camera to suppress the interference light outside the band. Figure 6a shows the deformed fringe image when one single such filter is added in front of the camera, in which a large area is overexposed due to the strong laser intensity of the additive manufacturing system. After the superposition and combination of two filters, the deformed fringe image of the processing process is obtained again as shown in Figure 6b. It can be seen that the superposition of two filters can filter out the influence of visible light, and the subsequent 3D shape reconstruction of the processing workpiece can be carried out.

In the actual additive manufacturing process, there are often splashes of metal powder. Figure 7a shows a frame of a deformed fringe image in the additive manufacturing process. Considering the detection speed requirement in additive manufacturing, PMP, which has a higher reconstruction accuracy for complex objects, could not be chosen as the phase analysis method because a large number of acquired images (at least three frames) is required to complete a single measurement. In this work, FTP was chosen as the means of calculating and acquiring phase information. Additionally, the Fourier fringe analysis method also can filter out the high-frequency noise from the interference of dust on the actual measurement. By using Fourier fringe analysis, the corresponding wrapped and unwrapped phase is shown in Figure 7b,c, respectively. Figure 7d−f show the section line of the Fourier spectrum and phase information, respectively. The usage of the selected light source, filter and Fourier fringe analysis algorithm can calculate the unwrapped phase of each recorded fringe pattern by two cameras, and provides a basis for the matching of homonymous points in the binocular system.

The phase unwrapping starting point can be marked on the grating to guide the acquisition of the same spatial point in the measurement field of view as the same starting point during phase unwrapping processes of two-phase distribution captured by cameras. However, the work environment in the actual measurement is complex, and the accurate positioning of the marked point is more difficult. Therefore, the feature with little change on the measured object surface can be directly selected as the phase unwrapping starting point. At the same time, the known priori information of the system calibration can be used as a constraint to correct the possible phase unwrapping errors. It is worth noting that we generally select an obvious feature point that can be recognized in both two cameras’ image as the starting point in the process of spatial phase unwrapping. Otherwise, when the object surface fluctuates greatly, and the fringe level dislocation occurs in the deformed fringe image of the left and right cameras, it may lead to the error of 3D reconstruction data.

After verifying the feasibility of the selected hardware, a binocular measurement system, using the projection of UV light source shown in Figure 4, is built.

2.4. Calibration of Measuring System

The camera calibration method based on the 2D planar target is used to calibrate the left and right cameras separately, and then the system is calibrated with the calibration results of two cameras. Since the filters are installed in front of both two cameras, the signal-to-noise ratio of the obtained target image under the condition of visible light illumination is low. Therefore, two extra UV light sources with a power of 10 W are used to assist the camera calibration process with the enhanced contrast of the target image.

3. Experiment and Discussion

After establishing the system, the spectral distribution in laser additive manufacturing process is measured. Then, a ball-bar gauge standard part was used to evaluate the accuracy of this system. Finally, 3D shape changes in the manufacturing process of metal parts were also restored, and the measurement results were analyzed and discussed.

3.1. Determination of Spectral Distribution in Laser Additive Manufacturing Process

For the steel alloy manufactured by the laser additive manufacturing process, the spectral characteristics of the laser additive manufacturing process are obtained by a spectrometer and a two-color pyrometer, and its intensity and light source wavelength window of optical measurement are analyzed.

The DED equipment (LSF1500) developed by the Automation Research Institute of China Ordnance Equipment Group Co., Ltd. located in the Mianyang city of China is used in this work and shown in Figure 8. It employs a semiconductor laser with the wavelength of 1064 ± 1 nm and the maximum power of 4 kW. The forming size is 1500 × 1000 × 600 mm³ with the accuracy of 0.2~0.5 mm. The used material for 3D printing is high strength alloy steel, 30CrMnSiNi2A. The process parameters are: laser power 0.8~2 kW, spot size 4mm, and scanning speed 10 mm/s. The measurement spectrometer for the test is the ANDOR-echelle 5000 model from the Oxford Instruments Company located in the Oxford city of England, with an accurate measurement range of 200~975 nm and a measurement accuracy of ±0.05 nm. The measurement range of the used two-color pyrometer is 600~2300 °C.

The laser power was gradually increased during the printing and measuring process. When the laser power is 1200 W, the intensity of the visible light region is high enough, and the change range of the wavelength intensity of other bands is very small, so the main distribution of the spectrum can be clearly identified. The maximum temperature of the 30CrMnSiNi2A material during laser additive manufacturing is 1900~2050 °C, and the temperature of the substrate and the deposited part is less than 600 °C. The measurement results of the corresponding spectrometer are shown in Figure 9.

For 30CrMnSiNi2A steel, the measured spectral distribution in the visible and infrared bands is significantly stronger than that in other bands. However, the intensity fluctuation of the spectrum below the wavelength of 380 nm is not significantly related to the additive manufacturing process, and the intensity is much lower than that of other bands. This also means that we can choose a suitable spectral band to avoid the radiation interference of all temperature segments in the DED process. Therefore, a UV light source below 380 nm is selected as the measurement light source in this paper.

3.2. Accuracy Evaluation of Measurement System

Since the actual morphology of a tested workpiece in the DED process cannot be directly and simultaneously measured using other proven technology to obtain a ground truth for the comparison analysis of our developed system, we have compared the measurement results of standard parts with the proposed method before we test the system on the site. Two balls of the gauge were measured separately. The results are shown in Figure 10. The comparison between the measurement results and the ground truth of this standard specimen is shown in

Table 1. Accuracy evaluation results of dumbbell gauge standard parts (unit: mm).

Ball-Bar Gauge Standard Parts	Ground Truth Value	Measurement Result
		Measured Value	Error
Ball A diameter	50.7991	50.7507	0.0484
Ball B diameter	50.7970	50.8436	0.0466

. Figure 10a and d, respectively, shows the deformed fringe images by two cameras when ball A and ball B are measured separately. Figure 10b,e show their height distribution. Figure 10c,f are the standard deviation distribution between the measuring results and their fitting sphere in Geomagic software, which is 0.0484 mm and 0.0466 mm, respectively.

According to the above accuracy evaluation results, the measurement error of the developed measurement system is within 0.05 mm, which can meet the requirements of 3D shape measurement in the manufacturing process of metal parts.

3.3. Measurement Results of Additive Manufacturing Process of Metal Parts

The additive manufacturing environment for the metal plate is shown in Figure 11. As there will be a large amount of metal powder splashing during the printing process of the DED system, to avoid the metal powder from adhering to the surface of the camera filter and the lens of the projection device, as well as to ensure the quality of the projection fringe and the recorded deformed fringe image, three blowpipes (numbered Outlet 1–3 in Figure 11) are placed on the upper end of the camera filter and the lens of the projection system, respectively. Air is supplied to cool the light source during the experiment, as shown in Outlet 4 in Figure 11. The whole measuring system is fixed on a tripod. The working distance was adjusted to ensure that the printing area is located at the projection focal plane, and the baseline distance and respective focal length between two cameras were adjusted in order to guarantee that the printing area can be clearly imaged by two cameras, respectively. The protective gas blown by the print head can effectively ensure the cleanliness of the measured object surface without dust.

The additive manufacturing process of metal parts were measured. Figure 12a shows the deformed fringe image of the metal plate after the first layer is printed. The additive manufacturing is carried out in the dotted area of the metal plate, and a total of 15 layers are printed. Figure 12c shows the deformed fringe image of the metal plate after printing, and Figure 12b shows the fringe change of the corresponding area before and after additive manufacturing. The deformed fringe image of the printing area after the printing of each layer is shown in Figure 12d.

All of the obtained deformed fringe images have been used for 3D reconstruction according to the aforementioned manner, and the height change after the printing of each layer can be identified.

Figure 13 shows the reconstructed height distribution of 3D print metal at 15 times shown in Figure 12d. The above number of each subfigure is the corresponding printing layer.

It can be seen from Figure 13 that when the number of printing layers is greater than 7, the height distribution of metal parts cannot be fully obtained with part of the region data missing. This is because the height changes greatly when the printing layer increases, resulting in a local shadow of images and mutual occlusion of binocular cameras, and finally leading to the loss of phase and height data.

Figure 14a shows the height change 15 groups along the 500th row and the 250–500th column. Figure 14b shows the height change at pixel (600, 375) after different printing layers, and Figure 14c shows the height variation of Figure 14b.

To further analyze the height changes of metal parts with different printing layers, Figure 15 shows the height data in the square area of the 250–500th rows and 550–650th columns. Similarly, based on restored 3D data after the first printing, as shown in Figure 15a, the height difference distribution between each layer and the first layer is shown in Figure 15b–o, from which the surface shape change process of metal parts can be clearly seen. It is consistent with the above analysis that the surface shape of printed metal parts becomes increasingly uneven with the increase of the numbers of printing layers.

The analysis shows that the binocular system, based on the projection of a UV light source, can reconstruct the 3D shape of the additive manufacturing process of metal parts in the high temperature and strong light environments. However, when the height variation of metal parts is large, local shadows block or two cameras block each other, and local point clouds will be missing and not accurate. This is future work to be addressed.

4. Conclusions

This paper looked at 3D shape measurement of metal parts during the additive manufacturing process in a DED printing system under high temperature and a strong light environment. A binocular measurement system based on ultraviolet light source projection was built using fringe projection and Fourier analysis. The UV projection light source and physical grating were used as fringe projection devices. Fourier analysis is carried out by using a single deformed fringe, and phase unwrapping is carried out with the spatial phase unwrapping algorithm, which is used as the guiding basis for the binocular same point matching process for 3D shape measurement and reconstruction. The results show that the system can measure 3D shape changes of metal parts in the extreme additive manufacturing process, and the obtained 3D data can be used to analyze the relationship between the parameters of the printing system and the quality of printed metal parts.

In this paper, 3D imaging technology is extensively studied, and the scheme based on binocular structured light imaging is improved. Fourier fringe analyses are integrated, making the system more efficient and suitable for the monitoring of the additive manufacturing process. Compared with the traditional phase-shifting methods, the number of captured images is reduced, and the tested workpiece morphology can still be well restored. Therefore, the developed measuring system and presented morphology reconstruction in this paper is of theoretical value and pragmatic significance for the monitoring and imaging field of industrial production. In future, a motion mechanism with multiple degrees of freedom could be added into this system by choosing an appropriate measuring azimuth angle to suppress the shadow/occlusion occurring.