Quality Assessment Method Based on a Spectrometer in Laser Beam Welding Process

Abstract

For the automation of a laser beam welding (LBW) process, the weld quality must be monitored without destructive testing, and the quality must be assessed. A deep neural network (DNN)-based quality assessment method in spectrometry-based LBW is presented in this study. A spectrometer with a response range of 225–975 nm is designed and fabricated to measure and analyze the light reflected from the welding area in the LBW process. The weld quality is classified through welding experiments, and the spectral data are thus analyzed using the spectrometer, according to the welding conditions and weld quality classes. The measured data are converted to RGB (red, green, blue) values to obtain standardized and simplified spectral data. The weld quality prediction model is designed based on DNN, and the DNN model is trained using the experimental data. It is seen that the developed model has a weld-quality prediction accuracy of approximately 90%.

1. Introduction

The advent of Industry 4.0 has brought significant enhancements to manufacturing processes, which are now based on smart and autonomous systems, and are incorporated with data and machine learning (ML) [1]. The enhancements to smart manufacturing processes, especially concerning welding technology, must be extended and established, in the context of modern manufacturing [2]. Among various welding technologies, laser beam welding (LBW) has a higher precision, productivity, flexibility, effectiveness, and numerous other advantages, which include a deeper penetration, higher welding speed, and lower distortion, compared to other welding technologies, owing to the properties of the power sources in LBW [3,4,5,6]. In addition, the LBW is a suitable joining technology for welding dissimilar metals and advanced materials such as NiTi alloy [7,8,9]. The LBW presents a substantial potential for manufacturing applications [10], and more study on the LBW process is required to increase the benefits of the LBW. In particular, there seems to be a need of research in the field of monitoring and controlling the welding process and weld quality, which is attributed to the complexity of LBW systems and the characteristics of the LBW process. Considering that the technologies in this field are essential to develop a smart manufacturing system for LBW, extensive research must be carried out to achieve this objective.

The monitoring of a process, and consequently the quality assessment, can be divided into three categories: pre-process, in-process, and post-process [11]. The pre-process monitoring concerns the weld seam tracking before welding or ahead of the laser heat source. The in-process monitoring focuses on the monitoring of the phenomena on the welded zone during welding, such as the keyhole’s shape stability. The post-process monitoring, generally, is concerned with the weld defect detection, and the measurement of the form of the weld seam, after welding, or behind the weld pool. In the pre-process and post-process monitoring methods, ultrasonic and camera-based techniques have been dominantly used. In the case of in-process monitoring, a wider range of measuring methods, including optical (ultraviolet, visual, infrared) and acoustic detectors, X-ray radiography, and camera-based methods, have been used to adequately deal with welding phenomena generated by the high-energy-density laser.

According to Stavridis et al. [10], the in-process quality assessment, i.e., in-process monitoring of LBW, can be further sub-divided into four parts, based on the monitoring techniques used, such as image processing, acoustic emission, X-ray radiography, or optical signal techniques. Vision and thermal images are mainly used in image processing techniques to observe the weld pool [12], keyhole [13], plume, and spatters [14]. The acoustic emission techniques have been applied for the measurement of the melting, vaporization, plasma generation, keyhole formation [15], and propagation of cracks [16], and the X-ray techniques have been used to observe the welding phenomena and defects, including slag inclusions, blow holes, incomplete penetration, and undercuts [17]. In the case of optical signal techniques, they use vision systems (charge-coupled device and complementary metal–oxide–semiconductor cameras), photodiodes [18], and spectrometers (which are highly related with the topic of the study).

Sibillano et al. [19] studied the dynamics of the plasma plume, produced in LBW of 5083 aluminum alloy, with the help of correlation spectroscopy, and presented the results of the influence of welding speed on the loss of alloying elements. Rizzi et al. [20] investigated the spectroscopic signals, produced by the laser-induced plasma optical emission, together with energetic and metallographic analyses of CO₂ laser-welded stainless-steel lap joint, using the response surface methodology (RSM). This statistical approach allowed the study of the influence of the laser beam power and laser welding speed, on the plasma plume electron temperature, joint penetration depth, and melted area. Konuk et al. [21] used a spectrometer to collect the optical emissions of the welding area, and calculate the electron temperature, and the data measured and calculated were used to determine the weld quality and to control the laser power. Sebestova et al. [22] designed a sensor to monitor the pulsed Nd:YAG laser welding process, based on the measurement of the plasma electron temperature, and this sensor was used to detect the weld penetration depth. Zaeh and Huber [23] investigated the radiation emission of the laser-induced plume during LBW of aluminum and steel alloys, and they developed a system to control the chemical composition of the melt pool, during the active welding process. Chen et al. [24] proposed a spectroscopic method based on a support vector machine (SVM) and artificial neural network (ANN) for the detection and classification of fiber laser welding defects. The spectral data captured by the spectrometer was processed by selecting sensitive emission lines and extracting features of the evolution of the spectral data; the SVM and ANN models were designed based on the processed spectral data. It has been verified that these two models were quite effective in detecting and classifying the weld defects. Zhang et al. [25] designed a multiple-sensor system that includes an auxiliary illumination visual sensor system, an ultraviolet- and a visible-band visual sensor system, a spectrometer, and two photodiodes to capture the real-time welding signal and determine the laser welding quality. A deep learning framework based on the stacked sparse autoencoder (SSAE) was established to model the relationship between the multi-sensor features and their corresponding welding statuses; a genetic algorithm (GA) was used to optimize the parameters of the SSAE framework. Lee et al. [26] developed an in-situ monitoring system using a spectrometer for laser welding on galvanized steel. They applied Fisher’s criterion to rank several features for extraction of the most valuable features, and then used the K-nearest neighbors and SVM algorithms to classify welding conditions related to the welding defects. Although some studies have been carried out to investigate weld quality assessment using a spectrometer in LBW process, till date, little attention has been paid to the quality assessment methodology, in LBW, based on the spectrometer and ML algorithms.

This study presents a deep neural network (DNN)-based quality assessment method in spectrometry-based LBW. We designed a spectrometer, that can measure and analyze the light reflected from the welding area in LBW process. The weld quality of LBW was classified through welding experiments, and the spectral data were analyzed using the spectrometer, according to the welding conditions and weld quality classes. Subsequently, the spectral data were converted to CIE 1931 RGB values, to standardize and simplify the spectral data. A weld quality prediction model was designed based on DNN, and the DNN model was trained using the experimental data. The weld-quality prediction accuracy of the model was calculated.

The remainder of the paper is organized as follows. Section 2 describes the experimental procedure and setup. The obtained results are discussed in Section 3, and Section 4 concludes the paper with a brief summary.

2. Experimental System and Procedure

2.1. Spectrometer Development

Figure 1 shows the schematic and prototype of the spectrometer designed in this study, which includes a collimator, an optical fiber, a connector, a reflective diffraction grating, a focusing mirror, and a complementary metal-oxide-semiconductor (CMOS) linear sensor. The optical fiber, through which the light enters, is connected to the collimator with an SMA (SubMiniature version A) connector. Since the round-to-linear fiber optic bundle (model No.: BFL200HS02, Thorlabs Inc., Newton, MA, USA) is used as the optical fiber, there is no need to add a separate slit as an entrance aperture. This indicates that the output terminal of the fiber optic bundle, i.e., the linear bundle, acts as the slit. The collimator (model No.: PAF2S-7A, Thorlabs Inc., Newton, MA, USA) is used to collimate the light, coming from the fiber optic bundle, and then send a collimated beam towards the grating. The reflective diffraction grating (model No.: GR13-0605, Thorlabs Inc., Newton, MA, USA) splits the photons coming from the collimator, depending on the wavelength, and then spreads the light across the focusing mirror (concave). The focusing mirror directs the light, at each wavelength, onto the CMOS linear image sensor. The line camera (model: USB line camera 8M, Coptonix GmbH, Berlin, German) which consists of a main circuit board, and the CMOS linear image sensor (model No.: S11637-1024Q, Hamamatsu, Hamamatsu City, Japan) is used as the detector. Each pixel of the CMOS linear image represents a portion of the spectrum that is translated into a measurable value, by a spectroscopy software. A dedicated software was also developed to operate the spectrometer, display the measured data, and store measurement results. The spectral response range of the developed spectrometer was 225–975 nm, and its sampling frequency was 5 kHz.

2.2. Experimental Setup and Material

The experiment was conducted using a 4-kW disk laser welding system (model No.: TRUMPF HLD 4002, procured from Trumpf GmbH + Co. KG, Ditzingen, Germany). Figure 2 shows a diagram of the experimental setup, which was constructed by reference to the study of You and Katayama [27]. The scanner laser head was equipped with a six-axis robot arm, and was connected with two optical fibers. One of them was the round-to-linear fiber optic bundle connected to the spectrometer, and the other was the optical fiber for laser beam input transmission from the laser power source. In the former case, the scanner laser head and the fiber optic bundle were assembled with a connector, shown in Figure 3, which also has a function to condense the light entering the scanner laser head. The light emission of visible light and laser light from the welding area was transmitted to the spectrometer, through the scanner laser head (having a partially transmitting mirror) and optical fiber. The transmitted visible light emission and laser beam reflection were collimated by the collimator, and detected by the line camera. The image data obtained by the CMOS linear image sensor were sampled and digitalized at a frequency of 5 kHz. Subsequently, the processed data were collected, displayed, and stored by the computer software. Figure 4 shows a plot of the spectrum intensity variation, in LBW of lap joint configuration, with a welding speed of 1.5 m/min and a laser power of 3500 W. To test the weld quality and collect data for development of weld quality prediction models, the experiment was carried out using typical automotive high strength steel (uncoated 780 MPa-grade dual-phase (DP) steel) sheets of 1.2 mm thickness. The workpiece was cut to dimensions of 100 × 30 mm, and welding was conducted in the overlap joint configuration.

2.3. Experimental Procedure

In this study, we tried to classified the weld quality into four types as shown in Figure 5:

(Class 1) Unwelded: No weld is formed between the workpieces

(Class 2) Incomplete penetration: Weld is formed between the workpieces but not full penetration

(Class 3) Full penetration: Weld fully penetrates all workpieces

(Class 4) Unwelded by a gap, between the workpieces in lap joint configuration

To evaluate the weld quality and as well as to obtain experimental data, which can be qualified according to the predefined types and are enough to develop a quality prediction model, the welding parameters were initialized with values presented in

Table 1. Experimental factors and their values.

Welding Parameter	Value
Laser power (W)	1000, 1500, 2000, 2500, 3000, 3500, 4000
Welding speed (m/min)	1.5, 2.0, 2.5
Gap (mm)	0, 0.8

. For the laser power and the welding speed, welding was carried out twice (2 replicates) in all possible conditions (21 combinations), at a gap of 0 mm. At a gap of 0.8 mm, welding was carried out only at laser powers of 3000 and 4000 W for all welding speeds, which means that welding was performed under six conditions for two replicates. All welding runs were carried out with a length of 50 mm in the lap joint configuration. The diameter of a focused laser beam was 0.6 mm, and the wavelength of disk laser was 1030 nm. The defocused distance was fixed at 0 mm. Argon (Ar) inert gas was used as shielding gas, and the flow rate of the shield gas was set to 20 L/min. The metal back plate was set under the workpieces as shown in Figure 1.

3. Results and Discussion

3.1. Weld Quality Evaluation

In this study, we evaluated the weld quality, by examining the cross sections of the weld joints, shown in

Table 2. Cross-sections of welded joint according to the laser power and welding speed.

Laser Power (W)	Welding Speed (m/min)
	1.5	2.0	2.5
1000	Unwelded	Unwelded	Unwelded
1500		Unwelded	Unwelded
2000
2500
3000
3500
4000

. Three pre-defined types of joints were observed: unwelded, incomplete penetration, and full penetration, according to different welding speeds and laser power. The “unwelded” type was mainly observed in case of a relatively low power range of 1000–1500 W, and the type “incomplete penetration” was observed in the range of 1500–2000 W. The type “full penetration” was observed in the range of 2500–4000 W. Because the back plate was used in this study (shown in Figure 2), it seems that burn-through was not created when a high laser power was used. That might have led to a larger range of the “full penetration” type. Figure 6 shows the top, bottom, and side views of the weldment, having a gap of 0.8 mm. Although a welding speed of 1.5 m/min and laser power of 3000 W produced a “full penetration” joint, in the case of zero gap (Table 2), a weld joint was not seen, in the case of 0.8 mm gap. This is because the molten metal was insufficient to fill the gap, which was larger in size. This results in the formation of the last type of joint, “unwelded,” which is due to the unfilled gap between the workpieces in the lap joint configuration.

3.2. Signal Analysis

Table 3. Color-coded images of spectrum, according to the laser power and welding speed; in each image, the horizontal axis is time (s), the vertical axis is wavelength (nm), and intensity is mapped in color (color scale: Viridis, date from [28]).

Laser Power (W)	Welding Speed (m/min)			Color Scalelegend
	1.5	2.0	2.5
1000
1500
2000
2500
3000
3500
4000

shows the color-coded images, which helps to visualize a 3D image of the spectrum intensity (Figure 4) in a 2D form of the spectrum during 0.2 s (0.4–0.6 s). In the color-coded image, the horizontal and vertical axes represent the time and wavelength, respectively, and the intensity is mapped in color. The used color scale was the same used by Viridis [28], and the minimum and maximum values of the color scale were set as 0 and 65535, respectively. It was seen that higher intensities were observed, mainly in the wavelength range of 350–650 nm. In general, when the laser power was kept constant, a higher welding speed increases the intensity. In terms of the size of weld bead, i.e., the penetration and width, this signifies that smaller the weld bead, more was the light, reflected into the spectrometer. Similarly, in case of constant welding speeds, the higher the welding power, the larger was the intensity. However, in this case, even if the size of the weld bead increased, the intensity increased, as the laser power increased. This means that the intensity is influenced more by the laser power, than the weld bead size. These trends were quite clearly observed, in the range of 1000–2500 W of the laser power. However, these trends were rarely observed in all values of laser power more than 3000 W, and rather, all color-coded images in these conditions were almost identical. Additionally, in these conditions, the size of the weld beads, at each welding speed, was almost identical, regardless of the laser power. The growth of the weld bead seems to saturate at a laser power above 3000 W. It is thought that two experimental results, observed when the laser power was more than 3000 W, are related to each other.

Table 4. Color-coded images of spectrum at a gap of 0.8 mm according to the laser power and welding speed: in each image, the horizontal axis is time (s), the vertical axis is wavelength (nm), and intensity is mapped in color (color scale: Viridis, data from [28]).

Laser Power (W)	Welding Speed (m/min)			Color Scalelegend
	1.5	2.0	2.5
3000
4000

presents the color-coded images of spectrum at all conditions, at a gap of 0.8 mm, and the other configurations except the gap setting are same as those of Table 3. In this case, the pattern of the spectrum intensity was irregular, and the intensity level was smaller, compared to those of Table 3. This result indicates that the irregular geometry of the welding area, caused by the gap, caused a higher irregular reflection of the visible light and laser light. The 3D images (Figure 4) and color-coded images (Table 3 and Table 4) may be difficult to be interpret, and their color and form can also change quite drastically, when compared to a referenced color scale and other configurations. Therefore, in this study, we used the color space created by the International Commission on Illumination in 1931 (CIE 1931 color space), to convert the measured spectrum data to a standard form. The measured spectrum data were converted into CIE 1931 XYZ values, and then these values were further converted into CIE 1931 RGB values by Colour, an open-source Python package, providing algorithms and datasets for color science [29]. The RGB format is effective for showing the features of data measured by the spectrometer and it consists of much smaller data converted using the spectrum data, which allows to estimate models and handle data, with less computation time. Additionally, the maximum value of wavelength for each sample was searched and added.

To investigate the effect of the weld penetration on the spectrum intensity, the maximum wavelength and RGB values, at a laser power of 2000 W and different welding speeds, are shown in Figure 7. It is seen that the welding heat input (expressed in kJ/mm) increases, as the welding speed decreases, when the laser power was kept constant. Likewise, the weld penetration increased with increasing heat input, as the welding speed decreased (Table 2). The RGB value, which is converted from the intensity data, decreased when the penetration was increased, which is similar to the results shown in Table 3. It might be inferred that the amount of light reflected from the welding area decreases, as the penetration depth increases, when the laser power is limited to 3000 W, as used in this study. The maximum values of the wavelength were found to be distributed in the range of 500–540 nm, and these values in the case of welding speed 1.5 m/min were slightly lower, when a different welding speed was used. Figure 8 shows the maximum wavelength and RGB values wavelength, at a welding speed of 2.5 m/min, according to different laser powers. The RGB values increased, as the laser power increased, even though the size of weld bead increased due to the high laser power. In addition, the RGB values were almost same, in the conditions of the laser power more than 3000 W. These results were found to be in good agreement, with those of Table 3. The maximum values of wavelength were distributed in the range of 530–540 nm. Figure 9 shows the RGB values and maximum wavelength at a gap of 0.8 mm, and a 3000 W laser power, according to the welding speed. In this case, the maximum wavelength and RGB values were almost same as those shown in Figure 8c, which had conditions of a 3000 W laser power and a 2.5 m/min welding speed. However, overall waveforms of the RGB curves were found to be uneven and irregularly-shaped, and the variations of the RGB and maximum wavelength values were relatively small, as compared to those of the cases when the laser power was above 3000 W. For this reason, the green curve is clearly separated from the red and blue curves, as seen in Figure 9.

3.3. Weld Quality Prediction Model

To predict the weld quality, a DNN model was used as the prediction model. As shown in Figure 10, we modeled a neural architecture, as a seven-hidden-layer DNN, and assigned 256 neurons to each of the hidden layer. The neurons of every preceding layer were fully connected, to those of the succeeding layer. The eight values, which were the average and standard deviation of the RGB values and the maximum wavelength, were used as the inputs to the network (see Nomenclature). Especially, considering the weld size and speed of welding, the model was designed to predict the weld quality, per 0.5 mm of the weld length. The outputs are the four joint types, described in Section 2.3: unwelded (Y₁), incomplete penetration (Y₂), full penetration (Y₃), and unwelded by a gap (Y₄). The other specifications of the DNN implementation model is summarized in

Table 5. Summary of DNN model implementation.

Item	Description
Structure	Input Layer	Hidden Layer	Output Layer
Number of nodes	8	256	4
Learning rate	0.001
Epoch	1000
Batch size	100
Activation function	ReLU
Function of output layer	Softmax
Cost function	Cross-entropy
Optimizer	Adam optimizer

. All the hidden layers utilized rectified linear unit (ReLU) functions, to calculate their respective intermediate outputs. The output layer had four nodes, and the output layer with the Softmax activation function computed and returned the type probability with respect to each of the output types. The prediction of the weld quality, by the model, was determined based on the type probability. A total of 5400 data sets (4320 training sets and 1080 test sets) were used to generate the DNN model. Each dataset contained the given eight input variables, and four output variables. The backpropagation algorithms were carried out with a batch size of 50, with 500 epochs. Figure 11 shows the training results, including the cost, training accuracy, and test accuracy. After training, the cost value was calculated as 0.2806, the training accuracy was 0.8993, and the test accuracy was 0.9083.

To test the developed model, we produced 100 test datasets, for each class, by carrying out an additional welding experiment. Figure 12 shows the result of testing the developed DNN model using 400 test datasets, the confusion matrix and prediction result according to weld seam length. In Figure 12, the vertical and horizontal axes represent the predicted value of the DNN model and the data sequence, respectively. The four symbols, green, gray, red, and blue, represent the class to which each dataset belongs. It is seen that most of the errors occurred owing to the false prediction of Y₂ as Y₁. That is, 22 of the 100 datasets of class 2 (Y₂) made false predictions, and among them, the 21 datasets were predicted as class 1 (Y₁). Although Y₁ and Y₂ are divided into unwelded and incomplete penetration, respectively, in this study, Y₁ and Y₂ can be the same case, which is not full penetration. In particular, in the lap joint configuration of the two same metal sheets, if the penetration depth is less than 50% of the sum of the thickness of the two sheets, the weld quality is classified Y₁. If the penetration depth is more than 50% but less than the sum of thickness of the two sheets, the weld quality is classified Y₂. It is considered that this relationship between Y₁ and Y₂ causes this error. In addition, most errors occurred at the beginning of welding, which could be due to the relatively unstable spectral signal at the beginning of welding. In the case of Y₄, 9 of the 100 datasets made false predictions. The errors, which occur, by predicting Y₄ as the other classes, might be due to excessively irregular signal of Y₄. In the case of Y₃, there is no error in predicting the weld quality because its signal was stable and had a distinct distribution in the RGB space. In summary, the biggest prediction error was seen to have occurred in predicting Y₂. This error could be explained by the relatively small amount of available training data needed to increase the prediction accuracy of Y₂. That is, it is assumed that there were insufficient training data that otherwise could have made the quality prediction model more accurate in classifying Y₂ and Y₁. As presented in Table 3 and Table 4, the range of the welding conditions in which the data for Y₂ can be obtained is smaller than that of other classes, and these welding conditions exist in the range of laser power of 1500–2000 W. If sufficient training data is obtained through more detailed experiments in the same range of laser power, the obtained data can reduce the prediction errors by more clearly learning a prediction model the boundary of Y₂ and Y₁.

4. Conclusions

In this study, we proposed the DNN-based quality assessment method based on a spectrometer in the LBW process, conducted the relevant experiments to analyze the features of the measured data, and verified the developed method. Notable developments and outcomes from this study are as follows.

• We designed and developed a spectrometer that can measure and analyze the light reflected from the welding area in an LBW process. The spectral response range of the developed spectrometer was 225–975 nm, and its sampling frequency was 5 kHz.
• The spectral data were converted to the CIE 1931 RGB color space to analyze the features of the spectral data and obtain the standardized and simplified spectral data.
• The prediction model that can classify the weld quality for LBW using data measured by the spectrometer was also developed. The weld quality prediction model was designed based on DNN, and the DNN model was trained using the converted RGB data and maximum frequency values. The developed model had a weld quality prediction accuracy of approximately 90%.

The results of this study substantially contribute to the state-of-the-art, regarding the automation of the welding process and monitoring technology, for LBW processes. Despite our study’s contributions, some limitations are worth noting. Although our quality prediction model, based on the spectrometer, effectively estimates the weld quality of laser welding, the application of the model is limited to the scope of this study. Future work will focus on the generalization of the quality assessment method proposed in this study. For this purpose, we will collect sufficient data on various materials and welding conditions, and then further improve the data processing techniques and quality assessment algorithms using advanced ML algorithms. In parallel, we will also improve our spectrometer and its software developed in this study.