A CNN-LSTM and Attention-Mechanism-Based Resistance Spot Welding Quality Online Detection Method for Automotive Bodies

Abstract

Resistance spot welding poses potential challenges for automotive manufacturing enterprises with regard to ensuring the real-time and accurate quality detection of each welding spot. Nowadays, many machine learning and deep learning methods have been proposed to utilize monitored sensor data to solve these challenges. However, poor detection results or process interpretations are still unaddressed key issues. To bridge the gap, this paper takes the automotive bodies as objects, and proposes a resistance spot welding quality online detection method with dynamic current and resistance data based on a combined convolutional neural network (CNN), long short-term memory network (LSTM), and an attention mechanism. First, an overall online detection framework using an edge–cloud collaboration was proposed. Second, an online quality detection model was established. In it, the combined CNN and LSTM network were used to extract local detail features and temporal correlation features of the data. The attention mechanism was introduced to improve the interpretability of the model. Moreover, the imbalanced data problem was also solved with a multiclass imbalance algorithm and weighted cross-entropy loss function. Finally, an experimental verification and analysis were conducted. The results show that the quality detection accuracy was 98.5%. The proposed method has good detection performance and real-time detection abilities for the in-site welding processes of automobile bodies.

1. Introduction

The automobile industry, as one pillar industry of the national economy, plays an important role in promoting the development of China’s economy and society. In automobile manufacturing, welding is the main process used to join the weldments to form an automotive body. Welding quality can greatly affect the quality of the entire automobile. Resistance spot welding (RSW), as a common welding method, is widely used. Its welding principle is that under the pressure and heating of the two metal electrodes of the welding gun, the automobile body weldments are melted to ultimately form a reliable welding nugget with which to achieve the connection of metal plates. The detailed RSW process is shown in Figure 1. RSW is widely used in automobile body connections due to its advantages of energy concentration, low cost, high production efficiency, and the ease with which it is automated. According to statistics, there are approximately 4000 to 6000 welding spots from RSW on one automobile body, accounting for over 90% of its connections. In these spots, even though optimal process parameters are set, in-site welding defects, such as overburning and cold welding, can also be generated during the actual production process. Therefore, dynamically ensuring the quality of RSW is extremely important [1,2]. In this process, quality detection (and also inspection) is foundational. Nowadays, most enterprises still use manual offline inspection for RSW, such as total destruction inspection or manual sampling inspection. These can lead to low detection efficiency, high detection cost, and an inability to ensure whole detection for each welding spot. Moreover, the traditional sampling inspection process is susceptible to randomness. Many defects can be ignored or not identified, which may not fully ensure the quality of the welding. Therefore, it is urgent to investigate more efficient and convenient methods to fulfill detection capabilities for ensuring RSW quality [3].

Fortunately, along with the development of new generation information and artificial intelligence technology [5], data-driven welding quality online detection methods [6,7] have attracted the attention of researchers. The data used in this technology involve monitored dynamic resistance [8,9,10], electrode displacement [11,12], welding voltage [13,14], dynamic power [15,16] and welding spot image data [17,18,19,20,21]. Various machine learning and deep learning models have been developed and applied using these data. Although most of this research can implement online quality detection for RSW, there are still the following shortcomings and limitations:

(1)

Traditional machine learning methods require manual feature extraction and rely heavily on expert knowledge and manual experience. Improper features and poor knowledge can lead to poor detection accuracy.

(2)

Deep learning methods can automatically extract features, avoiding the shortcomings of manual extraction. However, these models are regarded as black boxes, and their interpretability is relatively poor. Meanwhile, the proposed deep learning models mainly focus on vision detection and other combined dynamic data are scarcely considered. This can lead to lower detection accuracy for the inner defects of welding nuggets.

(3)

In resistance spot welding, in-site quality data are extremely imbalanced. Large amounts of data are normal and defect data, such as cold welding and overburning, are relatively rare. However, most studies have not considered the impact of imbalanced data on model detection performance.

In addition, automobile production is a typical large-scale flowline production mode, which can generate a large amount of production data at the same time. For example, a typical welding workshop can have more than 150 welding robots fulfilling welding operations simultaneously. A higher concurrency of tasks and a large volume of sensor data can be generated. In these cases, the traditional cloud-based centralized processing methods with huge data storage, high task calculations and high transmission latency can make it difficult to implement the timely detection of each welding spot. Thus, it is necessary to explore novel detection methods to meet the real-time requirements for resistance spot welding processes [22].

In response to the above shortcomings, this paper proposes a combined CNN-LSTM and attention-mechanism-based RSW quality online detection method for automotive bodies under an edge–cloud collaboration framework. It aims to develop a novel method to improve detection efficiency and detection accuracy compared with traditional manual inspection and machine learning methods. Moreover, different to conventional visual detections with deep learning, this paper attempts to utilize dynamic current and resistance data to improve detection ability for inner nuggets, and not surface images. Meanwhile, the interpretability and data problems are also integrated to enhance detection performance. The main innovations and contributions of the method include:

(1)

The constructed edge–cloud collaboration framework enables real-time, fast, and accurate detection in RSW processes. On basis of the “cloud-based model training, edge-based online detection” mode, cloud pressure and transmission delay can be reduced. Meanwhile, the deployed edge detection model can directly input the monitored dynamic time-series welding current data and resistance data for the timely identification of quality defects.

(2)

The CNN and LSTM models are combined and applied from the integrated dynamic current and resistance data, and not by using static process parameters or vision image data. The CNN is applied to automatically extract the local detail features of the input data. The LSTM is integrated to learn the time-series correlation features of the data. This combination not only avoids improper manual extraction, it also considers the time-series characteristics from welding processes. Moreover, unlike traditional vision detection methods, inner defects can also be avoided.

(3)

The interpretability of the model and imbalance problems from the data are all considered and integrated within the method. In our paper, the attention mechanism is introduced to improve the interpretability of the model. The synthetic minority oversampling technique for multiclass imbalance (SMOM) algorithm and weighted cross-entropy loss function are also applied to solve the problem of imbalanced data.

(4)

The proposed model and method are verified with a real in-site welding production process, and exhibited higher detection accuracy, better detection performance and real-time detection abilities.

The remainder of the paper is organized as follows: Section 2 reviews the related works and Section 3 presents the online quality detection operation framework for RSW. In Section 4, the proposed online quality detection model for RSW is presented. The experimental validation and discussion are presented in Section 5. Finally, Section 6 presents the conclusion of this study.

2. Related Research

In resistance spot welding, the formation process of the welding nugget (as shown in Figure 1) is fast and invisible. In order to detect welding quality, it is necessary to reveal the real-time correlation between the welding condition data/result images and welding quality. In recent years, most researchers have found that the dynamic welding current, dynamic resistance, electrode displacement, welding voltage, dynamic power, welding sound and welding spot image data can greatly influence or reveal the quality results. Some related machine learning and deep learning methods have also been developed [23]:

(1)

In machine learning, Xing et al. [8] used dynamic resistance data as an input to classify the quality of RSW using random forests and evaluated the importance of features in the data. Zhao et al. [9] extracted 20 features from dynamic welding resistance, and thus built a multiple linear regression model to predict welding joint strength with errors of less than 10%. Zhang et al. [11,12] utilized the electrode displacement condition to implement a detection process based on a genetic k-means algorithm, probabilistic neural network, and Chernoff faces. Wan et al. [13] analyzed the correlation between voltage changes and the formation process of welding nuggets and manually extracted the critical factors to implement the quality classification with a generalized regression neural network. Wang [24] combined data from the dynamic welding current, welding voltage and electrode pressure to develop a hidden Markov model to determine welding quality. Zhou et al. [25] applied logistic regression, support vector machine, and a recurrent neural network to detect the splashing quality problem with dynamic resistance data. From the above research, it can be seen that most scholars focus on one or multiple monitored datasets to ensure the quality of detection processes. However, manual feature extraction is also critical [26,27] for machine models. In these models, a huge amount of real in-site welding data and knowledge are lacking, which leads to difficulties in determining the most effective features. Therefore, detection results and precision can be poor. In response to the above shortages, novel deep learning methods have been gradually applied to RWS processes.

(2)

Using deep learning, Dai et al. [18] proposed a small object detection network model based on YOLOv3 (You Only Look Once) and achieved the resistance spot welding vision inspection of an automobile body. Yang et al. [28] combined GoogleNet, transfer learning, and a multi-layer perceptron network to automatically extract welding image features to implement quality classification. Xiao et al. [20,21] proposed a convolutional neural network AcmNet and RswNet with automotive welding images and introduced a dual-scale attention mechanism to improve the model’s performance. Ding et al. [19] proposed an in situ weld surface defect recognition method based on an improved lightweight MobileNetV2 algorithm where the image feature information was refined automatically with a convolutional block attention module. Dai et al. [10] focused on the dynamic resistance signals of automotive production lines and integrated welding process stability with a one-dimensional convolutional neural network model to predict welding quality. Bogaerts et al. [29] combined autoencoder deep learning and Gaussian process regress methods to predict resistance spot welding quality with a dynamic resistance curve. Zhou et al. [30] considered the vibration excitation response signals of welded joints and constructed the real spatial–temporal attention denoising network, a deep learning model. Compared with traditional machine learning methods, deep learning can obtain better detection results. Vision detection is widely used in these processes.

Overall, deep learning methods have become the main research trend in welding processes with real-time monitoring data. However, it can be seen that most of this research mainly focuses on vision detection from image signals [18,19,20,21,28]. Real-time data, such as welding current, dynamic resistance, and dynamic power, researched in the machine learning processes, have been considered less frequently in deep learning up to now. Even though the image signals are greatly useful, they can only be applied to identify surface defect problems; critical inner defects cannot be detected. Meanwhile, the lower interpretability for deep learning [10,20,30] and imbalanced data for RSW processes [7] are gradually drawing more and more attention from researchers. However, research on these processes are still scarce. In sum, considering the above shortcomings, this paper focuses on dynamic current and resistance data, and proposes a novel combined CNN-LSTM and attention mechanism-based resistance spot welding quality online detection method. This method also integrates interpretability and imbalanced data problems. Meanwhile, the edge–cloud collaboration framework of the model application and the time-series correlation characteristics of monitoring data are also considered.

3. Operation Framework under Edge–Cloud Collaboration

In order to address the shortcomings of traditional cloud-based centralized data processing methods, edge computing has been introduced into real-time online detection processes. Edge computing can not only reduce the pressure of the cloud center, but also improve the real-time performance of the system by transferring the storage and calculation of some data to edge equipment. The edge–cloud collaboration can fully utilize the characteristics of the low latency of edge and the strong computing ability of cloud [31,32], and thus to enable online quality detection for RSW. Therefore, based on the edge–cloud collaboration, an operation framework of online quality detection for RSW is proposed, as shown in Figure 2.

In the above framework, the core idea is “cloud-based deep learning model training and edge-based online quality detection”. Concretely, in terms of model training, edge computing equipment collects real-time welding data and saves the data in edge databases. The edge databases periodically upload the welding sample data to the corresponding cloud database to update the data used for model training. When the data upload is completed, the RSW quality detection model is trained through the cloud’s powerful computing ability, and is then updated and saved to the cloud model. In terms of online quality detection, the model in the cloud model library can be downloaded and deployed to the edge computing equipment. In this case, once the cloud model is updated, the deployed edge model can be synchronized. During the production welding stage, the real-time welding data collected by the edge computing equipment are input into the corresponding edge model for online detection. Furthermore, if defects are identified, the resulting data are uploaded to the cloud database for the next model training and updating process.

4. Online Quality Detection Model for RSW

This Section proposes an online quality detection model for RSW by combining the CNN-LSTM network and attention mechanism, as shown in Figure 3. The model first uses the CNN-LSTM network to automatically extract local detail features and time-series correlation features of the welding data, and then uses the attention mechanism to focus on the critical main features that are most useful for the output results, improving the model’s effectiveness and endowing the model with interpretability [33,34]. The proposed model consists of five parts: an input layer, convolutional neural network (CNN) layer, long short-term memory network (LSTM) layer, attention mechanism layer and output layer.

4.1. Input Layer

The inputs of the online quality detection model are two time-series datasets: welding current data and welding resistance data. These reveal the different welding characteristics and welding defects at different stages. For example, at the initial stage, the current can generate fluctuations. If the fluctuation is abnormal, the potential defects can be identified. Meanwhile, at the middle stage, the current is more stable and abnormal variations in the resistance can occur. Therefore, it is necessary to fuse the two datasets to reveal the entire welding characteristics, and thus improve the detection abilities. Detailed data descriptions are given in Section 5.1.

To achieve the fusion of the two types of data, the input layer splices and fuses the one-dimensional current data C = (c₁, c₂, ..., c_n), resistance data R = (r₁, r₂, ..., r_n) where the c_i and r_i are the instantaneous values at the sampling time t_i, and n is number of samplings for each welding spot under the frequency of 1 KHz. This fusion generates two-dimensional time-series data X = (C; R) = [x₁, x₂, ..., x_n] (x_i = (c_i, r_i)) as the subsequent input for the model to identify the quality results.

4.2. CNN Layer

The second part is the CNN layer. CNN is one of the most commonly used networks in deep learning and is widely used in fields such as image recognition and fault diagnosis. CNN has the characteristics of sparse connections and weight-sharing and has superior capabilities in extracting local features for the original data [21]. The typical CNN includes convolutional layers, pooling layers, and a full connection layer.

In our model, the proposed CNN layer takes 2D time-series data X (2 × 240 × 1, as shown in Figure 3) as an input to automatically extract local detail features, reduce data dimensions and retain main features, and finally forms the optimal sequence feature matrix (1 × 58 × 24, as shown in Figure 3). In this process, the CNN concatenates two convolutional layers (Conv1 and Conv2) and two maximum pooling layers (Pool1 and Pool2) with the structure of Conv1-Pool1-Conv2-Pool2. The convolutional layer utilizes the data X and convolutional kernel to fulfill the convolutional operations, and then implements the feature extractions with the activation function. The pooling layers combine the extracted feature to realize the feature compression and information filtering, and to finally retain most of the main features. The detail layer relations and feature extractions are displayed in Figure 3.

Note that in the convolutional process, the sigmoid and tanh activation function are replaced as leaky ReLU. This not only avoids the problems of gradient disappearance and gradient explosion caused by sigmoid and tanh, but also avoids the problem of neuron death in the negative half-axis of the traditional ReLU function. The detailed leaky ReLU is formulated as:

$$f(x)=\left\{\begin{array}{ll}\alpha x\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}& x\le 0\\ x\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}\hspace{0.33em}& x>0\end{array}\right.$$

(1)

where α is the leaky ReLU negative axis slope.

In addition, to inhibit overfitting of the model, the dropout layers are added after each pooling layer in the training process, improving the generalization of the model by randomly deleting neurons under the given probability.

4.3. LSTM Layer

The third part is the LSTM layer. Since RSW data are time-series data, and the CNN has poor ability in processing time-series information, in order to better extract the time correlation characteristics of the data, the LSTM network is introduced to further learn the time correlations of features. The LSTM network is an improvement of the recurrent neural network RNN, which can effectively avoid the problem of gradient vanishing and exploding by selectively adding and forgetting previous information through three gate control units: an input gate, forgetting gate, and output gate. Details of the LSTM methods can be referenced in the related research [35,36].

In our paper, the input of LSTM is the extracted feature matrix (1 × 58 × 24) from CNN, while the output is denoted as H = [h₁, h₂, ..., h_r] where r is set as 58. The h_t of step t is calculated as follows [33]:

$$\left\{\begin{array}{l}{i}_t=\sigma (W_i{h}_{t-1}+U_i{p}_t+b_i)\hfill \\ f_t=\sigma (W_f{p}_t+U_f{h}_{t-1}+b_f)\hfill \\ {\tilde{c}}_t=\tanh (W_c{p}_t+U_c{h}_{t-1}+b_c)\hfill \\ c_t=f_t\otimes c_{t-1}+i_t\otimes {\tilde{c}}_t\hfill \\ o_t=\sigma (W_o{p}_t+U_o{h}_{t-1}+b_o)\hfill \\ h_t=o_t\otimes \tanh (c_t)\hfill \end{array}\right.$$

(2)

where σ is the sigmoid activation function; p_t (1 × 24) is the input of LSTM cell of step t; i_t, f_t, and o_t represent the input gate, forget gate, and output gate; W_*, U_*, and b_* (*∈i, f, c, o) denote the LSTM network parameters to be learned; and $\tilde{\mathrm{c}}$_t is the long-term state obtained by the nonlinear function.

4.4. Attention Mechanism Layer

The fourth part is the attention mechanism layer. The attention mechanism can enable the model to focus its attention on information that is more useful for the output results, suppressing useless interference information, improving the detection accuracy of the model, and endowing the model with interpretability [37,38]. The structure of the attention mechanism module in our model is shown in Figure 4.

First, the time-series data H = [h₁, h₂, ..., h_r] are inputted to a full connection layer to obtain the attention weight matrix E = [e₁, e₂, ..., e_r]. The weight value e_t is calculated with the attention-scoring function, as shown in Equation (3). Second, the obtained attention weight matrix is normalized through the softmax layer to obtain the normalized weight α_t of each input feature h_t. Third, the input features and their weights are multiplied and summed to obtain the final output y_att of the attention mechanism. The calculation formula is:

$$\begin{array}{l}{e}_t=V\tanh (W{h}_t+b)\hfill \\ {\alpha }_t=soft\max (e_t)=\frac{\exp (e_t)}{{\displaystyle \sum _{t=1}^r\exp (e_t)}}\hfill \\ y_{att}={\displaystyle \sum _{t=1}^r{\alpha }_t}{h}_t\hfill \end{array}$$

(3)

where V and W are the weight coefficients, and b denotes the bias coefficient.

Finally, the output y_att obtained by the attention layer is used to realize the final classification of RSW quality through the following softmax activation function in the output layer.

5. Experimental Validation and Analysis

To evaluate the effectiveness and accuracy of the proposed method, this paper takes the RSW process of a real automobile manufacturing enterprise as the case and obtains actual welding production data to conduct the experimental validation. The experimental algorithm programming is based on Python 3.7 and TensorFlow version 2.1.0, with the hardware configuration of Intel Core i5-10400F (2.9 GHz) CPU and 16 GB RAM.

5.1. Description of RSW Data

The enterprise workshop uses welding robots for welding, with current and voltage sensors integrated inside the welding gun. The real-time current and voltage data of the welding process can be obtained at a frequency of 1 KHz, and the dynamic resistance data can be calculated by the formula R_t = U_t/I_t. The obtained data will be saved to the welding controller. The real-time data of the welding process can finally be obtained by parsing the data from the welding controller. The data collection devices are shown in Figure 5.

In the RSW process, the quality defects of welding spots mainly include normal, cracking, splashing, stomata, cold welding, overburning, deep indentation, and welding slag. Among these, the most common defect is splashing, which accounts for about 40% of total welding defects. However, most splashing defects do not affect the connection strength of automotive bodies. In fact, the defects that have the greatest impacts on strength only account for about 0.5% of total welding spots. Therefore, the defects data of the welding spots are extremely imbalanced. Among these defects, cold welding and overburning (as shown in Figure 6) are the main defects, accounting for about 85% of defective welding spots. Therefore, this paper intends to identify three welding quality categories, namely normal, overburning, and cold welding from real-time welding data. To detect the above quality detects, data from three in-site monitored operations are shown in Figure 7.

From Figure 7, it can be seen that, compared with normal data, the data from overburning welding spots mainly present two different states. The first state is that the initial dynamic resistance is relatively high. It will undergo a significant increase or decrease as the welding proceeds. At the same time, the current will have significant fluctuations. The reason for this is that the initial contact resistance is relatively high due to the low electrode pressure during welding, and the local input of the welding heat is too fast. This can lead to overheating during the welding process, causing perforation of the welding plates and resulting in a sudden increase in dynamic resistance and significant fluctuations in the current. The second state is that the initial resistance is relatively small, and the dynamic resistance gradually decreases as the welding proceeds, resulting in a smaller final resistance value. The reason for this is that excessive heat is generated, which leads to excessive melting of the welding plate, generating a lower final resistance. The current data for cold welding spots are smaller than that of normal welding spots. At the same time, the dynamic resistance of cold welding spots decreases less during the welding process, which is due to the fact that the welding plate has not melted or only partially melted. Through the analysis, it can be concluded that welding current data and dynamic resistance data can be fused as inputs for feature extraction in deep learning models, and thus reveal whole welding characteristics, and improve the detection abilities for RSW.

5.2. Dataset Construction

The data collected from welding workshop for the three quality categories of normal, overburning, and cold welding were 2000, 100, and 100, respectively. The dataset is seriously imbalanced. To prevent the imbalanced dataset from causing poor detection performance, this paper uses an oversampling method to expand the defect data at the data level. Meanwhile, to avoid incorrect sampling at the corresponding class, the improved weighted cross-entropy loss function was also used to increase the cost of data misclassification. A detailed description is shown in Section 5.3.

The synthetic minority oversampling technique for the multiclass imbalance algorithm [39] is an oversampling method based on k-nearest neighbors. Different to traditional oversampling methods based on k-nearest neighbors (the synthetic instances are generated randomly in the direction of k-nearest neighbors), SMOM gives a corresponding selection weight for each nearest neighbor direction, indicating the probability that it is used as the direction for synthetic instance generating. The selection weight for the nearest neighbor direction mainly considers the problem that the generation of synthetic instances along the nearest neighbor direction may have a negative impact on other classes (especially minority classes) and thus lead to overgeneralization. A smaller selection weight will be given to the neighboring directions with severe overgeneralization, so that the SMOM can establish a mechanism to avoid overgeneralization. The flowchart of the SMOM algorithm is shown in Figure 8.

On the basis of the above SMOM algorithm, the original defect data are expanded at a ratio of 1:5. The results are shown in Figure 9. Finally, the 2000, 500 and 500 sample profiles of normal, overburning, and cold welding are obtained to complete the construction of the RSW dataset. These data are divided into a training set, verification set and test set at a ratio of 3:1:1.

5.3. Model Training Process

To train the classification model, the most commonly used loss function is the cross-entropy loss function, which is calculated as follows:

$$J(w)=\frac{1}{m}{\displaystyle \sum _{i=1}^m{\displaystyle \sum _{c=1}^k{y}_{ic}}}\log (h_w({\mathit{X}}_i))$$

(4)

where m is the number of sample welding spots in the training set; k is the number of classification categories; y_ic indicates the quality label; X_i denotes the time-series input data vector of welding spot I; and h_w represents the model function.

Considering the imbalance between normal data and defective data, the traditional cross-entropy loss function can lead to low classification accuracy for the defective data. Therefore, on the basis of the idea of cost-sensitive learning, this paper introduces the weighted cross-entropy loss function as Equation (5) to enhance the misclassification cost of defective data so as to lower the overgeneralization of oversampling data and improve the classification accuracy of defective data:

$$J(w)=\frac{1}{m}{\displaystyle \sum _{i=1}^m{\displaystyle \sum _{c=1}^k{w}_c{y}_{ic}}}\log (h_w({\mathit{X}}_i))$$

(5)

where w_c is the weight of category c. Equation (5) shows that the loss of the category with the larger weight contributes more to the overall loss. Therefore, when the above loss function is used for the model training, it will lead to an increase in the detection accuracy of the model for defective data.

Combined with the loss function, the Adam optimization algorithm is applied in our paper to minimize the loss costs. Meanwhile, in the proposed model, the main parameters are shown in

Table 1. Model parameters.

Hyper Parameters	Value	Hyper Parameters	Value
Conv1 kernel size	2 × 3 × 12	Loss function weights	1:6:2
Pool1 kernel size	1 × 2 × 12	Dropout probability	0.5
Conv2 kernel size	1 × 3 × 24	Neurons number of LSTM	10
Pool2 kernel size	1 × 2 × 24	Optimization function	Adam
Activation function	Leaky ReLU	Max epoch	2000
Leaky ReLU negative axis slope	0.2	Initial Learning rate	0.0006

.

5.4. Result Analysis and Validation

5.4.1. Result Analysis

In order to quantitatively evaluate the effectiveness of the proposed model, the results of the model are evaluated in terms of accuracy Acc, precision P, and recall R, which are calculated as follows:

$$\begin{array}{l}Acc=\frac{TP+TN}{TP+TN+FP+FN}\\ P=\frac{TP}{TP+FP}\\ R=\frac{TP}{TP+FN}\end{array}$$

(6)

where the TP, TN, FP, FN are the number of true positives, true negatives, false positives, and false negatives. The TP and TN indicate that the model correctly predicts the positive class or negative class as a positive class or negative class. The FP and FN indicate that the model incorrectly predicts negative class or positive class as a positive class or negative class [10].

Combined with the training processes and evaluation indicators, the loss and accuracy of the training set and verification set are shown in Figure 10. The final training accuracy reaches 97.6%, which indicates that the training results are good.

To further verify the effectiveness of the model, the trained model was tested with the test set. The confusion matrix of the results is shown in Figure 11, and the evaluation indicators are shown in

Table 2. Evaluation Indicators of Test Set.

Items	Acc	P	R
Overall data	98.5%	/	/
Normal	/	98.7%	99%
Overburning	/	95.9%	95%
Cold welding	/	100%	100%

.

From the results, it can be seen that the overall accuracy of the test set reaches 98.5%, which is basically equivalent to the training set. At the same time, the accuracy and recall for three different categories are all above 95%. This indicates that the model proposed in this paper can still achieve good results in the face of an imbalanced dataset.

5.4.2. Analysis of Interpretability

The attention mechanism can focus attention on features that are more useful for the output results of the model and suppress the interference of useless information. The distribution of attention weights demonstrates the interpretability of the model. To investigate the impact, the input data of three quality categories were weighted according to the attention weight in the model and the visual representations are shown in Figure 12.

Attention weights are assigned based on the time period of the welding data. Therefore, the values of the attention weights are determined by the combined characteristics of the dynamic resistance and current in that time. From Figure 12, it can be seen that the model has high attention weights at the positions of the initial maximum resistance value and the final resistance value for three quality categories. This indicates that these two features are important factors in evaluating the quality of resistance spot welding. It is worth noting that the initial high attention weight period is not only located at the initial maximum dynamic resistance period but is also located at the period when the welding current reaches its maximum value. This characteristic is greatly important for determining defects in RSW. In addition, for overburning data, there are some high attention weights in the period when the resistance value is at a minimum and rises during the middle stage. Meanwhile, for cold welding data, there are high attention weights during the period when the dynamic resistance is at a maximum in the middle stage. The results indicate that the model assigns different attention weights to the different time periods and the different categories. The visualization of attention weights can help us to understand the final determination. Therefore, the attention mechanism endows the model with certain interpretability.

5.4.3. Analysis of Imbalanced Datasets

In the training phase, based on the cost-sensitive idea, this paper uses the weighted cross-entropy loss function as the model training evaluation metric to improve the detection performance for the imbalanced dataset. In order to analyze the effect of the loss function, this paper compared the results that either used the weighted cross-entropy loss function with the model or did not. The results are shown in

Table 3. Comparison of the models.

Items	No Weighted Cross-Entropy Loss Function			Using Weighted Cross-Entropy Loss Function
	Acc	P	R	Acc	P	R
Overall data	97%	/	/	98.5%	/	/
normal	/	96%	99.3%	/	98.7%	99%
overburning	/	97.6%	84%	/	95.9%	95%
cold welding	/	100%	99%	/	100%	100%

. It can be seen that, while using the weighted cross-entropy loss function, the recall rates for overburning and cold welding have increased by 11% and 1%, respectively. The accuracy of the overall data has also been improved from 97% to 98.5%. This reveals that the weighted cross-entropy loss function can improve the detection performance for minority and imbalanced data.

5.4.4. Comparisons and Analysis

To further verify the superiority of the proposed method, the quality detection results of the proposed model were compared with the manual sampling method, and the CNN, LSTM and CNN-LSTM models, which are shown in

Table 4. Comparisons of the quality detection results of different methods.

Model	Acc	Normal		Overburning		Cold Welding		Time (ms)
		P	R	P	R	P	R
Manual	91.0%	-	-	-	-	-	-	>1000
CNN	92.7%	95.0%	95.0%	82.6%	81.0%	91.0%	93.0%	13.6
LSTM	95.1%	96.6%	94.8%	81.4%	92.0%	100%	95.0%	14
CNN-LSTM	98.0%	98.0%	99.0%	95.8%	92.0%	100%	100%	15.8
Proposed	98.5%	98.7%	99.0%	95.9%	95.0%	100%	100%	7.3

.

From Table 4, it can be seen that the model proposed in this paper has the highest detection accuracy, 98.5%, which is 13.3%, 6%, 3.5%, and 1.5% higher than the manual, CNN, LSTM, and CNN-LSTM models, respectively. Note that manual inspection is a partial sampling method. In it, the Acc of the sampled data is 100%. However, the other data are not detected, and are all predicted as the normal quality. Therefore, the final Acc from all the tested data in this method is susceptible to randomness, which relies on the ratio of sampling and system stability. Moreover, the average detection time for test samples from the proposed model is obviously shorter, with 7.3 ms, which considers the network data transmission and model deployment. The manual method is not detected automatically, which spends much time. The other models are deployed in cloud, while the proposed model is deployed in edge. This reveals that the edge–cloud collaboration is useful for improving detection efficiency. In addition, in terms of defect data, the proposed model has recall rates of 95% and 100% for overburning and cold welding, respectively. The recall rate for overburning is 14%, 3%, and 3% higher than the CNN, LSTM, and CNN-LSTM models, respectively. The recall rate for cold welding is 7% and 5% higher than the CNN and LSTM models, respectively. Meanwhile, the cold welding is more inclined to be identified than overburning welding using our method.

5.4.5. Case Application and Validation

According to the operation framework showed in Figure 1, the online quality detection model for RSW is deployed to the edge computing equipment to verify the detection effect after model training is completed. The real-time online sample detection results are displayed in the management system, as shown in Figure 13. It involves the real-time online detection results and quality statistics results.

To verify the detection performance, the historical welding experiments data, including 20 normal, 10 overburning, and 10 cold welding experiments, were obtained and analyzed. Figure 14 shows the detection results and real in-site results for the three quality categories. On the basis of the detection model and interface exhibition, all the overburning and cold welding experiments were correctly identified. Nineteen out of the 20 normal welding models were correctly identified, and one was identified as overburning. The overall detection accuracy was 97.5%, and the average detection time was 7.32 ms.

From the above results, it can be seen that in the actual industrial production with our proposed method, the quality detection accuracy is higher and the efficiency is faster. Once one welding spot is fulfilled, the real-time quality detection result can be classed from the proposed model and given in the system interface. Compared with the manual sampling detection method, it greatly avoids randomness and implements the total detections for each welding spot without the use of destruction operations. This enhances detection efficiency, reduces detection costs, and improves detection accuracy.

In sum, to realize the model integration and practical application on in-site manufacturing systems, some strategies and suggestions are provided: (1) The edge–cloud collaboration network environment should be configured in the actual production field, which is the basis on which to implement the model application; (2) On the basis of the environment, the proposed model should be trained in cloud and deployed in edge. The model also needsto be updated regularly to ensure its adaptations for in-site welding processes; (3) The management system that integrates the detection model needs to be developed and deployed. It should clearly display the detect results, operation data, statistical results and alert information; (4) The detected quality defects and alert information should be considered to further implement the corresponding controls, such as modifying the welding process parameter, replacing electrode caps and adjusting robot postures, thus improving the manufacturing stability.

6. Conclusions

In response to the problem of poor detection performance using traditional manual methods and conventional data-driven methods, this paper proposed an RSW quality online detection method for automotive bodies based on CNN-LSTM and an attention mechanism under edge–cloud collaboration. The main contributions are given:

(1)

An online welding quality detection framework for RSW is proposed under edge–cloud collaboration. It contributes to improve the detection efficiency and real-time ability to solve the higher concurrency and huger data problem for multiple welding spots;

(2)

A novel online quality detection model for RSW is established. The model extracts local features and time-series related features of RSW data by CNN-LSTM network. The attention mechanism is introduced to make the model focus on more useful features, improving the interpretability and detection performance. Moreover, the integrated SMOM oversampling algorithm and weighted cross-entropy loss function improve the detection ability of the model on imbalanced data;

(3)

The case validation was conducted with actual data from the welding production process and the results were compared with other methods. The results indicate that the overall detection accuracy of the proposed method can reach 98.5%, and the recall and accuracy for defective welding spots are both above 95%. Meanwhile, the recall rate of the model proposed for overburning is 14%, 2%, and 2% higher than that of models using CNN, LSTM, and CNN-LSTM. The recall rate for cold welding is 6% and 5% higher than that of models using CNN and LSTM. Moreover, the overall detection accuracy can be improved by 5.9%, 3.4%, and 0.5%, respectively.

In sum, the proposed method has good detection performance and can meet the real-time detection requirements for RSW. Subsequent work will further analyze parameter sensitivity, and verify and apply the model in actual production processes. Meanwhile, the dynamic quality controlling strategies and methods, with the detection results, could also be studied.