Intelligent Fault Diagnosis of Broken Wires for Steel Wire Ropes Based on Generative Adversarial Nets

Abstract

The quantitative identification of broken wires is of great significance to maintain the safety of mechanical systems, such as steel wire ropes. However, in order to achieve high accuracy recognition results, a large number of fault samples are necessary, which is difficult to achieve in practical industrial detection. In this paper, a novel quantitative identification approach, based on generative adversarial nets (GANs) and a convolutional neural network (CNN), is proposed to solve the broken wire recognition problem in situations where real inspections have generated only a small sample of broken wires for analysis. One-dimensional original signals of broken wires are transformed into two-dimensional time-frequency images by continuous wavelet transform (CWT). Next, these time-frequency images are used for quantitative identification of various defects by combing GANs and CNN with limited samples. The main innovation of this paper is that the identification accuracy of broken wires can be improved by generating fault samples through GANs. The experimental results demonstrate that the proposed method achieves better recognition rates for broken wires compared with the existing detection methods.

1. Introduction

Steel wire ropes play an important load-bearing role in many mechanical systems [1]. The occurrence of broken wires may lead to unexpected security issues [2]. Regular replacement is a commonly used in engineering to avoid wire rope accidents, as such accidents usually cause huge economic waste [3]. However, the rope is still potentially safety hazard before the replacement period [4,5], therefore, it is necessary to explore an intelligent diagnosis method to safely maintain the operation of steel wire ropes.

In recent years, many studies have been carried out to realize the quantitative identification of broken wires, which includes signal processing methods, feature extraction, recognition models, and so on [6,7,8,9]. Zhang et al. [6] proposed a denoising method based on ensemble empirical mode decomposition (EEMD) to reduce the noise of the remanence signals caused by the defects. A back propagation (BP) neural network was used for the identification of different broken wires. Zhang et al. [7] built a two-step neural network to distinguish internal and surface defects of the wire rope, in which the MFL signals of broken wires are received by both the induction coil and Hall sensor. Liu et al. [8] present a surface detection method for steel wire ropes using deep learning and computer vision techniques. Liu et al. [9] combined signal analysis and a convolutional neural network (CNN) for the identification of wire rope defects.

However, a large number of fault samples are required for the methods mentioned above to find the relationship between damage information and various defects, which is difficult to implement in practical detection. The testing process must be controlled to minimize machine downtime and ensure orderly production. Therefore, intelligent fault diagnosis methods are urgently demanded to monitor the condition of steel wire ropes in the case of small samples of faults.

Generative adversarial networks (GANs) have been attracting considerable attention since they were proposed in 2014 [10,11]. New samples, different from the original data but with similar distribution, can be generated through the GANs, which provides a new method for fault diagnosis under the condition of small samples [12]. GANs have been successfully applied in the domain of machine fault diagnosis [13,14]. Liang et al. [13] proposed an intelligent fault diagnosis for rotating machinery based on wavelet transform (WT), GANs, and a convolutional neural network (CNN). Langevin A et al. [14] developed an auxiliary classifier GAN framework for machine fault diagnosis, which was verified through induction motor vibration signal datasets. However, there are few related studies in the field of wire rope fault diagnosis. Hence, in this paper, the GANs are first attempted for the wire rope detection, and the effectiveness of the network is evaluated through commonly broken wire experiments.

The main contributions of this paper can be summarized as follows:

(1)

A novel intelligent detection method is presented by combing GANs and CNN, which can be applied to the quantitative identification of surface and inner broken wires of wire ropes in the case of small samples.

(2)

By using the GANs for the expansion of the fault data set to achieve higher accuracy of quantitative detection, which can adapt to the actual complex conditions of broken wire inspection.

The remaining part of the paper is organized as follows. The theory of GANs is introduced in Section 2. In Section 3, the detailed description of the whole framework based on GANs is illustrated. Section 4 presents the experimental results of the proposed method. Finally, the conclusions and discussion are drawn in Section 5.

2. Theoretical Background

2.1. Generative Adversarial Nets (GANs)

Due to the great success in data generation, GANs have become one of the deep network models that has the most potential [15,16]. GANs are inspired by a competitive game [10]. It consists of two networks trained together, a generative model (generator, G) and a discriminative model (discriminator, D), shown in Figure 1. The generator is trained by using random noise to produce fake samples, trying to deceive the discriminator, while the discriminator is trained to distinguish between generated and real data. Through the continuous adversarial learning of the two models, the performance of the generator and the discriminator can be improved simultaneously. The goal of the optimization is to achieve Nash equilibrium between G and D [17,18]. The objective function can be expressed as follows:

$$\underset{G}{\min }\underset{D}{\max }V\left(D,G\right)=E_{x\sim p_{data}(x)}\left[\log D\left(x\right)\right]+E_{z\sim p_z(z)}\left[\log \left(1-D\left(G\left(z\right)\right)\right)\right]$$

(1)

where $G$ and $D$ represent the generator and the discriminator; $p_{data}$ is the distribution of real sample; and $p_{\mathrm{z}}$ is the prior distribution of the input noise variables. During the training process, $D$ is trained to maximize the probability of the discriminator to distinguish both real and generated data, while $G$ is trained to minimize $\log \left(1-D\left(G\left(z\right)\right)\right)$ to generate realistic data.

2.2. Convolutional Neural Network

Convolutional neural networks (CNN) have become one of the most commonly used deep network models, as they can automatically extract feature information from the original data [19]. CNN is mainly composed of a convolutional layer, a pooling layer and a fully connected layer. Its typical structure is displayed in Figure 2.

As feature extractors, the convolutional layers can learn the feature information of the input images. The core idea of convolution is to reduce the number of parameters to be trained by using local receptive fields and sharing weights. The calculation process of the convolutional layer can be expressed as follows [20]:

$$x_j^l=f({\displaystyle \sum _{i\in M_j}{x}_i^{l-1}\times k_{ij}^l+b_j^l})$$

(2)

where the $\times$ represents the convolution operation; $x$ is the input image; $k_{ij}^l$ represents the weight matrix; $M_j$ represents the selection of the input feature map; $b_j^l$ is the bias.

Followed by one or more convolution layers, the pooling layer is used to reduce the dimension of the feature maps and ensure the spatial invariance of the feature. The most commonly used pooling functions are max pooling function and average pooling function. Average pooling refers to calculating the average value of all units in the pooled receptive field as the new eigenvalue, while max pooling refers to selecting the maximum value as the new eigenvalue in the pooled receptive field [21].

After several combined convolution layers and pooling layers, there is usually a fully connected layer to realize classification or logical regression [22]. In order to distinguish $T$ categories, the number of neurons in the fully connected layer should be set to $T$.

3. Proposed Method

Usually, it is necessary to train the deep learning model with enough fault samples, which is difficult to implement in practical detection. Since the wire rope works properly most of the time, it is easy to acquire normal samples, while it is relatively difficult to obtain fault data. Therefore, it is difficult to achieve accurate recognition from the limited sample size of fault data.

In this paper, a new method is proposed for quantitative identification of broken wires based on GANs. The framework of the proposed method is illustrated in Figure 3. The specific process of fault diagnosis is displayed as follows:

(a)

The broken wire signals of the steel wire rope are collected by a probe using the magnetic flux leakage (MFL) method.

(b)

In order to make the broken wire signals suitable for model training, these original damage signals are transformed into time-frequency images through continuous wavelet transform (CWT).

(c)

This image data are divided into a training set and a testing set. In order to meet the requirements of deep learning model training, the training set is used to generate more fault samples based on GANs. These generated data and original data are combined and input into the CNN model for training.

(d)

The trained CNN model is used to classify the testing set to achieve broken wire diagnosis of the wire rope.

For the purpose of data generation, a deep convolutional generative adversarial networks (DCGANs) are selected in this paper because of its advantage in generating high-quality images [18]. The structure of DCGANs can be seen in Figure 4. The training process of GANs is very unstable, which often leads to meaningless output of the generator [23]. Compared with GANs, some structural changes were made in DCGANs to improve the quality of samples. The CNN is used in the discriminator and generator of the DCGANs to replace the multi-layer perceptron in the GANs. At the same time, to make the whole network differentiable, the pooling layer in the CNN is removed. In DCGANs, deconvolution is used in the generator and the convolution architecture is adopted in the discriminator.

As shown in Figure 5, a CNN is used in this paper for fault classification [9]. This CNN consists of three convolutional layers with the convolution kernel size of 11 × 11. Between the convolutional layers, the max pooling layer is deployed for feature dimensionality reduction. Following the convolution layers and pooling layers, there is a fully connected layer to realize classification of T-type defects.

4. Experimental Study

4.1. Experimental Setup

To validate the effectiveness of the proposed method, the steel wire rope broken wire experiment of is conducted on the safety detection test rig in Qingdao University of Technology. Six various defects are hand-crafted on the wire rope: the first three injuries are surface broken wires and the last three injuries are the internal broken wires, which is illustrated in Figure 6. The fracture length of each broken wire is 13 mm, and the number of broken wires varies from 1 to 3. According to the number of broken wires and different positions, these defects are divided into six patterns, shown in

Table 1. Data description.

Label	Fault Description
1	1 surface broken wire
2	2 surface broken wires
3	3 surface broken wires
4	1 internal broken wire
5	2 internal broken wires
6	3 internal broken wires

.

The steel wire rope test rig is shown in Figure 7, which is composed of the framework, a movable tray, a detector, and a loading device. During the testing process, the damaged wire rope is fixed to the framework and loaded through the loading device. The detector follows the movable tray to scan the rope and collect signals. This detector is designed and manufactured by our laboratory based on the magnetic flux leakage principle [4]. It is a magnetic concentrating sensor, which is used to collect the magnetic flux leakage (MFL) signals generated by broken wires. The damage signals obtained by the detector are analog-to-digital (A/D) converted by National Instrument (NI) 4496 Card, and displayed and stored in real time by a designed LabVIEW program, shown in Figure 8.

4.2. Data Preprocessing

The original oscillation signal collected in the experiment is shown in Figure 9. It can be clearly seen that there are 6 mutations (MFL signals) in the figure below, corresponding to 6 kinds of defects in Figure 6. However, it is very difficult to distinguish the different faults in the time domain. In order to realize the identification of various broken wires, each MFL signal with 1024 data points is cut into a data segment. Then, these data segments are transformed into time-frequency images by continuous wavelet transform (CWT). The conversion is implemented in MATLAB through the “cwtfilterbank” function, where the analytic Morse wavelet is used in the filter bank. The conversion result is shown in Figure 10.

A total of 200 groups of oscillation signals were collected in the detection experiment. Therefore, for each pattern in Table 1, the MFL signals are converted into 200 time-frequency images. There are 1200 samples in total for 6 kinds of injuries, of which 900 samples are used for data generation and CNN training, and the remaining 300 samples are used for testing.

The training set is used for generating more samples by the GANs mentioned in Section 3. For each type of damage, 50 fake images are generated, which is shown in Figure 11. As can been seen from Figure 11, these generated images are similar to the original images. At the same time, these fake pictures contain some new features. By this operation, the fault sample is expanded, the training of the classifier can be strengthened, and the recognition accuracy of the model can be further improved.

The training set and the generated data are used to train the CNN. After the training, the testing set is used to test the effectiveness of fault recognition. As shown in Figure 12, a confusion matrix is employed to visualize the fault classification effect of the proposed method. Obviously, almost all faults have been correctly identified, the recognition accuracy is 99.3%.

In addition, the proposed method is compared with the back propagation (BP) neural network and the traditional CNN. For the BP network, some artificial features are extracted from MFL signals (Figure 9) for training. For the conventional CNN, the training set in this paper is used for training without generating data, and the testing set is used for identification. The accuracy of each fault and the total identification accuracy are revealed in Figure 13. Obviously, there is a fluctuation in the identification accuracy of different fault cases. However, the proposed method has the highest classification accuracy for each fault. Compared with the BP network and the traditional CNN, the GAN+CNN method has the best recognition accuracy.

In order to show the powerful ability of the proposed GAN-based model in broken wire recognition, the features learned by the sofmax layer of the CNN are represented by t-distributed stochastic neighbor embedding (t-SNE), which is compared with manual features in the BP and a single CNN, as shown in Figure 14. It can be seen from Figure 14 that artificial features failed to cluster different faults, and a serious coverage occurs between different broken wire samples. For a single CNN, the clustering of faults is more obvious, and only a few errors occur. For the proposed GAN+CNN method, the boundaries of different faults are more clearly distinguished. At the same time, compared with the other two methods, the coverage phenomenon of various fault samples is lowest using the proposed method.

5. Conclusions

In this paper, an intelligent fault diagnosis method based on GAN is proposed to identify various broken wires of steel wire ropes. The MFL signals of broken wires are transformed into time–frequency images by CWT. Data generation is carried out through the GAN, and fault identification is realized by a CNN. By combining GAN and CNN, the recognition accuracy of broken wires is improved when compared with a BP network or a conventional CNN. The experimental results show that the accuracy of the proposed method can reach 100% for 6 kinds of internal and external broken wires. The GAN-CNN based model has great application prospects for faults diagnosis of steel wire ropes.

As for the GAN based model, this paper only studies the identification effect of several internal and external faults under laboratory conditions. The recognition performance of the model for more faults under complex working conditions has not been explored. The effective application of this model to practical detection will be the key research content in future work.