Algorithm Development for the Non-Destructive Testing of Structural Damage

Abstract

Monitoring of structures to identify types of damages that occur under loading is essential in practical applications of civil infrastructure. In this paper, we detect and visualize damage based on several non-destructive testing (NDT) methods. A machine learning (ML) approach based on the Support Vector Machine (SVM) method is developed to prevent misdirection of the event interpretation of what is happening in the material. The objective is to identify cracks in the early stages, to reduce the risk of failure in structures. Theoretical and experimental analyses are derived by computing the performance indicators on the smart aggregate (SA)-based sensor data for concrete and reinforced-concrete (RC) beams. Validity assessment of the proposed indices was addressed through a comparative analysis with traditional SVM. The developed ML algorithms are shown to recognize cracks with a higher accuracy than the traditional SVM. Additionally, we propose different algorithms for microwave- or millimeter-wave imaging of steel plates, composite materials, and metal plates, to identify and visualize cracks. The proposed algorithm for steel plates is based on the gradient magnitude in four directions of an image, and is followed by the edge detection technique. Three algorithms were proposed for each of composite materials and metal plates, and are based on 2D fast Fourier transform (FFT) and hybrid fuzzy c-mean techniques, respectively. The proposed algorithms were able to recognize and visualize the cracking incurred in the structure more efficiently than the traditional techniques. The reported results are expected to be beneficial for NDT-based applications, particularly in civil engineering.

1. Introduction

Structural health monitoring (SHM) using non-destructive inspection (NDI) has been known as one of the technologies in which the material properties can be evaluated without the occurrence of any damages A. NDI and Non-destructive Testing (NDT) methods rely on the employment of electromagnetic waves and intrinsic characteristic attributes of materials to test samples. One of the most commonly known NDI/NDT techniques is ultrasonic inspection, which engages transducers to generate mechanical oscillations through the inspected material, and receives the reflected ultrasonic waves from the material [1]. The analysis of this data has the potential to detect structural damage in the early stages, by minimizing errors, and thus reducing the asset management costs. Despite the vast number of studies in this field, few robust methods have been proven to indicate an adverse condition of a structure in service. Also, many researchers have concentrated on the automated detection of defects [2,3,4,5,6,7,8,9,10,11,12], but few studies have been done into the comprehensive or well-ordered data analysis of systems, and hence this is the motivation for the present research.

The solution requires a classifier that can discriminate structures as either being damaged or intact. Concerning AI techniques, the most commonly used classifiers are based on Discriminant Analysis, Artificial Neural Network, K-Nearest Neighborhood, Support Vector Machine (SVM), Support Vector Regression, Adaptive Neuro-Fuzzy System (ANFIS), Fuzzy Inference System (FIS) and Decision Trees. These algorithms have successfully been utilized to solve engineering problems [3,4,5,6,13,14,15]. The SVM method has been extensively used in many research studies, and it is accepted as an efficient classifier for detecting damage [7,8,9]. However, there had been a substantial increase in utilizing advice sets for enhancing SVM performance. However, applying and expressing this knowledge in terms of its constraints still has some difficulties. These techniques require new parameters and operations that can enhance the SVM computational cost. In this regard, the first contribution of this study is to develop non-iterative SVM-based algorithms for analyzing the sensor (smart aggregate) data, which can improve the performance of SVM, rather than traditional SVM.

Furthermore, the local structural health monitoring based on microwave and millimeter-wave imaging is an efficient approach for visualizing and localizing detected damage in structures [10,11,12,16,17,18]. These methods offer cost-effective methods of local inspection, to identify the exact location of the crack [19,20]. Microwave- and millimeter imaging techniques are non-contact and one-sided techniques which can be efficiently applied for the non-destructive evaluation of non-transparent materials, as their signals can penetrate the dielectric materials and provide information about the entire structure [21,22]. In other words, these techniques examine microwave pulses or continuous wave signals, to offer comprehensive descriptions of the systems. In common microwave imaging inspections, the reflection from the test specimen is recorded by the radio frequency (RF) detector, and it is used for the detection of damage inside the specimen [21,23,24]. In general, to obtain the desired range resolution and 3D images, relatively complex wide-frequency range measurement systems and image processing algorithms are required. In many applications, wide band techniques may not be applicable, due to specific requirements, such as the depth of penetration in composite materials, and the cost of the techniques. However, there are various microwave- and millimeter-wave imaging systems that have been used for NDT in structures, including relatively simple single-frequency reflectometers [10,21,22,25], and also relatively complex wide-band reflectometers, such as synthetic aperture radar (SAR) [11,26]. However, most of the generated images have very low resolution, in which the investigating object is not recognizable. Therefore, the proper image processing techniques [27] are required to handle the imaging data. In this regard, the second contribution of this study is to develop algorithms using image processing techniques to create enhancement and segmentation of obtained microwave images of civil structures for the recognition of anomalies, and to obtain the area of interest. In contrast to the results of prior research [22], this study performs a systematic investigation of the problem, and this is followed by various image processing approaches. The methodology of this research relies on modern computer vision techniques, instead of traditional photogrammetric methods.

The remainder of the paper is organized as follows: The damage detection technique based on ML is described in Section 2 of this paper. Then, damage detection and visualization, using the microwave- and millimeter-wave imaging systems are presented in detail. Third, experimental setups are explained, and then the experimental results are investigated, to evaluate the validity of the proposed data analysis procedure, in the Results and Discussion. This is followed by a discussion, and finally conclusions.

2. Damage Detection Using a Machine Learning Approach

2.1. Support Vector Machine

Support vector machine (SVM) is a discriminative classifier which was proposed in 1995 by Vapnik, to analyze data for classification and regression purposes [28]. SVM provides the optimal hyperplane by separating two classes with the maximum margin from the hyperplane. Assume an SVM binary classification with N samples in the data training set $\left({\mathrm{x}}_1,{\mathrm{y}}_1\right),\dots ,\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{y}}_{\mathrm{i}}\right),\dots ,\left({\mathrm{x}}_{\mathrm{N}},{\mathrm{y}}_{\mathrm{N}}\right)\in {\mathrm{R}}_{\mathrm{n}}\times \left\{\pm 1\right\}$, where ${\mathrm{x}}_{\mathrm{i}}$ is a feature vector and ${\mathrm{y}}_{\mathrm{i}}$ is the label of its class. The hyperplane is mathematically defined as the pair (w, b) in the form: w^Tx + b = 0, where w is a weight vector and b is the bias. The hyperplane should meet the following condition as described in Equation (1), when linearly separating the training data [29]:

$${\mathrm{y}}_{\mathrm{i}}\left(\mathrm{w}\times {\mathrm{x}}_{\mathrm{i}}+\mathrm{b}\right)\ge 1,\mathrm{i}=1,\dots ,\mathrm{N}$$

(1)

The distance between the hyperplane and each of the training data ${\mathrm{x}}_{\mathrm{i}}$ is defined through Equation (2) as [29]:

$${\mathrm{d}}_{\mathrm{i}}=\frac{\mathrm{w}\times {\mathrm{x}}_{\mathrm{i}}+\mathrm{b}}{\Vert \mathrm{w}\Vert }$$

(2)

By considering both Equations (1) and (2), the following result is obtained for all ${\mathrm{x}}_{\mathrm{i}}$ as [29]:

$${\mathrm{y}}_{\mathrm{i}}{\mathrm{d}}_{\mathrm{i}}\ge \frac{1}{\Vert \mathrm{w}\Vert }$$

(3)

where $\frac{1}{\Vert \mathrm{w}\Vert }$ is considered as the lower distance bound between the hyperplane and the training data ${\mathrm{x}}_{\mathrm{i}}$. The maximum margin of the hyperplane is obtained through the minimization of Equation (4):

$$\mathrm{z}=\frac{1}{2}\mathrm{w}\times \mathrm{w}$$

(4)

Therefore, the final decision function is given as [30]:

$$\mathrm{f}\left(\mathrm{x}\right)=\mathrm{sign}\left({\displaystyle \sum }_{{\mathsf{\alpha }}_{\mathrm{i}}>0}{\mathrm{y}}_{\mathrm{i}}{\mathsf{\alpha }}_{\mathrm{i}}<\mathrm{x},{\mathrm{x}}_{\mathrm{i}}>+\mathrm{b}\right)$$

(5)

where $\left({\mathsf{\alpha }}_1,{\mathsf{\alpha }}_2,\dots ,{\mathsf{\alpha }}_{\mathrm{N}}\right)$ represents the non-negative Lagrange multipliers related to the constraints (see Equation (1)). However, most of these α are usually zero, leading to the small proportion of the training data x being considered as vector w. Since these points are considered as the closest points to the hyperplane, they are named the Support Vectors (SV). SVs are the training patterns that fall on the margin boundaries. In general, a small portion of the training sample SVs are used by SVM for the purpose of classification. However, if the SVs consist of the training data beyond the corresponding margins, they are referred to as misclassified data [31,32,33]. If the linear separation of data is not feasible, then the hyperplane is pointless. Therefore, N non-negative $\left({\mathsf{\xi }}_{\mathrm{i}},\mathrm{i}=1,\dots ,\mathrm{N}\right)$ variables are considered, such that:

$${\mathrm{y}}_{\mathrm{i}}\left(\mathrm{w}\times {\mathrm{x}}_{\mathrm{i}}+\mathrm{b}\right)\ge 1-{\mathsf{\xi }}_{\mathrm{i}}$$

(6)

This variable, ${\mathsf{\xi }}_{\mathrm{i}}$, will include a very small quantity of misclassified points, and this will be zero if the Equation (1) condition is met; otherwise the term $-\mathsf{\xi }$ will be added to Equation (1), and this generates Equation (6). However, the tolerance parameter may also cause the ignorance of some training data in order to make the linear hyperplane. Then, the generalized separation hyperplane is expressed by minimizing Equation (7):

$$\mathrm{z}=\frac{1}{2}\mathrm{w}\times \mathrm{w}+\mathrm{c}{\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{N}}{\mathsf{\xi }}_{\mathrm{i}}$$

(7)

The expression $\mathrm{c}{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{N}}{\mathsf{\xi }}_{\mathrm{i}}$ will supervise the number of misclassified points. For smaller values of c, the above solution maximizes the minimum distance of 1/w, and for large values of c, it minimizes the number of misclassified data. It is noted that the use of misclassified data can help to enhance the performance of classifiers.

2.2. Developed Support Vector Machine

In the implementation of a traditional SVM algorithm, the first type of support vectors or hyperplane position is the only information that is utilized in the test phase of SVM from the training phase, while the purpose of this algorithm is utilizing the second type of support vectors to provide the subsequent knowledge by the purpose of enhancement in SVM performance. The second type of support vectors is selected, as there is a lot of misclassified data, even in the existence of the optimized hyperplane [31]. Two potential sources of this misclassified data are the outliers and the non-linear separable data when using kernels. Traditional SVM do not consider non-linear separable data during the training phase. This happens in defining the constraints and also the tolerance parameters of the objective functions. The main concern herein is that if the data in the test set appears to be practically the same as these misclassified data in the training set, they will be essentially classified incorrectly as being misclassified data. This misclassification may reduce accuracy, as it happens because the SVM ignores the information in the training phase. Therefore, in order to obtain the benefits from misclassified data, the outlier’s effect should be taken into account. Searching to find more similar data to the misclassified data, as proposed in this research, can enhance SVM performance. In this method, after determining the misclassified data in the training phase, the SVM will provide the advice weights, which are used in conjunction with the decision values in the testing phase. These defined advised weights will help SVM to improve its accuracy, while eliminating the outlier data. Equation (8) expresses the obtained misclassified data of the training phase as [29]:

$$\mathrm{MD}=\underset{\mathrm{i}=1}{\overset{\mathrm{N}}{{\displaystyle \cup }}}{\mathrm{x}}_{\mathrm{i}}|{\mathrm{y}}_{\mathrm{i}}\ne \mathrm{sign}\left({\displaystyle \sum }_{{\mathsf{\alpha }}_{\mathrm{j}}>0}{\mathrm{y}}_{\mathrm{j}}{\mathsf{\alpha }}_{\mathrm{j}}\mathrm{k}\left({\mathrm{x}}_{\mathrm{i}},{\mathrm{x}}_{\mathrm{j}}\right)+\mathrm{b}\right)$$

(8)

The right side of the above equation may include any SVM decision function. Although there is the possibility that misclassified data may be null, experimental results have shown that the incidence of the misclassified data is prevalent. For each misclassified data in the MD set, the length from the corresponding k-nearest neighbor (k = 10) that has been correctly classified is computed as:

$$\mathrm{CL}\left({\mathrm{x}}_{\mathrm{i}}\right)={\mathrm{Minimum}}_{{\mathrm{x}}_{\mathrm{j}}}\left(\Vert \mathrm{cl}\left({\mathrm{x}}_{\mathrm{i}}\right)-\mathrm{cl}\left({\mathrm{x}}_{\mathrm{j}}\right)\Vert \right)$$

(9)

$$\mathrm{N}=1,2,\dots ,10,{\mathrm{y}}_{\mathrm{i}}\#{\mathrm{y}}_{\mathrm{j}}$$

$$\mathrm{cl}\left({\mathrm{x}}_{\mathrm{i}}\right)=\frac{{\mathrm{x}}_{\mathrm{i}}}{{\mathrm{M}}_{\mathrm{d}}}$$

(10)

$$\mathrm{cl}\left({\mathrm{x}}_{\mathrm{j}}\right)=\frac{{\mathrm{x}}_{\mathrm{j}}}{{\mathrm{M}}_{\mathrm{d}}}$$

(11)

$${\mathrm{M}}_{\mathrm{d}}=\frac{1}{\mathrm{N}}{{\displaystyle \sum }}_{\mathrm{j}=1}^{\mathrm{N}}({\mathrm{x}}_{\mathrm{i}}-{\mathrm{x}}_{\mathrm{j}})$$

(12)

where $\mathrm{CL}\left({\mathrm{x}}_{\mathrm{i}}\right)$ is the dispersion of the misclassified data from the 10-nearest neighbors that have been correctly classified. ${\mathrm{M}}_{\mathrm{d}}$ represents the average of the distances within the k-nearest neighbor (k = 10) of ${\mathrm{x}}_{\mathrm{i}}$. However, by mapping the training data into a higher dimension, Equation (13) can be used with respect to kernel k to compute the length as:

$$\left(\Vert \mathsf{\theta }\mathrm{cl}\left({\mathrm{x}}_{\mathrm{i}}\right)-\mathsf{\theta }\mathrm{cl}\left({\mathrm{x}}_{\mathrm{j}}\right)\Vert )\right)=\left(\mathrm{k}\left(\mathrm{cl}\left({\mathrm{x}}_{\mathrm{i}}\right),\mathrm{cl}\left({\mathrm{x}}_{\mathrm{i}}\right)\right)\right)+\left(\mathrm{k}\left(\mathrm{cl}\left({\mathrm{x}}_{\mathrm{j}}\right),\mathrm{cl}\left({\mathrm{x}}_{\mathrm{j}}\right)\right)\right)-2\mathrm{k}{\left(\mathrm{cl}\left({\mathrm{x}}_{\mathrm{i}}\right),\mathrm{cl}\left({\mathrm{x}}_{\mathrm{j}}\right)\right)}^{0.5}$$

(13)

where $\mathrm{k}\left(\mathrm{cl}\left({\mathrm{x}}_{\mathrm{n}}\right),\mathrm{cl}\left({\mathrm{x}}_{\mathrm{n}}\right)\right)=ø\left(\mathrm{cl}\left({\mathrm{x}}_{\mathrm{n}}\right)\right)·ø\left(\mathrm{cl}\left({\mathrm{x}}_{\mathrm{n}}\right)\right) \left\{n=i or j\right\}$, and represents the Kernel function. It defines the inner product in the transformed space, and reduces the complexity of finding the mapping function.

By computing the CL for each x_k in the test set, the self-weightings (SWs) are assigned as:

$$\mathrm{W}=\{\begin{array}{l}0\forall {\mathrm{x}}_{\mathrm{i}}\in \mathrm{MD}\mathrm{if}\mathrm{MD}\mathrm{is}\mathrm{empty}\mathrm{or}\left(\left|\left|\mathrm{cl}\left({\mathrm{x}}_{\mathrm{k}}\right)-\mathrm{cl}\left({\mathrm{x}}_{\mathrm{i}}\right)\right|\right|\right)>\mathrm{CL}\left({\mathrm{x}}_{\mathrm{i}}\right)\\ 1-\frac{{{\displaystyle \sum }}_{\mathrm{xi}}\left|\left|\mathrm{cl}\left({\mathrm{x}}_{\mathrm{k}}\right)-\mathrm{cl}\left({\mathrm{x}}_{\mathrm{i}}\right)\right|\right|}{{{\displaystyle \sum }}_{\mathrm{xi}}\mathrm{CL}\left({\mathrm{x}}_{\mathrm{i}}\right)}{\mathrm{x}}_{\mathrm{i}}\in \mathrm{M}\mathrm{if}\mathrm{MD}\mathrm{is}\mathrm{empty}\mathrm{or}\left(\left|\left|\mathrm{cl}\left({\mathrm{x}}_{\mathrm{k}}\right)-\mathrm{cl}\left({\mathrm{x}}_{\mathrm{i}}\right)\right|\right|\right)\le \mathrm{CL}\left({\mathrm{x}}_{\mathrm{i}}\right)\end{array}$$

(14)

The obtained SWs show the closeness of the test data to the misclassified data in the training phase. Therefore, the developed self-weighting SVM (SW-SVM) can be concluded as follows:

• In the training phase, perform the SVM training.
• Use Equation (8) to find the misclassified data (MD).
• Investigate the existence of misclassified data maintained in a MD structure; if the MD includes the data, the CL is computed through Equation (9) for each member of MD; otherwise the normal SVM procedure is continued.
• In the testing phase, compute the self-advised weights of each x_k in the test set.
• For each x_k in the test set, the absolute values of the SVM decision values are computed, and normalized or scaled to [0,1].
• The SVM labelling is followed, based on the conditions in Equation (14). The normal SVM labelling is performed if SW (x_k) < decision value (x_k).

3. Damage Visualization

3.1. Developed Image Processing Algorithms

The proposed algorithms are used to visualize and detect the suspected critical cracks, using a spot measurement on the structures. The basis of this section relies on the fact that the structural components provide pixels that are visible to the high-resolution sensors, which can demonstrate the topographical appearance characteristics of structural integrity. The changes in structural integrity can be recognized by the variation of image intensities at different intervals. To begin with, upon the antenna measurements of the specimens, the information of the returned signals from the target was gathered as the raw data. The recorded data included the magnitude and phase at different frequencies. This data was processed to achieve a two-dimensional image, through Fourier Transform (FT). The phase and magnitude at each frequency were multiplied by a ‘Kaiser’ window function to minimize the FFT affect over non-integer numbers. The windowed frequency data was transformed to the time-domain, by employing an inverse Fourier transform. Therefore, the antenna time-domain measurement result was subsampled as an image in Matlab.

3.1.1. Pre-Processing

The obtained images were converted into greyscale, in which the minimum intensity value was equal to 0, and the maximum intensity value was equal to 255 [34]. Then, the averaging filter and contrast enhancement techniques were used to precede the enhancement of the raw images. Enhancement in this case, consists of smoothing the image while preserving as much of its grey tonality as possible. Moreover, computational efficiency is an essential consideration in image processing, due to the size of data sets. Some filters, such as average filter, can be implemented by division into two simpler filters, lead to substantial time savings [27]. Assume an n × n filter that can be separated into two smaller sizes of n × 1 and 1 × n. The performance of the n × n filter involves n² multiplications, and also n² − 1 additions for every pixel in an image, while the performance of n × 1 filter only requires n multiplications and n−1 additions. In total, the two smaller filters involve 2n multiplications and 2n − 2 additions. This efficiency savings is dramatic when n is large [27,35]. Therefore, in this study, a rectangular averaging filter size of 3 × 3 was considered and implemented by division into two smaller filters. Firstly, 3 × 1 average filters were applied to the raw image, and then a 3 × 1 average filter preceded the result. The new grey values of the corresponding pixel were calculated by taking the average of all nine values in the 3 × 1 and 3 × 1 masks. This process was followed by a Gamma transformations function to adjust the intensity. With Gamma transformations, the grayscale components can be curved to either brighten or darken the intensity [36]. The MATLAB function, imadjust (), was used herein to create Gamma transformations. Since our images were of the class ‘uint8′, the function multiplied the values by 255 to specify the actual values of an image. After pre-processing of images, cluster analysis was carried out for steel plate (SP), composite materials (CM), and metal plate (MP) images.

3.1.2. Cluster Analysis

3.1.2.1. Segmentation of Steel Plate (SSP) Images

The gradient magnitude was calculated on pre-processed SP images by computing the gradients in four directions of an image with the Prewitt filter. The edges of the crack were returned at those points where the gradient of the image was maximized. Therefore, the edges were obtained by using the Prewitt approximation to the derivative. Assuming the profile of the image as f(x), the detailed pseudocode of this algorithm is described as follows.

Pseudocode 1. The proposed FT-based algorithms for defect segmentation in microwave imaging.

Input: Phase and Magnitude

Output: True Target Segmentation

1. Pre-processing of SP images as in Section 1.

2. Computation of the derivative of the input microwave image as f′(x) to identify the direction of the maximum changes: $$\frac{\mathrm{dy}}{\mathrm{dx}}=\underset{\mathrm{h}\to 0}{\lim }\left(\frac{\mathrm{f}\left(\mathrm{x}+\mathrm{h}\right)-\mathrm{f}\left(\mathrm{x}-\mathrm{h}\right)}{\mathrm{h}}\right)$$ where h has the value of ‘1’ as the smallest possible value that can occur in an image; therefore, the discrete version of the derivative expression is the form of: $$\mathrm{f}\left(\mathrm{x}+1\right)-\mathrm{f}\left(\mathrm{x}-1\right)$$

3. Consideration of the horizontal, vertical, and diagonal filters based on Prewitt approximation in four directions: These filters find horizontal, vertical, and diagonal edges in an image while smoothing the results. $${\mathrm{P}}_{\mathrm{x}}=\left(\begin{array}{ccc}-1& 0& 1\\ -1& 0& 1\\ -1& 0& 1\end{array}\right)\mathrm{Vertical}\mathrm{filter}$$ $${\mathrm{P}}_{\mathrm{y}}={\mathrm{P}}_{\mathrm{x}}^{\prime }\mathrm{Horizontal}\mathrm{filter}$$ $${\mathrm{P}}_{\mathrm{d}1}=\left(\begin{array}{ccc}0& 1& 1\\ -1& 0& 1\\ -1& -1& 0\end{array}\right)\mathrm{Diagonal}\mathrm{filter}(\mathrm{Right}\mathrm{to}\mathrm{left})$$ $${\mathrm{P}}_{\mathrm{d}2}=\left(\begin{array}{ccc}-1& -1& 0\\ -1& 0& 1\\ 0& 1& 1\end{array}\right)\mathrm{Diagonal}\mathrm{filter}(\mathrm{Right}\mathrm{to}\mathrm{left})$$

4. Assuming that Px, Py, Pd1, and Pd2 as the grey values produced by applying the above filters to an image, the gradient magnitude is computed by: $$\mathrm{GM}=\sqrt{{\mathrm{p}}_{\mathrm{x}}^2+{\mathrm{p}}_{\mathrm{y}}^2+{\mathrm{p}}_{\mathrm{d}1}^2+{\mathrm{p}}_{\mathrm{d}2}^2}$$

5. Computing the threshold value of the image by Otsu’s thresholding method.

6. Generating the binary image that contains edges by only applying the threshold value.

3.1.2.2. Segmentation of Composite Material (SCM) Images

Three algorithms are proposed for this purpose;

Algorithm 1: In the first algorithm, the 2D-FFT, which is one of the most fundamental tools in image analysis and an appropriate candidate for grey level images [37], is computed on CM images. It provides a more precise separation of the desired information than the 1D transform [38]. It can be mathematically formulated as:

$${\mathrm{Y}}_{\mathrm{p}+1,\mathrm{q}+1}={\displaystyle \sum }_{\mathrm{j}=0}^{\mathrm{m}-1}{\displaystyle \sum }_{\mathrm{k}=0}^{\mathrm{n}-1}{\mathrm{w}}_{\mathrm{m}}^{\mathrm{jp}}{\mathrm{w}}_{\mathrm{n}}^{\mathrm{kq}}{\mathrm{X}}_{\mathrm{j}+1,\mathrm{k}+1}$$

(15)

$${\mathrm{w}}_{\mathrm{m}}={\mathrm{e}}^{-2\mathsf{\pi }\mathrm{i}/\mathrm{m}}$$

(16)

$${\mathrm{w}}_{\mathrm{n}}={\mathrm{e}}^{-2\mathsf{\pi }\mathrm{i}/\mathrm{n}}$$

(17)

where $\mathrm{Y}$ is a discrete Fourier transform, X is a matrix of m × n (2D-image), ${\mathrm{w}}_{\mathrm{m}}$ and ${\mathrm{w}}_{\mathrm{n}}$ denote complex roots of unity, and i indicates the imaginary unit. The obtained output is shifted to locate the zero-frequency component at the center of the output. The element-wise multiplication of the obtained FFT frequency components with a Wavelet transform is performed to suppress the effect of noise. This convolution was performed one, two, and three times, to investigate its optimization effects on the segmentation results [35]. This process is followed by taking the inverse transform and multi-thresholding to perform segmentation. Finally, the obtained binary image is eroded to remove small objects. This algorithm is summarized as Pseudocode 2.

Pseudocode 2. The proposed FT-based algorithms for defect segmentation in microwave imaging.

Input: Phase and Magnitude

Output: True Target Segmentation

Pre-processing of CM images as Section 1. Computation of the 2D-FFT of the on the input microwave image as “y”. Element-wise multiplication of Wavelet transform and “y”. for i: 1 to 3 // Separately perform one-, two-, and three-fold convolution xy [i] = x*y // Convolution of x and y: (x*y): Integral of the product of the two matrices after one is reversed and shifted. Computation of the inverse Fourier Transform to get the resulting image “I”. Computation of multi-thresholding on “I” to obtain “S” as a segmented image. Performing the morphological operation (erosion) on “S” to remove small objects.

Algorithm 2: The second proposed algorithm, in this section follows the Pseudocode 2 procedure, in which the Wavelet transform function will be replaced by a Gaussian function.

Algorithm 3: The third algorithm is based on fuzzy c-mean (FCM) clustering, followed by Multi-thresholding and erosion. This algorithm has been formed in this paper only for a comparison with Algorithms 1 and 2, as it uses the FCM procedure.

3.1.2.3. Segmentation of Metal Plate (SMP) Images

Three algorithms are proposed for this segmentation. The first and second algorithms almost completely follow the traditional approach, and are investigated for comparison.

Algorithm 1: In the first algorithm, the performance of the traditional gradient magnitude based on Sobel’s gradient operator [36] in the x and y axes is investigated. The edges of the notch were returned at those points where the gradient of the image was at its maximum. Therefore, the edges were achieved by using the Sobel approximation to the derivative.

Algorithm 2: The second algorithm calculates the gradient by using the derivative of a Gaussian filter, and it employs two thresholds; the local maxima of the gradient are considered to be edges.

Algorithm 3: In the third algorithm, the effective selection of the threshold value is performed on pre-processed MP images. The threshold value is obtained by Otsu’s thresholding method, and it is employed, to improve the threshold value of the Fuzzy c-mean technique. The center value of the thresholds was considered to be the desired value, which is more likely to detect true edges. The edges that were not stronger than the threshold value were ignored. Figure 1 shows the flowchart of this algorithm.

The approaches, introduced above, allow experts to accurately visualize the defects that are significantly associated with any subset or linear combination of cracks. The methods can greatly simplify health monitoring in steel, composite materials, and metal structures, respectively. These algorithms can be generalized by investigations on other materials.

In the following section, the experimental setup and results are discussed.

4. Experimental Setup

4.1. ML-Based Damage Detection for Concrete and RC Beams

Six concrete beams and four reinforced concrete (RC) beams are fabricated and tested. The 3-point and 4-point bending tests are carried out on beams until failure occurs on the specimens.

Table 1. Specifications for experimental setup.

Concrete Beam	Size (mm)	Loading Capacity (kN)	Displacement Movement (mm/min)	Loading Cell Starting Point
Simple	400 × 100 × 100	200	0.01	0
Reinforced	1700 × 150 × 250	1000	0.009	0

depicts the specifications of our experimental setup.

The size of the concrete beams is 400 mm × 100 mm × 100 mm, with a distance of 200 mm between the sensor and actuator, and the size of the RC beams is 1700 mm × 150 mm × 250 mm, with a distance of 400 mm between the sensor and actuator [39]. An N16 steel bar was used in this experiment as the longitudinal and transverse reinforcement size. The Australian Portland cement type GB, Natural river sand (Fine aggregate), Natural river gravel (Coarse aggregate), and Tap water are used as materials of concrete mixture [39]. The experimental tests are carried out through the Instron high bandwidth system commonly known as the “Universal Test Machine (UTM)”, consisting of a loading cell, for monitoring in real time. Figure 2a,b shows our bending test setup on the beams.

4.2. Microwave Imaging for Damage Detection

The measurement system consisted of a computerized imaging system, including a millimeter-wave reflectometer with a single frequency oscillator, isolator, magic tee, detector, variable short, the OEWA, and a scanning mechanism providing a raster scan of the reflectometer over the sample under test. The open-ended flange-mounted rectangular waveguide probe was used as an open-ended waveguide antenna (OEWA). The OEWAs are robust, reliable, efficient, and relatively simple antennas used for crack detection [10,21,22,23]. The OEWA irradiated the specimen under testing, and picked up the signal reflected from the specimen. This reflected signal was combined with the reference signal in the magic-tee, and the combined data was sent to the detector; the computer recorded its output voltage for every data point over the plane for each specimen. The images were generated through the recorded data in Matlab software, and processed with the proposed image processing algorithms. Figure 3 shows the measurement setup of this study.

To highlight the utility of the single-frequency reflectometer with the OEWA, along with the proposed image processing approaches for imaging different structural materials, various samples were prepared and imaged, as follows.

• Steel Plate

The first specimen was a section of a 12 mm-thick A-36 steel plate with a few through-fatigue cracks that were created during cyclic loading. A W-band open-ended flange-mounted rectangular waveguide probe with aperture dimension of 2.54 mm × 1.27 mm, and with a square (19 mm × 19 mm) flange was used in this investigation. Two scanned areas, with dimensions of 50 mm × 70 mm and 30 mm × 30 mm, included cracks that were obtained at a standoff distance of d ~ 0 mm, d ~ 0.8 mm, and d ~ 1.7 mm.

• Composite Material

The second specimen consisted of six equally thick layers of construction foam with dimensions of 240 mm by 240 mm by 90 mm, as shown in Figure 4a. Four metal discs (each with a diameter of 25 mm and a thickness of 2 mm), in two groups of two, were placed at interfaces between layers 1–2 and 5–6, respectively, as shown in Figure 4. Sample 1 was scanned with a 3 mm step size (along the x- and y-axes), while the complex reflection coefficient was measured at a frequency of 11.5 GHz (X-band) and a 30 mm standoff distance (the distance between the OEWA aperture and the top surface of Sample 1).

• Aluminum Plate

The third specimen consisted of an aluminum plate with dimensions of 150 mm × 60 mm × 10 mm, and was covered by a rubber layer. Two notches were made on the surface of the plate, both with a length of 15 mm and a depth of 1 mm, and with widths of 1 mm and 0.5 mm in notch 1 and notch 2, respectively, as shown in Figure 5. Two scanned areas were made, with dimensions of 40 mm × 20 mm and 100 mm × 40 mm, including notch 1, Figure 5a, and notches 1 and 2, Figure 5b, respectively. The step sizes (sampling) of 1 mm and 0.5 mm along the length and width of the notches, respectively, were used in this investigation.

5. Results and Discussion

5.1. Estimating the Performance Indicator for Traditional and Developed SVM

Experimental data analysis was carried out, considering the signals obtained by SAs mounted on the concrete and RC beams. The signals were recorded every minute throughout the loading period.

Table 2. Comparison of accuracy between the traditional and the developed Support Vector Machine (SVM).

Accuracy (%)
No. Observations	Concrete Beam		RC Beam
	Simple SVM	Developed SVM	Simple SVM	Developed SVM
1	84.72	87.23	85.26	86.19
2	84.63	87.19	85.42	87.14
3	84.81	87.22	85.39	86.19
4	84.76	87.23	85.4	86.18
5	84.83	87.19	85.39	86.15
6	84.46	87.23	85.28	86.19
7	84.91	87.22	85.2	86.5
8	84.63	87.24	85.48	86.19
9	84.65	87.35	85.4	86.19
10	84.61	87.22	85.52	86.56
11	84.91	87.2	85.38	86.25
12	84.52	87.18	85.29	86.19
13	84.66	87.2	85.43	86.66
14	84.87	87.21	85.29	86.16
15	84.8	87.21	85.42	86.18
16	84.81	87.19	85.46	86.19
17	84.82	87.23	85.44	86.12
18	84.58	87.23	85.31	86.19
19	84.88	87.22	85.27	86.19
20	84.76	87.23	85.44	86.21
21	84.55	87.23	85.32	86.19
22	84.8	87.28	85.43	86.38
23	84.65	87.19	85.32	86.19
24	84.6	87.17	85.37	86.19
25	84.87	87.19	85.47	86.47
26	84.58	87.22	85.45	86.19
27	84.83	87.22	85.24	86.72
28	84.93	87.35	85.42	86.14
29	84.68	87.19	85.4	86.19
Average	84.72	87.22	85.38	86.29

shows the comparative analysis between the traditional and the developed SVM of this study. The classifier learnt from the training features [40] was used to make a decision when the testing data was presented. The performance accuracy was computed through an average of 4-fold cross-validation for 100 runs, and performed for 29 observations. The performance of the algorithms was obtained through computing the average accuracy.

According to the above results, the analysis conducted through an estimation of the performance of the developed SVM obtained an accuracy of 87.22% rather than the traditional SVM, with an accuracy of 84.72%. The suggested method could outperform the benchmark. Further analysis of this performance was carried out by computing the p-value of the T-test, for the average of the accuracies (

Table 3. T-Test: Simple SVM and developed SVM sample for the means.

	Simple SVM	Developed SVM
Mean	84.73	87.22
Observations	29	29
t statistic	−101.443
P (T ≤ t), one-tailed	8.78×10−3
P (T ≤ t) two-tail	1.76×10−3

).

The obtained p-values in the above cases demonstrate that the difference between the accuracies of the simple SVM and the developed classifier are statistically significant.

5.2. Evaluating the Results of the Proposed SSP Algorithm

This section details an investigation and an analysis of the image results for steel plate sample using the proposed SSP Algorithm in Section 3.1.2.1. The original images herein are 90 GHz images of cracks in a steel plate at different standoff distances with an open-ended waveguide probe. The resulting images were enhanced and segmented at different distances. Figure 6 shows the signals, which were recorded through an open-ended waveguide antenna under 50 × 70 scanning, obtained at three standoff distances of 0 mm, 0.8 mm, and 1.7 mm.

Figure 7 and Figure 8 show the generated images of a crack in Matlab for two scanned areas of 50 × 70 and 30 × 30, with standoff distances of d ~ 0.2, d ~ 0.8 and d ~ 1.7, respectively. As it is evident from Figure 7 and Figure 8, the original images are blurry, and limited information can be obtained.

The first step of the SSP algorithm, as explained before, is to enhance the formed microwave images. Figure 9 shows the enhanced images after performing the enhancement procedure on the original images.

Figure 10 shows the images that were obtained through the employment of the proposed algorithm on the enhanced image (Figure 9), with the gradient magnitude being calculated in six directions of x, y, d₁, and d₂, followed by Otsu’s thresholding method.

The comparison of the imaging results and the original image, Figure 9 and Figure 10, confirmed that the developed algorithm could identify the crack edges while eliminating noises on the target region. As is evident from the experimental results, the algorithm could provide the image with pure texture and strong edges. Furthermore, Figure 11 shows the 2D grid coordinates that were plotted, based on the coordinates (vectors x and y) of the segmented image. The color change in the middle of grid indicates the occurrence of the abnormality in the data set. The grid-based data representation is an effective way to approximate the amplitude at any location between the grid points.

Table 4. Properties of the image region, results, and descriptions.

Property	Result	Description
Centroid	[38.7045 27.9205]	Mass center of the region
BoundingBox	[35.5000 2.5000 7 64]	Smallest rectangle containing the region
FilledImage	[64 × 7 logical]	Image with the size of bounding box for the region
FilledArea	176	Number of ‘on’ pixels
MajorAxisLength	68.7051	Number of pixels for the major axis of the ellipse
MinorAxisLength	5.8664	Number of pixels for the minor axis of the ellipse
Orientation	88.0654 (degree)	Angle between the MajorAxisLength and the x-axis

shows the properties of the imaged region, along with the results and descriptions. The results have been computed by using the regionprops () command in MATLAB.

5.3. Evaluating Results for the Proposed SCM Algorithms

This section evaluates results of the proposed SCM Algorithms in Section 3.1.2.2. The original images of the sample for composite material are shown in Figure 12, and they demonstrate that the raw images do not provide much observable information, while four defects exist and must be seen.

To enhance the indications of the desired targets, three proposed algorithms were run using the same condition. The first algorithm was based on one-, two-, and three-fold multiplication of the wavelet transform, with the Fourier transform of the image, followed by the multi-threshold segmentation method and morphological operation “erosion”, to remove the small elements from the image to achieve the target. Figure 13 demonstrates the total process of this segmentation. The first row of images in Figure 13 shows the process as performed by a “single” multiplication of the wavelet and Fourier transform, while the second and third rows show the process as performed by two- and three-fold multiplication of the wavelet and Fourier transform, respectively. However, the obtained result through the single multiplication (row 1), Figure 13d, shows the best result in this section, as it determines three out of four defects, while the other results only detect two defects.

Figure 14 shows the results obtained by the second algorithm. This figure also demonstrates the total process for one-, two- and three-fold multiplication of Gaussian, with the Fourier transform of the image, followed by the multi-threshold segmentation method, and morphological operation “erosion”, to remove the small elements from the image to achieve the target. The first row of images in Figure 14 shows the process performed by a “single” multiplication of the Gaussian and Fourier transform, while the second and third rows of images show the process performed by two- and three-fold multiplications of Gaussian and Fourier transforms, respectively. However, the results obtained through two-fold multiplication (Figure 14d, row 2) shows the best-segmented result, in this section, as it determines four out of four defects, with less noise than the other results in Figure 14.

Figure 15 shows the results that are obtained through the Algorithm 3. This figure also shows the total process of target segmentation using fuzzy c-mean clustering, followed by Multi-thresholding and erosion with two different sizes, 10 and 15, of the structuring element. The results show that the higher size of the structuring element makes the image cleaner, but they change the detected defects sequentially, and make the image unreal. However, the obtained result in Figure 15d shows the best-segmented result; in this section, as it determines three out of four defects and the skeleton of the fourth defect.

The comparison of segmentation results using the above three algorithms shows that the results achieved by “Algorithm 2”, Figure 14, outperform the other two algorithms. In Figure 14, the best demonstration is for the “two-fold multiplication of Gaussian with the Fourier transform” which is followed by a multi-threshold segmentation method and morphological operation “erosion” (Figure 14, row 2b). The image shows a clear view of four separated defects with less noise and redundant objects.

Also, this section presents the surface plots for the magnitude and phase of reconstructed images as a further comparison of Algorithms 1 and 2. Figure 16a shows the surface plot for the wavelet transform basis and Fourier transform (Algorithm 1). The plots in the first row of Figure 16a demonstrate the magnitude and phase of the 2D data, containing random phases. The plots in the second row of Figure 16 demonstrate the magnitude and phase of the 2D data containing random magnitudes. Figure 16b presents the surface plots for the magnitude and phase of a reconstructed image, based on Gaussian filtering and Fourier transform (Algorithm 2).

The comparison of Figure 16a,b shows that the basis had little effect on the phase, while the magnitude changed from [0,0.03] in Figure 16a to [0,1] in Figure 16b, which demonstrates the better performance of the second algorithm over the other algorithm. Figure 17 shows the results obtained from the setup on Figure 5b.

Both the proposed Algorithms 1 and 2 show almost similar results (rows 1 and 2), while they outperformed the Fuzzy c-mean algorithm (row 3). As is obvious from the Figure 17 (row 3a), Figure 17 (row 3b), and Figure 17 (row 3c), the outcome of FCM is dependent on the selection of the threshold value. The threshold value of the first figure is 0.206500, while the second and third figures have threshold values of 0.211538 and 0.480769, respectively.

From all of the above results obtained through employing the algorithms on experimental setups in Figure 5a,b, it is concluded that the proposed algorithms were successfully adapted to determine the notch regions. However, the algorithms discriminated the boundaries of the notches, where Algorithm 3 (benchmark) was less successful (Figure 16, row 3a).

5.4. Evaluating Results for the Proposed SMP Algorithms

This section evaluates results of the proposed SMP Algorithms in Section 3.1.2.3. Figure 18 show the original (left) and enhanced (right) images of notch 1 under the 2 mm-thick rubber layer, obtained at three standoff distances of 0.1 mm, 0.25 mm, and 4 mm. As is evident from Figure 18, the original images (left) were blurry, and no information could be gained from the images of notches under the 2 mm-thick rubber layer with a standoff distance of 4 mm, as shown in Figure 18c. However, the right-hand-side images show the results of image enhancement over the original images. It is obvious that the enhanced images can show up the existence of notches.

Figure 19 shows the original and enhanced image of notches 1 and 2 under the 4 mm-thick rubber layer obtained at a standoff distance of 0.1 mm. As is evident from Figure 19a, the original image was blurred, and no information could be gained from an image with a standoff distance of 0.1 mm, while the notches were clearly observable on the enhanced images (Figure 19b).

By applying the image-enhancement algorithm, the notches in the specimens became clear, as shown in Figure 18 (a(right), b(right), c(right)), and Figure 19b, thus meeting one of the goals of this investigation. In order to make progress with the algorithms, the enhanced image of notch 1, Figure 18a, was used to apply three image processing algorithms, as presented in the Section 3.1.2. To save space, the corresponding images obtained for Figure 18b,c after the SCM process, are provided in Appendix A. Figure 20 shows the images obtained for notch 1 in each step of Algorithm 1.

Figure 21 shows the images obtained for notch 1 in each step of Algorithm 2.

Figure 22 shows the images obtained for notch 1 in each step of Algorithm 3. In order to facilitate the discussion, in Algorithm 3, the developed Fuzzy c-mean thresholding algorithm was applied, to segment the image. However, the obtained threshold value was 2.05 × 10⁻¹, while for Otsu’s thresholding method and the traditional fuzzy-c-mean thresholding algorithm, it was 1.7 × 10⁻¹ and 2.4 × 10⁻¹, respectively.

Figure 23 shows the comparison of the segmented images of notch 1, obtained with Algorithms 1–3. According to the segmented images, the developed Fuzzy c-mean thresholding algorithm (Algorithm 3) outperformed the other two algorithms. As is evident from Figure 23c, the true edges, which were detected by the developed Fuzzy c-mean thresholding algorithm, were more accurate with less noise, while Figure 23a,b include redundant edges and noises, which represent the segmentation error.

A further investigation of the developed algorithm was carried out, by comparing its performance with the traditional Fuzzy c-mean thresholding algorithm and Otsu’s thresholding algorithm. Figure 24 depicts this comparison. However, the obtained results show that the developed Fuzzy c-mean thresholding algorithm performed better than the other two traditional algorithms. To save space, the obtained corresponding segmented results with respect to a proposed algorithm have been provided in Appendix A.

Imaging results confirmed that the developed Fuzzy c-mean thresholding algorithm can identify the notch edges while eliminating the noises on the target region, which proves that the algorithm could achieve the images with pure textures and strong edges.

6. Conclusions

This paper first proposed a damage detection algorithm, using the developed ML approaches for concrete and RC beams, and then investigated the performance of microwave imaging on the local detection and visualization of damage in a steel plate, a composite material, and a metal plate. The SVM-based algorithm was developed and compared with traditional SVM, through the response data of various numbers of specimens mounted on SAs, to validate its effectiveness. Accuracy and a t-test were used for this validation. The proposed algorithm outperformed the performance of traditional SVM. It can be concluded that the extraction of subsequent knowledge from the training phase, and its fusion with unsupervised machine learning (k-nearest neighbor), can help to improve the performance of traditional SVM.

Then, the microwave- and millimeter-wave imaging approach was used as the second approach of this paper. The obtained images had a very low resolution, and in some cases, no information was observable. The images were processed through different proposed image-processing algorithms. The algorithms were based on the Fourier transform, Wavelet transform, Fuzzy c-mean, Gaussian filter, Median filter, and Gradient magnitude. The proposed algorithms made it possible for clear observations of anomalous occurrences in structures to be made. The results prove that the algorithms have the potential to successfully enhance microwave images with poor resolution, and to identify areas of interest, specifically in structural health monitoring.

The purpose of this study was to minimize the need for manual inspections of civil structures, and it sought to complement edge-based damage detection methods for hazardous scenarios. However, the proposed algorithms of this study can be applied to different imaging applications where the enhancement of quality, and obtaining the area of interest are critical approaches.