Research on Feature Identification and Trajectory Planning of Pavement Cracks

Abstract

As the most common method to detect pavement cracks, manual detection has uncontrollable factors such as low efficiency, inconsistent standards and easy to be interfered with by external forces, so it is not suitable for pavement crack detection in today’s intricate traffic network. In order to improve the efficiency of pavement repair and reduce the labor cost of the repair process, this paper proposes an intelligent pavement crack detection and repair algorithm. The algorithm uses image numerical parameters to classify cracks with different geometric features and extracts texture geometric features of various types of cracks based on different filtering strategies. It solves the problem that traditional single filtering algorithms are difficult to extract features according to the different characteristics of the collected image, which leads to the loss of information. Finally, the algorithm establishes a mathematical model for efficient trajectory planning combined with the nozzle size of the crack-repairing machine. In this paper, the robustness and efficiency test of the algorithm is carried out on the pavement image dataset with various types of cracks, and the experiment is carried out on the intelligent pavement crack detection and repair prototype, which verifies the accuracy and reliability of the planned trajectory.

1. Introduction

As a complex of intricate engineering facilities, motor vehicle lanes require constant maintenance and repair to ensure a high-quality and safe traffic environment. Crack detection utilizes the type, length, and severity of cracks to quantify the condition of the pavement, analyze the cause of the deterioration of the road condition, and is the best indicator for assessing whether the road needs preventive maintenance treatment. Carrying out crack detection at the initial stage is conducive to the proper maintenance of the pavement, so as to save a large amount of cost of repairing in the later stage. According to the degree of manual intervention required, crack detection methods can be divided into manual detection, semi-automatic detection and automatic detection [1]. Even though many scholars have proposed automatic crack detection models, manual detection methods are still the most common in practice. Nevertheless, due to the limitation of manpower, there are many unavoidable problems in manual detection methods, which make it difficult to detect the details of cracks consistently. Therefore, in today’s increasingly developed traffic system, manual detection methods are no longer suitable for large-scale road maintenance projects.

In the past decades, the government and scientific research institutions have made great efforts to build an automated pavement crack detection system. Safaei et al. [2] proposed a comprehensive crack extraction algorithm based on local threshold, morphological manipulation and preprocessing to extract the crack image of pavement damage and calculate the crack width. The researchers also used other methods based on autocorrelation screening, such as two-dimensional pavement image crack detection method based on the weighted neighborhood pixel segmentation algorithm, and Gaussian cumulative density function(CDF) as an adaptive threshold to overcome the disadvantage of fixed threshold in noisy environment [3]. Aiming at the problems of low crack image recognition rate and difficult classification, Ouyang et al. [4] proposed a crack recognition method based on image enhancement and image segmentation. Akagic et al. [5] proposed a crack detection method based on the Otsu thresholding method and image segmentation, which does not need to know the morphological characteristics of cracks in advance, but uses the spatial distribution characteristics of crack pixels to analyze the image. Nevertheless, for complex and diverse pavement crack detection work, such traditional detection algorithms based on image segmentation algorithms have problems of low efficiency and poor robustness [6].

Machine learning methods can efficiently and accurately classify images through image spatial distribution, color tone, pixels and other features before specific crack identification, so are more and more frequently used to solve pavement crack detection problems. Othman et al. [1] proposed an improved Otsu-Canny edge detection algorithm, and the crack detection based on the adaptive multi-resolution threshold technology improved the accuracy of crack extraction. Sari et al. [7] proposed a method for identifying cracks in asphalt pavement based on Support Vector Machine and Otsu thresholding method. As an efficient feature extraction technology, Hough transformation is also very helpful to solve the problem of crack identification. By comparing a variety of edge detection, Kaur et al. [8] proposed a crack detection method based on Canny–Prewitt operator and Hough transformation, and proved that the algorithm has the best robustness and efficiency through experiments. Nasser [9] proposed a road trajectory fitting method based on Hough space transformation and the K-means clustering algorithm, which greatly reduces the number of feature points while fitting the trajectory efficiently. Neural network is the core of trainable algorithm for crack detection, which has advantages over threshold-based techniques and morphological tools. Now, many scholars have proposed to train complex neural networks to obtain a model that can accurately identify pavement cracks and fit the trajectory [10,11,12]. However, although these methods could provide extremely fine fitting effect, the main drawback of learning-based approaches is that the build learning step is usually accompanied by a tricky manual labeling step, which does not provide fast and fully automatic analysis compared to image segmentation-based methods [2].

Most crack identification algorithms today have relatively single scene quality requirements for the processed image features. However, affected by objective factors such as light and shooting angle, the crack images identified by the machine inevitably have different contrasts, textures and shades. At present, there is still a lack of a systematic pavement crack detection and trajectory planning algorithm, which can balance high accuracy and fast execution efficiency, and is not limited to the complex manual labeling work of the dataset. In order to solve this problem, this paper proposes an intelligent pavement crack detection and repair algorithm. The algorithm establishes a targeted filtering model for different types of crack feature images, and realizes intelligent trajectory planning based on the Hough space transformation and K-Means clustering algorithm. Through a large number of experiments, we find that under the condition of fixed size, the crack images collected by the camera can be divided into two types according to the morphological characteristics of the main crack and the distribution characteristics of the noise, namely the high-density noise wide crack type and the low-density noise narrow crack type. According to this, the algorithm firstly performs global variance calculation on the binarized original crack image and accurately classifies two typical crack features based on the threshold discrimination method. For the high-density noise wide crack type images which tend to have wide main cracks and dense noise distribution, the algorithm uses the median filter and Otsu thresholding method, which greatly eliminates the image dense noise groups while retaining the main crack feature relatively completely. For the low-density noise narrow crack type images which tend to have narrow main cracks and scattered noise distribution, the algorithm accurately extracts the crack features through the eight-neighborhood maximum connected component extraction algorithm, thus avoiding the loss of narrow crack feature information caused by strong filtering. After crack feature extraction operation, the algorithm uses Hough space transformation and K-Means clustering algorithm to establish a mathematical model that can be combined with the nozzle size of the crack repair model for efficient trajectory planning. Finally, the robustness test of the algorithm is carried out on pavement image dataset containing various types of cracks, and experiments are carried out on the intelligent pavement crack detection and repair prototype to verify the accuracy and reliability of its planned trajectory.

2. Research Significance

This paper aims to propose an intelligent pavement crack detection and repair algorithm, which can establish a targeted filtering model for different types of crack feature images to extract the main crack feature quantity more completely, and establish a mathematical model for efficient trajectory planning based on the crack feature quantity combined with the crack repair model nozzle size.

3. Intelligent Pavement Crack Detection and Repair Algorithm

3.1. Algorithmic Framework

In this paper, an intelligent pavement crack detection and repair algorithm is proposed. The algorithm consists of four parts to realize pavement crack feature identification and trajectory planning, and its framework is shown in Figure 1. First, the algorithm calls the system camera to capture the scene images with a fixed size, and performs a preliminary binarization operation on the images collected by the camera one by one at the system terminal to obtain the binarized crack image. For the binarized scene image, the algorithm adopts the threshold discrimination method, and compares the preset threshold with the global variance of the pixel matrix, and then divides the corresponding original crack image into a high-density noise wide crack type and a low-density noise narrow crack type. The algorithm proposes targeted filtering policies for the two types of graphs, respectively. For the high-density noise wide crack type image, the algorithm extracts the original image again, smoothes the texture features of the original image based on the median filtering method, and uses the Otsu threshold algorithm to eliminate the dense noise group and capture the main crack feature. For the low-density noise narrow crack type image, the algorithm obtains the binarized crack image based on the Otsu thresholding algorithm and uses the eight-neighborhood maximum connected component extraction algorithm to achieve accurate extraction of fine crack features. In the Hough space transformation stage, the algorithm performs Canny edge detection and Hough lines fitting on the feature-extracted image and collects all the endpoints of the fitted feature lines. Finally, according to the complexity of the crack shape, the algorithm uses the K-Means clustering algorithm to divide the fitting feature endpoint set into several clusters, and connects each cluster center in the output stage to form a crack repair trajectory.

3.2. Image Filtering Stage

In the image filtering stage, the algorithm firstly calls the system camera to take pictures of the cracks and upload them to the system terminal for subsequent analysis. Since the image signal of the camera sensor is easily affected by pulse currents, analog-to-digital converter or bit transmission errors occur, and the crack scene image collected by the camera usually contains a lot of salt and pepper noise. However, the pavement crack repair algorithm based on traditional morphological segmentation methods such as the Otsu thresholding method and histogram equalization method only uses a single mode of filtering method and is difficult to establish a highly adaptive filtering model for different types of crack images. In order to solve this problem, the algorithm divides the original crack images into two categories as shown in Figure 2 according to the distribution of the probability density function and the relative size of the main crack feature. Namely, high-density noise wide crack type image and low-density noise narrow crack type image, and we purpose the targeted filtering policies for these two types of crack images, respectively.

3.2.1. Image Classification Policy Engine

Due to unstable environmental conditions such as light, angle and background tone, the relative size of the main crack feature points and noise points contained in the images collected by the system camera, and the degree of noise dispersion is often uncontrollable, which in turn affects the subsequent image filtering and feature extraction. In 2021, Sheerin er al. [13] conducted an integrated analysis of mainstream crack detection and classification techniques, and proposed the necessary conditions for a pavement crack feature identification algorithm. That is, the morphological characteristics of cracks and noise clusters in different types of scene images should be comprehensively considered, and a targeted filtering model should be established. Accordingly, we experimented with global variance calculation on multiple crack images. Figure 3 shows the comparison of the original binarization effect of the two types images, while

Table 1. Global Variance Calculation Results.

	High Density Noise Wide Crack Type				Low Density Noise Narrow Crack Type
Sample	Sample I	Sample II	Sample III	Sample IV	Sample I	Sample II	Sample III	Sample IV
Global Variance Value	898.5224	947.9831	806.3760	1124.1458	129.5714	116.9045	287.2469	221.6327

shows the global variance calculation results of the images. After a large number of experiments, we found that under the condition of fixed size, the image in Figure 3a has relatively narrow cracks and the noise distribution of this type of image after binarized is also relatively scattered, meanwhile, the global variance of the calculated pixel matrix is usually in the range of 0–300 as shown in Table 1. However, the image in Figure 3b has a relatively wide crack and the noise distribution of this type of image after binalization is concentrated on the edge side of the crack. The amount of noise of this type is much larger than that of the narrow crack type image, and the global variance of the calculated pixel matrix is usually greater than 800. Therefore, for a certain size of the crack scene binarized image, the threshold discrimination method is introduced as the image classification policy engine, and 500 is selected as the appropriate classification threshold. According to the difference of the global variance value of the pixel matrix, the image is divided into high-density noise wide crack type and low-density noise narrow crack type, and the corresponding processing strategies are adopted, respectively.

3.2.2. Wide Crack Feature Extraction Policy

In road repair work, high-density noise wide crack type images are a common type of image captured by system cameras. In this type of image, the main crack is often thick, and dense noise points are commonly distributed at both ends, which brings a certain degree of difficulty to the extraction of main crack features. In general, there are the following feature extraction methods for crack scene images: 1. Otsu thresholding method [5]; 2. Image enhancement combined with segmentation methods [7]; 3. Canny operator edge detection [8]; 4. grayscale morphological filter [14]; 5. histogram equalization [15]; 6. median filter [16]. However, the above traditional filtering methods are all based on pixel neighborhood feature values to filter images indiscriminately. Applying it directly to filter this type of image is prone to the situation that the noise group and the feature cannot be separated or the crack feature is damaged during filtering. Therefore, in order to filter the dense noise point groups and retain the main crack feature, the algorithm in this paper combines the median filter and the Otsu thresholding method, and proposes a feature extraction policy for high-density noise wide crack type images.

High density noise wide crack type images have Tamura features with large roughness, small contrast and scattered directions. Compared with its opposite category, the cooccurrence variance of its Gray-level Co-occurrence Matrix is larger, so it is not suitable to directly use traditional morphological segmentation method for filtering [17]. The algorithm in this paper firstly uses the median filter method to smooth the original image texture, neutralizes the image roughness, and greatly reduces the possibility of noise clustering. The median filter replaces each pixel in the image with the median value of the sequence of its eight neighborhoods, so that the surrounding pixels are close to the real value, thereby eliminating isolated noise point. As shown in Figure 4, the algorithm uses a 3 × 3 two-dimensional rectangular sliding window to sort the pixel values in the window to generate a monotonically increasing data sequence. The filtering effect is shown in Figure 5. The median filter can smooth the texture of the original crack image on the basis of retaining the main crack feature to the maximum extent.

After median filtering, the image has already have good contrast and regularity. At this time, the traditional morphological segmentation method can be used to accurately extract the main crack feature. The algorithm uses the Otsu thresholding method to binarize the image. By setting the threshold to maximize the variance between the crack and the background class, a binarized image that only contains crack features is obtained [18]. As shown in Figure 6, the Otsu thresholding method accurately removes all the noise groups in this type of crack image, and retains the main crack feature relatively completely. This paper verifies the reliability of the model by comparing the actual effects of various feature extraction and combination methods, as shown in Figure 7.

3.2.3. Narrow Crack Feature Extraction Policy

Corresponding to the high-density noise wide crack type image, the low-density noise narrow crack type image is another common collected scene image, which has the characteristics of long and narrow main cracks and scattered noise distribution. Nevertheless, compared with wide crack images, the feature size of cracks in this type of image is similar to the size of noise points, and there is a risk of filtering noise points and feature points together by using the same feature extraction policy. Therefore, it is necessary to design a feature extraction policy for this type of image, which can filter the sparsely distributed salt and pepper noise without losing the feature quantity of fine cracks. The algorithm in this paper binarizes the original image by the Otsu thresholding method, and uses the eight-neighborhood maximum connected component extraction algorithm to extract the features of the narrow crack sparse noise type image.

In order to ensure that the feature quantity of the narrow crack would not be affected by the filtering algorithm to the greatest extent, this paper proposes a maximum connected component extraction algorithm based on the pixel features of the eight neighborhoods of the image. Based on the sparse distribution of image noise, the algorithm extracts the largest area connected component to achieve image filtering without feature loss. The algorithm flow is as shown in Figure 8, which could be divided into 5 steps.

(i)

Establish a label matrix of the same size and a one-dimensional array for storing the area size of each connected component according to the pixel matrix of the incoming image.

(ii)

Traverse the pixel points of the binarized image row by row or column by column to obtain their pixel values.

(iii)

If the values of the four adjacent pixels at the upper left of the traversed pixel are not all 0, assign the same label value to the pixel and the adjacent pixels that are not 0, and record the number of consecutive pixels currently traversed. The working principle of the algorithm is shown in Figure 9.

(iv)

Loop through the pixel matrix until the adjacent pixels of the traversed pixels are all 0, treat the pixel set with the same label value as a connected component, store its area in the area array, and update the label value to record the next connected components.

(v)

Traverse the image multiple times until the elements of the label matrix and the area array would no longer change. Retrieve the largest area value in the area array, which corresponding connected component is just the desired crack feature quantity.

The execution result of the algorithm is shown in Figure 10. The algorithm removes the scatteredly distributed noise points in the original image while completely retaining the narrow crack feature.

3.3. Canny Marginalization and Hough Space Transformation

In the image filtering stage, for the high-density noise wide crack type image and the low-density noise narrow crack type image collected by the system camera, the algorithm adopts corresponding feature extraction policies, respectively, and obtains a binarized image containing only crack features. In order to further fit the crack to obtain the coordinate points of the repair trajectory, the algorithm uses the Canny edge detection algorithm to extract the edge of the binarized image, and adopts the polar coordinate Hough space transformation method to perform the Hough straight lines fitting on the feature lines of the edged crack image.

The Hough space transformation, as a very important method for detecting the shape of discontinuous point boundaries, transforms the image coordinate space into the parameter space, and realizes the fitting of straight lines and curves. The algorithm in this paper adopts the generalized polar coordinate Hough space transformation [19], and the algorithm flow is as follows:

a.

The algorithm uses the normal form to represent each line in the image coordinate space: $r=xcos\theta +ysin\theta$, where r represents the distance from the origin of the coordinate system to the straight line, and $\theta$ represents the angle formed by the coordinate point and the x-axis. Therefore, in Hough space, any straight line can be represented by a $(r,\theta )$ parameter pair and points on the same line have the same $(r,\theta )$ parameter pair. If many feature points in a parameter space hold the same $(r,\theta )$ parameter pair, it means that in the Cartesian coordinate system the corresponding number of feature points in are collinear. After this step, each line in the binarized image is represented in the form $(r,\theta )$.

b.

For each coordinate point in the parameter space, the algorithm counts the corresponding $(r,\theta )$ parameter pair in the Hough space, then we could establish an array which contains the count values of all $(r,\theta )$ parameter pairs.

c.

The algorithm uses a voting mechanism to traverse the $(r,\theta )$ parameter pairs corresponding to all feature points with non-zero pixel values in the image matrix, and vote for the model parameters. When there are feature points with the same $(r,\theta )$, the corresponding parameter pair is increased by one vote. After the execution of this step, we can obtain the global statistics of the parameter pairs, so that the parameter pairs with high statistical times can be set as the contour lines recognized by the Hough space algorithm.

d.

An appropriate threshold is set according to the size of the pixel matrix and the $(r,\theta )$ parameter pair with a high number of votes is retained, and the corresponding straight line constitute the Hough space fitting line set of the image.

The Hough space transformation uses the global characteristics of the image to connect the edge pixels and form a closed boundary of the region. While directly performing Hough space line fitting on the binarized crack feature image will introduce a lot of unnecessary redundant calculations. In order to achieve the best fitting effect, the algorithm introduces the Canny edge detection algorithm to marginalize the crack feature image in the early stage of Hough line fitting.

The Canny edge detection algorithm is a multi-level edge detection algorithm, which smooths the image through Gaussian filtering, calculates the image intensity gradient, performs non-maximum suppression of the gradient amplitude and double threshold processing to achieve the optimal edge detection of the image [20]. Since in the image filtering stage, the algorithm has performed filtering operations according to different types of crack feature images, the Canny operator introduced in the algorithm would only include the operation of calculating image gradient and non-maximum suppression. The Canny algorithm would firstly introduce the x-direction convolution operator $S_x$ and y-direction convolution operator $S_y$ as shown in Equation (1), and calculate $d_y$, $d_x$ by Equation (2) for each pixel in the image matrix. Then the algorithm would calculate the image gradient magnitude $M(x,y)$ according to Equation (3), and finally get the gradient direction angle ${\theta }_M$ by Equation (4). The specific mathematical model for calculating the image gradient ${\theta }_M$ is shown in Equations (1)–(4), where Equation (1) represents the convolution operators in the x and y directions, respectively, and Equations (2)–(4) represent the process of calculating the image gradient direction angle through the convolution operator.

$$S_x=\left[\begin{array}{cc}-1& 1\\ -1& 1\end{array}\right],S_y=\left[\begin{array}{cc}1& 1\\ -1& -1\end{array}\right]$$

(1)

$$d_x=f(x,y)\times S_x,d_y=f(x,y)\times S_y$$

(2)

$$M(x,y)=\sqrt{d_x^2(x,y)+d_y^2(x,y)}=|d_x(x,y)|+d_y(x,y)|$$

(3)

$${\theta }_M=arctan\left(\frac{d_y}{d_x}\right)$$

(4)

After obtaining the image intensity gradient, the algorithm performs non-maximum suppression on the amplitude along the gradient direction to eliminate the non-edge point set. The specific process is as follows:

i.

Through the image coordinate angle, the edge directions are divided into four directions: vertical, horizontal, 45°, and 135°. The corresponding gradients are also four directions orthogonal to the edge direction.

ii.

For a horizontal edge, its gradient is vertical, and the gradient direction angle satisfies: ${\theta }_M\in [-22.5°,22.5°]\cup [157.5°,180°]\cup [-180°,-157.5°]$; for a 135°edge, its gradient is 45°, and the gradient direction angle satisfies: ${\theta }_M\in [22.5°,67.5°]\cup [-157.5°,-112.5°]$; for a vertical edge, its gradient is in the horizontal direction, and the gradient direction angle satisfies:${\theta }_M\in [67.5°,112.5°]\cup [-112.5°,-67.5°]$; while for a 45° edge, its gradient is 135°, and the gradient direction angle satisfies: ${\theta }_M\in [112.5°,157.5°]\cup [-67.5°,-22.5°]$.

iii.

As shown in Figure 11, along the above-mentioned four types of gradient directions, the algorithm compares the value of the traversed pixel with the values of its neighborhood. For each pixel traversed by the algorithm, if its pixel value is the maximum value in the neighborhood centered on it, its pixel value is retained, otherwise, its pixel value is set to 0.

After processing by the Canny edge detection operator, the algorithm uses the Hough space transformation to perform a line fitting operation on the boundary of the image crack feature, and the obtained fitting trajectory is shown in Figure 12.

3.4. K-Means Clustering and Fitting Trajectory Output

In real road crack inspection work, the shape of the camera-collected image is often uncertain, resulting in different lengths and widths of crack features. However, the Hough space line fitting algorithm would include all the feature points located inside the crack boundary one by one into the statistical category. For cracks with larger widths, directly using the line set fitted by the algorithm as the coordinates of the repair trajectory is prone to the situation of trajectory redundancy as shown in Figure 12. To solve this problem, this paper proposes a trajectory-fitting method based on the K-Means clustering algorithm, which can significantly reduce the number of trajectory endpoints while efficiently fitting the crack feature.

The K-Means clustering algorithm is a clustering algorithm based on the Euclidean distance of feature endpoints. Its core idea is to determine that the closer the distance between two targets, the greater the similarity. By iterative calculating Equation (5), each point in the endpoint set is divided into corresponding clusters by the algorithm [21], where S represents the set of all cluster center points while $S_i$ represents the ith cluster divided by the algorithm, and x represents the coordinates of any feature point located in cluster $S_i$.

$$\underset{S}{arg}min\sum _{i=1}^k\sum _{x\in S_i}\left|\right|x-{\mu \left|\right|}^2=\underset{S}{arg}min\sum _{i=1}^k\left|S_i\right|\times Var{S}_i$$

(5)

The trajectory fitting process based on the K-Means clustering algorithm proposed in this paper is shown in Figure 13.

The K-Means algorithm performs cluster analysis on all trajectory endpoints generated in the Hough space line fitting process, and divides the endpoint set into a corresponding number of clusters according to the morphological complexity of the crack, as shown in Figure 14.

Finally, the algorithm identifies the k cluster centers obtained as the endpoints of the road repair trajectory, and connects them end to end based on the shortest Euclidean distance of the endpoints of the trajectory to obtain the complete fitting trajectory. The fitting results are shown in Figure 15.

4. Experimental Design

4.1. Construction of the Intelligent Pavement Crack Detection and Repair Prototype

In order to verify the efficiency of the algorithm in this paper, a motion control card, servo driver, servo motor, laser sensor, high-definition camera and laptop computer were used to build an intelligent pavement crack detection and repair prototype. As shown in Figure 16, the prototype consists of two parts: a servo motor with three axes and a screw rod with one axis. The activity range of the servo motor is 1500 mm × 400 mm × 500 mm, and each active axis is equipped with three laser ranging sensors with an effective range of 8000 mm and an accuracy of 1 mm, including two limit sensors and one origin sensor. The mechanical parameters of the equipment are shown in

Table 2. Prototype Mechanical Parameters.

Parameters	Description
Sensor Type	BL-100NZ-485
X-axis Origin Sensor Location	250 mm (from the left limit sensor)
Y-axis Travel Range	150 mm (from the left limit sensor)
Z-axis Travel Range	50 mm (from the left limit sensor)
W-axis Travel Range	250 mm (from the left limit sensor)
Laser-type	635 nm
Sensor measurement accuracy	1 mm
Pulse volume of servo motor	1412.7 (pulse volume/ms)

. The prototype is controlled by a four-axis high-precision motion control card, as shown in Figure 17. The x, y, and z axes correspond to the feed axes of the three motion directions of the servo motor, respectively, and the fourth axis screw is used to move the original station of the crack image. The original crack image is placed in the lead screw tray, and the side end of the tray is equipped with a laser sensor. The servo motor can be positioned to the workpiece coordinate system through the laser sensor, and the crack scene image can be identified and captured by the high-definition camera at the front of the motor and uploaded to the model terminal, so as to perform the fitting of the road repair trajectory. The specific equipment parameters of the prototype are shown in

Table 3. Prototype equipment parameter.

Equipment	Model
Four-axis motion control card	Googoltech GT2-400-ACC2-VB-G-A
High-definition camera	Hikvision B12- V2-I(POE)
Servo driver	Delta-China ASDA-B2-0721-B 750W

.

4.2. Experimental Results and Analysis

In this paper, the robustness, efficiency and accuracy of the algorithm are comprehensively tested through the terminal algorithm simulation fitting verification and the experimental prototype actual scene verification.

4.2.1. Algorithm Simulation Fitting Verification

In the terminal algorithm simulation fitting verification stage, this paper uses 30 crack sample images captured by the high-definition camera of the prototype during the actual operation for analysis. Each group of experimental samples was collected under different scene conditions such as walls and pavements. The crack lengths, widths, colors and shade depths contained in each image also varies. At the same time, in order to standardize the measurement fitting results, manual ground truth labeling was carried out for all the 30 crack sample images selected. Part of the experimental results is shown in Figure 18. The algorithm can accurately extract and identify trajectories for both types of camera images under light and dark backgrounds.

The crack repair work has high requirements for the recognition accuracy and execution efficiency of the automatic algorithm. Therefore, in order to comprehensively reflect the advancement of the proposed algorithm with the traditional recognition method based on morphological segmentation and the recognition method based on deep learning, the algorithm proposed is compared with the algorithm in the literature [5,12] in terms of recognition accuracy and computational efficiency. The detailed analysis of this stage is as follows.

Algorithm Accuracy Analysis

In order to measure the performance in accuracy between the pavement crack repair trajectory fitted by the algorithm and the real crack feature, the RMSE metric is selected as the evaluation metric of the algorithm fitting effect. By calculating and comparing the RMSE values between the labeled ground truth and the fitted trajectory lines in the sample crack image, the accuracy of each algorithm can be comprehensively analyzed, and the results are shown in Figure 19.

Algorithm Execution Efficiency Analysis

In order to evaluate the performance in execution efficiency between the proposed algorithm and the algorithm in the literature [5,12], this paper takes the above-labeled crack image dataset as the target, and calculates the time cost required by the proposed algorithm and the algorithm in the literature [5,12] to identify crack images with different complexity, respectively, and the results are shown in Figure 20.

Performance Analysis

Through the above performance analysis and comparison results, it can be seen that the proposed algorithm has relatively excellent performance, which proves the advancement of the proposed algorithm. Although the algorithm of literature [12] is slightly better than the algorithm of this literature in terms of accuracy, the time cost of the algorithm in the literature [12] is much higher than that of the proposed algorithm when dealing with the same samples. On the other hand, our proposed algorithm holds a similar execution efficiency with the algorithm of paper [5] while a better fitting accuracy compared to it. In summary, at the cost of a small performance overhead, the proposed algorithm basically meets the requirements of crack repair work, and is more suitable for the actual scene of pavement crack repair work.

4.2.2. Experiment Verification

The built intelligent pavement crack detection and repair prototype is used to verify the fitting effect of the algorithm in this paper in practical application scenarios. After completing the camera calibration and establishing the workpiece coordinate system, a crack scene image would be placed on the screw tray of the prototype, and the crack image would be captured by the front-end camera of the servo motor and uploaded to the model terminal. After algorithm processing, the servo motor receives the coordinate points of the feedback crack repair trajectory and is controlled by the motion control card to drive a fixed 2mm gray marker to draw a fitting trajectory in the scene image to simulate the real crack repair work. Finally, this paper evaluates the accuracy of the algorithm by analyzing the radial distance error between the trajectory points drawn by the servo motor and the real crack feature points during the operation of the prototype.

As shown in Figure 21, the fitted trajectory drawn by the servo motor is highly coincident with the feature of the scene crack image. In order to compare the accuracy of the algorithm fitting horizontally, this paper selects 7 cluster centers generated during the K-Means clustering process as the fitting trajectory endpoints for each experiment and uses the orthogonal distance to the original crack feature as the evaluation metric for the accuracy of the fitted trajectory.

Table 4. Radial error of fitted trajectory points.

Crack Image	Type 1 (mm)	Type 1 (mm)	Type 1 (mm)	Type 1 (mm)	Type 1 (mm)	Type 1 (mm)	Type 1 (mm)	Average Error (mm)	Accuracy
Crack Type 1	1.624	1.858	0.855	1.825	2.601	1.724	1.023	1.644	100%
Crack Type 1	1.530	1.942	0.874	1.005	1.429	0.743	0.962	1.212	85.8%
Crack Type 1	1.211	1.047	0.753	0.969	1.953	1.277	0.857	1.152	100%
Crack Type 1	0.742	0.249	1.105	0.791	0.412	0.363	0.693	0.622	100%
Crack Type 1	0.653	1.673	1.692	2.052	1.559	0.901	0.612	1.306	85.8%

summarizes the radial errors between the fitting track points of the five images used in the experiment and the real crack feature. The calculation method of the radial error is as follows: with the clustering feature points generated by the algorithm as the center of the circle, a circle is made which is tangent to the actual repair trajectory marked in the binarized crack image (the actual repair trajectory is the ground truth pre-marked in the robustness test of the algorithm), and the radius of the circle is the radial error between the fitting point and the feature point of the actual repair trajectory. It can be seen from the table that almost all the errors between the fitted 7 feature endpoints and the original crack feature are less than the radial range of 2 mm, while the nozzle diameter of the pavement repair prototype is within the range of 2–10 mm, which further reflects the accuracy and stability of the algorithm in this paper in the actual model application.

5. Conclusions

To deal with the dilemma of the balance between the high accuracy and the execution efficiency of the crack detection algorithm, this paper proposes an intelligent pavement crack detection and repair algorithm based on the morphological characteristics of image cracks and noise. The algorithm uses the global variance of the pixel matrix as the policy engine, adopts targeted processing policies for different types of crack images, and efficiently generates high-precision fitting trajectories. The main contributions of this paper to the research on crack detection, regarding the proposed method, are as follows: (1) Firstly, the crack images collected under different environmental conditions are effectively classified, and feature extraction strategies are proposed for each type of crack image, which greatly reduces the amount of data and necessary calculation; (2) For high-density noise wide crack type image, median filter and Otsu algorithm are combined to solve the problem that the high-density noise group and the main crack feature can not be separated by using Otsu algorithm alone, and the crack detection accuracy is improved; (3) For the low-density noise narrow crack type image, an eight-neighborhood maximum connected component extraction algorithm is proposed, which can effectively remove all the sparse distribution noise and realize the lossless extraction of the main crack feature; (4) In the trajectory fitting stage, the Hough space transformation is combined with the K-Means clustering algorithm, which greatly eliminated the generation of redundant trajectories and improved the execution efficiency of the algorithm. Finally, through the simulation and fitting verification of the terminal algorithm and the actual scene verification of the experimental prototype, the robustness, efficiency and accuracy of the algorithm are comprehensively tested. In the future, a crack trajectory fitting algorithm that is more efficient and more suitable for real road repair application scenarios will be designed and implemented for complex network crack images and diversified noise backgrounds.