Sparse Haar-Like Feature and Image Similarity-Based Detection Algorithm for Circular Hole of Engine Cylinder Head

Abstract

If the circular holes of an engine cylinder head are distorted, cracked, defective, etc., the normal running of the equipment will be affected. For detecting these faults with high accuracy, this paper proposes a detection method based on feature point matching, which can reduce the detection error caused by distortion and light interference. First, the effective and robust feature vectors of pixels are extracted based on improved sparse Haar-like features. Then we calculate the similarity and find the most similar matching point from the image. In order to improve the robustness to the illumination, this paper uses the method based on image similarity to map the original image, so that the same region under different illumination conditions has similar spatial distribution. The experiments show that the algorithm not only has high matching accuracy, but also has good robustness to the illumination.

1. Introduction

If faults, such as distortion, cracks and defects, occur in engine parts, it will damage mechanical equipment and cause trouble for engineering applications. Many kinds of signals have been used for mechanical fault diagnosis, such as vibration signals [1] and acoustic signals [2]. In recent years, in order to quickly and accurately detect faults, image detection, on-line monitoring, infrared thermography [3] and automatic target recognition have been proposed. Among them, the feature point matching algorithm is the key algorithm in image detection for the faults of cylinder head hole.

In this algorithm, feature selection is an important issue. Image features can be roughly divided into color, texture, shape and spatial relationships. The methods for extracting color features include: color coherence vector, color set, color histogram and color correlogram. The methods for extracting texture features include: statistical methods, such as gray-level co-occurrence matrix [4], geometric methods, such as Voronio chessboard characteristic method and structure method [5], model methods, such as Markov random field [6] and conditional random field, and signal processing methods. The methods for extracting shape features include: boundary feature extraction methods, such as Hough transform [7] and Canny method [8], geometric parameter methods and shape invariant moments methods. The methods for extracting shape features include region segmentation and object segmentation.

Based on the combination of color, texture, shape and space features, some feature matching detection algorithms which are more efficient have been developed. The classic methods among them are: determinant of Hessian (DOH) features [9], Haar features [10], features from accelerated segment test (FAST) [11], local binary pattern (LBP) features [12], scale-invariant feature transform (SIFT) [13] and speeded-up robust features (SURF) [14].

The SIFT algorithm has strong scale invariance and rotation invariance, as well as robustness in transform of blur, view angle, illuminance and image compression. It was improved by Lowe in 2004 [13]. However, this algorithm is not in real time. Bay and Tuytelaars improved SIFT to SURF, which greatly improved the efficiency of the algorithm [14]. First, this algorithm uses different scales Hessian matrices to generate the key points of image stabilization, and then uses the determinant of Hessian to extract local maxima points as the interest points. Then, all the interest points obtained by Hessian matrix are compared with 26 points in its three-dimensional space neighborhood, and the largest interest point in its three-dimensional space neighborhood is selected as the primary key point, so that the key points of weak energy are filtered out. Finally, the direction corresponding to the maximum value in the direction histogram is selected as the main direction of the feature point, and the feature point descriptors are obtained. Both SIFT and SURF are highly robust, but do not have excellent real-time performance. SURF is not as complex and time-consuming as SIFT, but SURF still requires a lot of calculation to obtain the main direction and feature descriptors.

In order to improve the speed of extracting the features and obtaining feature descriptors to meet the requirements of practical tasks, more algorithms have been proposed, such as Haar, FAST, binary robust independent elementary features (BRIEF), Oriented FAST and Rotated BRIEF (ORB) [15], LBP, etc.

In 2002, Lienhart and Maydt extended the Haar feature library using diagonal characteristics [16]. In 2014, Zhang et al. proposed a pedestrian detector based on Haar-like features, which has good robustness in many situations [17]. Furthermore, Rubbe et al. proposed a fast feature extraction algorithm, ORB, based on FAST and BRIEF [15]. The algorithm is widely used in many fields that need feature point matching because of its fast computing speed. Then, Kral et al. proposed a method for breast tumor detection based on LBP in 2016 [18]. Although the LBP operator is good in real time, its robustness is poor.

In addition to those mentioned above, there are many algorithms that have strong robustness of scale and rotation transformation or fast real-time performance too, such as the nonlinear feature detection algorithm named KAZE [19], and the center-symmetric local binary pattern (CS-LBP) [20]. In recent years, algorithms based on deep learning have also made achievements in image similar block detection. For example, aiming at the problem of optical flow estimation, Bailer and Varanasi proposed a matching network based on pixel blocks [21]. Si and Xiaodan proposed a multi-column convolutional neural (M-CNN) network to predict the confidence and displacement of an optimal matching region in the test image of a particular semantic region in the k-Nearest Neighbor (KNN) image [22].

For detecting the faults of the circular hole on the surface of engine cylinder head, this paper proposes a detection method based on feature point matching, which can reduce the large detection error caused by distortion and light interference. First, improved sparse Haar-like features are used to reduce the effect of scratches and speckles on the same feature in Haar. Then we combine Euclidean distance and activation number to calculate the similarity of feature vectors. Next, we use image similarity to map the images of different brightness into the same gray space, to reduce the influence of external light on local texture. Finally, this paper compares the results of the improved Haar-like features with those of the traditional Haar features in detail, and shows the anti-light-interference capability based on image similarity and the detection results of the surface of the engine cylinder head. Experimental results show that the algorithm proposed in this paper can overcome the texture difference caused by the camera angle and illumination, and has high detection accuracy.

2. Methods

The workflow of the feature-matching algorithm based on sparse Haar-like feature and similarity calculation is shown in Algorithm 1. In the basic feature-matching process, feature selection and similarity calculation are the most critical parts. Moreover, a robustness enhancement method of illumination is added to the algorithm.

Algorithm 1: Flow of feature-matching algorithm based on sparse Haar-like feature and similarity calculation.

Input: Model image Im, test image It, key point loc in the model image, image pyramid layer number Np and scaling factor Zp, convolution kernel layer number Nc and scaling factor Zc, similarity adjustment factor k, adjustment factor α. Output: The similarity point Ps in the test image It which is most similar to the key points loc in the model image Im.

1. The brightness-enhanced images It’ and Im’ of It and Im are respectively obtained according to Equation (11). 2. Construct the image pyramid on It’ and Im’ according to Np and Zp; construct the sparse Haar-like feature detection operator pyramid according to Nc and Zc. 3. Extract the features of each pixel of each layer of the pyramid according to Equation (1), and then merge them by inverse pyramid scaling to obtain the sparse Haar feature F of each pixel of It and Im. 4. Normalizing the features according to Equation (2). $\tilde{F}$ is gotten. 5. The heat map $H_{i,j}^s$ of the key point loc in Im and It is obtained according to Equation (7). The maximum point of the heat map is the similarity point Ps in It that is most similar to loc in Im.

2.1. Improved Sparse Haar-Like Features

Viola and Jones proposed four Haar-like features [10]. Figure 1 shows the texture features of these four basic directions.

Lienhart et al. proposed an improved Haar-like feature [16], pointed out that the feature D proposed by Viola was invalid, and divided the new Haar-like feature into edge features, linear features, and surround features, as shown in Figure 2, where white means 1 and black means −1.

As for the actual problem of engine cylinder head detection, the area to be detected can be located based on the relationship between a certain point of the circular edge and the eight directions around it, but the accuracy is insufficient, as shown in Figure 3, where the blue points indicate the location to be detected and the green represents the location that is actually detected.

From the above, we can see that Haar-like features are not accurate enough, and are obviously influenced by the illumination. In order to improve the detection accuracy, a sparse Haar-like feature is proposed in this paper, as shown in Figure 4, where green stands for 1, black for −1, and white for 0. In the figure, the first convolution kernel of edge feature and linear feature rotates 90°, 45° and 135° to get the last three.

Let the size of convolution kernel be n × n, the rotation angle be θ, Ω_b denote the black region, Ω_g denote the green region, p_b and p_g denote the pixel points of the black and green regions. Thus, the calculation formula of the convolution feature is as follows:

$$F_{i,j}={\displaystyle \sum _{p_b}^{{\Omega }_b}{s}_b\cdot I_{(i,j)-p_b}}+{\displaystyle \sum _{p_g}^{{\Omega }_g}{s}_g\cdot I_{(i,j)-p_g}},$$

(1)

where s_b denotes the value of the black region in the convolution kernel, which is −1; and s_g denotes the value of the green region in the convolutional kernel, which is 1. Lastly, (i, j) − p_g represents the relative position of the convolutional kernel centered on the (i, j) point of the image.

Finally, the convolution features are normalized to ensure that each feature contributes the same to the overall matching.

$$\{\begin{array}{c}{m}_a=\max (F)\\ m_i=\min (F)\\ \tilde{F}=(F_{i,j}^m-m_i)/(m_a-m_i)\end{array}$$

(2)

Suppose we take two convolution kernels of size 8 and 26, as shown in Figure 5. The convolution kernels in some areas are so small that it is difficult to distinguish certain pixels from the surrounding pixels, which will cause inaccuracy. Larger convolution kernels can suggest more information in the region and improve the detection accuracy, but take a long time to calculate.

Therefore, in order to obtain more regional information and improve the calculating speed, the image pyramid and operator pyramid, which is a convolution kernel pyramid, are used to widen the local area, as shown in Figure 6.

2.2. Calculation of Feature Similarity

For calculating the similarity of feature vectors, let ${\tilde{F}}^s$ denote the feature map obtained from the model image, and ${\tilde{F}}^t$ denote the feature map obtained from the test image. Moreover, H and N denote the difference of the features at each pixel point of the model image and the test image, and the number of similar features, respectively. Suppose:

$$v_{i,j,k}=\left|{\tilde{F}}_{i,j,k}^t-{\tilde{F}}_{loc,k}^t\right|,$$

(3)

where ${\tilde{F}}_{i,j,k}^t$ denotes the k-th feature at the pixel point (i, j) of the test image and ${\tilde{F}}_{loc,k}^t$ denotes the k-th feature at the pixel point loc of the model image.

Then, calculate the difference at the k-th feature of the pixel (i, j) of the model image and the test image, which is h_i,j,k, and calculate the similarity, which is m_i,j,k:

$$h_{i,j,k}={\Vert {\tilde{F}}_{i,j,k}^t-{\tilde{F}}_{loc,k}^t\Vert }_2$$

(4)

$$m_{i,j,k}=\{\begin{array}{c}1,v_{i,j,k}<0.5\\ 0,v_{i,j,k}\ge 0.5\end{array},$$

(5)

where “0.5” is the activation threshold, which can be adjusted according to the actual situation.

Next, the heat map H and the similar quantity map N can be obtained:

$$\{\begin{array}{c}{H}_{i,j}=\sqrt{{\displaystyle \sum _{k=0}^n{h}_{i,j,k}}}\\ N_{i,j}={\displaystyle \sum _{k=0}^n{m}_{i,j,k}}\end{array}.$$

(6)

Generally, the smaller H_i,j is, the more similar the test image pixel point (i, j) is to the model image pixel point loc. The larger N_i,j is, the more similar features the test image pixel point (i, j) and the model image pixel point loc have. Thus, the heat map H and the similar quantity map N can be fused into the final matching heat map:

$$H_{i,j}^s=H_{i,j}\cdot {\mathrm{e}}^{-k\cdot N_{i,j}/n},$$

(7)

where k is the adjustment factor and n is the number of features. The maximum point of the heat map is the similarity point P_s in the test image that is most similar to the key point loc in the model image.

2.3. Robustness Enhancement Method of Illumination

Similarity calculation has been used in many research fields of image processing [23]. In this paper, the similarity between pixels and the center point in the image is used for the robustness enhancement method of illumination.

Let I denote the input image, w denote the width, and h denote the height. First, subtract the mean value of the image from the input image:

$$\{\begin{array}{c}u=\frac{1}{w\times h}{\displaystyle \sum _i^w{\displaystyle \sum _j^h{I}_{i,j}}}\\ I_{i,j}^m=I_{i,j}-u\end{array}.$$

(8)

Then, the image is normalized by calculating maximum value and minimum value:

$$\{\begin{array}{c}{m}_a=\max (I)\\ m_i=\min (I)\\ \tilde{I}=(I_{i,j}^m-m_i)/(m_a-m_i)\end{array}.$$

(9)

The difference image is calculated by using the normalized image. Let c be the center of the image and B be the difference image. Thus, the formula is as follows:

$$B_{i,j}=I_{i,j}-I_c$$

(10)

Next, the difference image is mapped to [0, 1] space by a sigmoid function. After this, the final similarity image $I_{i.j}^{\prime }$ can be obtained as following, where α is the adjustment factor:

$$I_{i.j}^{\prime }=\frac{1}{1+e^{-\alpha \times B_{i,j}}}.$$

(11)

For different adjustment factors, the similarity image will also be different. However, by using an adaptive adjustment factor α, the similarity images of the images with different illuminations could be more similar, as shown in Figure 7. In the figure, u denotes the absolute value of the difference between the center point value of the image and the mean value, and v denotes the variance of the pixel values of the image. It is obvious that the images of the same region with different illuminations are very similar, when α = 8 or α = 12.

3. Experimental Results and Analysis

In practical engineering applications, an algorithm based on feature point matching should be used where the detection environment can be guaranteed to be the same at different times. The experimental platform built in this work is shown in Figure 8. In the figure, in red box 1 is an experimental bracket, a nearly sealed iron box with a camera and an illuminant in it. Sealing reduces the impact of the environment on the algorithm. In red box 2 is the host computer. In the experiment, the radius of the circular hole on the surface of engine cylinder head is detected.

First, in this section, the advantages of the improved Haar-like texture feature compared with the traditional Haar feature are analyzed in detail. Secondly, the effect of space mapping method based on image similarity on illumination robustness enhancement is demonstrated. Finally, the detection results of the surface of the engine cylinder head are shown.

3.1. Analysis of Improved Haar-Like Feature

Figure 9 shows the detection results of the circles with different illuminations, using ORB, KAZE, traditional Haar feature, and improved Haar feature, respectively. Model_1, Model_2, and Model_3 on the left side of the dashed line are original circular models for different regions. On the right of the dashed line are the images of three models under two different illumination conditions. The pink line is the connecting line between the original feature point and the center, while the red line is the connecting line between the matching feature point and the center. It is obvious that under different illumination conditions the matching effect of the improved Haar-like feature is better than that of ORB, KAZE, and the traditional Haar feature.

Table 1. Comparison of average pixel errors under different illuminations by three matching algorithms.

Model	KAZE		Haar		Improved Haar
Model_1	12.52	10.01	3.12	4.50	2.75	3.75
Model_2	14.78	16.09	8.12	13.25	4.87	6.80
Model_3	11.62	13.98	4.01	6.625	2.65	3.25

compares the average pixel errors of the matching results of three circular model feature points under two illumination conditions, using KAZE, the traditional Haar feature, and the improved Haar-like feature, respectively. It can be found from the table that the average pixel error of traditional Haar feature algorithm is smaller than that of the KAZE algorithm, and the average pixel error of improved Haar-like feature algorithm is much smaller than that of the traditional Haar feature.

Table 2. Comparison of circle radii average pixel errors under different illuminations by three matching algorithms.

Model	KAZE		Haar		Improved Haar
Model_1	3.43	3.76	1.48	1.96	0.74	0.90
Model_2	4.20	5.59	1.85	8.00	1.52	1.93
Model_3	3.35	3.24	2.53	1.79	0.88	1.22

compares the average pixel errors of circle radii obtained by the matching results of three circular model feature points under two illumination conditions, using KAZE, the traditional Haar feature, and the improved Haar feature, respectively. It can be found from the table that the average pixel error of the improved Haar feature algorithm is much smaller than that of the KAZE algorithm and the traditional Haar feature algorithm, which is approximately 1~2 pixels.

3.2. Analysis of Detection Robustness

Figure 10 shows the matching results of the circles with different illuminations by the method without any light processing and by the method based on image similarity, where “General” indicates that the image has not undergone any light processing. It is obvious that the method based on image similarity can improve the matching effect of the images with different illuminations and enhance the robustness of detection.

Table 3. Comparison of average pixel errors under different illuminations by robust enhancement algorithm.

Model	General		Similarity
Model_1	3.25	3.00	1.625	1.75
Model_2	4.87	5.50	2.68	2.75
Model_3	5.50	5.63	3.35	1.75
Model_4	14.22	10.74	5.65	5.48

compares the matching average pixel errors of the circle models with different illuminations by the method without any light processing and by the method based on image similarity. It can be seen from the table that the matching results of the original image with different illuminations have large average errors, and the matching error can be greatly reduced by using similarity mapping to preprocess the original image.

Table 4. Comparison of circle radii average pixel errors under different illuminations by robust enhancement algorithm.

Model	General		Similarity
Model_1	1.96	1.73	0.83	0.64
Model_2	1.47	3.15	1.15	0.74
Model_3	1.32	2.64	0.90	0.69
Model_4	9.46	9.12	1.93	1.80

compares the average pixel errors of circle radii of three circular models, which are obtained by the method without any light processing and by the method based on image similarity. It can be seen from the table that the average pixel error of image detection method based on similarity mapping is approximately 2 pixels, which means that its detection results are excellent.

3.3. Comprehensive Experimental Results and Analysis

Figure 11 shows the detection results of the circular holes on the surface of the same engine cylinder head. It is clear that the matching line is very close to the actual connecting line.

Figure 12 shows the average pixel errors between the detected radii and the actual radii in eight directions of the detected circles in Figure 11. It is easy to see that the average pixel errors of the detection radii is in the range of [0, 1], which shows that the detection results are excellent.

4. Discussion

According to the algorithm proposed in this paper, the dimension of the cylinder head circular hole can be accurately detected, which means this method is suitable for this specific problem. However, detecting the faults of other cylinder parts still needs further research. Moreover, the models accumulated in the experiment are not sufficient. Thus, there is still some room for development in the engineering practice.

5. Conclusions

In this paper, a detection algorithm based on feature point matching is proposed. First, the feature vectors are obtained based on the improved sparse Haar-like feature. Then, the feature similarity is calculated by Euclidean distance and activation number to obtain the most similar matching point in the region. However, Haar features are easily affected by illumination conditions. In order to improve the robustness of detection, from the point of view of algorithm theory, a method based on image similarity is adopted in this paper, which maps the images with different illuminations to the same space to reduce its interference. In engineering practice, the model index database is established, and model images with different illuminations are added. Experiments show that the improved sparse Haar-like feature has smaller detection errors and the space mapping method based on image similarity has strong illumination robustness.

After accumulating enough data in engineering practice, by using artificial intelligence and deep learning methods, a network model can be trained in the future. It can automatically select valid features without human intervention. In addition, the faults of other parts of the engine will also be taken into account.