The Detection of Railheads: An Innovative Direct Image Processing Method

Abstract

This study presents a fully automated railhead detection method based on a direct image processing algorithm for use on a railway track. This method functions at a much faster pace than artificial intelligence algorithms that process rail images on embedded systems or low-power devices, as it does not require the use of significant computing resources. With the use of this method, railheads can be analyzed to identify the presence of cracks and other defects. We converted color images to halftone images, performed histogram equalizations to improve the contrast, applied a Gaussian filter to reduce the presence of noise, utilized convolutional filters to extract any vertical and horizontal lines, applied the Canny method and Sobel filters to refine the boundaries of the extracted lines, applied the Hough transform technique to extract lines belonging to the railhead images, and identified the segments with the highest brightness values to process the images of the railheads under study. The method of railhead separation described in this article will allow for further comprehensive diagnostics of the condition of rail threads to ensure the safe and sustainable operation of railway transport. The implementation of intelligent maintenance systems and effective monitoring of railway track conditions can reduce the negative impact on the environment and contribute to the advancement of rail transport as a sustainable, safe, and more environmentally friendly mode of transportation.

1. Introduction

In the modern world, railway networks play an increasingly important role in ensuring the safe transport of passengers and freights to meet the growing economic and military needs. Infrastructure, development, and maintenance are integral to the railway industry. The railway track is one of the most important parts of the railway system and is influenced by various factors, such as high train speeds and vibrations and climatic conditions, which can cause severe damage and create problems related to transport safety and crucial social, economic, and sustainability factors that must be considered.

When the rail and wheel interact, particularly on high-speed and heavily loaded tracks, defects and damage to the rail are inevitable. A significant number of these defects appear on the rolling surface of a railhead. These defects can develop rapidly and pose a serious threat to train safety.

Figure 1 presents defective and acutely defective rails detected over one month on a 6147.7 km railway track at one of the regional branches of JSC Ukrzaliznytsia (Ukraine). To ensure the trouble-free operation of the track and the timely detection of defects and damage to the rails and turnout elements, domestic railways have developed and implemented a multi-stage non-destructive testing method based on technical diagnostics, including the use of primary and secondary inspection equipment.

This concept presents a synergistic property when the overall efficiency of a rail diagnostic system is higher than the sum of the efficiencies of each diagnostic system. The analysis of the information received from different flaw detection devices assessing the same track section over a certain period of time allows for the real-time tracking of the rail condition and introduces preventive measures for systematic track rehabilitation purposes, eliminating (minimizing) the chance of sudden track failure occurring due to rail defects, thus ensuring the incorporation of necessary train safety measures.

A clear and well-coordinated operation of the non-destructive testing complex and systematic analysis of the causes of defects are some of the key aspects of ensuring the safety of train traffic on the railway transport system.

Detachable tools that move along the tracks to simultaneously inspect two rail strands and self-propelled flaw detection vehicles equipped with automated control equipment are some of the devices used at present for rail flaw detection purposes.

Detachable rail inspection tools require the direct involvement of track maintenance personnel who walk along the railway track and conduct non-destructive rail inspections. This method allows operators to visually detect any defects on the rolling surface of the railhead.

Non-destructive testing and technical diagnostics of rails at high speeds can be performed using mobile rail inspection equipment. The use of mobile flaw detection equipment significantly reduces the cost of rail operations due to lower inspection costs, increased frequency of use, and the timely detection of defects.

However, the non-destructive testing of rails requires the emitters and receivers of the diagnostic system to be in continuous contact with the railhead surface. However, since railhead surfaces can experience different types of damage and defects, this continuous contact is not always achieved, resulting in the difficulty and, at times, impossibility of performing the non-destructive testing process. These are called non-testable rails. According to the Classification and Catalogue of Defects and Damage of Rails in Ukrainian Railways [1], surface defects include:

• crumbling and delamination of metal on the railhead’s rolling surface due to deficiencies in rail manufacturing technology (code 10.1–2) and the crumbling of metal on the side working bend of the railhead due to insufficient contact fatigue strength of the metal (code 11.1–2);
• slippage of rails by locomotive wheels (code 14.1–2);
• crumbling and delamination of metal on the rolling surface in the hardened layer (in the absence of surfacing) (code 17.1–2);
• crumbling of the welded layer on the rolling surface of the railhead, included as a result of the violations of welding rail connector technology (code 18.1–2) and non-testable due to the presence of a significant number of defects and damage on the rolling surface of the rail (code 19.1–2).

These problems frequently occur on railway tracks (see Figure 1) and are exacerbated by passing tonnage and numerous technological and technical factors. For the timely detection of defective areas, it is necessary to perform additional inspections using removable ultrasonic inspection equipment that detects internal damage.

Therefore, the aim of this study is to develop a direct rail image processing method for railhead extraction purposes.

In view of the aims presented above, our study pursues the following objectives:

• Section 2 analyzes the current technical diagnostic methods for the physical and mechanical characteristics of rails, which are used during different temperatures and conditions.
• Section 3 describes the development of the direct rail image processing method algorithm used to perform railhead extractions.
• Section 4 presents a comparsion of our developed railhead segment extraction method and the method presented in the work of [2].
• Section 5 provides the conclusions and a discussion of the results of our investigation and additional steps to improve the method developed in this study.

2. Analysis of the State of the Problem under Study

Based on the above considerations, and in accordance with the requirements for identifying internal and surface rail defects described in [3], it can be concluded that there is no universal method of the non-destructive testing of rails. Each method has its own advantages and disadvantages. It is necessary to combine inspection systems to increase the probability of detecting defects in rails. For example, paper [4] proposes a combined study using electromagnetic detection and multi-frequency excitation, which will allow detecting surface and near-surface defects in the heads. The authors of [5] propose to combine Ground-Penetrating Radar (GPR) and Interferometric Synthetic Aperture Radar (InSAR) methodologies and continue to explore the possibilities of integrating machine learning to predict the condition of railway tracks and related maintenance. The use of deep machine learning methods and three-dimensional recurrent models based on neural networks for defect localization was investigated in [6]. The application of the decision tree (DT) method, the use of automated systems, and modeling the service life of railway sleepers to ensure railway transport safety are presented in [7,8].

Visual inspection is used to increase the probability of detecting surface rail defects. For example, a general visual inspection is used to detect the presence of surface defects while walking along the railway track. To create the best conditions for detecting defects during such control, the rail should be illuminated, if necessary, with auxiliary lighting to achieve a minimum of 160 lux, and the distance between the inspection site and the rail surface may be more than 600 mm [3].

It is most effective to use automatic visual inspection for rail inspection, which would allow the automatic detection of defects on the rolling surface of the railhead. Visual inspection can detect the following defects [3], which are classified according to [9]:

• horizontal/vertical cracking;
• corrosion;
• squats;
• wheel burns;
• head checks;
• shelling;
• corrugation;
• missing or damaged components associated with the rail (for example, fish plates, insulated joints, chairs, or clips).

At present, several different technical diagnostic methods assessing the physical and mechanical characteristics of metal structures under long-term operation at different temperatures and under different environmental conditions exist [10].

The non-destructive testing of rails can be divided into methods [11,12] based on eddy currents, as well as ultrasonic, visual inspection, thermal, X-ray, magnetodynamic, and direct current (DC) methods, all of which have their advantages and disadvantages.

The eddy current method involves eddy current monitoring and magnetic flux dissipation. This method measures the response of a material to an induced electromagnetic field; the presence of a surface or internal (below-the-surface) defect creates a distributed electromagnetic field that can be measured. This is a non-contact method and is very sensitive to the detachment of the probe from the surface of the samples being tested [12]. This method can detect the presence of surface or near-surface transverse cracks; however, it is not suitable for the detection of longitudinal cracks, and its effectiveness is affected by the selected scanning speed [13].

The railway industry uses ultrasonic methods for the mobile diagnostics of railway track defects to check for any internal defects and to monitor rails in operation [14,15]. The disadvantages of this method include the contact between the rail surface and the piezoelectric emitter, which prevents the detection of surface defects on rails. The possibility of obtaining inaccurate results increases when rail surfaces become rough due to corrosion. Surface irregularities can also create dead zones that cannot be read by the sensors. Standard sensors do not distinguish linear defects that run parallel to their beams, and control sensors are not suitable for processing coarse-grained materials, as evidenced by the research materials of Zetec, a world leader in the field of non-destructive testing (NDT).

Thermal methods employ the change in the thermal properties of the rail to detect the presence of a defect. A single frame or a video of the surface temperature distribution of the test sample is used to determine the presence of any defects [16].

Radiographic methods allow for a visual analysis of the internal structure of a rail or weld. This method detects cracks, defects, and thickness reductions in great detail, but it is a hazardous, expensive, and time-consuming approach [13,15].

The magnetodynamic method, also known as the magnetic flux leakage (MFL) method, is currently used in countries, including the United Kingdom, Iran, and the USA, for the detection and characterization of defects on railway tracks [17].

The eddy current (EC) testing method is also widely used in many countries. However, despite its advantage of everyday use, its results are negatively affected in the presence of multiple cracks located close to each other on a rail [18].

The operation of flaw detectors using the magnetodynamic method has also revealed their main limitation: the presence of an air gap between magnet poles and the rail. Increasing this gap to 20–23 mm for traffic safety reasons leads to a significant weakening of the magnetic flux in the rail. At the same time, due to an increase in the scattering flux, all the structural elements of the rail track become clearly visible, and the operator receives a false impression of the normal functioning of the structure. Moreover, the dimensions of the electromagnets of the rail magnetization system, which completely occupy the inter-wheel space of the inductor trolley, and their power consumption values may also be affected.

Visual inspections can be performed by a railway worker by walking along the railway tracks and searching for surface defects, since methods for the automatic recognition of defects on the rolling surface of rails need to be developed.

The surfaces of railheads can have various forms of damage and defects that can make non-destructive testing difficult, or even impossible, to perform. Areas with complicated or impossible contact control are referred to as non-testable rails. The number of these areas on a railway track increases with the volume of passing tonnage and various other technological and technical factors.

Video inspection using mobile rail testing equipment is one way of detecting non-testable defects on the rolling surface of rails [19]. The primary task of this method is to isolate the rail and its head in the image. Existing rail detection solutions using artificial intelligence (neural networks) [20,21,22], which requires the use of many test images with highlighted features and their descriptions for training, demand substantial resources for implementation purposes. As a result of these shortcomings, the potential for the widespread adoption of this method and its use in a real-time setting are complicated.

Study [23] presented a laser scanning methodology that enables the automatic determination of the position and geometry of rails determined from the three-dimensional (3D) data of the rail point cloud. By comparing it with rail position standards (IFCs), the method can identify position deviations smaller than 3 cm, but it does not have the capability to identify defects within the rails themselves.

The detection of cracks in railway structures is the primary task in railway analysis. In study [24], the authors proposed a cost-effective solution to address crack-related issues using radio frequency control. This approach detects track defects that occur periodically, whether by natural or artificial means. Natural causes include significant rail expansion that occurs due to temperature increases or flooding, while artificial causes include events such as terrorist attacks. Figure 2 presents common surface defects on and damage to the sections of main railway tracks that are considered non-testable.

To summarize, it can be argued that the development of flaw detection tools, together with measures aimed at improving the processing and systematization of data on the causes of rail defects, is ultimately one of the main ways to improve train safety, cargo delivery times [25], and passenger schedules [26].

The analysis shows that, over the years, various methods have been developed to diagnose the condition of rails and turnouts. The solutions available at present do not fully meet the requirements of railway infrastructure operators for traffic safety monitoring purposes; thus, this area requires further research to improve railroad efficiency, increase the probability of detecting defects, and allow for the predictive maintenance of railway infrastructures.

3. Materials and Research Methods

Recently, methods based on the detection of surface defects on rails, also known as the machine vision, have gained importance in the field. This method consists of the preliminary processing of rail images, which consists of removing unnecessary objects, such as parts of sleepers, rail soles, crushed stone, or other loose material on which the rail pair is laid. In addition, it is necessary to remove the background from the image and highlight the rail and its head to effectively present the damage present on its surface.

We selected a direct image processing method for isolating the rail and its head in the image, as the algorithm on which this method is based does not require significant computational resources and can operate on embedded systems or low-power devices. This is particularly important for real-time image processing or when the processing speed is a critical factor. An additional advantage of the method we selected is the independence of the direct image processing algorithm from complex learning algorithms, and, as a consequence, it produces fewer errors.

To perform the railhead detection study using the described algorithm, we selected color images of rails at 500 × 500 pixels. At the initial stage of the assessment, the original image was converted into a grayscale image, which contained information about the brightness and did not contain information about the color of the pixels, in order to reduce the influence of colors on further processing stages.

In the color image, each pixel was a combination of three values ranging from 0 to 255 corresponding to red, blue, and green. To convert the image from color to grayscale, it was necessary to transform the color values of each pixel into a single gray value within the same range of 0 to 255. As a result, each pixel will have one color value instead of three. The simplest method is to select the average of the three values of red, green, and blue for each pixel. This study presents the developed algorithm for the direct image processing method, which is schematically depicted in Figure 3.

The first stage of this algorithm, the conversion of a color image into a halftone image, aims to increase the efficiency of image processing by reducing the number of operations required to process it, which will improve the processing quality. The grayscale, which has 256 gradations of brightness, can accurately represent an object in an image.

The second stage of the algorithm, image histogram equalization, is performed to increase the contrast between image objects, which allows for a more accurate selection of the required objects.

The third step of the algorithm, Gaussian filtering, removes small objects from the image and reduces the impact of noise, which also improves the quality of railhead detection.

The subsequent step is convolutional filtering, which is used to remove lines from the image that may not belong to rail images. As a result of this filtering activity, the processed image usually presents lines that can belong to rails, rail soles, and sleepers.

The application of the Canny method and Hough transform, which are the subsequent steps of the algorithm, allow for the selection of lines that belong to the rails.

The selection of median lines, created from group of lines that are closely spaced and/or overlap, allows the removal of unnecessary selected lines and only the lines that delimit the railheads remain; the selection of the brightest segment between the lines is the railhead. However, the human eye perceives colors differently, for example, green is perceived to be ten times brighter than blue. The conversion of a color image to grayscale is conducted using the following formula:

$$Y=0.2126R+0.7152G+0.0722B$$

(1)

where Y is the original color value of a pixel in a halftone image, R is the value of the red color of the pixel, G is the value of the green color of the pixel, and B is the value of the blue color of the pixel. As a result of the conversion stage, each pixel has a single value in the range from 0 to 255. Figure 4 presents one of these images and its halftone transformation.

The subsequent step of the image processing algorithm is the enhancement of the image in terms of increasing the contrast of the edges and contours in the images. The equalization (leveling) of the brightness histogram was performed [27,28]. Since the brightness level is a discrete function used for processing halftone images, the probability of occurrence of the brightness level, $r_k$, is calculated using the following formula [29]:

$$p\left(r_k\right)=\frac{n_k}{M\times N}, k=0, 1, \dots ,m-1$$

(2)

where $M\times N$ is the size of the input image, $n_k$ is the number of pixels with a brightness level, $r_k$, $m$ is the number of possible brightness levels of an image (for example, 256 for an 8 bit image), and p is an empirical probability density function. A histogram of the number of pixels, $n_k$, with brightness levels, $r_k$, is shown in Figure 5.

The cumulative probability density function of the brightness levels can be calculated using the following formula [28,29]:

$$h\left(r\right)=\sum _{j=0}^{k}p\left(r_j\right)$$

(3)

Based on the results obtained for the analysis of images and their luminance histograms, it can be concluded that an image with a uniform distribution of the luminance range will be better for perception purposes. The histogram equation requires the re-estimation of the cumulative probability density function of the brightness levels using the following formula [29]:

$$s_k=\left(m-1\right)\sum _{j=0}^k{p}_k\left(r_j\right)=\frac{\left(m-1\right)}{M\times N}\sum _{j=0}^k{n}_j , k=0, 1, 2, \dots , m-1$$

(4)

Figure 6 presents histograms of the stages before and after the equalization was performed.

Indeed, following the histogram equalization process, function p(x) presents a linear form. The image with improved contrast is presented in Figure 7.

The subsequent step of the algorithm is the application of a Gaussian filter to reduce noise and remove small details from the image, which is based on the use of the Gaussian as a kernel for image convolution purposes [30]:

$$G\left(x\right)=\frac{1}{2\pi \sigma }{e}^{\frac{x^2+y^2}{2{\sigma }^2}}$$

(5)

where x, y is the distance from the origin to the pixel, and σ is the standard deviation of the Gaussian distribution.

To speed up the calculations, the central pixels [31,32] are usually selected for each image element. The subsequent step of the algorithm requires the extraction of the contours of the vertical and horizontal lines, which is performed by applying convolutional filters to the image [33,34]. These filters are called ‘horizontal’ and ‘vertical’ filters, respectively, and have a dimension of 5 × 5 (Figure 8); the results of applying vertical and horizontal convolutions to the image with the rail are shown in Figure 9.

In the convolution process, the input image is convolved pixel by pixel with the appropriate filter (kernel) based on the multiplication and addition of values. This operation produces an output image (or feature map) that reflects the important features (in our case, lines) detected in the input data.

Many lines can be selected when using convolutions. They can overlap and/or be spaced very close to each other (Figure 10).

We used the Kenny method to refine the boundaries (contours) of the selected lines [35,36]. This method is one of the most widely used in the field and is based on the application of the Sobel operator [37]. Its concept is as follows: Let the 3 × 3 envelope of a certain image element be represented by brightness values, as shown in Figure 11.

The Kenney method is used to move the mask from one point in the image to another. The movement occurs in the direction of the brightness gradient, which indicates the direction of its fastest change. The Sobel operator uses masks, as shown in Figure 12.

The approximate values of the derivatives at the (x, y) point in a digital image are used to calculate the gradient:

$$\begin{gathered} G_x=\left(z_7+{2z}_8+z_8\right)-\left(z_1+{2z}_2+z_3\right) \\ {G}_y=\left(z_3+{2z}_6+z_9\right)-\left(z_1+{2z}_4+z_7\right) \end{gathered}$$

(6)

Then, the gradient is expressed as follows:

$$G\left(x\right)=\sqrt{G_x^2+G_y^2}$$

(7)

To determine the direction of the gradient, the angle between its direction and the axis, $O_x$, is calculated as follows:

$$\mathsf{\Theta }={\mathrm{t}\mathrm{a}\mathrm{n}}^{-1}\left(\frac{G_x}{G_y}\right)$$

(8)

The direction of the path at the $\left(x, y\right)$ point is always perpendicular to the gradient. This improves the method’s ability to detect the boundary. Subsequently, the threshold filtering technique is applied, in which pixels with a gradient that exceeds the threshold are retained, and the remainder are discarded. The threshold can be set by the user or automatically selected using the algorithm [35,38]. To determine straight lines for the boundaries, which correspond to the rails in the images, we used the Hough transform method [39,40]. Each point of the boundary is represented as $\left(x, y\right)$ in the image, for which possible parameters (angle and/or radius) are generated that define a geometric shape that passes through this point, such as a line or a circle (in our case, a line), i.e., an ‘accumulation matrix’ or ‘accumulation space’ is created, in which each possible set of parameters (angle, radius, etc.) is represented as a value in this space.

The local maxima are determined for each point of the boundary corresponding to certain values of the selected parameters. The peaks corresponding to the local maxima indicate the parameters (number, position, etc.) that best fit the detected geometric shape—in our case, a line.

The Hough method does not distinguish among lines that belong only to railheads but can also detect extraneous lines, such as those formed by the shadow of a rail. Therefore, after applying the Hough transform method, it is advisable to filter the several lines that appear in the image (Figure 13a,c).

To perform this step, the lines are grouped according to their distance from each other, forming groups of closely spaced lines, and the median line is calculated in each group. Thus, one line is selected from each group of lines (Figure 13b,d). The selected lines belong to the rails (Figure 14).

The next stage of computation is performed to distinguish between the straight lines, the parts of the image that represent railheads. For this stage, we compared the brightness of the selected segments. Since the railheads present the highest brightness level relative to the entire rail, the average brightness level of the segments between the lines is calculated, and the segment with a brightness level that is higher than the average is selected.

The brightest segment of the image between the two lines corresponds to the image of the railhead (Figure 15).

4. Comparison of the Results of the Developed Method and the Prototype

The railhead segment extraction method developed in this study was compared to the method presented in the work of Kang Zhao [2]. His work [2] also described a direct image processing method used for detecting railheads. Kang Zhao’s algorithm is based on the image threshold method. The segmentation threshold is determined based on the grayscale diagram of the image. It was observed that the grayscale value of the rail surface was high and the change in values in the adjacent area was small compared to the transition from rail to non-rail surfaces. Therefore, the grayscale level at the junction changes abruptly (Figure 16). The image threshold method was applied to the input image (Figure 17a), which created a binary image (Figure 17b).

Subsequently, the output image was produced by comparing the pixels of the input and binary images. The pixels of the input image remained unchanged if they corresponded to white pixels in the binary image, and the pixels of the input image that corresponded to black pixels in the binary image were converted to black. The result is an image with a highlighted railhead (Figure 18).

One of the disadvantages of the described algorithm is its high dependence on the brightness of other elements present in the image. This means that bright details may appear in the original image that are not actually related to the rail. During the image processing step, such elements can be mistakenly interpreted as forming part of the railhead. In comparison, the algorithm presented in this study has an important advantage: it does not depend on the brightness of other elements that are not related to the rail object in the image. This allows maintaining the accuracy and reliability of the processing results, avoiding erroneous interpretations and ensuring the clear identification of the railhead regardless of the additional, bright details present in the input image. In addition, the machine vision system developed by Kang Zhao for railhead detection purposes is directly dependent on the image quality; therefore, it has high requirements for image acquisition and rail illumination and should be approximately 4600 lux, unlike the developed system, which is independent of the illumination of the rail.

To summarize, it can be argued that the research focusing on developing flaw detection tools, coupled with initiatives aimed at enhancing the processing and systematization of data regarding the causes of defects, is ultimately a key factor influencing sustainable railway transport, train safety, cargo delivery times [25], and passenger transport schedules [26].

5. Conclusions

This study analyzed the existing problems in rail image processing methods and identified the main disadvantages that affect their speed and quality. For example, the use of neural networks in image processing techniques requires a high computation and complex learning algorithms that increase the likelihood of errors. The use of ultrasonic rail inspection methods, electromagnetic waves, and eddy currents can detect internal rail defects but do not detect surface cracks or surfaces with deep faults. The method proposed in this study allows the selection of railheads for further processing in order to identify defects located on rail surfaces based on the direct image processing method.

As a result of the experiment, it was determined that the developed method can identify the railheads, regardless of the placement of the rail in the image; however, if the position of the rail in the image is known in advance, i.e., it is horizontal or vertical, the processing time of the rail image can be accelerated by applying a specific convolution of the boundary search. If the rail is photographed from above, perpendicular to the rolling surface of the railhead, the images are processed faster, because less time is spent on correcting the position of the rail. The dependence of the processing speed on the position of the rail in the image (Figure 10, Figure 13 and Figure 14) is presented in Figure 19.

The railhead detection method we developed was compared with the prototype method [2]. The main disadvantage of the prototype method is that the method focuses on the brightness of the rail surface and can mistakenly highlight bright elements that are not related to the rail.

It was experimentally proven that the presence of other bright elements in the image (Figure 13 and Figure 14) did not affect the processing result, since the algorithm focused on clear longitudinal lines (straight lines) and did not pay attention to other details (Figure 14 and Figure 15).

The developed method serves not only as an effective tool for processing rail images to extract railheads but also as a basis for further scientific research. The main emphasis in the future will be placed on automating the detection of various types of defects on the rolling surface of railheads and assessing their impact on train safety. The development of research in this area presents additional opportunities to improve the sustainability and efficiency of railway transport.

We implemented software based on the direct processing method of rail images for railhead detection and the algorithm described in detail in this study. The results are illustrated in the figures presented above.

To summarize, it can be argued that research focused on developing a suite of flaw detection tools, coupled with initiatives to enhance the processing and systematization of data regarding causes of defects, is ultimately one of the key factors influencing sustainable railway transport, train safety, cargo delivery times [25], and passenger transport schedules [26].