UAV 3D Modeling and Application Based on Railroad Bridge Inspection

Abstract

Unmanned aerial vehicle (UAV) remote sensing technology is vigorously driving the development of digital cities. For experimental objects such as large, protruding, and structurally complex steel truss railway bridge structures, commonly used oblique photography and cross-circular photography techniques can lead to blurring, missing, or lower accuracy of fine texture in the models. Therefore, this paper proposes a real-scene three-dimensional modeling method that combines oblique photography with inclined photography and compares it with oblique photography and cross-circular photography techniques. Experimental results demonstrate that the model generated by combining oblique photography with inclined photography exhibits clearer textures, more complete lines, and higher accuracy, meeting the accuracy requirements of 1:500 topographic map control points. This method plays a beneficial auxiliary role in the inspection of ailments such as steel structure coating corrosion and high-strength bolt loss in steel truss railway arch bridges.

1. Introduction

Railway bridges are structures built to traverse obstacles such as rivers and valleys for railway crossing, as well as to facilitate three-dimensional crossings with railways or roads. China’s high-speed railway implementation of closed management, no one can not enter during operation, which also led to the railway line on the bridge inspection is very tricky, railway bridge [1,2,3] inspection can only be carried out at night, which led to the railway bridge disease inspection work is difficult, difficult to comprehensive analysis of the overall bridge disease. To address these limitations, UAV three-dimensional modeling technology is commonly introduced to assist bridge inspections. To cater to diverse needs of real-scene three-dimensional modeling, UAV flight planning encompasses various approaches, including vertical photography, oblique photography, and circular photography.

Unmanned aerial vehicles are widely utilized across various domains, such as agriculture [4,5], surveying [6,7], environmental and safety applications [8], due to their convenience, flexibility, relatively low costs, and ability to acquire high-resolution images through automated means. This has facilitated collaborative development among different fields. In the evolution of UAV-based modeling, numerous scholars have conducted investigations into aspects including model accuracy [9,10,11] and texture effects [12,13].

Wang et al. [14] focused on a primary school as their research area, utilizing oblique photography from four different levels of unmanned aerial vehicles at varying flight altitudes, and comparing the performance of different types of drones in three-dimensional reconstruction. Li et al. [15] introduced the innovative 3D reconstruction method, Edge3D, which restores building edges in the form of 3D lines. They enhanced line segment matching accuracy through geometric constraints and progressive filtering techniques, thereby reducing technological costs and enabling precise building modeling. Che et al. [16] proposed a method based on triangular mesh cutting to restore the edge lines of 3D building models captured through UAV oblique photography, aiming to enhance the realism of buildings and improve model rendering quality. Chen et al. [17] combined deep learning technology with unmanned aerial vehicles to facilitate maintenance in various construction projects involving roads, bridges, and buildings. Through automatic image classification and detection, they effectively identified changed areas in construction sites, contributing to construction management support within the development of intelligent construction technologies.

The route planning approach for UAV real-scene three-dimensional modeling typically employs oblique photography techniques, using different angles to plan UAV flight paths for image acquisition from varying perspectives, thus producing rich images and high-resolution textures. However, for prominent bridge structures that require detailed modeling [18], oblique photography techniques still have limitations. These limitations manifest in fixed gimbal pitch angles, flight altitudes, lateral and longitudinal overlap ratios, and insufficient connection points, resulting in regions of the generated model exhibiting blurred or even missing textures. To address this issue, some researchers have proposed the use of cross-circular photography methods [19] for refined modeling of protruding bridge structures. Surrounding flight is directed solely towards photographing the architecture itself, reducing redundancy in image acquisition. However, for railway bridges where ground control points (GCP) cannot be deployed on the bridge, the three-dimensional error in models created through cross-surround methods often tends to be higher.

Currently, both domestically and internationally, there is a wide variety of steel bridges. In comparison to conventional concrete beams, steel bridges possess advantages such as superior span capabilities, high strength, and shorter construction periods. However, issues such as apparent corrosion, stress corrosion, and corrosion fatigue can lead to surface defects, reduced lifespan, and even loss of operational capacity in these large steel bridge components. To mitigate these concerns, protective measures such as corrosion-resistant coatings are often employed for the preservation of steel bridges.

For steel truss arch bridges, the UAV oblique photography technique cannot guarantee the texture quality of the building, and the cross-circular photography cannot guarantee the ground image control point accuracy. This paper addresses the challenges of 3D modeling for prominent structures like railway bridges by proposing a flight route planning strategy that combines oblique photography with near-range oblique photography (inclined photography) [20]. It offers solutions for the modeling of oblique photography parallel to the bridge deck but at a farther distance, as well as for the processing of near-range oblique photography vertical to the bridge deck. A comparative analysis is conducted between model accuracy, detailed textures, and overall model effects of 3D modeling using oblique photography combined with near-range oblique photography, oblique photography alone, and cross-circular photography. This method can solve the problem of fuzzy or even missing texture of large abrupt buildings such as arch bridges and also ensure the accuracy of the model. Finally, the application and prospects of UAV modeling results in railway bridge inspection are discussed.

2. Principles of UAV Three-Dimensional Modeling

In recent years, with the consolidation of UAV remote sensing technology, the technological advantages of unmanned aerial vehicles have been increasingly highlighted [21].

The process of UAV three-dimensional modeling involves importing captured UAV imagery into modeling software and generating models and data through post-processing within the software (Figure 1). The fundamental principle entails initially utilizing the UAV’s own positioning system to obtain position and orientation system (POS) data of the imagery, which is then employed as the initial orientation element of the images. Subsequently, the imagery and POS data are imported into the modeling software, where algorithms such as SIFT, PMVS, and structure from motion (SFM) are employed to establish a dense 3D point cloud of the target through aerial triangulation. Subsequently, ground control points are incorporated to correct model accuracy discrepancies caused by systematic errors through bundle adjustment, thereby restoring the true 3D model of the object. Finally, through multi-image dense matching of the 3D model, a triangulated irregular network (TIN) and solid 3D model are constructed. Optimal image textures are selected for texture mapping, culminating in the completion of the three-dimensional modeling of the real scene.

3. Workflow of UAV Three-Dimensional Modeling Technology

3.1. Overview of the Experiment

The chosen subject of this experiment was the San’an Yongjiang Extra-Large Bridge (a three-span continuous steel truss arch bridge with spans of 132 m + 276 m + 132 m), located in the southeastern outskirts of Nanning City, Guangxi Province, China near the San’an Horticultural Park. The central geographic coordinates of the experimental area were approximately 108.432° longitude and 22.788° latitude, with an elevation ranging between approximately 70 m and 170 m. The experiment was conducted on 6–7 November 2022, under clear weather conditions, high visibility, and favorable satellite signal strength [22].

The DJI Phantom 4 RTK quadcopter, developed by DJI Innovations (Shenzhen, China), was selected as the UAV model for aerial photography equipment. Phantom 4 RTK is equipped with a 1-inch 20-megapixel CMOS sensor camera, featuring a field of view of 84°, an aperture range of f/2.8–f/11, and an autofocus lens. This UAV model offers a flight endurance of up to 30 min and boasts a hovering accuracy of ±0.1 m in both vertical and horizontal directions.

Control points were established using Real-Time Kinematic (RTK) measurements, with the Zhonghaida V5 mobile station selected as the equipment model. The static positioning plane accuracy of Zhonghaida V5 is ±(2.5 + 0.5 × 10 − 6D) mm, and the elevation accuracy is ±(5 + 0.5 × 10 − 6D) mm, where D represents the distance between the measured points. Field data collection was based on the 2000 National Geodetic Coordinate System (CGCS2000), with a central meridian of 108 degrees and a geodetic height.

3.2. Arrangement of Ground Control Points

Ground control points, also referred to as image control points, are control points strategically placed within the aerial imaging area to correct aggregate errors and geospatial information discrepancies in the aerial images. Factors such as quantity, distribution, and accuracy of these image control points significantly impact the accuracy of the generated three-dimensional model. In this experiment, a total of nine image control points were uniformly arranged at prominent and clear locations within the imaging area, as depicted in Figure 2.

3.3. Image Acquisition and Flight Path Planning

To validate the improvement in bridge model quality resulting from the introduction of oblique planning, three sets of experiments were conducted in accordance with different UAV aerial photography methods: oblique photography, cross-circular photography, and a combination of oblique and inclined photography, as outlined in this study. Oblique photography flies the drone according to the “S-shaped” route, from the southeast, west, north, south, north and central five directions to take a comprehensive picture of the building; cross-circular photography flies the route in a circle to take detailed aerial photography of the center of the circle; inclined photography flies the route parallel to the building façade to take aerial photography of the façade of the building and to obtain the image of the side of the building.

3.3.1. Parameter Configuration for Oblique Photography

Oblique photography technology overcomes the limitations of traditional aerial surveying, which only captures images from vertical angles. It provides a more realistic and comprehensive representation of object appearance, texture, and positional data. This greatly facilitates fieldwork collaboration in surveying, inspection, and related domains. The oblique photography parameters for this experiment were set as depicted in

Table 1. Main Parameters of Oblique Photography.

Main Parameter	Numerical	Main Parameter	Numerical
Lateral Overlap/%	80	Gimbal Pitch Angle/°	−60
Longitudinal Overlap/%	80	Flight Altitude/m	120
Oblique Lateral Overlap/%	80	Main Route Angle/°	0
Oblique Longitudinal Overlap/%	80

.

3.3.2. Parameters for Cross-Circular Photography

Compared to oblique photography, the cross-circular photography approach adopts a multi-angle and omni-directional shooting method tailored for detailed modeling of individual structures. This method offers advantages in capturing more intricate texture details of buildings and enhancing the efficiency of model creation. In this experiment, different circular photography parameters were set for both the side span and the main span of the railway steel truss arch bridge. Cross-circular photography parameters for this experiment were set as depicted in

Table 2. Main Parameters of Cross-Circular Photography.

Main Parameter (Main Span)	Numerical	Main Parameters (Side Span)	Numerical
Flight Altitude/m	120	Flight Altitude/m	100
Circular Radius/m	100	Circular Radius/m	80
Gimbal Pitch Angle/°	−45	Gimbal Pitch Angle/°	−45

.

3.3.3. Parameters for Inclined Photography Combined with Oblique Photography

Inclined photography is conducted by forming a flight plane using three points on the UAV. It is applicable to scenarios such as slopes and building facades, enabling comprehensive and detailed data collection of building facades to enhance the texture accuracy of real-world 3D modeling. In the case of railway steel truss arch bridges, combining inclined photography with oblique photography can supplement more side texture information of the bridge while maintaining model accuracy. Due to the overall height and width of the bridge, it is essential to analyze the principles and parameter settings of inclined photography in advance to achieve appropriate image overlap and object distance. The schematic diagram of inclined photography is illustrated in Figure 3. The correspondence between the corresponding parameters In the figure can be seen in (1) to (3).

$$d=f\cdot GSD/a,$$

(1)

$$X=\frac{A_2{B}_1}{A_1{B}_2}=2d\cdot \tan \left(\frac{\alpha }{2}\right)-h_1/2d\cdot \tan \left(\frac{\alpha }{2}\right)+h_1,$$

(2)

$$Y=\frac{C_2{D}_1}{C_1{D}_2}=2d\cdot \tan \left(\frac{\alpha }{2}\right)-h_2/2d\cdot \tan \left(\frac{\alpha }{2}\right)+h_2,$$

(3)

where d is the distance from the drone to the side of the bridge in meters; f is the lens focal length in millimeters; GSD is Ground Sampling Distance, representing spatial resolution; a is pixel size in millimeters; h₁ is the vertical separation between adjacent images in the lateral direction in meters; h₂ is the horizontal separation between adjacent images in the longitudinal direction in meters; α is the camera’s vertical Field of View (FOV) in degrees; β is the camera’s horizontal Field of View in degrees; A₂B₁ and C₂D₁, respectively, represent lateral and longitudinal overlap regions. We let the lateral and longitudinal overlap rates of the drone be denoted as X and Y, visually represented in the diagram by the ratios of A₂B₁ to A₁B₂ and C₂D₁ to C₁D₁.

In this experiment, the same approach was applied to the railway steel truss arch bridge, where different inclined photography parameters were set for both the side span and the main span. The inclined photography parameters for this experiment were set as depicted in

Table 3. Main Parameters for the Inclined Photography Flight Paths.

Main Parameter (Main Span)	Numerical	Main Parameters (Side Span)	Numerical
Flight Overhead Extension Distance/m	50	Flight Overhead Extension Distance/m	60
Distance to Bridge/m	25	Distance to Bridge/m	20
Lateral Overlap/°	80	Lateral Overlap/°	80
Longitudinal Overlap/°	80	Longitudinal Overlap/°	80

.

3.4. Aerial Triangulation

So far, there is a proliferation of UAV modeling software, each with its strengths in terms of model accuracy and the effectiveness of three-dimensional mesh construction. In this experiment, image processing software Context Capture 20.0 (formerly known as 3D Smart) developed by Bentley Systems was employed for analysis and processing. The images from the three proposed methods were imported into the Context Capture software for aerial triangulation.

For oblique photography and cross-circular photography, due to the similarity in flight altitude differences, adjacent spacing, photography time, and grayscale values of captured images, all images were grouped into the same image set for the aerial triangulation process. This enabled the successful calculation of optical attribute parameters, such as principal point and distortion, as well as position information, for all images through aerial triangulation computation.

For the combined oblique and inclined photography flight method, as oblique photography employs flight paths parallel to the bridge deck while inclined photography uses flight paths perpendicular to the bridge deck, there exist significant differences between the two in terms of capture angles, image-to-bridge distances, inter-image spacing, and grayscale values. For the traditional aerial triangulation process, it is difficult for the software to identify and compare all the image information, and there are often a lot of images with unsuccessful aerial triangulation. Therefore, it is difficult to process the images of tilt and oblique photography in the software, and this paper improves on the traditional processing method to ensure that the oblique photography combined with inclined photography method can be applied. Therefore, this study proposes to divide the slanted images into two groups: upstream and downstream faces of the bridge. High-precision GPS and the onboard RTK positioning module of the unmanned aircraft are employed to separately conduct multi-view matching and dense matching for the upstream and downstream faces. This approach yields varying optical attribute parameters after steps such as iterative least squares adjustment and distortion correction. The oblique, upstream slanted, and downstream slanted image groups are then merged and assigned to three distinct image sets. Optical attribute parameters are imported for each set, and aerial triangulation computation is performed for the respective images.

After the initial aerial triangulation, the majority of image parameter information can be successfully computed. For those images that remain unresolved, manual tie points are added, and iterative aerial triangulation is performed until the accuracy requirements are met. Following this, the process proceeds to three-dimensional reconstruction.

4. Model Quality Assessment and Application

4.1. Texture Evaluation and Analysis of the Models

In order to examine the impact of three distinct aerial photogrammetry techniques on the visual quality of realistic 3D models of railway steel truss bridges, an evaluation and analysis of the modeling outcomes was conducted. This assessment involved comparing the overall shape and completeness of the models, as well as the texture fidelity of their local details, to gauge the efficacy of the three modeling approaches.

The overall effect is illustrated in Figure 4. It can be observed from the figure that the models reconstructed using the cross-circular photography and oblique combined with inclined photography approaches exhibit relatively complete overall effects. However, in the case of oblique photography, the camera’s proximity to the bridge’s apex results in insufficient matching connection points, ultimately leading to the appearance of voids and significant texture loss in the model. It can therefore be shown that the introduction of inclined photography has greatly improved the integrity of oblique photographic models.

The three approaches are arranged according to oblique photography, cross-circular photography, and oblique combined with inclined photography for comparison, analyzing the modeling effects of fine textures on the bridge (Figure 5). A large span steel truss arch bridge is a kind of structural system between beams and arches, and a large number of trusses are subjected to axial force to locate the stability of the structure, so it is particularly important to display the images of different truss joints. Meanwhile, comparing the textures of the detailed models of the three methods, it can be seen that the introduction of inclined photography makes the truss model clearer, and it can be seen that the oblique combined with inclined photography model is more rich in texture than the other two methods, including the fact that the sunlight, shadows, and the corrosion of steel trusses can be shown very well. The linearity of the internal trusses of the bridge is more complete, without any voids or deformations.

In the detailed model image of cross-circular photography, it can be observed that although the bridge’s edge-span truss is relatively intact, the corrosion on the steel truss’s side surface is discernible but somewhat blurry. The visibility of node plates and high-strength bolts is not well reflected, and the linearity of the internal trusses of the bridge is not ideal.

The feedback regarding the corrosion on the side surfaces of the bridge in the oblique photography model is similar to that of cross-circular photography. The visibility of node plates and high-strength bolts is not very satisfactory, and there is a phenomenon of texture loss in the internal trusses of the bridge. For complex structures like the steel truss bridge’s internal framework, tilted photography is not ideal.

The performance of the internal trusses of the bridge is not particularly ideal. By incorporating oblique flight paths for close-range photography of the bridge, it is possible to better supplement the model with detailed textures and enhance the overall completeness of the model.

4.2. Model Accuracy Evaluation and Analysis

After aerial triangulation was completed in this experiment, the overall accuracy of the three approaches was assessed by comparing the errors of control points in the X, Y, and Z directions. Additionally, a comparison was made between the three approaches and the precision of the bridge model was evaluated by comparing it with the check points on the bridge side obtained from a total station target.

4.2.1. Control Point Accuracy Assessment

From the error plots (Figure 6), it can be observed that the distribution of errors between the same control points is similar among the three schemes, but it can be roughly determined that the control point errors for the cross-circle flight scheme are relatively larger, while those for the oblique photography scheme are smaller. The error curve for the scheme combining oblique and inclined photography falls between the other two.

In order to analyze and compare the level of dispersion between the actual and predicted values of ground control point coordinates, the Root Mean Square Error (RMSE) formula is introduced to calculate the errors in the horizontal and vertical directions of ground control points, as shown in

Table 4. Root Mean Square Errors of Ground Control Points.

Approach	RMSE in Horizontal Direction (m)	RMSE in Vertical Direction (m)
Oblique Photography	0.0102	0.0089
Cross Circular Photography	0.0202	0.0152
Integration of Oblique Photography and Inclined Photography	0.0155	0.0081

.

Comparative analysis of the RMSE of ground control points in both horizontal and vertical directions reveals that the RMSE in the horizontal direction aligns with the representation in the azimuth error plot. Thus, with the introduction of oblique imagery through inclined photography, the horizontal accuracy of ground control points experiences a slight decrease. However, the accuracy still remains higher than that of cross-circle circumnavigation aerial photography. When considering the RMSE in the vertical direction, the inclusion of oblique photography in the inclined photography approach results in a slight improvement over the original oblique photography scheme. Among the three approaches, this combined scheme demonstrates the optimal error precision in the vertical direction. From the accuracy of the control points, it can be seen that the three schemes can meet the accuracy requirements of the control points of the 1:500 topographic map mapping, but for bridges such as large abrupt structures, the introduction of inclined photography to improve its vertical progress, the bridge has a very good role in helping detection.

4.2.2. Checkpoint Accuracy Assessment

After controlling the accuracy of the bridge model, the accuracy of the bridge structure was analyzed and compared by introducing checkpoints on the bridge side. According to the symmetry, 20 checkpoints were placed on the downstream right side of the bridge, as shown in Figure 7, and their positions were measured using a total station. Subsequently, the position data of the bridge side checkpoints collected by the total station were imported into Context Capture for individual point fitting. The horizontal and vertical errors of the 20 checkpoints were then plotted as box plots, as shown in Figure 8, for analysis and comparison.

The box plots of horizontal and vertical errors of model checkpoints on the bridge side obtained from the three different approaches show that the modeling scheme combining inclined photography with oblique photography significantly improves the accuracy of the bridge side. This photographic modeling approach outperforms the accuracy of the bridge model compared to the cross-circular photography method, with some improvement in horizontal direction accuracy.

4.3. Application of Realistic 3D Models

In this experiment, the steel structure coating and corrosion condition of the railway steel truss arch bridge were examined using UAV-based realistic 3D modeling (as depicted in Figure 9). From the image, it is evident that the bridge members have experienced varying degrees of corrosion, with some components exhibiting severe corrosion. The inspection and analysis of corrosion on high-strength bolts were conducted using the realistic 3D model, as shown in Figure 9. The images clearly reveal that significant corrosion has affected several high-strength bolts subjected to substantial loads, posing a potential threat to both the structural integrity of the bridge and the safety of vehicular traffic passing beneath it. Furthermore, the absence of bolts on the junction plates of the steel truss arch bridge can also be distinctly discerned. The diameter of the high-strength bolts on the bridge is roughly 3 cm, so it is known that for the oblique photography combined with inclined photography UAV 3D modelling method the resolution can be at least 3 cm, and from the model it is known that the resolution can be up to 7.5 mm for one pixel.

In conclusion, this study employs a combination of drone-based oblique photography and inclined photography to carry out real-time 3D modeling of a railway steel truss arch bridge. This approach enables the examination of bridge structural issues, facilitating the clear identification of corrosion and detachment of steel structure coatings, as well as the condition of high-strength bolts. For large-span railway bridges, UAV 3D modelling is shorter and less costly, and for larger construction workloads, the choice of UAV modelling for bridge disease detection has a very good auxiliary role. However, drone-based inspection also has limitations; its precision is relatively lower compared to that of sophisticated instruments such as total stations and 3D scanners. To address this, integration with point clouds from 3D scanners can be employed to enhance the precision and texture of the self-generated point cloud, thereby improving the overall model accuracy.

5. Conclusions

This study investigated the impact of different aerial route modes, including oblique photography, cross-circular photography, and the integration of oblique photography and inclined photography, conducted by a multi-rotor single-lens UAV, on the texture and accuracy of real-world three-dimensional models of bridge structures. Furthermore, the study proposed the application of the developed method to the detection of railway steel truss arch bridges. The results are as follows:

(1)

In terms of model texture effects, the integration of oblique photography and inclined photography exhibited the highest model completeness, showcasing smoother overall linearity and more distinct detail feedback.

(2)

Regarding model accuracy, the integration of oblique photography and inclined photography demonstrated the highest model accuracy on the bridge’s side, with horizontal accuracy slightly lower than that of oblique photography. Nonetheless, it effectively met the accuracy requirements for 1:500 topographic map control points.

(3)

In terms of model application, the incorporation of inclined photography significantly enhanced the texture accuracy of the real-world 3D bridge model, proving to be effective for the detection of structural issues such as steel corrosion and missing bolts in bridge components.