A Comparative Study of a Fully-Connected Artificial Neural Network and a Convolutional Neural Network in Predicting Bridge Maintenance Costs

Abstract

The cost assessment of bridge maintenance is a difficult topic to study, but it is critical for a bridge life cycle cost analysis. The maintenance costs sample database was established in this study according to actual engineering data, and a bridge maintenance cost prediction model was developed using a fully-connected artificial neural network (ANN) and convolutional neural network (CNN), respectively. First, eight main factors affecting maintenance costs were evaluated based on the random forest method, and the evaluation results were verified by an exploratory data analysis. The original data were then screened based on the isolation forest principle, and the recent gross domestic product (GDP) growth rate was used to illustrate the relationship between economic development and bridge maintenance costs. Finally, these two neural networks were used to establish maintenance cost prediction models, respectively. The results from the two models were compared and their prediction accuracies were analyzed. The prediction performance of the CNN model for bridge maintenance costs was found to be better than that of the traditional fully-connected ANN model. The results of this study will enhance the opportunity for bridge managers to balance lifecycle maintenance costs.

1. Introduction

Bridges are important parts of transportation infrastructure; the expected service life of a bride is (at least) several decades and possibly more than 100 years. Bridges need to be maintained continuously to ensure their safety and applicability during the service period. The 2021 Infrastructure Report Card by the American Society of Civil Engineers (ASCE) states that about 7.5% of bridges in the USA were structurally deficient in 2020 [1]. An estimated USD 123 billion was invested into bridge maintenance in the USA, at all levels of government, in 2019 [2]. In 2016, there were 86,000 bridges in need of maintenance in China, which is approximately 10% of the total number of bridges [3], roughly equivalent to the proportion of structurally deficient bridges in the USA. The total number of highway bridges in China reached 912,800 at the end of 2020, which brought tremendous pressure on bridge maintenance [4]. Both China and the USA are facing difficulties in having such a large number of aging bridges. Maintenance is arduous and requires huge capital investments. Correct maintenance cost estimations are important in the planning of bridge maintenance [5].

Current maintenance cost studies primarily focus on the optimization of cost-oriented maintenance scenarios. Barone and Frangopol considered the interactions among components in the system, and selected the most cost-effective maintenance measures according to detection results and preset damage threshold [6]. Sabatino et al. combined maintenance costs, failure losses, and choice preferences to optimize multiple objectives in the maintenance planning simultaneously, with the goal of finding a set of Pareto optimal solutions [7]. Condition states and reliability indicators were defined as performance indicators, and a multilinear model was used to describe the degradation process of performance indicators under different maintenance scenarios. Ghodoosi et al. selected the most cost-effective one by comparing different maintenance scenarios based on a genetic algorithm engine [8].

Many domestic and foreign researchers have carried out extensive research in the cost assessment of bridge maintenance theory, making significant progress. There are cost estimation methods based on numerical analyses, for example, maintenance cost estimations based on the bridge performance degradation model and binary programming algorithm [9], bridge maintenance cost allocation based on the priority index [10], prediction of daily maintenance costs of bridges based on linear regression and time series analyses [11], and the cost assessment of bridge maintenance based on system reliability analyses and multiple regression algorithms [12]. Such evaluation models are established based on the statistical analysis and parametric analysis results of maintenance costs. As for their advantages: they do not require large amounts of samples and the evaluation of maintenance costs could be achieved with fewer computational steps/computational effort. However, these models have some drawbacks. For example, the model prediction accuracy is not high because the parametric analysis does not fully learn the characteristics of the data.

At the same time, there are cost prediction studies based on data analyses and the intelligent algorithm. For example, Bayzid et al. established a cost prediction model through the particle swarm algorithm based on big data analysis. This method can predict the short-term maintenance costs of the structure under the current performance [13]. Echaveguren and Dechent allocated the costs of bridge maintenance based on prioritization indices, according to the condition, strategic importance, and vulnerability of the bridge. This method involves simple/a small amount of calculations, which is helpful in short-term maintenance decision-making [14]. Shi et al. established a database of the daily maintenance costs of bridges in the United States, with the logarithm of the maintenance cost as a dependent variable and the service life of the bridge as an independent variable, and they proposed a prediction model based on a backtracking search algorithm [15]. Zhu et al. proposed a decision-making method based on performance detection and maintenance cost records of Chinese bridges, and used an annealing algorithm to evaluate the future maintenance costs and expected losses of bridges [16]. Then, the genetic algorithm was used to find the decision-making scheme with the lowest maintenance costs in the whole life cycle of the bridge. This is a decision-making method combining bridge performance and costs but does not explain the quantitative relationship between bridge performance and maintenance costs. Most of the existing bridge maintenance cost prediction models are based on the maintenance cost prediction of a single type of input indicator with a small sample base. This affects the applicability and generalization ability of the model. Bridge maintenance cost prediction is a typical and complex nonlinear regression problem. The prediction models established based on traditional statistical analysis methods and simple data-driven intelligent algorithms have certain defects in prediction accuracy and generalization performance. It is necessary to analyze, in detail, the degree of influence of many factors that affect the costs of bridge maintenance and, on this basis, use intelligent algorithms to deeply learn and extract the characteristics of actual engineering data. Thus, a set of computing processes and methods that combine data preprocessing, influencing factor analyses, and intelligent predictions are established. An effective “reference” for bridge managers would involve quantitative prediction of bridge maintenance costs using conventional bridge maintenance data.

Based on the above, a database of the maintenance costs in bridges was constructed in this study, based on practical engineering data, containing the maintenance cost samples of 268 bridges in Chinese inland and coastal areas. The database includes many factors that affect maintenance costs, for example bridge location, bridge technical condition, bridge maintenance time. The fully-connected ANN and CNN were used to establish the intelligent prediction model of maintenance costs, respectively, and the prediction accuracies were compared. We also propose a new method to optimize bridge life cycle maintenance costs and predict the time-varying law of maintenance cost.

2. Bridge Maintenance Cost Database

A variety of external factors could affect the maintenance costs of bridges [17]. The bridge maintenance-related costs in China’s coastal and inland areas are highly discrete and influenced by multiple factors. The obtained data are classified according to the seven main factors that affect the maintenance costs (maintenance time, bridge technical condition, bridge location, bridge age, bridge grade, superstructure, highway grade). Classifications of all data in the database are shown in Figure 1. The database contains a total of 399 maintenance cost samples.

The classification standards of highway grades are based on the Chinese “Technical Standard of Highway Engineering (JTG B01-2019)” and are divided according to the design service life of the highway on which the bridge is located [18]. The details are presented in

Table 1. Standards for dividing highway grade indicators.

Highway Grade	Highway Traffic Grade	Design Service Life
High	Expressway, First class highway	20 years
Medium	Second class highway	15 years
Low	Third class highway, Fourth class highway	10 years

. The technical condition grades are classified according to the Chinese “Highway Performance Assessment Standards (JTG 5210-2018)” and are divided according to the weighted score of the overall technical conditions of the bridge [19]. The details are presented in

Table 2. Indicators classification of bridge technical conditions.

Weighted Mark for Overall Technical Condition of Bridges	Bridge Technical Condition Grade Dj
	First Class Bridge	Second Class Bridge	Third Class Bridge	Fourth Class Bridge	Fifth Class Bridge
Dr	(95,100)	(80,95)	(60,80)	(40,60)	(0,40)

; “Highway Bridge Design standard” is the reference standard for classification of bridge grades and the details are presented in

Table 3. Indicators classification of bridge grade indicators.

	Extra-Large Bridge	Large Bridge	Medium Bridge	Small Bridge
Full length (m)	(1000,+∞)	(100,1000)	(30,100)	(8,30)
Single span (m)	(150,+∞)	(40,150)	(20,40)	(5,20)

[20].

In China, bridge maintenance costs are considered public expenditures. All levels of government investments in bridge maintenance are affected by economic development. GDP growth rate was used as the eighth influencing factor in this study in order to assess the impact of economic effects on investments in bridge maintenance. GDP growth rates from previous years could be queried on government websites; its predicted values were calculated using the prediction models developed by economists [21]. The prediction model for Chinese GDP growth rate used in this study was established by Jiang [22]. The values of the Chinese GDP growth rate in past years are presented in Figure 2 [23].

3. Neural Network-Based Bridge Maintenance Cost Prediction Framework

In recent years, with the development of computer technology, neural networks have become more mature and diverse. For example, the fully-connected ANN is widely used to solve engineering problems. The deep learning proposed by Geoffrey Hinton in 2006 broke the limitations of traditional neural networks on the number of layers and further broadened the application scope of neural networks [24].

Therefore, this section introduces a comparative study of a fully-connected ANN and CNN for predicting bridge maintenance costs. The calculation framework of the two models is shown in Figure 3. It was necessary to preprocess the dataset of bridge maintenance costs prior to establishing the neural network model. This included an exploratory data analysis, selection of input indicators, and sample screening. The database was divided into a training set and a test set, which were used to train the neural network model and verify the prediction performance of the model, respectively.

3.1. Selecting Input Indicators Based on Random Forest

Random forest is a classification index test method based on decision trees. It is widely applied and achieves good results in the index selection of low feature dimensions and low sample size data [25,26]. The VIM is used to express the importance of indicators, and the Gini index [27], which represents the probability that a randomly selected sample in the sample set has been classified by mistake, is used to calculate the VIM value.

Suppose that there are n indicators X₁, X₂, X₃… X_n, the Gini variable importance measure of each indicator X_j is calculated and represented by $VI{M}_j^{\left(Gini\right)}$, which represents the average change of node splitting impurity in the j-th variable in all decision trees in the random forest [28]. The detailed calculation process of $VI{M}_j^{\left(Gini\right)}$ is as follows:

(1)

Calculate the Gini index of the node m in a certain category K. The formula for calculating the Gini is:

$$G{I}_m={\displaystyle \sum }_{k=1}^K\widehat{P_{mk}}\left(1-\widehat{P_{mk}}\right)$$

(1)

where K is the number of categories in the sample set and $\widehat{P_{mk}}$ is the estimated probability that the sample of node m belongs to the k-th category; when the sample belongs to one of categories, K = 2.

(2)

The decision trees in the random forest used in this study only had two branches. Therefore, the Gini index of node m could be calculated as:

$$G{I}_m=2\widehat{P_m}\left(1-\widehat{P_m}\right)$$

(2)

where $\widehat{P_m}$ is the estimated probability that the sample belongs to any category at node m.

(3)

Calculate the VIM value of node m. The importance of indicator X_j at node m; that is, the variable quantity in the Gini index before and after the branch at node m is:

$$VI{M}_{jm}^{\left(Gini\right)}=G{I}_m-G{I}_l-G{I}_r$$

(3)

where $G{I}_l$ and $G{I}_r$ represent the Gini indexes of two new nodes split by node $m$.

(4)

Calculate the VIM value of indicator $X_j$ in the random forest. If the indicator $X_j$ appears $M$ times i-th in the tree, the importance of the indicator $X_j$ in the i-th tree is:

$$VI{M}_{ij}^{\left(Gini\right)}={\displaystyle \sum }_{m=1}^{M}VI{M}_{jm}^{\left(Gini\right)}$$

(4)

The VIM of the indicator $X_j$ in the random forest is defined as:

$$VI{M}_{ij}^{\left(Gini\right)}=\frac{1}{n}{\displaystyle \sum }_{i=1}^{n}VI{M}_{ij}^{\left(Gini\right)}$$

(5)

where n is the number of decision trees in the random forest.

3.2. Bayesian Optimization

To improve the prediction performance of the model, this study uses Bayesian optimization to determine some important hyperparameters in random forest, isolation forest, and fully-connected ANN and CNN. The hyperparameters herein are the non-trainable parameters in the model. Those parameters (for example, the number of neurons and the structure of the network layer) are set before the training process starts. Nevertheless, a few hyperparameters, such as the number of trees and the number of predictor variables to sample at each node, are important hyperparameters in random forest and isolation forest. These parameters may have an important impact on the calculation performance of the forest [29]. Other parameters, such as the regularization coefficient and the learning rate, are important hyperparameters in neural networks. These parameters are very important for the neural network model to improve robustness to databases with small sample sizes.

Since Bayesian optimization has the advantage of sample efficiency, a relatively good hyperparameter combination can be found with a very small number of cycles. The calculation principle of adopting it to determine hyperparameters is as follows:

The basic assumption that the objective function $E_v\left(\overline{\theta }\right)$ is adopted for Bayesian optimization is obtained, a priori, by the Gaussian process (GP), $E_v\left(\overline{\theta }\right)~N\left(0,K\right);\overline{\theta }$is the hyperparameter vector considered in this study. Since $E_v\left(\overline{\theta }\right)$ is corrupted by Gaussian noise with a mean value of zero and a standard deviation of ${\sigma }_{noise}$, the kernel matrix can be expressed as [30]:

$$\mathrm{K}=\begin{array}{ccc}k\left({\overline{\theta }}_1,{\overline{\theta }}_1\right)& \cdots & k\left({\overline{\theta }}_1,{\overline{\theta }}_t\right)\\ ⋮& \ddots & ⋮\\ k\left({\overline{\theta }}_t,{\overline{\theta }}_1\right)& \dots & k\left({\overline{\theta }}_1,{\overline{\theta }}_t\right)\end{array}+{\sigma }_{noise}^2\mathrm{I}$$

(6)

where $k\left({\overline{\theta }}_t,{\overline{\theta }}_t\right)$ is the covariance function. The results of previous iterations are expressed as: $D_{1:t}=\left({\overline{\theta }}_{1:t},E_{1:t}^V\right)$, where $E_{1:t}^V=E_v\left({\overline{\theta }}_{1:t}\right)$. Assume ${\overline{\theta }}_{t+1}$ as the next point to estimate and the value of the function at ${\overline{\theta }}_{t+1}$ as $E_{t+1}^V=E_v\left({\overline{\theta }}_{t+1}\right)$. $E_{1:t}^V$ and $E_{t+1}^V$ are joint Gaussian expressions under GP prior. The predictive distribution can be obtained as follows [31]:

$$E_{t+1}^V|D_{1:t}~N(\mu \left({\overline{\theta }}_{t+1}\right),{\sigma }^2\left({\overline{\theta }}_{t+1}\right)+{\sigma }_{noise}^2$$

(7)

where

$$\mu \left({\overline{\theta }}_{t+1}\right)=k^T{\left(K+{\sigma }_{noise}^{2}I\right)}^{-1}{E}_{1:t}^V$$

(8)

$${\sigma }^2\left({\overline{\theta }}_{t+1}\right)=k\left({\overline{\theta }}_{t+1},{\overline{\theta }}_{t+1}\right)-k^T{\left(K+{\sigma }_{noise}^2\mathrm{I}\right)}^{-1}k$$

(9)

$$k={\left[k\left({\overline{\theta }}_{t+1},{\overline{\theta }}_1\right)k\left({\overline{\theta }}_{t+1},{\overline{\theta }}_2\right)\cdots k\left({\overline{\theta }}_{t+1},{\overline{\theta }}_t\right)\right]}^T$$

(10)

It can be seen from the above formulas, that both the predicted variance function ${\sigma }^2\left({\overline{\theta }}_{t+1}\right)$ and the predicted mean function $\mu \left({\overline{\theta }}_{t+1}\right)$ depend on the choice of the covariance function $k\left({\overline{\theta }}_t,{\overline{\theta }}_t\right)$, and they jointly determine the choice of the covariance function $E_{t+1}^V|D_{1:t}$.

This study uses the automatic relevance determination (ARD) Matérn 5/2 kernel [32]. This choice adapts to the problem of the ARD squared index kernel. It is impractical for such a covariance function to appear in the actual sample function optimization problem [33]. For $t=1,2,\dots N$, the detailed calculation steps of the hyperparameter determination algorithm based on Bayesian optimization are shown in Figure 4 [34].

3.3. Exploratory Data Analysis

We performed an exploratory data analysis on bridge maintenance cost data to verify the effectiveness of the random forest method. Exploratory data analysis methods focus on the true distribution of data and emphasize the visualization of data. It can help users see the relationship between the data and the selected feature at a glance, so as to detect the validity of the feature [35]. EDA is different from the initial data analysis (IDA) in that it focuses more on checking the assumptions required for the model fit and hypothesis testing, deals with missing values, and performs variable transformations as needed [36].

3.4. Sample Screening

Anomaly detection is an important step in pre-processing the training data. The main function of anomaly detection is to check whether there are input errors and unreasonable data. Ignoring the existence of outliers in engineering practices is very dangerous. Incorporating outliers into the calculation and analysis processes of data without excluding them will adversely affect the training and prediction results. For the outlier detection of actual engineering data, scholars have carried out valuable research. For example, Xu et al. adopted wavelet transform and generalized Pareto distribution for data processing and proposed a two-level anomaly detection method, namely the threshold-based anomaly detection and anomaly trend detection [37]. Zhang et al. proposed an online detection method for structural health monitoring data anomalies based on the Bayesian dynamic linear model. This method has better calculation accuracy and higher calculation efficiency [38].

The isolation forest method was proposed by Zhou et al. in 2008 [39]. In 2011, the method was applied to the anomaly detection of data samples [40]. In isolated forests, anomalies are defined as points that are more sparsely distributed and further away from high-density groups. In the feature space constructed by the isolation forest, if the data distribution in a certain area is sparse, it means that the probability of normal data values appearing in that area is very low. Therefore, if a certain data frequently appears in these areas, it is judged as an anomaly. Isolation forest is an unsupervised anomaly detection method suitable for continuous numerical data. That is, it does not require labeled samples for training, and the method has good robustness to data with low feature dimensions. Taking into account the data characteristics of the bridge maintenance costs in this study, it was necessary to choose an anomaly detection method that was suitable for small datasets with low feature dimensions. The isolation forest method has been widely used in the detection of anomalies in actual engineering data. For example, smart grid data [41] and optical emission spectroscopy data [42]. The isolation forest method has been proven to process data quickly and efficiently. It is particularly suitable for small and medium datasets with low feature dimensions. Therefore, this study adopted it as a method of anomaly detection.

Consider a dataset $X=\left\{x_1,x_2,\dots ,x_n\right\}$ with n samples and d features. First, decision trees are built. Then, a feature q and its segmentation threshold p are selected randomly. Finally, the dataset X is segmented recursively until there is only one sample $x_i$ on the node. The length of the segmentation path of sample $x_i$ from the root node to the leaf node in the decision tree is represented by $h\left(x\right)$. For the dataset X, the average path length of the decision trees is:

$$c\left(n\right)=\{\begin{array}{ll}2H\left(n-1\right)-2\left(n-1\right)/n& for n>2\\ 1& for n=2\\ 0& otherwise\end{array}$$

(11)

where n is the number of samples and $H\left(k\right)$ is the harmonic number, which can be estimated by $H\left(k\right)=\ln k+\xi$, where $\xi$ is Euler’s constant. As $c\left(n\right)$ is the average of $h\left(x\right)$ for the given n, we use it to normalize $h\left(x\right)$. The anomaly score $s$ of an instance $x$ is defined as:

$$s\left(x,n\right)=2^{-\frac{E\left(h\left(x\right)\right)}{c\left(n\right)}}$$

(12)

where $E\left(h\left(x\right)\right)$ is the average value of $h\left(x\right)$ is the average value of. The following conditions apply to the three special values of abnormal scores:

(a)

When $E\left(h\left(x\right)\right)\to 0,s\to 1$;

(b)

When $E\left(h\left(x\right)\right)\to n-1,s\to 0$;

(c)

When $E\left(h\left(x\right)\right)\to c\left(n\right),s\to 0.5$.

We can make the following assessment based on the anomaly score $s$. If an instance returns a value of s that is very close to 1, then it is definitely an anomaly. If an instance has an s value much less than 0.5, then it can quite safely be regarded as a normal instance. If all instances return $s\approx 0.5$, then there are no significant anomalies for all data in this sample set.

3.5. The Fully-Connected ANN Model

The fully-connected ANN is an information system constructed by theoretically abstracting, simplifying, and simulating the structure, function, and basic characteristics of the real neural network in the human brain [43]. The BP neural network, currently the most widely used fully-connected ANN, is a multi-layer feedforward neural network trained according to the error backpropagation algorithm. The transfer function used by the neurons of the BP neural network is usually a differentiable function that can realize any nonlinear mapping between the input and the output. Therefore, the BP neural network has a wide range of applications in pattern recognition, risk evaluation, intelligent prediction, etc. [44].

The BP neural network has arbitrarily complex pattern classification capabilities and excellent multi-dimensional function mapping capabilities. The objective function of it is the square of the network error; the minimum value of the objective function is calculated through the gradient descent method [45]. The detailed calculation process is as follows:

(1)

The connection weight value and threshold are initialized based on random values.

(2)

The output of each unit of the hidden layer and the output layer is calculated according to the parameters selected by the input mode and output mode. In this study, the ReLU [46] is used as the activation function of the hidden layer. The Tanh [47] is used as the activation function of the output layer.

The ReLU can be defined as

$$f\left(x\right)=\max \left(0,x\right)$$

(13)

The Tanh can be defined as

$$f\left(x\right)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$$

(14)

(3)

The new connection weights and thresholds are calculated using the Equations (13)–(16). The modification of the neuron threshold:

$${\theta }_k\left(t+1\right)={\theta }_k\left(t\right)+{\eta }_t{\sigma }_k,\left(t=1,2,\dots ,p;k=1,2,\dots ,l\right)$$

(15)

$${\theta }_j\left(t+1\right)={\theta }_j\left(t\right)+{\eta }_t{\sigma }_j,\left(t=1,2,\dots ,p;j=1,2,\dots ,m\right)$$

(16)

where ${\theta }_j$ and ${\theta }_k$ are the threshold values of the j-th node in the hidden layer and the k-th node in the output layer, respectively$;{\sigma }_j$ and ${\sigma }_k$ are errors of the j-th node in the hidden layer and the k-th node in the output layer respectively.${\eta }_t$ is the learning rate of the t-th training iteration.

The update formula of the weight is as follows:

$${\mathsf{\omega }}_{jk}\left(t+1\right)={\mathsf{\omega }}_{jk}\left(t\right)+{\mathsf{\Delta }\mathsf{\omega }}_{jk}\left(t\right),\left(t=1,2,\dots ,p\right)$$

(17)

$${\mathsf{\nu }}_{ij}\left(t+1\right)={\mathsf{\nu }}_{ij}\left(t\right)+{\mathsf{\Delta }\mathsf{\nu }}_{ij}\left(t\right),\left(t=1,2,\dots ,p\right)$$

(18)

where t is the number of iterations in the training process; ${\mathsf{\nu }}_{ij}$ is the weight value updating from the input layer to the hidden layer; ${\mathsf{\omega }}_{jk}$ is the weight value updating from the hidden layer to the output layer.

(4)

Return to the second step to train the neural network and update the learning input mode continuously until the number of training times reaches the preset value. The basic flow of the above calculation process is shown in Figure 5:

3.6. CNN Model

CNN is currently one of the most widely used deep learning algorithms. Compared to the fully-connected ANN, the most important features of CNN are local perception and parameter sharing [48]. The main parts of a CNN are the convolutional layer, pooling layer, and activation function [49]. CNN reduces the complexity of the network model through three strategies: a local receptive field, weight sharing, and downsampling. They are widely used in the fields of image classification, speech recognition, and intelligent prediction [50].

CNN has proven to be a reliable technique for extracting hidden features because it can complete the automatic creation of filters [51]. By using the convolution filter to act on all the feature spaces of the input data, for each perception area, the data at the corresponding position are multiplied by the weight in the convolution filter, and then summed as the input of the next layer of neurons. In the same convolution layer of CNN, all neurons are set to have no connection and only the weight of the filter is shared. Therefore, CNN is more effective than multi-layer perceptron training with the same structure. Each convolution layer can be expressed as [52]:

$$y_i=f\left({{\displaystyle \sum }}_{i=1}^n{W}_i{x}_i+b\right)$$

(19)

where $y_i$ indicates the results after convolution, f denotes the activation function, $W_i$ indicates the weights of the convolution filter, b indicates the bias terms.

The CNN model was built based on Python and TensorFlow in this study. On the basis of the aforementioned fully-connected ANN model, a convolution operation was added, and the other parts were similar to the fully-connected ANN model. The CNN consisted of an input layer, a convolutional layer, a flattening layer, a fully connected layer, and an output layer. The convolution kernel size was 2 × 1, the channels of the convolutional layer were 64. After convolution, the number of nodes changed from 9 to 576, realizing the deep extraction of features. In view of the few input parameters of this model, pooling operation was not performed.

ReLU was used as the activation function of the convolutional layer and the fully connected layer. Tanh was used as the activation function of the output layer. Adam [53] was the optimization algorithm in this study.

Adam can be defined as

$$m_t=\mu \times m_{t-1}+\left(1-\mu \right)\times g_t$$

(20)

$$n_t=\nu \times n_{t-1}+\left(1-\nu \right)\times g_t^2$$

(21)

$$\widehat{m_t}=\frac{m_t}{1-{\mu }^t}$$

(22)

$$\widehat{n_t}=\frac{n_t}{1-{\nu }^t}$$

(23)

$${\theta }_t=-\frac{\widehat{m_t}}{\sqrt{\widehat{n_t}}+ϵ}\times \eta$$

(24)

where $\mu$, $\nu$ indicate the exponential decay rates for the moment estimates, $g_t$ indicates the gradients of stochastic objectives at timestep t, $m_t$ indicates the biased first moment estimation of gradients, $n_t$ indicates the second raw moment estimation of gradients, $\widehat{m_t}$ and $\widehat{n_t}$ respectively, indicate the unbiased estimate after correcting $m_t$ and $n_t$, ${\theta }_t$ indicates the result parameter. After Bayesian optimization, the hyperparameters in the neural network are $\eta =0.001$, $\mu =0.9$, $\nu =0.999$ and $ϵ={10}^{-8}$. The basic flow of the above calculation process is shown in Figure 6:

4. Implementation Results of a Fully-Connected ANN and CNN

4.1. Influencing Factors of Bridge Maintenance Costs

The random forest method in Section 3.1 was used to calculate the VIM values of eight influencing factors (GDP growth rate, maintenance time, bridge grade, bridge technical condition, bridge location, bridge age, superstructure, and highway grade). The calculation results are shown in Figure 7.

It can be seen from Figure 7 that the importance values of the four indicators of maintenance time, technical conditions, highway class, and GDP growth rate are all greater than 0.2. The importance of these four indicators is significantly higher than the other indicators, and the importance value of the other indicators is less than 0.1. Therefore, we select GDP growth rate, bridge technical conditions, highway grade, and maintenance time as the input parameters of the prediction models.

The scatter graph matrix was adopted to conduct an exploratory data analysis on the database, to test the correlation between the four input parameters selected in the previous section and bridge maintenance costs. The probability histogram is on the diagonal line of the matrix, which represents the frequency of occurrence of a corresponding label for a certain feature. The scatter diagram is on the non-diagonal line of the matrix, which represents the correlation between labels, corresponding to different maintenance cost characteristics.

A total of five characteristics of the four selected indicators, plus maintenance costs, were involved in the exploratory data analysis. Thus, the matrix consists of 5 probability histograms and 20 scatter plots. As can be seen from Figure 8, there is a certain correlation between the selected indicators and maintenance costs, while ensuring certain independence.

4.2. Sample Classification Based on Selected Indicators

All samples are classified according to four input indicators, as shown in Figure 9. The samples only contain three indicators: maintenance time, bridge technical condition, and highway grade.

In Figure 9, the maintenance cost samples are divided into three categories according to the highway grade. According to the technical conditions of the bridge, each category was divided into four sub-categories and the samples were sorted by maintenance time. The sample values increased significantly as the technical conditions of the bridge worsened. As the highway grade worsened, the sample values also tended to increase, and they gradually increased with maintenance time. These results demonstrate the feasibility of predicting the maintenance costs based on the classification of these indicators.

4.3. Sample Screening Based on Isolation Forest

Twenty-seven maintenance samples (6.8% of the total) were identified as outliers by the isolation forest method. We replaced the outliers with the average of the same type of data to ensure that the prediction accuracy improved without reducing the sample size. The sample distribution after replacement is shown in Figure 10.

4.4. Structure of the Sample Database

The constructed database contained a total of 399 maintenance cost samples. All samples were classified according to the highway grade and bridge technical condition. The 80% of samples with the earliest maintenance times in each type of data were selected into the training set for training. The 20% of samples with the latest maintenance times constituted the prediction set as the verification of the calculation results. A total of 320 samples were selected into the training set and 79 samples were selected into the prediction set. The detailed classification of the database is shown in

Table 4. Classification of the database.

Highway Grade	Bridge Category	Training Set	Prediction Set
High	Second	14	3
	Third	23	5
	Fourth	54	13
Medium	Second	5	2
	Third	11	3
	Fourth	84	21
	Fifth	27	7
Low	Second	64	16
	Third	30	7
	Fourth	6	1
	Fifth	2	1
Total		320	79

.

4.5. Topology of the Fully-Connected ANN Model and CNN Model

In this study, a fully-connected ANN model was designed for maintenance cost prediction. It contained three layers: an input layer, a fully connected hidden layer, and an output layer. There are 9 neurons in the input layer, 980 neurons in the hidden layer, and 1 neuron in the output layer. The topology of the proposed model is shown in Figure 11.

It can be seen from Figure 12 that the output of the convolutional layer is a 9 × 64 matrix. The third layer is a flattening layer containing 576 neurons. The fourth layer is a fully connected layer containing 1280 neurons, and the final layer is the output layer.

4.6. Accuracy Analysis of Prediction Model

The established fully-connected ANN model and CNN model were used to predict the cost of bridge maintenance in the future. To verify the prediction accuracy of the model, the prediction result of the model was compared to the actual bridge maintenance costs. Figure 13 shows a comparison between the predicted values given by the two prediction models and the actual maintenance costs under the corresponding indicators.

The relative error distribution of the fully-connected ANN model is shown in Figure 14. It can be seen from Figure 14 that the relative error is less than 20% for 84.8% of the prediction data and less than 30% for 93.6% of the prediction data.

The relative error distribution of the CNN model is shown in Figure 15. It can be seen from Figure 15 that 87.4% of the prediction results have a relative error of less than 20% and less than 30% for 98.7% of the prediction data.

To compare the predictive performance of the two models, a linear regression model was added as a baseline for comparison. Linear, quadratic, and exponential curve fitting equations can be established based on regression analysis. Finally, the quadratic curve with the highest average goodness of fit (R²) was selected as the fitting form, a prediction model was established, and the regression coefficient was calculated. The regression analysis model form is defined as follows:

$$y=a{t}^2+bt+c$$

(25)

where y indicates the maintenance costs for the bridge; a, b, and c are parameters for regression analysis; t indicates time.

The relevant prediction error indicators for the fully-connected ANN model, the CNN model, and regression model are presented in

Table 5. Relevant error indicators of the fully-connected ANN model, CNN model prediction, and regression model.

Prediction Model	Mean Absolute Error (Yuan/m2)	Maximum Error (Yuan/m2)	Minimum Error (Yuan/m2)	Root Mean Square Error (Yuan/m2)
ANN	16.8	59.4	0.2	19.7
CNN	12.5	52.6	0.3	16.7
Regression model	20.6	57.8	3.6	21.9
Prediction Model	Average Relative Error (%)	Maximum Relative Error (%)	Minimum Relative Error (%)	Mean Absolute Percent Error (%)
ANN	12.05%	52.71%	0.28%	11.89%
CNN	9.45%	29.83%	0.24%	9.35%
Regression model	15.95%	46.51%	3.15%	15.22%

.

It is apparent—from the prediction results and error analyses of the two models—that the CNN model provides better predictions of the bridge maintenance costs than the fully-connected ANN. Through the convolution operation, multi-dimensional feature extraction can be applied to each maintenance cost indicator, improving the prediction accuracy.

5. Conclusions

This paper established a sample database of bridge maintenance costs over the period 2010–2019, based on the actual engineering data of 268 bridges in coastal and inland areas of China. Two bridge maintenance cost prediction models were developed based on fully-connected ANN and CNN approaches, respectively, and their prediction results were compared. The conclusions of this study are as follows:

(1) The eight main factors of bridge maintenance costs were evaluated based on the random forest method. The VIM of four indicators (technical condition, maintenance time, GDP growth rate, and highway grade) were greater than 0.2. The VIM values of the bridge age, superstructure, bridge grade, and bridge location were all less than 0.1.

(2) An outlier detection method based on the isolation forest method was adopted to identify maintenance cost samples that were significantly different from the bridge maintenance costs. As a result, 27 maintenance samples, accounting for 6.8% of the total, were identified as outliers and replaced by the average values of similar data.

(3) Prediction models were established based on fully-connected ANN and CNN, respectively. The mean absolute error, root mean square error, average relative error, and mean absolute percent error of the fully-connected ANN model were 17.4 Yuan/m², 21.2 Yuan/m², 13.02%, and 12.59% respectively. The mean absolute error, root mean square error, average relative error, and mean absolute percent error of the CNN model were 12.5 Yuan/m², 16.7 Yuan/m², 9.45%, and 9.35%, respectively. It can be concluded from the comparative analysis that the prediction accuracy of the CNN model is higher than that of the fully-connected ANN model.

6. Prospect

There are several limitations in this paper, which could guide the directions of future studies. The sample size of the database was limited, and the coverage was not wide enough. As a result, indicators, such as geological conditions and extreme weather conditions, were not fully considered. A more comprehensive bridge maintenance cost database is needed to support future studies. Social and economic development is a relatively broad concept and a complex economic problem. How to quantify and find the relationship between it and bridge maintenance costs is a topic worthy of in-depth study. This study used GDP growth rate as an indicator of social and economic development (and achieved good results). Further research could explore other economic indicators to quantify this impact based on an in-depth analysis of the correlation between economic development and bridge maintenance costs.