Research on Mining Maximum Subsidence Prediction Based on Genetic Algorithm Combined with XGBoost Model

Abstract

The extreme gradient boosting (XGBoost) ensemble learning algorithm excels in solving complex nonlinear relational problems. In order to accurately predict the surface subsidence caused by mining, this work introduces the genetic algorithm (GA) and XGBoost integrated algorithm model for mining subsidence prediction and uses the Python language to develop the GA-XGBoost combined model. The hyperparameter vector of XGBoost is optimized by a genetic algorithm to improve the prediction accuracy and reliability of the XGBoost model. Using some domestic mining subsidence data sets to conduct a model prediction evaluation, the results show that the R2 (coefficient of determination) of the prediction results of the GA-XGBoost model is 0.941, the RMSE (root mean square error) is 0.369, and the MAE (mean absolute error) is 0.308. Then, compared with classic ensemble learning models such as XGBoost, random deep forest, and gradient boost, the GA-XGBoost model has higher prediction accuracy and performance than a single machine learning model.

1. Introduction

Coal mining subsidence can not only easily cause surface damage, building collapse, road, and railway damage, but also a series of safety and environmental problems such as landslides and surface water loss. At the same time, it can seriously damage the ecological environment of the mining area [1,2,3]. By predicting the damage degree of surface subsidence, the maximum subsidence value, and other basic data information in advance, can not only guide mine design in terms of the combined production plan and designated disaster prevention measures, but can also reduce mining damage and alleviate resources by pre-optimizing the mining plan and treatment measures. The contradiction occurs between development and utilization, environmental protection, and social public security. The research on subsidence damage caused by mining has always been a research hotspot of mining scientists. In recent years, the main research methods of mining subsidence include logistics similarity simulation, computer numerical simulation, and other methods. Similar simulation can simulate the stress distribution of surrounding rock caused by excavation. It is characteristically intuitive and simple in the study of complex problems such as mining and provides a valuable reference for engineering applications and scientific research. However, it has disadvantages such as high cost, long periods, and a lack of qualitative parameter output for simulated excavation. Particularly when the model size is large and the mining conditions are complex, it is difficult to simulate the real mining subsidence process jointly [4,5,6,7]. Computer numerical simulation can simulate the complex dynamic change process and mining conditions in the mining process, with low cost and quick results, and has also been widely used in the mining field. However, the reliability of its simulation results is closely related to the modeling analyst’s mastery of numerical software and understanding of the actual mining process [8,9,10]. At present, the main theoretical method for predicting mining subsidence in China is the probability integral method, which is widely used because of its rich theory, rigorous mathematical theoretical foundation, and convenient calculation. However, due to complex mining conditions and geological conditions, there is a certain level of error between the theoretical prediction of the probability integration method and a real engineering case. Detailed parameters to improve the accuracy and reliability of the overall prediction are provided [11,12,13].

In recent years, with the development of computer science and technology, artificial intelligence methods have been successfully applied in the field of rock mass engineering [14,15]. More and more mining technicians have begun to introduce artificial intelligence technology to solve the highly nonlinear and complex prediction problem of mining subsidence prediction and have achieved fruitful research results [16,17,18,19]. With the wide application of artificial intelligence methods such as machine learning, more and more scholars use machine learning methods to evaluate land subsidence problems, use machine learning methods to analyze a large number of land subsidence data, mine the subsidence laws in the data, and study its influencing factors and corresponding countermeasures [20,21]. The successful application of machine learning methods provides new ideas for the study of land subsidence, and its powerful processing of complex nonlinear problems shows great potential [22]. For the subsidence problem caused by coal mining, the main direction of much of the research is to use intelligent algorithms and machine learning methods to optimize the predicted parameters of traditional theoretical models and improve the accuracy and reliability of the prediction [23,24]. There are many research methods of mining subsidence, involving a wide range of theoretical aspects, and each method shows certain applicability and defects. With the wide application of machine learning methods, it has become an effective method to study complex problems by analyzing data, mining the essential principles of physical phenomena, and analyzing relevant influencing factors. In view of the above analysis, this paper selects the classic XGBoost ensemble algorithm to deal with the highly nonlinear problem of mining settlement prediction. Through a comprehensive analysis of mining factors, geological factors, etc., based on on-site observation data, bypassing the theoretical assumptions of mining models and complex mining, geological and mining factors are used to study mining subsidence problems. At the same time, given the problems of many XGBoost hyperparameters and complex models, a genetic algorithm (GA) is introduced to optimize the hyperparameters of XGBoost. A GA-XGBoost combined model with adaptive adjustment of prediction accuracy is established, which improves the generalization ability and prediction accuracy of a single XGBoost model. The combined model method proposed in this paper can help mining scientists use artificial intelligence to predict mining subsidence problems and provide a reference for the optimization of mining plans and the formulation of disaster prevention measures.

2. Methodology

2.1. Model of Mining Subsidence

With the mining of the coal seam, the roof is broken and collapsed, and the overlying stratum appears to be deformed and bent and destabilized after a series of complex operations. The mining impact is transmitted from the downhole to the surface, which eventually leads to significant subsidence [25]. The mining subsidence model is shown in Figure 1.

As shown in Figure 1, the movement and damage of the overlying strata caused by coal mining destroys the stress state and stable structure of the original strata, and eventually evolves into severe land subsidence [26]. The main factors affecting mining subsidence are mining parameters (mining height: m, mining width: l), and geological conditions (coal depth: H, topsoil thickness: h, etc.)

2.2. Extreme Gradient Boosting (XGBoost)

XGBoost is a typical ensemble learning algorithm. Its basic idea is to combine several weak estimators, and finally form a strong estimator for predictive analysis by continuously fitting the prediction residuals of a single weak estimator [27]. The common weak learner integrated by XGBoost is a tree model. Since the traditional loss function and model complexity are considered when establishing the objective function, and a regular term is added to prevent overfitting, XGBoost has high computing performance and prediction accuracy [28]. XGBoost shows excellent performance and generalization ability in dealing with nonlinear problems. It has been widely used in the field of machine learning for tasks such as classification and regression. Its objective function is as follows [29].

$$Ob{j}^{(r)}={\displaystyle \sum _{i=1}^{m}L(y_i,{\widehat{y}}_i^{(r)})}+{\displaystyle \sum _{k=1}^r\Omega (g_r)}$$

(1)

where $i$ represents the ith sample of the data set; $m$ represents the total amount of imported data imported into the kth tree; $r$ represents all trees established; $y_i$ represents the true value; ${\widehat{y}}_i^{(r)}$ represents the predicted value of the rth prediction; $g_r$ represents the structural item of the tree model; $L(y_i,{\widehat{y}}_i^{(r)})$ represents the model loss function, which describes the difference between the true value and the predicted value; $\Omega (g_r)$ indicates the complexity of the model. The complexity formula of the model is as follows:

$$\Omega (g_r)=\gamma T+\frac{1}{2}\lambda {\displaystyle \sum _j^T{w}^2}$$

(2)

where $T$ represents the number of leaf nodes of the tree, $w$ indicates leaf weight, $\gamma$ parameter to control the number of leaves, $\lambda$ is the normalized coefficient. On the basis of considering the traditional error function, XGBoost takes into account the influence of model complexity, and at the same time optimizes the generalization error of the model in mathematical principles and seeks a balance point in the model’s variance-bias dilemma in order to generalize the model.

The error is the smallest and the operation speed is the fastest. XGBoost has been recognized as an extremely high-performance estimator for classification and regression problems. However, due to the complexity of the XGBoost algorithm and many hyperparameters, the prediction performance of the model is directly affected by the model hyperparameters, which is greatly limited in engineering applications. XGBoost has many hyperparameters, and the most significant ones in practical applications are learning rate (learning rate), number of weak evaluations (num_round), maximum depth of tree (max_depth), regularization parameter(lambda), etc. However, the optimization of hyperparameters is closely related to the analyst’s understanding of the physical meaning of the data and analysis experience. In practical applications, the parameters are generally adjusted by the initial default parameters, so the performance of the model is closely related to the modeler’s theoretical principles and engineering background. Usually, manual parameter adjustment is inefficient and often difficult to optimize the model quickly, and it is difficult to meet the growing requirements of engineering applications.

2.3. Genetic Algorithm (GA)

GA is a heuristic group search algorithm. Its basic idea is to simulate the principle of “survival of the fittest and survival of the fittest” in nature, and simulate the selection, replication, crossover, mutation, and other operations of biological population chromosomes with the help of computer programs. The elimination search is carried out in a group manner, and the optimal solution is finally searched in the solution space [30]. In the process of GA optimization, the potential possible solutions are first regarded as independent individuals, the individuals are encoded to form an initial population, and then the approximate optimal solution is generated through generation-by-generation evolution. The value of the selection operation and draw on the crossover and mutation operations of genetics are used to generate a new generation of populations. The whole process simulates the evolutionary process of nature, the descendant population is more adaptable to the environment and gradually outperforms the previous generation, and finally the optimal individual of the descendants is decoded to obtain the optimal solution to the problem [31]. GA has strong versatility and simple operation and has been widely used in optimization problems in many fields [32]. We first use the default parameter combination of the model as the initial solution for coding optimization and set a fixed number of iteration steps to stop the algorithm. By setting parameters such as different population numbers, it is used to study the genetic algorithm’s ability to optimize the hyperparameters of the XGBoost model. The basic operation process of GA is shown in Figure 2.

2.4. The Combined Model of GA-XGBoost

Aiming at the nonlinear regression prediction problem of mining subsidence, based on the XGBoost algorithm, and citing GA to optimize the hyperparameters of XGBoost, a combined prediction model for mining subsidence prediction is established called the GA-XGBoost prediction model. First, the dataset is divided into training set and test set, where the training set is used to train the model, and the nonlinear relationship between the features and labels in the data set is learned. The test set is used to evaluate the prediction performance of the model. Then, the hyperparameter vector of XGBoost is randomly initialized as an individual and encoded, and the prediction error of XGBoost is used as the objective function to perform evolution iteration on the hyperparameter vector through the genetic algorithm to search for the best hyperparameter combination within the solution space. The research framework of the GA-XGBoost model is shown in Figure 3.

When running the GA-XGBoost combined model, it is necessary to initialize the basic parameters of GA, such as the number of populations, the number of iterations, and the probability of mutation. As shown in Figure 2, the key step to optimize the XGBoost model using the GA algorithm is to select the solution vector of the optimal XGBoost hyperparameters through the GA. During the XGBoost training process, the GA realizes the adaptive adjustment of the XGBoost training, and at the same time through its global optimization, and the advantage of implicit parallelism, improves the optimization speed of XGBoost hyperparameters.

3. Materials

In order to verify the feasibility of GA-XGBoost model in mining subsidence prediction, some cases of domestic scholars’ research results of mining subsidence are collected for testing and analysis. In this paper, the measured data of domestic mines are used as the research data set, the characteristics (mining depth: H, mining height: m, the area of the goaf: s, and the thickness of the loose topsoil layer: h) are used as input variables, and the label (maximum subsidence value: w) is used as the output variable and as the dependent variable to study the relationship between mining subsidence and mining and geological factors [16,33,34]. The GA algorithm is used to optimize the hyperparameter vectors of XGBoost [num_round, max_depth, eta, $\lambda$].

3.1. Data Set

A dataset containing 78 coal mining subsidence data of previous studies on coal mining subsidence was collected to evaluate the feasibility of the GA-XGBoost model in this work [33,35]. The maximum subsidence caused by mining is mainly used as an indicator for the research and analysis in this paper and will be used as a prediction output. The distribution characteristics of the dataset are shown in Figure 4.

As shown in Figure 4, the median value of H is about 150 m, the lower quartile is 100 m, and the upper quartile is 300 m. The median value of m is about 2 m, the lower quartile is 1.9 m, the upper quartile is 4.3 m, the median value of s is about 5000 m², the lower quartile is 2000 m², and the upper quartile is 130,000 m². The median of h is 20, the lower quartile is 5 m, the upper quartile is 90 m, the median of w is about 1.8 m, the lower quartile is 0.8 m, and the upper quartile is 2.2 m.

3.2. Model Verification and Evaluation

In this work, a GA-XGBoost prediction model is built, the training set is used to train the prediction model, and the test set is used to validate the trained model. Evaluation metrics such as R², RMSE, and MAE are used to evaluate and analyze the reliability and accuracy of the predictive models developed in this article. These evaluation metrics are used to describe the relationship between the predicted and tested values of w. The formula for calculating the evaluation index is as follows [36,37]:

$$R^2=1-{\displaystyle \sum _{i=1}^N{\left(y_i-{\stackrel{⌢}{y}}_i\right)}^2}/{\displaystyle \sum _{i=1}^N{\left(y_i-{\overline{y}}_i\right)}^2}$$

(3)

$$RMSE=\sqrt{{\displaystyle \sum _{i=1}^N{\left({\stackrel{⌢}{y}}_i-y_i\right)}^2}/N}$$

(4)

$$MAE={\displaystyle \sum _{i=1}^N\left|y_i-{\stackrel{⌢}{y}}_i\right|}/N$$

(5)

where $y_i$ is the actual value, ${\stackrel{⌢}{y}}_i$ is the predicted value, ${\overline{y}}_i$ is the mean of the actual values, and N is the number of test samples.

3.3. Results and Discussion

In this paper, the optimization of the XGBoost model by the GA algorithm is analyzed by setting different population sizes. As the GA algorithm converges, the error of the XGBoost module gradually decreases. Each iteration of the genetic algorithm searches for a hyperparameter combination that makes the XGBoost error smaller. The model fitness value optimization curve is shown in Figure 5.

At different population sizes, the fitness value of the model is gradually lower, and the prediction performance is gradually improved. This is done in order to reasonably evaluate the optimization performance of the GA algorithm of different populations for the XGBoost model. We further use the TOPSIS method to analyze and evaluate the main evaluation indicators of the model [38,39]. When the population size is 40, the prediction performance score of the model is 0.2 and the ranking is the best. The detailed indicators and scores are shown in

Table 1. Scoring of GA-XGBoost models with different population sizes.

Swarm Size	RMSE	R2	MAE	Score	Rank
10	0.462	0.908	0.394	0.017	7
20	0.407	0.928	0.312	0.150	4
30	0.386	0.935	0.287	0.193	2
40	0.369	0.942	0.308	0.200	1
50	0.413	0.926	0.319	0.138	5
60	0.405	0.929	0.347	0.129	6
70	0.471	0.904	0.392	0.002	8
80	0.398	0.931	0.299	0.170	3

.

When the population number is 40, the model performs the best, and the optimal solution vector of the hyperparameters of the XGBoost model is [num_round = 108, max_depth = 7, eta = 0.40148499,$\lambda$ = 14.84633415]. As shown in Figure 6, in order to further compare the superiority of the GA-XGBoost combined model, in addition to the single XGBoost, the RFR, AdaBoost, Bagging, and GradientBoost models were also selected for comparison. With the optimization of the GA algorithm, the prediction performance of the XGBoost model is significantly improved, with R² from 0.819 to 0.941, RMSE from 0.648 to 0.369, and MAE from 0.38 to 0.308. It can be seen that it is feasible to use the GA algorithm to optimize the hyperparameters of XGBoost, find a better combination of hyperparameters, and improve the prediction performance of the XGBoost model.

In order to further compare the GA-XGBoost combined model with other single integrated algorithm models, the performance indicators of each model are comprehensively analyzed and evaluated in this paper [20]. The performance indicators of the GA-XGBosst model and other single models and their performance rankings are shown in

Table 2. Comprehensive evaluation of GA-XGBoost model and other single models.

Model	RMSE	R2	MAE	Score	Rank
XGBoost	0.648	0.819	0.38	0.171	5
GA-XGBoost	0.369	0.941	0.308	0.371	1
RFR	0.593	0.849	0.412	0.189	3
GradientBoost	0.65	0.818	0.424	0.147	2
AdaBoost	0.765	0.749	0.553	0.000	6
Bagging	0.666	0.809	0.451	0.122	4

.

In order to analyze the reliability of a single model more comprehensively, such as GA-XGBoost Ref. [40], the Taylor graph was used to evaluate and display the performance of all models, as shown in Figure 7. According to the Taylor plot results, GA-XGBoost was closer to the reference (REF), which shows that the prediction performance of GA-XGBoost is better than other single models.

In order to evaluate the generalization ability of the GA-XGBoost combined model, the test set data was used to make predictions and compared with the actual measured values. Figure 8 shows the distribution of the predicted and measured values of the GA-XGBoost model. As shown in the prediction curve in the figure, except for the large deviation of the predicted value of the local scatter points, the predicted values of the other scattered points were in good agreement with the actual value. When the actual subsidence value (w) was relatively large, the predicted value had a large deviation. It can be seen that the prediction performance of the GA-XGBoost model has a significant impact when the subsidence value is large. The main reason is that the data set studied in this paper is not large enough, and there are not many cases with a large subsidence value in the data set. In the process of training, it is difficult for the model to learn the regression prediction law.

4. Conclusions

In this study, the XGBoost model combined with the GA algorithm was applied to the prediction of mining subsidence. By combining GA algorithm and XGBoost, a GA-XGBoost combined prediction model was constructed. In these modeling processes, the dataset used consists of four inputs and one output. They use RMSE, MAE, R², and other indicators to evaluate the prediction performance of the model. In addition, the GA-XGBoost proposed in this paper was compared with this classic single ensemble algorithm model such as RFR, GradientBoost XGBoost, AdaBoost, and Bagging. The proposed GA-XGBoost combined model outperforms other single models. On the plain XGB model, the prediction performance was significantly improved (R² = 0.941, RMSE = 0.369, MAE = 0.308). Therefore, it is feasible to apply the GA-XGBoost model introduced in this study to the prediction of mining subsidence.

(1)

The prediction accuracy of the GA-XGBoost model is higher than that of a single integrated algorithm model such as XGBoost, RFR, Gradient Boost, etc., indicating that it is feasible to use the GA algorithm to optimize the hyperparameters of XGBoost to improve the prediction performance of the model. It is feasible to combine traditional machine learning models with intelligent algorithms to predict mining subsidence.

(2)

The essence of GA-XGBoost is to use the search ability of GA to realize the self-adaptation and self-optimization of the XGBoost model, thereby improving the prediction performance. With the continuous enrichment and accumulation of mine data sets, the application scenarios of this model will be more extensive, and more influencing factors can be considered such as: key strata, old empty areas, coal seam dip, dip change rate, thickness change rate, etc. The complex mining area has application value. It can supplement the prediction methods and theories in the field of mine subsidence and provide auxiliary support for the formulation and optimization of relevant mining plans and control measures.