Comparative Assessment of Three Nonlinear Approaches for Landslide Susceptibility Mapping in a Coal Mine Area

Abstract

Landslide susceptibility mapping is the first and most important step involved in landslide hazard assessment. The purpose of the present study is to compare three nonlinear approaches for landslide susceptibility mapping and test whether coal mining has a significant impact on landslide occurrence in coal mine areas. Landslide data collected by the Bureau of Land and Resources are represented by the X, Y coordinates of its central point; causative factors were calculated from topographic and geologic maps, as well as satellite imagery. The five-fold cross-validation method was adopted and the landslide/non-landslide datasets were randomly split into a ratio of 80:20. From this, five subsets for 20 times were acquired for training and validating models by GIS Geostatistical analysis methods, and all of the subsets were employed in a spatially balanced sample design. Three landslide models were built using support vector machine (SVM), logistic regression (LR), and artificial neural network (ANN) models by selecting the median of the performance measures. Then, the three fitted models were compared using the area under the receiver operating characteristics (ROC) curves (AUC) and the performance measures. The results show that the prediction accuracies are between 73.43% and 87.45% in the training stage, and 67.16% to 73.13% in the validating stage for the three models. AUCs vary from 0.807 to 0.906 and 0.753 to 0.944 in the two stages, respectively. Additionally, three landslide susceptibility maps were obtained by classifying the range of landslide probabilities into four classes representing low (0–0.02), medium (0.02–0.1), high (0.1–0.85), and very high (0.85–1) probabilities of landslides. For the distributions of landslide and area percentages under different susceptibility standards, the SVM model has more relative balance in the four classes compared to the LR and the ANN models. The result reveals that the SVM model possesses better prediction efficiency than the other two models. Furthermore, the five factors, including lithology, distance from the road, slope angle, elevation, and land-use types, are the most suitable conditioning factors for landslide susceptibility mapping in the study area. The mining disturbance factor has little contribution to all models, because the mining method in this area is underground mining, so the mining depth is too deep to affect the stability of the slopes.

1. Introduction

Landslides refer to general landslides in this research, which are the phenomenon of a downward or outward movement of rock, soil, man-made artificial fill, or these combinative materials. A generalized landslide includes collapse, landslide, and debris flow, which cause slope erosion and gravitational erosion [1,2]. Susceptibility is the probability of a spatial occurrence of a landslide, given a set of geo-environmental factors in mathematical form [3]. Landslide susceptibility provides the spatial distribution of locations that are favorable for landslide occurrences in the future [4], and is considered an efficient way of reducing the damage of social and economic losses caused by landslides [5]. Landslide susceptibility maps are crucial steps which can help decision-makers, planners, and local administrations in disaster planning [6]. Numerous methods have been produced to assess landslide susceptibility at a regional scale, including direct geomorphological mapping (analysis of landslide inventories), heuristic approaches, statistical methods, physically-based models [7,8,9], and newly-developed machine learning models [10,11]. More detailed information of different models for landslide susceptibility mapping can be found in the literature [8,12,13,14,15,16,17,18,19,20,21,22,23]. However, all the methods have both advantages and drawbacks, and no one method is accepted universally for the effective assessment of landslide hazards due to the complex nature of landslides [11]. Patriche et al. [15,16] compared methods of logistic regression and other different methods for landslide susceptibility, and that reported logistic regression was the best. Yilmaz [14,17] compared many methods from conventional models to new machine learning models, and found that the artificial neural network was the best. Yao [9] trained one-class and two-class support vector machine (SVM) methods to map landslide susceptibility, comparing their accuracies with logistic regression, and concluded that two-class SVM possesses the best prediction result. Kavzoglu et al. [18,19,20,21,22,23] evaluated landslide susceptibility mapping using GIS-based multi-criteria decision analysis, support vector machines, and many other methods, and showed that SVM outperformed the conventional logistic regression method in the mapping of landslides. The above studies show the results in different regions with their own geo-environmental factors, and selecting a proper model should be performed for landslide susceptibility mapping through comparisons of landslide models.

For geological disasters in coal mine areas, Xiao [24], Zhai [25], and Wang [26] studied the geological disaster characteristics, formation mechanisms, and countermeasures. Zhang [27] and Lee [28] analyzed the sensitivity of land subsidence by using an artificial neural network model. Suh [29] used the deterministic weighting model to analyze the sensitivity of surface subsidence caused by coal mining. Oh [30] used the frequency ratio model to analyze land subsidence sensitivity around coal mine areas. However, there are few studies on the impact of mining on landslide susceptibility.

It is well known that, in mining regions, slope instability and subsidence are frequent phenomena. Shanxi Province is a predominant coal resource base in China, which makes great contributions to the repaid development of national economy, but the geological disaster caused by large-scale exploitation of coal resources is becoming more and more serious, whereas the effect of severe coal mining activity in this region on geological disasters is seldom considered. In order to assess whether coal mining has a significant impact on landslide occurrence, this paper aims to investigate the high-risk areas of landslide occurrence and appraise the influences of coal mining on landslide occurrence. First, the mining disturbance factor was adopted, and then the traditional statistical method of logistic regression (LR) and two machine learning methods, artificial neural network (ANN) and SVM, were selected, and applied to landslide susceptibility modeling. The performances of the three models were assessed using repeated five-fold cross-validation with spatial and random subsampling. The five-fold cross-validation technique is adopted to improve the predictive capability. Then, a comparison among these three models was performed using the area under the receiver operating characteristics (ROC) curves (AUC), and the statistical evaluation measures, and the method with relatively higher prediction accuracies to predict the spatial distribution of landslides in the Huoxi Coalfield was chosen.

2. Study Area

The Huoxi Coalfield (35°40’28”–37°17’12” N, 111°5’43”–112°21’26” E), located at the center of Shanxi Province, China, is one of the six large coalfields (Datong, Ningwu, Hedong, Xishan, Huoxi, and Qinshui) of Shanxi Province (Figure 1). It covers approximately 10,000 km². This area belongs to a temperate monsoon climate with the annual average temperature of 8.6 °C and about 180 days of frost-free conditions. The average annual precipitation is 634 mm and rainfall mostly occurs in July and August. Slope failures are triggered chiefly by intense and prolonged rainfall in July and August.

The Huoxi Coalfield includes the Fenxi field with an area of 3000 km² and the Huozhou field with an area of 7000 km². Fenxi’s eastern boundary is marked by the Huoshan fault. This field is on the east margin of the Qi-Lv-He epsilon-type structure arc-fold and different forms of folds compose coal-bearing strata which belong to the Taiyuan Formation (Upper-Carboniferous System) and the Shanxi formation (Lower-Permian System). The average thicknesses of the Taiyuan and Shanxi Formations are about 90 m and 50 m, respectively, which are 810 m and 760 m deep in the ground, respectively.

The Huozhou field is located in Lvliang mountainous uplift belt, which is on the east margin and frontal arc of the Qi-Lv-He epsilon-type structure. The folds formed by poly-tectonic movement and a series of fracture structures make up the tectonic body of this region. Its exposure strata are Archean, Upper Proterozoic, Paleozoic, and Cenozoic. The main coal-bearing strata in the Huozhou field are the Taiyuan Formation of Upper Carboniferous and the Shanxi Formation of Lower Permian, and the average thicknesses of the Taiyuan and Shanxi Formations are about 85 m and 40 m, respectively, which are 190 m and 150 m deep in the ground, respectively.

The Huoxi Coalfield is an underground mining region, with blasting, etc. The coal mine excavation is varied with the coal mining technology and the geological conditions.

3. Landslide Inventory and Conditioning Factors

3.1. Landslide Inventory

Landslide inventory maps are necessary to assess the relationship between landslide distribution and predisposing factors. Figure 1 shows the landslide inventory map in the study area. The landslide points, including landslides, debris flows, and rock falls, collected by the Bureau of Land and Resources in 2009, are represented by the X, Y coordinates of its central point. A total of 338 landslide sites are mapped. The spatial distribution of landslides has been carried out using remote sensing (RS) and GIS spatial analysis methods, and the results show that landslides are mainly distributed in the elevation between 700 and 1500 m (Figure 2). The collapse, landslide, and debris flows are often interrelated in their cause, dependent on each other in time and space, with a chain of spatial characteristics. Therefore, these three types of evaluation can be characterized by a unified model [1,2,31]. The occurrence dates of these landslide locations are mostly unknown, and an analysis of the landslide inventory map shows that landslides mainly occur during and after heavy rainfalls.

3.2. Landslide Conditioning Factors

Different landslide susceptibility influence factors have different correlations with landslide occurrence. Landslide factor selection has a direct influence on the results of landslide susceptibility. Thus, we need to choose a set of reasonable and scientific factors which comprehensively reflect the landslide sensitivity objectively, scientifically, and systematically. Taking the Huoxi Coalfield of Shanxi Province as the research area, and according to regional characteristics and the experience of experts [32,33,34], the correlations were analyzed among the spatial distribution characteristics of landslides and the factors which affect the stability of slopes. Finally, choosing five categories (including the basic influence of landslide topography, geology, hydrology, land cover, and human activities) and 12 conditioning factors influencing landslides, especially the mining disturbance factor (

Table 1. Parameters list of 12 conditioning factors.

Parameters	Data Source	Data Class or Range	Variables Type
Elevation	Advanced Spaceborne Thermal Emission and Reflection Radiometer, the global digital elevation model (ASTER-GDEM) (30 m)	348–2346 m	Continues
Plan Curvature		0–82.03
Profile Curvature		0–46.85
Slope Angle		0–64.5°
Slope Aspect		Flat, N, NE, E, SE, S, SW, W, NW	Categorical
Stratigraphic Lithology	Department of Geological Survey (1:50,000)	rock, shale, sandy shale, limestone, dolomite, shale coal, mudstone, clay, clayey, sandy loam, sandy clay	Categorical
Distance to Fault		<200, 200–400, 400–600, 600–800, 800–1000, >1000 (m)
Distance to Drainage	Department of Land Resource (1:50,000)	<100, 100–200, 200–300, 300–400, 400–500, >500 (m)	Categorical
Distance to Road		<100, 100–200, 200–300, 300–400, 400–500, >500 (m)
Mining disturbance		Mining area/Non-mining area
Land-use 2	Thematic Mapper (TM) (30 m)	farmland; forest; grassland; esidential; industrial land; water body 1	Categorical
Normalized Difference Vegetation Index (NDVI)		−0.414~0.631	Continues

¹ Water body was removed from the study area because this type of region is free of landslides; ² Only part of the land-use types entered into the logistic regression (LR) model.

), we explored the impact of mining activities on landslide disasters.

Released by the National Aeronautics and Space Administration (NASA) and Japan’s Ministry of Economy, Trade and Industry, Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER), the global digital elevation model (GDEM) has a spatial resolution of 30 × 30 m, which can be used for extracting the landslide conditioning factors for the study area. Based on the GDEM, five topographical parameters were derived: elevation (Figure 3a), slope angle (Figure 3b), slope aspect (Figure 3c), plan curvature (Figure 3d), and profile curvature (Figure 3e). Stratigraphic lithology (Figure 3f) was digitized based on the geological maps from the Department of Geological Survey at a 1:50,000 scale. Fault network (Figure 3g), drainage network (Figure 3h), and roads (Figure 3i) were digitized based on the geological maps and other thematic maps from the Department of Land Resource at a 1:50,000 scale, and then buffers for faults, drainage, and roads were conducted. Mining disturbances (Figure 3j) were digitized based on the coal resources planning maps, and if the point falls in the mine area, it is proved to be disturbed by the mining disturbance; otherwise it is not affected. Normalized Difference Vegetation Index (NDVI) (Figure 3k) and land-use types (Figure 3l) were interpreted and computed from Landsat Thematic Mapper (TM) images.

4. Landslide Susceptibility Mapping

4.1. Preparation of Training and Testing Datasets

For two-class LR, SVM, and ANN models, it is necessary to generate “stable” samples (i.e., non-landslide sites), because the landslide modeling in this study is considered as binary pattern recognition. Non-landslide samples in this study were generated randomly outside a buffer zone of 400 m from existing landslide points, and the minimum distance between any random points was 200 m [9]. A total of 338 non-landslide points, equal to the number of landslide points (338 landslide points), were generated and converted to pixel format. All the landslide grid pixels denoting the presence of landslides were assigned the value of 1, whereas the non-landslide pixels were assigned the value of 0. In this study, in order to compare the three methods under the unified circumstances, the same training and testing datasets were used for each model. The five-fold cross-validation method was adopted and the landslide/non-landslide datasets were randomly split into a ratio of 80:20 and five subsets for 20 times were acquired for training and validating models by GIS Geostatistical analysis methods; all of the subsets were employed in a spatially balanced sample design. In a five-fold cross-validation, five randomly and spatially selected disjoint subsamples, or folds, are derived. The model was trained five times on the combined data of four folds and tested on the data of the remaining fold by applying the trained model to the test fold and calculating the performance measure [35]. The analysis was carried out using the IBM SPSS Modeler 18, which is a data mining and text analytics software application from IBM. It is used to build predictive models and conduct other analytic tasks. It has a visual interface which allows users to leverage statistical and data mining algorithms without programming.

4.2. Logistic Regression Model

Logistic regression, which is a kernel version of logistic regression that transfers the original input space into a high-dimensional feature space using kernel functions, uses independent variables to create a mathematical model that predicts the probability of an event occurrence [10,36]. Similar to linear regression, logistic regression detects the quantitative relationship between a dependent variable and one or more independent variables; the dependent variable values are transformed into the value of their corresponding probability ratio logarithm, which is the probability of the event variable using a logistic curve to fit. The solving method is also changed from the original least square estimation to the maximum likelihood estimation [16]. In the case of landslide susceptibility mapping, the dependent variable is a binary variable denoting the presence or absence of landslides, and the relationship between landslide occurrence and causative factors can be expressed as:

$$p=\frac{1}{1+e^{-z}}$$

(1)

where p is the estimated occurrence probability of a landslide and varies from 0 to 1 on an S-shaped curve; z is the weighed linear combination of the independent variables and varies from −∞ to +∞; z can be expressed as a summation of some constant value [10].

$$z=\mathit{ln}(\frac{p}{1-p})=\alpha +{\displaystyle \sum _{i=1}^n{\beta }_i{x}_i}$$

(2)

where p/1−p is the likelihood ratio and α is the intercept of the model; x_i represents the independent variables and β_i denotes their respective coefficients. In this study, when a binomial was selected and all of the factors were entered into the equation directly, the significance level was 95%. The tuning parameter and regularized parameter for LR obtained with the training data were 0.015 and 0.03, respectively [10].

4.3. Artificial Neural Network

An artificial neural network is defined as a set of interconnected neurons and the most particular characteristic of a neural network system is its ability to learn. ANN learning algorithms define how network weights are adjusted in the training process. In this study, a back-propagation learning algorithm was used for training the neural network because this algorithm has proven successful in various studies [37]. The ANNs generally consist of an input layer, one or more hidden layers, and an output layer [10,38]. The number of input neurons is the same as the number of selected landslide conditioning factors, whereas the number of hidden neurons is determined based on the specific training data [10]. A three-layer ANN structure with one input layer, one hidden layer, and one output layer was selected. Further, in order to transfer the network output between 0 and 1, the logistic sigmoid was used as the activation function [39]. The training parameters for the learning rate, momentum, and training time were selected as 0.3, 0.2, and 500, respectively [10,40,41,42].

4.4. Support Vector Machine

A support vector machine is a supervised learning method based on statistical learning theory and the structural risk minimization principle [36]. The SVM model performance is affected by the use of the kernel functions, such as linear, polynomial, sigmoid, and radial basis function (RBF, often called Gaussian) kernel. In the present study, these four kernels were all employed for the SVM model using the training datasets; the best one was chosen and then compared with the LR and ANN models. The prediction median accuracy of SVMs with RBF, polynomial, sigmoid, and linear kernels were 73.53%, 55.88%, 47.06%, and 66.91%, respectively, and, thus, the RBF kernel SVM was selected as it presented the best performance. The behavior of the SVM using the RBF kernel is influenced by the kernel width (γ) and the regularization (C) parameters. The best model parameters of the RBF kernel SVM model were a regularization parameter (C) of 0.8 and a kernel parameter of 0.95 [10,43,44]. With these optimized parameters, the SVM model was built using the training data and then applied to calculate the landslide susceptibility indices for the entire study area.

4.5. Accuracy Assessment and Comparison

Every model is a simplification of reality. The prediction skills can be analyzed quantitatively with various performance measures and estimation techniques; common quantitative performance measures are the success/prediction rate [45], confusion matrix or error rates [16], and cost curves [46]. Among the performance estimation techniques and measures, cross-validation using a single hold-out method and the area under the receiver operating characteristic curve (AUROC) value based on ROC plots are usually applied [35,45,47]. In order to obtain results that are independent of a particular partitioning, cross-validation was repeated 20 times resulted in 100 different estimates of the performance measures, and the median of the 100 outcomes was calculated for each of the three models.

In order to evaluate the performance of the trained landslide models, five statistical evaluation measures (i.e., accuracy, sensitivity, specificity, positive predictive value, and negative predictive value) [10] were calculated using the median value of the results in the training and validating stages, respectively. The equations of these five measures can be expressed as:

$$Accuray=\frac{TP+TN}{TP+TN+FP+FN}$$

(3)

$$Sensitivity=\frac{TP}{TP+FN}$$

(4)

$$Specificity=\frac{TN}{TN+FP}$$

(5)

$$Positive predictive value=\frac{TP}{TP+FP}$$

(6)

$$Negative predictive value=\frac{TN}{TN+FN}$$

(7)

where TP (true positive) and TN (true negative) are the median number of pixels that are correctly classified, whereas FP (false positive) and FN (false negative) are the median number of pixels that are mistakenly classified.

Additionally, the receiver operating characteristics (ROC) of the three models, as well as the areas under the ROC curves (AUC) of each model, which is a performance measure derived by comparing the sensitivity of a model to the specificity, were calculated. The ROC curve was often used for checking the spatial effectiveness of the susceptibility maps obtained in landslide assessment [10]. AUC is the statistical summary of the global performance of the landslide models. An AUC equal to 1 indicates excellent prediction accuracy of the model, whereas if it equals 0.5, this indicates that the model is non-informative [48].

Three fitted landslide models were acquired using SVM, LR, and ANN models by selecting the median value of the performance measures. Then, the three fitted models were compared using the ROC, AUC, and the performance measures.

4.6. The Importance of Landslide Conditioning Factor

Relative importance of the conditioning factors to a landslide model is affected by the use of the methods and evaluation criteria. Factors with a high contribution to a particular model may be useless for another, and vice versa, thus the importance of a conditioning factor may represent a significant variation [10]. Many methods have been proposed to calculate the importance of model factors. In the present research, the relative importance is assessed according to the decreased degree of uncertainty, which will occur during the prediction of the dependent variable after using a certain factor, and the prediction uncertainty is predicted in accordance with the distribution entropy. The relative importance of the three models is calculated as:

$$H_Y=-{\displaystyle \sum _{i}P(Y=i)\log P(Y=i)}$$

(8)

Based on the exploratory analysis of the predictive value (x), the conditional distribution ($f_{y|x}$) of the output value (Y) is calculated. By comparing the conditional distribution ($f_{y|x}$) and the distribution boundary (f_y), the information value (Y) is predicted by evaluating X. The difference between the entropy of the conditional distribution and the distribution boundary is expressed as:

$$\mathrm{\Delta }H(x_j)=H_Y-H_{Y|x_j}$$

(9)

where H_Y|xj is the entropy value of the conditional distribution probability (f_y|x); x_j is the information whether the conditional distribution probability (f_y|xj) is greater than the distribution boundary (f_y).

The relative importance is described by the decrease of the uncertainty when Y is predicted by using a random variable X, and can be expressed as the amount of interaction information between two variables:

$$\begin{array}{cc}\hfill M(Y,X)& ={\displaystyle {\sum }_i{f}_x}(x_j)\mathrm{\Delta }H(x_j)\hfill \\ \hfill & =H_Y-H_{Y|X}\hfill \\ \hfill & =H_Y+H_X-H_{Y,X}\hfill \end{array}$$

(10)

The fractal dimension, reduced with the change of X, is expressed as the mutual information factor that is the relative importance of the factor. It is calculated by Equation (11); the variation value is around 0–1. If the independent and dependent variables are independent of each other, it equals 0. Otherwise, if they can be expressed as a linear or nonlinear formula, it equals 1.

$$I_{y,x}=1-\frac{H_{Y|X}}{H_Y}=\frac{M_{Y,X}}{H_Y}$$

(11)

4.7. Landslide Susceptibility Map

The main focus for a susceptibility map is its ability to forecast future landslides [35]. Once the three fitted models are successfully built in the training process and validated in the validation process, they can be used to calculate the landslide susceptibility for all other pixels in the study area [45]. Three landslide susceptibility maps are obtained by classifying the range of landslide probability into four classes representing low (0–0.02), medium (0.02–0.1), high (0.1–0.85), and very high (0.85–1) probability of landslide, respectively. The performance of the landslide sensitivity regionalization can be represented in two aspects: (1) the surveyed landslide disaster points should mostly distribute in the regions with high landslide sensitivity, which shows high gradation accuracy of the landslide sensitivity; and (2) in all the surveys, the points with high sensitivity grades should account for a low portion, which can reduce the redundancy of landslide prediction with high sensitivity, so as to improve the accuracy in the sensitivity evaluation [40].

5. Results

5.1. Landslide Conditioning Factor Analysis

In order to carry out the comparison between the models, the median value of the relative importance of the different cross-variables of the three fitted models are plotted in Figure 4. It can be observed that lithology, distance from the road, slope angle, elevation, and land-use types contribute to each model. Therefore, these five factors are the most suitable conditioning factors for landslide susceptibility mapping in this area. In the three models, lithology and distance to the road had higher scores. The relative importance of lithology in the three models is, respectively, 57.7% (SVM), 31.8% (LR), and 17.8% (ANN); the relative importance of distance to roads is 18.9% (SVM), 16.3% (LR), and 11.4% (ANN); the relative importance of elevation is 5.6% (SVM), 8.3% (LR), and 9.0% (ANN); the relative importance of slope angle is 7.4% (SVM), 24.5% (LR), and 12.4% (ANN); and the relative importance of land-use type is 4.2% (SVM), 6.9% (LR), and 12.7% (ANN).

5.2. Model Fitting Results and Analysis

Using the 12 conditioning factors, SVM, ANN, and LR were built using the training and validating datasets of repeated five-fold cross-validation. The results (median value) are shown in

Table 2. The median of the performances for the three fitted landslide susceptibility models/%.

Model	Stage	Accuracy	Sensitivity	Specificity	Positive Prediction	Negative Prediction
LR	training	73.43	73.61	73.26	73.06	73.80
	validating	71.64	72.73	71.64	71.64	72.73
Artificial Neural Network (ANN)	training	78.23	73.68	84.93	87.82	68.63
	validating	67.16	64.20	71.70	76.12	56.72
Support Vector Machine (SVM)	training	87.45	87.45	87.45	87.45	87.45
	validating	73.13	74.60	71.83	77.61	76.12

.

From Table 2 it can be observed that in both the training and validating model stages, SVM has the best performance with the highest classification accuracy (87.45% and 73.13%, respectively). Moreover, SVM has higher values in sensitivity and specificity than the other two models. Furthermore, SVM also has more balance in terms of positive and negative predictive values (87.45%, 87.45% in the training stage, and 77.61%, 73.95% in the validating stage), which is followed by LR and ANN. Thus, SVM is the best model in landslide susceptibility prediction in this research.

5.3. ROC Curve and AUC Value Analysis

Cross-validation estimation of a model’s predictive performance is a crucial step in predictive modeling because estimation on the training set is always too optimistic [16]. Cross-validation results in bias-reduced performance estimates as the test partitions used in each repetition do not overlap with the training sample [16]. In particular, spatial cross-validation is recommended for spatial data, which may be subject to spatial autocorrelation [16]. The ROC curves and their AUC values of the three fitted landslide susceptibility models were plotted, and the AUC of each model was calculated in Figure 5.

With the training dataset, the results of AUC for the three fitted landslide susceptibility models are shown in Figure 5a. According to obtained AUC using the training datasets, the AUC values were all more than 0.7, showing that the three models all give promising results. SVM has slightly higher prediction performance (0.906) than LR (0.815) and ANN (0.807). This may be due to the fact that in SVM, increment on the number of parameters makes the data richer and this enrichment makes SVM slightly more successful than LR and ANN. The machine learning methods of SVM and ANN have a higher capability of modeling complex problems than the conventional method of LR.

Using the validating dataset, the results of AUC for the three fitted landslide susceptibility models are shown in Figure 5b. It can be seen that, as in the results of the building models stage, AUCs using the validating dataset range from 0.753 to 0.944. Same as the building model results, SVM has the highest AUC with 0.944, showing that SVM appears to be more accurate than other models. Another study [10] reported that no single model performed best for all evaluation metrics, and a higher AUC does not guarantee for a high spatial accuracy. Therefore, ignoring the slight difference, it could be evaluated that ANN and LR have relatively similar accuracies, and those with an AUC value of more than 0.7 can also be considered suitable for landslide susceptibility mapping in this study.

5.4. Landslide Susceptibility Map Analysis

Once the SVM, ANN, and LR models were successfully trained in the training process, they were used to calculate the median landslide susceptibility indices for all the pixels in the study area. The study area contains 12,581,787 pixels. In ArcGIS software, the pixels were converted into point type. A total of 12,581,787 sites were mapped. The median landslide susceptibility indices were reclassified into four susceptibility levels: very high, high, moderate, and low, using the equal area classification method. Based on the percentage of landslide pixels and the percentage of landslide susceptibility map, the four susceptibility classes in this study were determined as very high (0.85–1), high (0.1–0.85), moderate (0.02–0.1), and low (0–0.02), respectively. For the purpose of visualization, the four landslide susceptibility maps produced from the SVM, ANN, and LR models are shown in Figure 6 [11].

Figure 6 shows that the three models all predict that the southern region has a low or moderate landslide susceptibility. This feature is realistic, as the region is in the Linfen Basin, with a low elevation and plan ground, and landslides occurred rarely. On the contrary, they all predict an altitudinal increase in landslide susceptibility towards the mountain situated in the extreme northeast of the study area.

To analyze the spatial prediction results of these three fitted models comparatively, the distributions of landslides and the area percentage under different susceptibility standards are shown in

Table 3. Distributions of landslide and area percentage under different susceptibility standards.

Models		Low (0–0.02)	Medium (0.02–0.10)	High (0.10–0.85)	Very High (0.85–1)
ANN	Number of landslides	0	20	190	128
	Area percentage/%	0	17.79	43.80	38.42
LR	Number of landslides	1	8	181	148
	Area percentage/%	0.12	23.12	60.32	16.43
SVM	Number of landslides	1	21	67	249
	Area percentage/%	20.38	28.64	25.71	25.28

.

Table 3 shows that, in the landslide sensitivity region at very high and high degrees, there are 316, 329, and 318 landslide points for SVM, LR, and ANN, respectively. Regarding the area percentage, they are 50.99%, 76.75%, and 82.22% for the three models. According to the two rules: (1) the surveyed landslide disaster points should be mostly distributed in the regions with high landslide sensitivity; and (2) in all the surveys, the points with high sensitivity grades should account for a low portion [40], the SVM model is a little better than the LR model, and the two models are both evidently better than the ANN models.

6. Discussion

6.1. Advantages and Disadvantages of the Three Models

Regional landslide susceptibility mapping has been a hot issue because of its difficult and non-linear characteristics. Although various methodologies for producing landslide susceptibility maps have been developed, the prediction accuracy of these methods is still debated [49]. In the present study, the conventional method of LR as well as the machine learning methods of ANN and SVM were selected for susceptibility modeling and compared in a coal mining area. Through comparison, the advantages and disadvantages of the three models are the following:

The logistic regression model is simple and easy to implement, as it only considers the linear relationship between the evaluation factors and the landslide susceptibility, whereas the formation mechanism of landslide is a complex and non-linear issue [10,17]. ANN is effective with respect to nonlinear continuous data and is especially suitable for simulating complex phenomena with multiple factors and results. However, the machine learning language has natural defects, which is a black box model, so it is hard to judge the internal mechanism [10,17,43]. The SVM model keeps the points near the classification line away from the classification line as far as possible. The basic principle of classification is not only to classify the points correctly, but to keep the divided distance at a maximum [10,17,50]. SVM can be efficiently used in problems with two classes. In this research, the susceptibility evaluation in the Huoxi Coalfield belongs to the binary classification under multiple factors, so the SVM model has better results than LR and ANN.

6.2. Accuracy Comparison for the Three Models

It is clear that the prediction accuracy of a landslide model depends on the method used. Some new machine learning methods, such as SVM, have shown better results than conventional methods [11]. We addressed this issue in this paper with the evaluation and comparison of three methods, including LR, ANN, and SVM methods. According to the statistical evaluation measures, in both the training and validating models stages, SVM has the best performance with the highest classification median accuracy (87.45% and 73.13%, respectively). Moreover, SVM has higher median values in sensitivity and specificity than the other two models. Furthermore, SVM also has more balance in terms of positive and negative predictive median values (87.45%, 87.45% in the training stage, and 77.61% and 76.12% in validating stage), which is followed by LR and ANN and, generally speaking, the accuracies in the validating stage are lower than those in the training model stage for all three models [11,17]. Thus, SVM is the best model in landslide susceptibility prediction in this research.

Moreover, according to the obtained AUC using training and validating datasets, the AUC values were all around 0.7, showing that the three models all give promising results [10]. Comparatively, SVM has higher median values (0.906 and 0.944) than both LR (0.815 and 0.753) and ANN (0.807 and 0.761) in training and validating stages, which shows that SVM is more accurate than the other models.

6.3. Variable Importance

The selection of landslide conditioning factors is a key point that affects the quality of the landslide susceptibility models and has been previously discussed [7,35]. The selection of conditioning factors is mainly carried out based on the analysis of the landslide types and the characteristics of the study area [17,47]. In general, topography, geology, hydrology, geomorphology, and land use are widely accepted as conditioning factors in most landslide susceptibility modeling.

The assessment of the contribution of the conditioning factors to the models is of great interest in landslide analysis and has been previously discussed [10]. In this study, there are significant differences in the contribution of landslide conditioning factors to the landslide models, and this may lead to inconsistent conclusions.

Lithology is the most important factor, followed by distance from the road. The highest contribution of lithology may be because of the fact that the physical condition of landslide sliding is mainly a shear failure; therefore, the shear strength of the rock mass is a necessary condition to measure the stability of slopes [2]. From the influence of lithology on the mechanical properties, it is found that the hard and dense rock mass has higher shear strength and is not sensitive to the landslide [2]. While in the case of distance from the road, landslide sites are mostly (70%) distributed close to the road with a distance from 0 to 200 m. The distance from the road to the landslide is mainly due to the construction of the road and will change the original natural slope, forming a cutting slope, which may then lead to the occurrence of landslides.

Secondly, elevation and slope are important factors. In general, slope is widely reported among the more effective instability factors [51,52]. It is estimated that 50% of landslide sites occur in the regions around 1000 m (Figure 2) because the topography is one of the decisive factors of landslide development. Although elevation will not change the distribution of the slope stress line, it can control the slope stress value; with the increase of the slope height, the stress value will increase significantly [2,40]. From the point of view of regional geomorphic conditions, under normal circumstances, the higher the elevation is, the stronger the cutting is. The stronger the cutting is, the more chances there are for a landslide disaster to happen [2,40]. From the point of view of the local topography, the appropriate slope, height, and shape are convenient for the existence of the surface, which has a direct effect on the formation of the surface. According to the statistics in the region with past landslides, 53.25% and 28.11% of landslides occur in slopes of 5° to 15° and 15° to 25°, respectively. This agrees with the field inspection by Li et al. [53], which reveals that landslides are mostly limited to slope angles of 5° to 25°.

Other causative factors have different contributions depending on the model type. The ANN model yields a high contribution by distance from the faults (10.89%), while this factor has a much lower contribution to the other models: 2.63% for SVM and 2.09% for LR.

As a causative factor, mining disturbance has little contribution to all the models, perhaps because the main coal bearing strata in the Huoxi Coalfield are the Taiyuan Formation and Shanxi Formation. In the Fenxi field, the average thicknesses of Taiyuan and Shanxi Formations are about 90 m and 50 m, respectively, which are almost 810 m and 760 m deep in the ground, separately. In the Huozhou field, the average thicknesses of the Taiyuan and Shanxi Formations are about 85 m and 40 m, respectively, which are nearly 190 m and 150 m deep in the ground, separately. The mining method in this area is underground mining, so the mining depth is too deep to affect the stability of the slopes.

7. Conclusions

This study contributed to a comparison and evaluation of the conventional method (LR) and machine learning methods (SVM and ANN) for landslide susceptibility mapping, and then tested whether coal mining has a significant impact on landslide occurrence. Three topics are focused upon: (1) model assessment and comparison using the accuracy, ROC, and AUC; (2) which method is more accurate and reliable for landslide susceptibility mapping in coal mine areas; and (3) whether coal mining has a significant influence on landslide occurrence. According to the results, SVM is better and has more balance in terms of positive and negative predictive values than the other two models. Landslide susceptibility spatial distribution yielded by SVM is reliable and, therefore, SVM is considered as a promising model for landslide susceptibility mapping. Further, the five factors, including lithology, distance to road, slope angle, elevation, and land-use types, are the most suitable conditioning factors for landslide susceptibility mapping in this area. The mining disturbance factor has little contribution to all the models because the mining method in this area is underground mining, so the mining depth is too deep to affect the stability of slopes.