Machine Learning-Based Classification of Rock Bursts in an Active Coal Mine Dominated by Non-Destructive Tremors

Abstract

Rock bursts are dynamic phenomena in underground openings, causing damage to support and infrastructure, and are one of the main natural hazards in underground coal mines. The prediction of rock bursts is important for improving safety in mine openings. The hazard of rock bursts is correlated with seismic activity, but rock bursts are rare compared to non-destructive tremors. The five machine learning classifiers (multilayer perceptron, adaptive boosting, gradient boosting, K-nearest neighbors, and Gaussian naïve Bayes), along with an ensemble hard-voting classifier composed of these classifiers, were used to recognize rock bursts among the dominant non-destructive tremors. Machine learning models were trained and tested on ten sets of randomly selected data obtained from one of the active hard coal mines in the Upper Silesian Coal Basin, Poland. For each of the 627 cases in the database, 15 features representing geological, geomechanical, mining, and technical conditions in the opening as well as tremor energy and correlated peak particle velocity were determined. Geological and geomechanical parameters of the coal seams and surrounding rocks were aggregated into a single GEO index. The share of rock bursts in the database was only about 8.5%; therefore, the ADASYN balancing method, which addresses imbalanced datasets, was used. The ensemble hard-voting classifier most effectively classified rock bursts, with an average recall of 0.74.

1. Introduction

Rock burst is a catastrophic phenomenon occurring in underground mines and geotechnical openings during which rocks are violently ejected into the excavation space. As a result of this dynamic phenomenon, the underground opening or its part including support, machines, and devices are damaged or destroyed. In most underground mines or underground geotechnical structures, rock bursts have a stress cause. The stress level in the rock mass in the vicinity of the opening is so high that the strength of the rocks is exceeded, and they suddenly break. Of course, the ability of rocks to accumulate strain energy and release it rapidly is also essential.

In underground hard coal mines, a rock burst can be defined as a sudden release of strain energy accumulated in the rock mass, correlated with high-energy vibrations of rock mass, and acoustic effects and shock waves causing the structural destruction of a coal seam or its roof or floor rocks, and, simultaneously, an ejection of rocks into the excavation [1]. The dynamic destruction of an opening in a coal seam, having a stress origin, is usually referred to as a coal bump. However, due to its characteristic geological structure (i.e., thick layers of competent rocks), other types of rock bursts, called stroke rock bursts, can be found in coal mines [1,2]. Stroke rock burst is caused by the dynamic load pulse resulting from the fracture of thick layers of competent rocks, which affects the part of a coal seam close to an excavation [2]. However, the coal seam need not be stressed [2]. This dynamic load pulse may also impact the partially stressed sidewall part. Such a mixed rock burst is referred to as a stress-stroke [2].

The occurrence of rock bursts in coal mines is conditioned by many factors, making their prediction very challenging. The ability of the coal seam to accumulate strain energy and violent disintegration (i.e., the proneness of the coal seam to burst and the stress level in the coal seam) is of great importance [1,2]. The stress level in the coal seam may result from the thickness of the overburden, mining remnants, protective pillars, and geological disturbances [1,2]. Additionally, mining works performed in the same coal seam may significantly determine the risk of rock bursts in subsequent openings [2]. A neighborhood of goafs in the same coal seam or excessive bed cutting may affect the level of rock burst hazard [2]. A floor heave may occur when the seam is thick or weak rocks are deposited in its floor (e.g., shales). This phenomenon is often associated with a rock burst, especially when the top layer of a thick seam is mined first, and there is coal on the floor of the excavation. The presence of thick sandstone layers in the deposit structure is also a factor that may affect the occurrence of stroke or stress-stroke rock bursts. The proneness of the seam and the surrounding rocks for the occurrence of rock bursts may be considered together. In addition to those above-mentioned, mining and technical factors may also determine the occurrence of rock bursts. Releasing stress from the coal seam or fracture of strong roof rocks (e.g., by blasting) can significantly reduce the occurrence of rock bursts. Moreover, strengthening the support of openings may make them more resistant to rock bursts. In such cases, the rock burst may not occur, or its damage may be minimized. As presented above, the multitude of factors influencing and minimizing the occurrence of rock bursts is significant in coal mines.

The risk of rock bursts is one of the basic threats in coal mines in the Upper Silesian Coal Basin (USCB). In mines located in the USCB area, the mining hazard assessment method estimates the potential risk of rock bursts. This method is a part of the complex method for assessing the state of rock burst hazards in coal mines [2]. Factors such as the depth of mining, the thickness of the coal seam, the uniaxial compressive strength of coal, the presence of edges of other seams, geological disturbances, destressing mining, and others are taken into account, and each of them has an appropriate number of points. Finally, point values of all factors are summed up, and the opening or its part is qualified to one of four degrees of rock burst risk (i.e., none, low, medium, and high) [2].

In addition, when designing mining works, the analytical state of stress in the rock mass is usually calculated [3,4,5,6]. This allows us to estimate the amount of stress in the rock mass near structures such as edges and pillars in other seams, faults, seam splits, etc. There is a growing interest in modeling rock and support behavior in correlation with mining activities [7,8]. Although the phenomenon of rock bursts in coal mines has been studied for many years [9,10,11,12], it is still very challenging as many parameters and the nonlinear relationship between them influence this.

Machine learning algorithms are used increasingly in many fields including mining (e.g., for mining-induced seismicity [13,14,15] and mining subsidence [16,17]) and in mining geology [18,19]. This is due to their ability to solve nonlinear problems. Attempts were also made to apply these algorithms directly to assess the risk of rock bursts in mines of various rocks and minerals including coal mines.

1.1. Current Use of Machine Learning Algorithms for Rock Burst Prediction

The prediction of rock bursts using machine learning algorithms provides an alternative approach adopted by many researchers, which is based on learning by experience. Machine learning algorithm models are trained without the need to determine the cause of rock bursts. Supervised machine learning algorithms have mainly been used to classify stress rock bursts into appropriate intensity levels. This is understandable because rock bursts of this type most often occur during the underground mining of various mineral resources and tunneling. The parameters taken into account mainly concern the stress level and strength parameters of rocks in which the opening is drilled such as the maximum tangential stress, the uniaxial tensile strength, the uniaxial compressive strength, and the elastic energy index. For example, Bai et al. [20] trained an artificial neural network model based on 20 rock bursts that occurred during tunneling. Chen et al. [21] applied artificial neural networks for predicting rock bursts and their intensity, and the total database included 18 cases. Ge and Feng [22] proposed the use of artificial neural networks as weak classifiers in the AdaBoost algorithm to categorize and predict rock bursts and considered 36 rock burst cases. Su et al. [23] applied the k-nearest neighbor algorithm for rock burst prediction at a great depth in South Africa. Su et al. [24] proposed a rock burst intensity prediction model based on Gaussian process (GP). The model was trained on thirteen cases and tested on five cases of rock bursts during geoengineering works in different countries. Gong et al. [25] used a Bayes discriminant analysis method to predict the possibility and classification of rock bursts, and the database included 14 cases. Zhou et al. [26] used the support vector machine (SVM) algorithm for rock burst prediction. Their database contained 132 rock bursts in openings drilled in various rocks (e.g., granite, sandstone, diorite, rhyolite). Jia et al. [27] proposed the particle swarm optimization (PSO) algorithm and the general regression neural network (GRNN). The database contained 26 cases. Adoko et al. [28] combined neural networks with fuzzy logic for predicting the intensity of rock bursts, and the dataset consisted of 174 rock bursts. Zhou et al. [29] compared the ability of the following machine learning algorithms to predict rock burst categories: linear discriminant analysis, quadratic discriminant analysis, partial least-squares discriminant analysis, naïve Bayes, k-nearest neighbor, multilayer perceptron neural network, classification tree, support vector machine, random forest, and gradient-boosting machine, and their dataset included 246 rock bursts. Li et al. [30] applied Bayesian networks for rock burst prediction, and their rock burst database contained 135 cases. Shrihani Faradonbeh and Taheri [31] applied genetic algorithm-based emotional neural network (GA-ENN), gene expression programming (GEP), and decision tree-based C4.5 algorithm to predict the occurrence of rock burst in a binary condition and their dataset included 134 rock bursts. Li and Jimenez [32] used the logistic regression classifier for the long-term prediction of rock burst. The database contained 135 cases (i.e., 83 rock bursts and 52 non-rock burst events). Pu et al. [33] applied a decision tree model to predict rock burst potential in kimberlite at an underground diamond mine. The database contained 132 rock bursts including 108 complete samples and 24 incomplete samples. Zheng et al. [34] applied the entropy weight gray relational backpropagation (BP) neural network to predict the intensity grade of rock burst and the database used to train the model contained 20 rock bursts. Ghasemi et al. [35] applied a decision tree algorithm to predict the occurrence and intensity of rock burst and the database contained 174 rock bursts. Ke et al. [36] developed naïve Bayes (NB) models and optimized them using four weight optimization methods (i.e., forward, backward, particle swarm optimization, and evolutionary). The database contained 134 rock bursts from different projects (e.g., hydropower plants, underground coal and metalliferous mines, and powerhouse stations). Shukla et al. [37] utilized XGBoost, decision tree, and support vector machine to predict the occurrence of rock burst in various underground projects. A total of 134 rock burst events were gathered together from various published studies. Ullah et al. [38] employed t-distributed stochastic neighbor embedding (t-SNE), K-means clustering, and XGBoost to predict the short-term rock burst risk. A total of 93 rock burst patterns with six influential features from microseismic monitoring events of the Jinping-II hydropower project in China were used. Li et al. [39] combined feature selection (FS), t-distributed stochastic neighbor embedding (t-SNE), and Gaussian mixture model (GMM) clustering for data preprocessing and the dataset contained 344 rock burst cases.

1.2. Current State of Research for Hard Coal Mines

Machine learning algorithms have also been used to predict rock bursts and their intensity, strictly in underground coal mines. For example, Sun et al. [40] proposed a fuzzy neural network risk prediction model for rock burst trained with the improved backpropagation algorithm. There were 10 input parameters representing conditions in the openings. The model was trained on 16 samples and tested on 10 samples. Shi et al. [41] developed a rock burst prediction model based on the BP neural network. Eight input parameters were used representing geological and mining conditions and mining technology. The model was trained on seventeen cases and tested on six cases. Wojtecki et al. [42] classified cases of rock bursts and non-destructive tremors in one of the active underground coal mines. The database contained 150 samples, and the percentage of rock bursts was approximately 35%. These samples were characterized by 25 input parameters. Seven parameters described the geological conditions, ten parameters defined the mining conditions, and three parameters described the technical/technological conditions. There were also two seismic parameters and three parameters describing the rock burst hazard. The models of multilayer perceptron classifier and decision tree were the most effective. Wojtecki et al. [43] reduced the number of parameters in the rock burst database and replaced the geological and mining parameters with the rock mass bursting tendency index W_TG and the anomaly of the vertical stress component in coal seams, respectively. A total of eleven input parameters were used. The thickness of the seams, two seismic parameters, three technical/technological parameters, and three parameters describing the seismic hazard were left. Again, the models of decision tree and multilayer perceptron classifier classified the rock bursts best. Miao et al. [44] proposed a rock burst prediction index system containing 16 indicators. A total of 118 cases were used as sample data. However, the Boruta-random forest model had better predictive performance than the others [44].

Only about 1% of tremors with the energy that could lead to a rock burst in the opening end up in this way [45]. In hard coal mines in the USCB, the number of rock bursts is significantly smaller compared to the number of strong tremors that can lead to a rock burst. In recent years, the ratio of the number of rock bursts—classified by Polish mining regulations—to the number of strong tremors (i.e., tremors with energy greater than or equal 1 × 10⁵ J, so with a local magnitude M_L ≥ 1.68) in coal mines in the USCB was as follows: ~0.13% (2018), ~0.32% (2019), ~0.07% (2020), and ~0.13% (2021). It should be noted that in underground coal mines located in the USCB, there are also dynamic phenomena that are less destructive (i.e., they do not cause a loss of functionality of the openings, or they cause a slight loss of their functionality or a slight deterioration in the safety of their use). Such phenomena are not classified as rock bursts in official statistics. However, the number of all dynamic phenomena, regardless of the extent of destruction and damage to underground mine openings (i.e., classified as a rock burst or not), is still very small compared to non-destructive strong tremors. The prepared database at least partially tries to reflect this imbalance in the number of destructive and non-destructive phenomena in mining openings.

Based on the selected geological, geomechanical, mining and technical parameters as well as the tremor energy and the correlated peak particle velocity (PPV), an attempt was made to classify rock burst cases and non-destructive tremors in one of the active hard coal mines in the USCB. The rock mass proneness to rock bursts was assessed via the GEO system [46]. All geological and geomechanical parameters were included in the GEO index [46].

2. Materials and Methods

2.1. Characteristics of the Database

Parameters related to rock bursts in the underground openings of coal mines or preventing their occurrence were selected. The database contained a total of 627 samples, of which 53 were cases of rock bursts (labelled as 1), and in the remaining cases (labelled as 0), despite the tremors, the excavation was not damaged or destroyed. All dynamic phenomena that led to any damage to the opening were classified as rock bursts. Thus, samples labelled as 1 include all cases of damage or destruction of underground infrastructure on a local scale or with a range of several dozen to several hundred meters (e.g., floor uplift, damages of support (steel arches, mesh, props), and coal ejection into the opening). In this way, the definition of a rock burst was extended, which, according to Polish mining regulations, is considered only to be a case of complete loss of functionality by an opening.

Samples in the database represent points in underground mining excavations for which geological, geomechanical, mining and technical conditions were determined and correlated with the occurrence of tremor with known energy and the corresponding value of peak particle velocity. These data come from one of the hard coal mines in Poland, in the USCB. The risk of rock bursts in the selected mine remains high due to, for example, complex geological structure of the deposit (thick layers of sandstone, faults, folds, washouts etc.), mining remnants (edges of coal seams, pillars), and the proneness of coals to accumulate strain energy and burst. In the selected mine, coal seams are extracted using a longwall system with caving. Mining is carried out at large depths, exceeding even 1 km. Such large depths determine a high level of primary stress, which also adversely affects the occurrence of rock bursts. The previously mentioned thick sandstone layers are also very unfavorable in the selected mine. The fracturing of these sandstones is a source of strong tremors threatening underground openings. The data collected in the database embraced a period of the last 25 years of mining in the selected mine. Therefore, the geological, mining and technological conditions, rock burst prevention methods, and seismological observations can be considered comparable. For each of the 627 samples in the database, 15 parameters (features) related to the occurrence of rock bursts or their prevention were included.

2.1.1. GEO Index

The potential rock burst hazard is determined while planning new openings, which can be paralleled to a long-term rock burst prediction. First, the mechanical properties of coal and/or surrounding rocks (e.g., the uniaxial compressive strength), are considered. Moreover, places where the risk of rock burst is greater due to the properties of coal or surrounding rocks or the presence of factors affecting the stress level in the rock mass (e.g., edges of adjacent coal seams) are identified. Assessing the geomechanical properties of the excavated rocks is one of the basic methods of assessing the potential rock burst hazards. Numerous studies in this area have been carried out for many years in the mines of the USCB [46,47,48,49,50,51]. However, most of the indicators for assessing the rock burst proneness in coal mines refer mainly to different coal types and rocks surrounding the coal seams. Only a few of them are used to assess the rock mass proneness, with careful consideration of its geological features. Until the 1970s, the natural proneness to rock bursts was assessed in the USCB only in terms of quality [46]. Then, indicators of rock burst proneness were developed based on the pre-failure stress–strain characteristics obtained during the compression of the sample. The first studies concerned the strain energy storage index W_ET [47]. This index was intended for coal. The coal sample is loaded to 81–90% of the breaking stress, unloaded to a value of about 5%, and repeatedly loaded until the sample is destroyed [46]. The W_ET index is the dimensionless ratio of the elastic deformation energy to the irreversibly lost energy during the sample deformation. For the conditions occurring in coal mines in the USCB, an attempt was made to collectively analyze the factors determining the rock mass proneness to rock bursts and develop an appropriate assessment scale. A geological and geomechanical system for assessing rock mass proneness to rock bursts, GEO, has been developed [46]. This system combines many elements that comprise the rock mass structure and properties in the Upper Silesian Coal Basin. For mines in the USCB, a GEO system was proposed to assess the proneness of the entire seam-surrounding rock system to rock burst [46].

All information on geological conditions in the vicinity of selected points in openings has been involved in the GEO index (e.g., depth, coal seam thickness, and distance of the coal seam from the thick layer of sandstone) [46]. This index also involves information about the geomechanical properties of rock mass and the maximum energy of tremors that may occur with a given rock mass structure. In addition, the GEO index contains information about the rock burst proneness of the entire seam-surrounding rock system. It includes W_ET index, rock mass number L_g [48], and rock mass bursting proneness index W_TG [49]. For each parameter, classes are specified that correspond to the points of the GEO system. Finally, the points associated with each parameter are summed up to give a GEO index value. The minimum, maximum, and average values of the GEO index in the database equaled 44, 134, and 101, respectively. The maximum number of points allowed in the GEO system (i.e., 134) was reached in about 2% of the samples. According to the scale for assessing the natural proneness of a rock mass to rock bursts in the GEO system, for approx. 3.7% of the samples, the rock mass was not prone to rock bursts (GEO index below 68); for approx. 30.8% of the samples, the rock mass was slightly prone to rock burst (GEO index between 68 and 90); and for approx. 65.5% of the samples, the rock mass was strongly prone to rock bursts (GEO index higher than 90). The distribution of the GEO index values, with division into rock bursts (1) and cases without damage and destruction in the openings (0), is shown in Figure 1a. Considering only rock bursts in the dataset, the calculated GEO index ranged from 75 to 134 (mean 109). About 80% of the rock bursts in the dataset occurred in rock mass strongly prone to rock bursts. The remaining approx. 20% of rock bursts occurred in rock mass slightly prone to rock bursts. This may result from extending the rock burst criterion to any excavation damage. Considering the lack of a formal rock burst limit, such an approach is experimental for Polish mines and brings a new quality. The GEO system was designated for cases of rock burst, which, according to the mining regulations, meant significant damage to the openings and loss of functionality.

2.1.2. Seismic Energy and PPV

This study aimed to determine whether, given the geological, mining, and technical conditions occurring in a selected point of the opening, a tremor of a certain energy, generating peak particle velocity at this point, leads to a rock burst or not. Rock bursts in coal mines in the USCB are mostly correlated with strong tremors (i.e., energy greater than or equal to 1 × 10⁵ J). However, there have also been cases of rock burst after medium-energy tremors( i.e., with an energy of 10⁴ J). Low-energy tremors, mainly of the upper range of energy around 10³ J, may also be associated with damage to the openings. Such cases mainly concern drilled headings. Selected excavation points included in the database were associated with tremors of energies from 8 × 10² J (M_L = 0.58) to 6 × 10⁸ J (M_L = 3.67). Samples related to strong tremors accounted for approximately 94% of the database. Low- and medium-energy tremors each accounted for approximately 3% of the database.

Numerous studies have shown the relationship between high values of PPV and the occurrence of rock bursts. Such studies have been conducted in mines in the USCB [52,53,54]. The PPV values for rock bursts in USCB coal mines usually range from about 0.05 m/s to 1 m/s [54]. When mining activities are conducted in areas of concentrated stress, excavation damage may occur at lower values of PPV [54]. There is a high risk to the stability of openings if the PPV value exceeds 0.4 m/s [52]. The PPV value can be measured directly in the openings or calculated using the empirical formula [54]:

$$log\left(\mathrm{P}\mathrm{P}\mathrm{V}·R\right)=0.66 log\left(M_o\right)-7.4$$

(1)

where PPV is expressed in m/s, R is the hypocentral distance in [m], and M_o is the scalar seismic moment in [Nm].

PPV values, calculated based on Formula (1) for individual samples in the database, ranged from about 0.006 m/s to about 0.77 m/s (mean 0.08 m/s). Considering only rock bursts, the calculated PPV ranged from about 0.043 m/s to about 0.77 m/s (mean 0.15 m/s). The distribution of tremor energies and PPVs, with division into rock bursts (1) and cases without damage and destruction in the openings (0), is illustrated using violin plots and presented in Figure 1b and Figure 1c, respectively. The values of the parameters presented in Figure 1a–c for rock bursts and non-destructive tremors were in the similar range. However, a high value of a given parameter does not mean that a rock burst is certain.

2.1.3. Mining Parameters

A total of nine mining parameters related to the occurrence of rock bursts or preventing this phenomenon were selected. Six of them can be considered as unfavorable, in other words, the presence of the mining remnant, the vertical distance between the opening and mining remnant (in meters), the time since the remnant was left (in years), the neighborhood of goafs in the same coal seam, excessive bed cutting, and other negative factors, relatively rare in the analyzed mine (e.g., change of mining direction during longwall mining, drilling the heading across the bedding of a seam, split or folding of coal seam etc.). The mining remnant, left due to an earlier extraction of another coal seam, causes a stress increase in the rock mass [1]. Three values of this parameter were assumed: none (63% of samples), edge of the coal seam (32.5% of samples), and residual coal pillar (4.5% of samples). The presence of additional stress in the coal seam is an important factor determining the occurrence of rock bursts. Therefore, the vertical distance of the mining remnant from the excavation is important. According to Polish mining regulations, the occurrence of the remnant in other coal seams must be considered when its vertical distance is between 160 m above and 60 m below the opening. Among the samples in the database, this distance ranged from about 316 m above and about 106 m below the opening (mean 103 m above the opening). The stress increase due to the presence of mining remnants should fade over time. However, the increase in stress due to remnants can persist for several decades [1].

The pressure of the rock layers above the boundary of goafs may increase stress in the unmined coal body. An opening drilled in the vicinity of goafs may be influenced by additional stress and thus may have a higher level of rock burst hazard. A neighborhood of goafs was assumed when the distance between the excavation and boundary of goafs did not exceed 50 m. A neighborhood of goafs occurred in about 39% of the samples in the database. Another negative parameter is excessive bed cutting. Drilling several excavations at a short distance from each other causes weakening of the coal body and may lead to a rock burst. Samples with excessive bed cutting (i.e., where at least three excavations were next to each other) accounted for 6.5% of the database.

The next three mining parameters were related to the earlier mining of the adjacent coal seam, which can significantly reduce the rock burst hazard. These parameters include the vertical distance to the extracted adjacent coal seam, the thickness of this seam, and the time since the mining of the adjacent coal seam was undertaken. In this article, such a previously extracted coal seam was referred to as a destressing coal seam, ignoring its vertical distance. According to Polish mining regulations, a coal seam is classified as a lower rock burst hazard degree if a destressing coal seam, deposited not more than 50 m below or 20 m above, is mined with caving.

In the case of a thick coal seam, the destress effect is achieved by mining one of its layers. In general, the destress effect should be greater if the destressing seam is closer and thicker and a shorter time has elapsed since it was excavated. The parameters correlated with the so-called destressing seam were determined for about 71% of the samples in the database. The vertical distance between the excavation and the destressing seam for these samples ranged between 0 m (upper layer of thick coal seam) and about 309 m (mean 75 m). Therefore, the distance criterion of the destressing seam was met by only about 38.6% of these samples. The thickness of the destressing seam was between 1 m and about 4 m (mean 2.5 m), and the time elapsed since destressing mining ranged between 1 year and 34 years (mean 15 years). In the case of the remaining 29% of samples, other seams were certainly mined above them, but their vertical distance and the time that had elapsed since their extraction were large enough to be completely ignored.

2.1.4. Technological Parameters

Three parameters related to the technology of works were also selected (i.e.,: type of excavation, reinforcement of the support, and rock burst prevention). Based on the first of the mentioned parameters, the samples were classified as longwall galleries (90.4% of samples), drilled headings (7.4% of samples), and opening-out headings (2.2% of samples). Such an uneven share results from rock bursts occurring mainly in longwall galleries. Advancing the longwall face generates mining pressure in the unmined coal body, so a higher stress level can be expected close to the main gate and tailgate. Drilled headings are also at risk of rock bursts. Coal may be ejected from the exposed working face directly into the opening, and the possibilities to protect against it are limited. Opening-out headings have a transport and ventilation function. These excavations are mostly old and located in protective pillars with high-stress levels. Steel arches are the basic support for openings. This support can be reinforced, which may prevent rock bursts or minimize excavation damage. Support reinforcement was applied in 90.9% of the samples in the database. Such a large share resulted from the fact that the dataset came from a mine with a high risk of rock bursts, where support reinforcement, mainly by props, is commonly employed. This applies, in particular, to openings with seismic activity and an increased risk of rock burst. The share of types of support reinforcement is as follows: props (97.2%), props and bolts together (1.2%), wooden support (0.9%), steel horseheads (0.35%), and props and steel horseheads and reduced spacing between steel arches together (0.35%). In the selected mine, active rock burst prevention is used on a large scale, based mainly on destress blasting in coal seams and roof rocks. Destress blasting for rock burst prevention was performed for 57.9% of the samples in the database. In 54.8% of them, long-hole destress blasting in roof rocks was executed together with destress blasting in coal seams; in 40% of these samples, long-hole destress blasting in roof rocks was performed independently; and in 5.2% of them, only destress blasting in coal seams was executed.

2.1.5. Feature Correlation

The fifteen parameters presented above were generally not or slightly correlated with each other. The Spearman correlation coefficient was used as a measure. This coefficient shows any, including the nonlinear, monotonic relationship between two variables, ranging between −1 and 1. The sign of the Spearman correlation coefficient indicates whether it is a positive or negative monotonic relationship. Only a few parameters in the database were strongly positively correlated, for example, the distance from the remnant and the time since it was left, and the distance from the destressing seam and the time that had elapsed since its extraction. The Spearman correlation coefficient in these cases equaled 0.97 and 0.7, respectively. Such a correlation resulted from the fact that shallower seams are usually excavated earlier than those deposited deeper. The energy of the tremor and the associated PPV were moderately positively correlated, with the Spearman correlation coefficient being 0.53. This is somewhat understandable because the PPV is proportional to the energy of the tremor. Of course, distance from the tremor focus and the directionality of seismic wave radiation must also be considered. Considering only rock bursts classified as such following the applicable mining regulations, the Spearman correlation coefficient between the GEO and tremor energy was 0.44.

The correlation between the fifteen distinguished parameters and the occurrence of rock bursts in the database was also determined. It should be noted that rock bursts accounted for approximately 8.5% of the dataset. In most of the remaining cases, despite high PPV values and the rock mass’s proneness to rock bursts, rock bursts did not occur. This does not prove the lack of usefulness of the analyzed parameters, but a complicated and nonlinear dependence of the occurrence of rock bursts. Strong tremors are often not associated with rock bursts in underground mining openings, even when drilled in rock masses prone to rock bursts. Moreover, in the case of the GEO system, it should be emphasized that it was created based on rock bursts classified according to the Polish mining regulations. Individual classification criteria in the GEO system have been adopted for rock bursts regarding dynamic phenomena caused by tremors. As a result, openings lost their functionality and thus were significantly damaged or destroyed. In this article, even local damage to the excavation was considered a rock burst. In an excavation drilled in a rock mass prone to rock bursts, rock bursts do not have to occur due to a strong tremor. Moreover, such cases are rare in comparison to cases of non-destructive tremors. Additionally, reinforcement of the support or a destressing effect due to blasting can prevent a rock burst, even in rock mass prone to rock burst.

The highest value of the Spearman correlation coefficient was for PPV (0.26), followed by the type of excavation (0.22), and GEO index (0.12). The Spearman correlation coefficient for a neighborhood to goafs, excessive bed cutting, and reinforcement of the support was 0.11. For the remaining nine parameters, the absolute value of the Spearman correlation coefficient did not exceed 0.1. Negligible negative and positive monotonic relationships were found for five and four features, respectively. A RadViz plot was used to illustrate the cases of rock bursts and cases without destruction and damage to mining openings against the background of the distinguished features with normalized values (Figure 2). The RadViz plot enables the graphical presentation of multidimensional data. In this way, it is possible to determine existing regularities, identify clusters of similar objects, and indicate unusual observations. The separateness of the features was visualized this way.

The dataset was highly imbalanced. Using such data may have resulted in the incorrect classification of cases being in the significant minority. To balance the dataset, the adaptive synthetic (ADASYN) algorithm was used. In this algorithm, the oversampling of minority samples takes place. This algorithm generates synthetic data but does not just copy the minority samples. Instead, synthetic samples with parameters similar to real samples are generated, especially in the vicinity of the majority of samples. In the ADASYN algorithm, k-nearest neighbors are found for each minority sample, and the majority class’s dominance in its neighborhood is determined. As a result, more synthetic minority samples are generated for neighborhoods dominated by majority-class samples.

2.2. Machine Learning Algorithms

Machine learning is a branch of artificial intelligence that demonstrates intelligent behavior by finding relationships in the training data. Machine learning algorithms differ from classical algorithms, which are composed of a specific sequence of instructions. Instead, they create a model based on analyzing the provided training data. The process of creating knowledge about a solution is called learning. The three most frequently mentioned categories of machine learning algorithms are supervised, unsupervised, and reinforcement learning [55,56]. Supervised learning involves identifying a function that predicts future outcomes based on the data containing both the input features and expected outcomes. Predictive models receive instructions on how and what to learn. Supervised machine learning algorithms include, among others, artificial neural networks, adaptive boosting, gradient boosting, extreme gradient boosting, K-nearest neighbors, naïve Bayes, support vector machines, decision trees, and random forests. In unsupervised learning, the training set only consists of the input features and does not include labels (expected outcomes). Unsupervised machine learning algorithms look for patterns based on the provided examples. This group includes K-means, independent component analysis (ICA), and principal component analysis (PCA). Reinforcement learning, a trial-and-error method, occurs in a completely unknown, often dynamic environment. The algorithm uses feedback from its decisions and learns from its experiences by receiving a positive (reward) or negative reinforcement signal (penalty). Q-learning and SARSA are examples of reinforcement learning.

This article demonstrates the use of selected supervised machine learning algorithms such as artificial neural network, adaptive boosting, gradient boosting, K-nearest Neighbors, and naïve Bayes to assess potential rock burst hazards in underground openings of an active coal mine. All calculations were performed in the JupyterLab environment [57,58]. All scripts were coded in Python using the available machine learning library Scikit-learn (sklearn) [59]. The optimal hyperparameters of each machine learning algorithm were determined using a grid search approach. In the vast majority of cases, the standard values proposed in the machine learning library Scikit-learn were adopted.

2.2.1. The Multilayer Perceptron Classifier

The artificial neural network (ANN) is a computational model designed to simulate the network of neurons that make up a human brain. McCulloch and Pitts [60] proposed a mathematical model of a neuron. Hebb [61] described the principles of neural network learning. Subsequently, Rosenblatt [62] created a perceptron, a simple neural network made of independent McCulloch–Pitts neurons. It was innovative to apply the learning process as a method of programming. Minsky and Papert [63] proved that perceptrons could not learn linearly inseparable functions such as XOR, which led to a decreased interest in perceptrons. This situation persisted until Rumelhart et al. [64] proposed the backpropagation algorithm (BP). The use of a hidden layer in the neural network enabled nonlinear calculations. LeCun et al. [65] used neural networks with backpropagation, where layers were constrained to be convolutional.

One of the most commonly used types of unidirectional artificial neural networks is a multilayer perceptron (MLP). The multilayer perceptron classifier (MLPC) can classify datasets that are not linearly separable. Such a neural network was also used for the study presented in this article. The neurons in the MLP network are connected between layers peer-to-peer, with no connections between neurons within the same layer. There are three basic types of layers in the MLP network: input, hidden, and output. In the MLP network used, five hidden layers were applied. The more hidden layers a network has, the deeper dependencies it can find. The connections between neurons are assigned weights established during the network’s training. These weights help determine the importance of each variable; larger weights have a greater impact on the output than other inputs. All inputs are then multiplied by their respective weights and then added together. The results are then passed through an activation function, determining the output value. The role of the activation function is to introduce nonlinearity in the data processing path. Its output value is forwarded to the next layer in the network. Neurons in hidden layers use activation functions such as the rectified linear unit (ReLU) function, logistic sigmoid function, and hyperbolic tangent (Tanh) function. The ReLU function outputs the input directly if it is positive; otherwise, it returns zero. This was used for hidden layers in the constructed MLP network. The most common activation functions in the output layer are linear, logistic, and Softmax. The Softmax function, often used in the output layer of a multilayer perceptron adapted to solve classification problems, was used in the constructed MLP network. The Softmax is an exponential function whose value is normalized so that the sum of the activations for the entire layer equals one. According to this normalization, the output values can be interpreted as the probabilities of a given input signal belonging to particular classes [66].

The training of MLP networks is possible thanks to the backward error propagation method [64,67]. In this method, the error signal is calculated from the output layer, through hidden layers, toward the input layer. Then, the minimization of the sum of squares (or other function) of the learning error using the gradient descent optimization method takes place. The weights of connections between processing elements located in adjacent layers of the network are adjusted in this way. A stochastic gradient-based optimizer, ADAM [68], was applied to improve the weights in the trained MLP network. A constant learning rate (step size in updating the weights) of 0.001 was used. The input and output layers comprised fifteen and two neurons, respectively. The hidden layers between them contained fifteen, twelve, nine, seven, and four nodes, respectively.

2.2.2. Adaptive and Gradient Boosting Classifiers

Boosting is a general method to increase the effectiveness of any learning algorithm. However, it is often used with decision trees. The decision tree is a machine learning algorithm that divides the dataset into smaller subsets according to their features. The splitting process continues until all the remaining data belong to only one class. The boosting method belongs to ensemble learning, where subsequent classifiers of the same algorithm (e.g., decision tree) are trained, and an ensemble classifier is created based on them. Kearns and Valiant [69] discussed whether a weak learning algorithm, whose efficiency is slightly better than random guessing, could create an effective strong classifier. The boosting method was introduced by Freund and Schapire [70]. There are three main types of boosting algorithms: adaptive boosting (AdaBoost), gradient boosting (GB), and extreme gradient boosting (XGBoost). The first two were selected for use in this article.

The AdaBoost algorithm was proposed by Freund and Schapire [71]. This algorithm initially sets the same weights for all samples in the training dataset. Next, a new classifier is built based on the training dataset, considering the sample weights. This new classifier is added to the ensemble classifier, with a weight corresponding to its accuracy. Finally, the sample weights are updated; correctly classified samples receive a lower weight, and incorrectly classified samples receive a higher weight. In the AdaBoost algorithm, each subsequent classifier tries to classify samples that the previous ones misclassified. The final prediction in the AdaBoost algorithm is based on a majority vote of the weak classifiers’ predictions weighted by their accuracy. The entire process is repeated until the residual error, or the difference between the actual and predicted values, falls below the established threshold. The AdaBoost algorithm used in this article was based on an elementary form of decision tree known as stumps (i.e., the maximum depth of these trees equaled 1). The maximum number of decision trees at which boosting was terminated equaled 50. Stagewise additive modeling using a multiclass exponential loss function for real (SAMME.R) was applied during model training. This algorithm uses the class probabilities during training.

The gradient boosting algorithm was developed by Friedman [72], and is similar to the AdaBoost algorithm in that it is also based on a sequential training technique. All classifiers influence the final decision regarding the classification of individual samples (e.g., decision trees) used to build the algorithm model. However, this algorithm does not increase the weights of incorrectly classified samples. The contribution of the weak classifier to the ensemble classifier is based on the gradient descent algorithm, an iterative optimization algorithm that minimizes the loss function. The subsequent classifiers are maximally correlated with the negative gradient of the loss function associated with the whole ensemble classifier. The GB algorithm used in this article was based on decision trees, and the number of boosting stages was 100. The log loss function was optimized. The maximum depth, which limits the number of nodes in each tree, equaled 3. The learning rate, which determines the share of each tree in the ensemble classifier, equaled 0.1. Such a small learning rate value requires more trees but has better generalization ability. This regularization method is referred to as shrinkage. The Friedman mean square error (Friedman mse) criterion was applied to measure the quality of a split in decision trees.

2.2.3. k-Nearest Neighbors Classifier

The K-nearest neighbors (KNN) classifier is one of the most widely used classifiers in supervised machine learning. It was introduced by Fix and Hodges [73] and developed by Cover [74]. The KNN classifier belongs to the family of “lazy learning” classifiers, which means that it only stores the training dataset and does not undergo training. All calculations take place when a classification is being made. This classifier predicts that a given sample belongs to the class with the largest k of its nearest neighbors. It is assumed that samples with similar parameters belong to the same class. The KNN algorithm first determines the k nearest neighbors for a given sample based on a distance metric (e.g., Euclidean, Manhattan, Minkowski, or Hamming). Then, a majority vote is taken among the selected k neighbors of the analyzed sample and is assigned to the dominant class in its surroundings. In this article, the Minkowski distance was used, which is expressed by the formula:

$$d_{Minkowski}={\left({\sum }_{i=1}^n{\left|x_i-y_i\right|}^p\right)}^{\frac{1}{p}}$$

(2)

where x_i and y_i are the samples for which the distance is calculated, and the value of the hyperparameter p equaled 2 (Euclidean distance measure). Moreover, k = 5 was adopted as the number of nearest neighbors for classification, and all were weighted equally.

2.2.4. Gaussian Naïve Bayes Classifier

The naïve Bayes classifier is a probabilistic classifier based on Bayes’ theorem. The probability that the sample belongs to class (label) c takes into account the feature values x₁, x₂, … x_m (i.e., the posterior probability), and can be calculated according to the following formula:

$$P\left(c|x_1, x_2,\dots x_m\right)=\frac{P\left(x_1, x_2,\dots x_m|c\right)P\left(c\right)}{P\left(x_1, x_2,\dots x_m\right)}$$

(3)

where P(x₁, x₂, … x_m|c) is a probability of feature values x₁, x₂, … x_m taking into account that class c, P(c) is a class prior probability (i.e., the ratio of the number of samples of a given class c to the number of all samples in the database), P(x₁, x₂, … x_m) is a marginal probability (i.e., probability of feature values x₁, x₂, … x_m occurring across all possible classes in the database). Since the denominator in Formula (3), in other words, the marginal probability does not depend on the class, it does not change the classification result. It is assumed that the features are independent in the naïve Bayes classifier. This assumption is often incorrect, but it does not cause difficulties in preparing an effective classifier in practice. An assumption about the independence of feature values from within a class leads to simplification:

$$P\left(x_1, x_2,\dots x_m|c\right)\approx {\prod }_{k=1}^{m}P\left(x_k|c\right)$$

(4)

A statistical probability distribution can be assumed for each feature in a database. The most commonly assumed types of distributions are normal (Gaussian), polynomial, and Bernoulli. In this article, it was assumed that the features had a Gaussian distribution. Hence, the used classifier was Gaussian naïve Bayes (GNB). The distribution density function was in the form of the normal distribution:

$$P\left(x_k|c\right)=\frac{1}{{\sigma }_c\sqrt{2\pi }}{e}^{-\frac{{\left(x_k-{\mu }_c\right)}^2}{2{\sigma }_c^2}}$$

(5)

where µ_c is the mean value of the feature k for vectors belonging to the class c, and σ_c is the standard deviation of this mean.

The application of a naïve Bayes classifier amounts to the calculation of the posterior probabilities for each class and finally classifying the sample to the class for which the value of the calculated probability is the highest:

$$c_{NB}=\arg \underset{\mathit{cϵC}}{\max }P\left(c\right){\prod }_{k=1}^{m}P\left(x_k|c\right)$$

(6)

In the applied Gaussian naïve Bayes, the priors were adjusted according to the data.

2.2.5. Ensemble Classifier

In many cases, the collective answer is better than a single expert’s answer. This phenomenon is known as the wisdom of the crowd, and it can also be used when the predictions of a group of classifiers are combined. Very often, a group of classifiers predicts better than a single one, even the most efficient classifier. Such a group of classifiers is called an ensemble classifier. This type of classifier can be created based on a group of classifiers of one type (e.g., decision trees) for each separate subset of the training set. An example of one of the commonly used ensemble algorithms is a random forest, built from a set of decision trees, in which the final classification is determined by majority voting. Ensemble learning is also used in boosting algorithms, as mentioned earlier.

An ensemble classifier is most effective when the individual component classifiers are as independent as possible. One way to achieve this is to train the component classifiers of ensemble classifiers using different machine learning algorithms. This increases the probability that the component classifiers will not make the same errors, leading to greater efficiency of the ensemble classifier. In this article, after training, the MLP, AdaBoost, GB, KNN, and GNB classifiers, they were combined into one ensemble classifier. Hard voting was used (i.e., predictions calculated by individual classifiers were combined), and finally, the class that received the most votes was chosen.

3. Results

Based on the train–test split function, the dataset was randomly divided into training and test data in an 80% to 20% ratio. The training dataset was pre-processed using a standard scaler algorithm (i.e., the features were standardized by removing their mean and scaling them to unit variance). Then, the ADASYN algorithm was applied to balance the training dataset. The scaling function determined for the training dataset was used to scale the test dataset. In the manner described above, ten independent sets of training and test data were prepared. The size of randomly selected datasets is shown in

Table 1. The size of randomly selected datasets.

Dataset	Training Dataset		Test Dataset
	P	N	P	N
1	42 (462)	459	11	115
2	38 (457)	463	15	111
3	40 (465)	461	13	113
4	43 (449)	458	10	116
5	44 (465)	457	9	117
6	44 (463)	457	9	117
7	40 (462)	461	13	113
8	43 (462)	458	10	116
9	44 (468)	457	9	117
10	39 (474)	462	14	112

. P stands for the number of rock bursts, and N stands for the number of non-destructive tremors. The value in brackets indicates the number of rock bursts after balancing the training dataset. Ten models of each of the mentioned machine learning algorithms (i.e., MLP, AdaBoost, GB, KNN, GNB classifiers, and ensemble classifier) were trained and tested on these datasets.

The use of binary classification (i.e., positive for rock bursts and negative for cases without damage and destruction to the openings) enabled the use of basic model quality assessments such as true positive (T_P), true negative (T_N), false positive (F_P), and false negative (F_N). The T_P and T_N are the numbers of correctly classified positive and negative cases, respectively. The F_P is the number of samples incorrectly classified as positive but actually negative, representing the number of false alarms. In the case of F_N, the situation is reversed and concerns the underestimation of positive cases. The model efficiency parameters were calculated based on the mentioned basic parameters such as accuracy, recall (sensitivity, true positive rate), precision (positive predictive value), and F1-score. The first of these parameters is the ratio of all correctly classified samples to all samples in the dataset:

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}=\frac{T_P+T_N}{T_P+T_N+F_N+F_P}$$

(7)

The recall is the ratio of T_P to all positive cases in the dataset. It can be calculated as follows:

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}=\frac{T_P}{T_P+F_N}$$

(8)

Precision describes the degree to which positive predictions can be trusted. It is the ratio of T_P to all cases classified by the model as positive:

$$\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}=\frac{T_P}{T_P+F_P}$$

(9)

F1-score is the harmonic mean of precision and recall, which are assumed to be equally important. F1-score is calculated according to the following formula:

$$\mathrm{F}1\text{-}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=2\frac{\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}·\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}{\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}+\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}}=\frac{2T_P}{2T_p+F_N+F_P}$$

(10)

The efficiency characteristics of the models of selected classifiers: MLP, AdaBoost, GB, KNN, GNB, and ensemble hard-voting classifier are presented in

Table 2. Efficiency of MLPC models.

Dataset	TP	TN	FP	FN	Accuracy	Recall	Precision	F1-Score
1	6	112	3	5	0.94	0.55	0.67	0.60
2	12	106	5	3	0.94	0.80	0.71	0.75
3	6	108	5	7	0.91	0.46	0.55	0.50
4	6	110	6	4	0.92	0.60	0.50	0.55
5	6	109	8	3	0.91	0.67	0.43	0.52
6	6	109	8	3	0.91	0.67	0.43	0.52
7	9	106	7	4	0.91	0.69	0.56	0.62
8	7	112	4	3	0.94	0.70	0.64	0.67
9	5	108	9	4	0.90	0.56	0.36	0.43
10	7	106	6	7	0.90	0.50	0.54	0.52
mean					0.92	0.62	0.54	0.57

,

Table 3. Efficiency of AdaBoost classifier models.

Dataset	TP	TN	FP	FN	Accuracy	Recall	Precision	F1-Score
1	7	110	5	4	0.93	0.64	0.58	0.61
2	8	101	10	7	0.87	0.53	0.44	0.48
3	8	101	12	5	0.87	0.62	0.40	0.48
4	3	109	7	7	0.89	0.30	0.30	0.30
5	5	103	14	4	0.86	0.56	0.26	0.36
6	6	106	11	3	0.89	0.67	0.35	0.46
7	6	103	10	7	0.87	0.46	0.38	0.41
8	7	108	8	3	0.91	0.70	0.47	0.56
9	6	97	20	3	0.82	0.67	0.23	0.34
10	6	100	12	8	0.84	0.43	0.33	0.38
mean					0.87	0.56	0.37	0.44

,

Table 4. Efficiency of GB classifier models.

Dataset	TP	TN	FP	FN	Accuracy	Recall	Precision	F1-Score
1	6	110	5	5	0.92	0.55	0.55	0.55
2	7	104	7	8	0.88	0.47	0.50	0.48
3	9	106	7	4	0.91	0.69	0.56	0.62
4	6	108	8	4	0.90	0.60	0.43	0.50
5	7	107	10	2	0.90	0.78	0.41	0.54
6	7	109	8	2	0.92	0.78	0.47	0.58
7	8	103	10	5	0.88	0.62	0.44	0.52
8	8	110	6	2	0.94	0.80	0.57	0.67
9	7	104	13	2	0.88	0.78	0.35	0.48
10	4	107	5	10	0.88	0.29	0.44	0.35
mean					0.90	0.63	0.47	0.53

,

Table 5. Efficiency of KNN classifier models.

Dataset	TP	TN	FP	FN	Accuracy	Recall	Precision	F1-Score
1	3	104	11	8	0.85	0.27	0.21	0.24
2	12	99	12	3	0.88	0.80	0.50	0.62
3	10	91	22	3	0.80	0.77	0.31	0.44
4	5	105	11	5	0.87	0.50	0.31	0.38
5	6	106	11	3	0.89	0.67	0.35	0.46
6	6	96	21	3	0.81	0.67	0.22	0.33
7	11	99	14	2	0.87	0.85	0.44	0.58
8	5	108	8	5	0.90	0.50	0.38	0.43
9	5	100	17	4	0.83	0.56	0.23	0.32
10	6	100	12	8	0.84	0.43	0.33	0.38
mean					0.85	0.60	0.33	0.42

,

Table 6. Efficiency of GNB classifier models.

Dataset	TP	TN	FP	FN	Accuracy	Recall	Precision	F1-Score
1	7	82	33	4	0.71	0.64	0.18	0.27
2	12	87	24	3	0.79	0.80	0.33	0.47
3	9	78	35	4	0.69	0.69	0.20	0.32
4	5	96	20	5	0.80	0.50	0.20	0.29
5	6	85	32	3	0.72	0.67	0.16	0.26
6	8	86	31	1	0.75	0.89	0.21	0.33
7	9	87	26	4	0.76	0.69	0.26	0.38
8	7	100	16	3	0.85	0.70	0.30	0.42
9	6	88	29	3	0.75	0.67	0.17	0.27
10	7	99	13	7	0.84	0.50	0.35	0.41
mean					0.77	0.67	0.24	0.34

and

Table 7. Efficiency of ensemble classifier models.

Dataset	TP	TN	FP	FN	Accuracy	Recall	Precision	F1-Score
1	7	109	6	4	0.92	0.64	0.54	0.58
2	13	104	7	2	0.93	0.87	0.65	0.74
3	11	103	10	2	0.90	0.85	0.52	0.65
4	5	109	7	5	0.90	0.50	0.42	0.45
5	7	110	7	2	0.93	0.78	0.50	0.61
6	8	108	9	1	0.92	0.89	0.47	0.62
7	10	103	10	3	0.90	0.77	0.50	0.61
8	8	112	4	2	0.95	0.80	0.67	0.73
9	7	105	12	2	0.89	0.78	0.37	0.50
10	7	105	7	7	0.89	0.50	0.50	0.50
mean					0.91	0.74	0.51	0.60

, respectively.

For a comparison of the trained and tested models of machine learning algorithms, the receiver operating characteristic (ROC) curve was also used (Figure 3). The ROC curve is created by plotting the recall (true positive rate) against the false positive rate, and it illustrates the ability of a binary classifier as its discrimination threshold is varied. The more accurate the model, the more the ROC curve is shifted toward the upper-left corner. The diagonal represents a random classifier. The area under the ROC curve is an indicator describing the effectiveness of the model. The values of the area under the ROC curves for the models of the selected machine learning algorithms are shown in

Table 8. Area under ROC curves for models of the selected machine learning algorithms.

Dataset	MLP	AdaBoost	GB	KNN	GNB	Ensemble Classifier
1	0.76	0.80	0.75	0.59	0.67	0.79
2	0.88	0.72	0.70	0.85	0.79	0.90
3	0.71	0.75	0.81	0.79	0.69	0.88
4	0.77	0.62	0.77	0.70	0.66	0.72
5	0.80	0.72	0.85	0.79	0.70	0.86
6	0.80	0.79	0.86	0.74	0.81	0.91
7	0.82	0.69	0.76	0.86	0.73	0.84
8	0.83	0.82	0.87	0.72	0.78	0.88
9	0.74	0.75	0.83	0.71	0.71	0.84
10	0.72	0.66	0.62	0.66	0.69	0.72
mean	0.78	0.73	0.78	0.74	0.72	0.83

.

During classification, it is not uncommon to struggle between discovering all the positive cases in the data while reducing the erroneous results. However, it is often a process where a balance is sought between two opposing forces. Models should therefore be characterized by both high recall and precision. Figure 4 shows the precision–recall curves for all trained and tested models, and for all datasets. The more accurate the model, the more the precision–recall curve is shifted toward the upper-right corner. The large area under the curve means both high recall and high precision. In

Table 9. Area under precision–recall curves for models of selected machine learning algorithms.

Dataset	MLP	AdaBoost	GB	KNN	GNB	Ensemble Classifier
1	0.63	0.63	0.57	0.28	0.42	0.60
2	0.77	0.52	0.52	0.66	0.58	0.77
3	0.53	0.53	0.64	0.55	0.46	0.69
4	0.57	0.33	0.53	0.43	0.37	0.48
5	0.56	0.43	0.60	0.52	0.42	0.65
6	0.56	0.52	0.63	0.46	0.55	0.68
7	0.64	0.45	0.55	0.65	0.49	0.65
8	0.68	0.60	0.69	0.46	0.51	0.74
9	0.47	0.46	0.57	0.41	0.43	0.58
10	0.55	0.41	0.40	0.41	0.45	0.53
mean	0.60	0.49	0.57	0.48	0.47	0.64

, the areas under the precision–recall curves are shown.

4. Discussion

The risk of rock bursts is still a significant problem in mines located in the USCB and around the world. The article presents the possibility of classifying rock bursts in a selected hard coal mine using examples from the past. The investigation of rock bursts in coal mines in the USCB shows that they mostly occur in longwall galleries (maingates or tailgates) and drilled headings. However, in recent years, rock bursts have also occurred in the opening-out headings, usually in the protective pillars, especially when the longwall mining was ending in their area.

This article presents the results of the classification of rock bursts and non-destructive tremors in an active underground coal mine using machine learning algorithms. The methodology used allowed for the correct classification of rock bursts in the conditions of numerous non-destructive tremors and in the complex geological and mining conditions of an underground coal mine.

Geological, geomechanical, mining, and technical parameters were taken into account. The first two types of parameters were included in one GEO index, adapted to the conditions prevailing in underground mines in the USCB and a rock burst database collected following the Polish mining regulations. The selected points were then correlated with tremors in their vicinity, and the PPV values at these points were calculated. Such a comprehensive use of the selected parameters was applied to assess whether a rock burst may occur at a given point in the opening. However, the presented method could be applied to assess the potential risk of rock bursts for openings in underground coal mines.

Efforts were made to at least partially reflect the real ratio of rock bursts to most cases of tremors that did not cause damage to the openings. The database contained 627 samples. The share of rock bursts in the current database was only about 8.5%. As a result, the dataset was highly imbalanced, and cases of tremors without damage to the openings predominated.

The parameters related to the occurrence of rock bursts did not show a linear relationship (i.e., an increase in a given parameter does not necessarily have to be associated with the occurrence of rock bursts in the opening). Many parameters influence the occurrence of a rock burst in a specific configuration, and a large number of them were included. Balancing the dataset with the ADASYN sampling algorithm was necessary for the correct classification of minority rock burst events.

Since the occurrence of rock bursts and the range of damage are not linear, machine learning algorithms were used. Machine learning algorithms including MLP, AdaBoost, GB, KNN, and GNB classifiers are particularly suitable for solving nonlinear issues. The average accuracy for the models of applied algorithms equaled: 0.92 for MLP, 0.9 for GB, 0.87 for AdaBoost, 0.85 for KNN, and 0.77 for GNB. The average recall of the models equaled 0.67 for GNB, 0.63 for GB, 0.62 for MLP, 0.6 for KNN, and 0.56 for AdaBoost. The average precision of these models ranged from 0.24 (for GNB) to 0.54 (for MLP). The average F1-score of these models was between 0.34 (for GNB) and 0.57 (for MLP).

The use of an ensemble hard-voting classifier consisting of models of the five previously mentioned classifiers was proposed. The average accuracy value of the ensemble classifier models was 0.91, comparable to the highest average accuracy of the single classifier (i.e., 0.92 for MLP). Moreover, the average recall value of the ensemble classifier models was the highest (i.e., 0.74). The average precision of the ensemble classifier models equaled 0.51, and the average F1-score of these models equaled 0.6 and was the highest. The number of false alarms accounted for about 3.5% to 10.3% of all cases without damage to the openings (mean 6.9%). The ensemble classifier models had the highest average area under the ROC curve (i.e., 0.83). Additionally, the highest value of this parameter (i.e., 0.92) was in one of the models of this classifier. The area under the ROC curve for the remaining models ranged from 0.72 to 0.78 on average. The mean value of the area under the precision–recall curve was also the highest for the ensemble classifier models (0.64). However, only half of the ensemble classifier models had the highest area under the precision–recall curve. For the other models, it was smaller (3 cases) or equal (2 cases) in comparison with the single algorithm models.

This article proposed a new approach to assessing the potential state of rock burst hazards in openings. It would be necessary to determine the geological, geomechanical, mining, and technical conditions at the points of the selected opening. By making certain assumptions, the trained models could be used to assess the potential risk of rock bursts in planned mine openings. The maximum predicted energy determined in accordance with mining regulations (e.g., by analytical, empirical or statistical methods) could be assumed as the tremor energy. Some assumptions about the location of the tremors’ foci would also be necessary as this affects the calculated PPV. Usually, strong tremors are associated with the fracturing of a thick layer of sandstone deposited above the seam. Moreover, they usually occur close to remnants and edges in other seams, where rock layers are deformed.

An important issue to consider in machine learning is the availability (quantity) and quality of data. The amount of data affects the generalization efficiency of the model. If there is too little data, especially for complex models, overfitting may occur. The study conducted for the dataset used did not show evidence of this phenomenon. Another limitation is the representativeness of the data. The database used contains records from one coal mine, so the resulting models were adapted to local conditions. Applying them without additional training to other mines may prove problematic. However, some general patterns in the trained models cannot be ruled out either. Moreover, it cannot be ruled out that the effectiveness of the models will deteriorate as the number of non-destructive tremors in the database continues to increase (i.e., the actual ratio of non-destructive tremors to rock bursts approaches).

For some rock bursts and non-destructive tremors, the parameter configurations used were very similar. Considering such cases could be the reason for differences in the effectiveness of the trained models. It cannot be ruled out that using a different parameter configuration would result in a better distinction between rock bursts and non-destructive tremors. It should be noted that the GEO index was created on the basis of a database of rock bursts in accordance with mining regulations (i.e., on the basis of dynamic phenomena causing the loss of functionality of mine openings). In the database used for the study, phenomena causing local destruction and damage of the openings were also considered to be rock bursts. An additional problem is the directionality of energy radiation from the source of the tremor, which depends on the focal mechanism. This can have a significant impact on the PPV value in the opening. Studies of the focal mechanism are not conducted as standard practice in coal mines in the USCB.

5. Conclusions

Models of selected machine learning algorithms were trained to classify rock bursts and non-destructive tremors in a selected hard coal mine. (1) The dominance of strong and non-destructive tremors over rock bursts in the database was considered. (2) The database corresponding to real conditions required balancing, therefore the ADASYN algorithm was used. (3) The GEO index was used as one of the features, describing the proneness of the entire seam-surrounding rock system to rock burst. (4) Rock bursts were best classified by an ensemble classifier composed of MLP, AdaBoost, GB, KNN, and GNB classifiers. On average, almost three-quarters of rock bursts in the test datasets were correctly classified by the ensemble classifier. (5) Developing a machine learning-based rock burst classification could help prevent accidents by alerting miners and engineers in advance of potential rock bursts. However, further research in this area should be continued using various machine learning algorithms and their configurations in the ensemble classifier.