Initial Study into the Possible Use of Digital Sound Processing for the Development of Automatic Longwall Shearer Operation

Abstract

Competition on the local and global market forces enterprises to implement modern solutions and adapt to technological changes. Applying modern solutions allows an increase in the quality of the product and reduces production costs. The acoustic sensor, as a relatively cheap solution, allows signals to be obtained which, after appropriate processing, can be used to develop an automatic control of the longwall shearer, together with the recognition of the type of shale. This paper presents an introductory research, the goal of which has been to check whether acoustic signals carry useful information on what kind of material–shale or coal–is being cut by the cutting head of a longwall shearer. For this purpose, the fast Fourier transform and short-time Fourier transform functions implemented in MatLab were used. The results of the analysis are presented in the form of three-dimensional graphs and spectrograms. To sum up, the research carried out so far justifies the need for continuation in the form of systematic experiments, the results of which could be incorporated into the control system of an unmanned combine.

1. Introduction

Hard coal mines in Poland have been subject to restructuring processes for several years. In the current market situation, the prices of coal imported from foreign markets are lower than the prices offered by domestic producers. In order to stay on the market, coal mines must increase their efficiency, product quality, and reduce personnel costs or costs incurred for the purchase of new machines or their renovation [1,2,3]. Enterprises may be supported by the developing idea of Industry 4.0, which was initiated by the evolving omnipresent digitization. We can indicate several elements in the revolution without which Industry 4.0 could not function, such as Internet of Things (IoT), cloud computing, intelligent factories, automation, and robotization. Their presence has revolutionized the work of companies, that work mainly on data, connecting it with the IoT. It provides almost unlimited access to data processing and storage for factories, entrepreneurs, and representatives of other businesses. Smart factories are new generation factories using different values of Industry 4.0 depending on their demands and needs. There are four types of such factories: smart-automated and robotized plants, digital mass-individualization factories or so-called customer-centric plants, E-plants in a box or mobile modular factories, handmade with digital touch [4,5,6]. The progressive digitization of devices in coal mines allows information to be obtained on the operation of devices, as well as measurement systems in the field of geophysics, gasometry, and the quality of hard coal in real time. Most of the data acquired from technical systems is used mainly for the control and monitoring of equipment operating parameters or environmental parameters. The data sets are archived and occasionally used to explain or analyze a specific event [7]. The acquired data is transmitted to the surface via the existing fiber-optic infrastructure. It is currently used mainly in technological data transmission systems, such as data transmission from machine and device controllers, data transmission from 6 kV power protection systems, as well as from closed-circuit television camera systems [8].

Digital signal processing (DSP) has revolutionized many fields of science and technology. Among others, it found application, in space research, medicine, telephony, military, and industry. Sight and hearing are the two most important senses. That is why a significant part of DSP applications is related to image and sound processing [9].

Throughout many years of coal mining, mining plants have been forced to exploit deeper parts of deposits. The increase in the depth of extraction is accompanied by an increase in the amount and intensity of natural hazards, affecting the cost of extracted coal and work safety. The average depth of coal extraction in Poland is 720 m, and the deepest seams reach almost 1300 m. The increase in the occurring natural hazards appearing with the depth of mining leads to a higher risk of mining accidents and disasters. The most common natural hazards in the longwall are the accumulation and explosion of methane and coal dust, rock bursts, and outbursts of gases and rocks. Apart from natural hazards, there are technical and organizational threats in mines, which are affected by the equipment and technologies used [10]. Due to the high variability and complexity of the occurrence of natural hazards, usage of digital signal processing is hard or almost impossible to predict. In hard coal mining, there are systems which are used to prevent a mining disaster, such as automatic methane measurement systems, systems measuring the composition of the atmosphere, and geophysical systems measuring the strength of rock mass tremors. The most dangerous section of the production process line is the longwall shearer area. Limiting the number of people working in this area may improve the safety of employees. The development of a coal and rock recognition system will allow automation of the production process, where the presence of an employee controlling a longwall shearer is not required. The employee will supervise the process in a safe place, where he will not be threatened by, for example, rockfall.

Digital signal processing has also found an application in the diagnostics of machines and devices. One of the applications of DSP is the diagnostics of internal combustion engines, which is based on the analysis of vibroacoustic vibrations measured with the use of laser Doppler vibrometry and accelerometry. In vibration diagnostics, wavelet analysis (WPT) was used to detect simulated engine damage [11,12,13]. Another application of DSP is the recognition of acoustic signals from an induction engine. By means of pattern recognition, it is possible to recognize pre-failure states of an engine. The analysis was performed using the FFT, shortened method of frequencies selection (SMoFS-10,), and the linear support vector machine (LSVM) [11]. The conducted research shows that the above methods can be useful for diagnosis and early detection of failures in engines and machines [11,12,13].

The main goal of the research is to develop a reliable method of automatic detection of coal/rock border layers. Developing a coal–rock cutting pattern recognition is not an easy task. Many solutions have been proposed to overcome this problem. The most significant methods include γ-ray detection [14], infrared detection [15], image detection [16], mechanical vibration [17], and radar detection [18]. The above methods have both advantages and disadvantages. The system based on optical recognition requires good lighting and low air dustiness; it also forces the camera lens to be kept clean, which is very difficult considering the prevailing conditions during the combine operation. The gamma radiation method requires the collection of many samples to determine the amount of carbon radiation and rock radiation in the roof and floor layers. The method that uses vibration measurement by the installed accelerometers can be unreliable because a series of vibrations is generated during coal mining, which may affect the recognition result. Radar detection is very effective; however, it is quite an expensive method. For the above reasons, the development of the recognition of the cutting pattern with the use of an acoustic signal may find an application in practice. The acoustic sensor is cheap, and collecting samples does not cause many problems. The development of the recognition system must be preceded by the collection of measurement samples, feature extraction, and identification. Samples prepared in this way can be used for teaching the neural network.

The problems of coal–rock recognizing issues using acoustic signals are considered in some scientific works. The source acoustic signal was first collected through an industrial microphone and decomposed by adaptive ensemble empirical mode decomposition (EEMD). A 13-dimensional set composed by the normalized energy of each level was extracted as the feature vector. The swarm intelligence optimization algorithm inspired by the feeding behavior of bats was used to determine the key parameters of the traditional neural network with variable wave translation (VTWNN). VTWNN optimized by modified BA (VTWNN-MBA) was used as cut pattern recognition. The simulation accuracy result was 95.25% [19].

In another study [20], in order to analyze the signal deeply, an adaptive Hilbert–Huang transform (HHT) was applied to decompose the sound to several intrinsic mode functions (IMFs) to subsequently acquire 1024 Hilbert marginal spectrum points. The 1024 time frequency nodes were reorganized as a 32 × 32 feature map. The LeNet-5 convolutional neural network (CNN), with three convolution layers and two sub-sampling layers, was used as the cutting pattern recognizer. In the simulation example, there was a set of 10,000 training samples, to prove the effectiveness of the proposed method. The proposed method achieved an identification rate of 98.55%. Another method [21] used for coal–rock interface recognition is based on permutation entropy, calculated using the local mean decomposition (LMD) method and supervised Kohonen neural network (SKNN) by performing sound signal analysis using the Mel-frequency cepstrum coefficient (MFCC) method and neural network. Noise separation was also performed using independent component analysis (ICA) for the acoustic signal. The extracted MFCC, as the feature, recognized the coal–rock interface via BP (back propagation) neural network. The result shows that MFCC reflects the voice features of coal–rock more effectively, comparing to other features (frame energy and kurtosis) [22].

The article focuses on the comparison of sound samples recorded during the combine working. The approach implemented by the authors is based on the methodology developed within process diagnostics [23]. The model-based approach to detection of the coal/rock border takes advantage of a SISO (single input, single output) model identified by the use of the system identification methodology [24]. The input to the model carries information about the properties of the rock that is being cut. In the paper, the recorded sound of the scraper conveyor operation, the sound of coal mining, and the sound of a rock cutting by the head of the combine are analyzed in order to confirm the possibility of their differentiation in conditions of disturbances occurring in the excavated process, without the installation of any other special sensors. Using the fast Fourier transform, the signal from the time domain to the frequency was transformed, and the spectrograms for the collected samples were compared. Short-time Fourier transform was used to recognize the nature of nonstationary signal variability and to plot three-dimensional graphs.

2. Methods of Signal Analysis and Spectra Comparison

On the basis of opinions of the operators of combines, the authors concluded that the human operator controlling the cutting process acquires information from noise he faces during the work. Therefore, the authors carried out frequency analyses of acoustic signals registered in the wall where the longwall shearer was operating.

The analysis of acquired signals was carried out twofold: by the use of typical frequency analysis, and using the STFT. All the analyses were performed in the MatLab environment using standard toolboxes.

A simple dictaphone, recording the sound with a frequency range of 50 Hz ÷ 20 kHz, was used to record the sound of the shearer’s cutting. The measurement was made during the operation of a coal and rock shearer, and during the operation of the face conveyor without shearer cutting. The recording device was installed at a distance of 2 m from the mining head at a height of 1.3 m from the bottom of the excavation. A longwall complex with Glinik-20/41-POz sections and an FS 400 longwall shearer with a cutting diameter of 2.0 m were installed in the longwall. The PSZ-950 “NOWOMAG” scraper conveyor was 228 m long. A coal seam in the wall had a thickness of 2.9–4.3 m. The deck consisted of sandstone. Shale with a thickness of 0.1–0.30 m was present at the bottom of the seam, and the layer below contained sandstone. The location of the voice recorder installation is shown in Figure 1.

Short-time Fourier transform is used in the analysis of nonstationary signals, i.e., signals whose parameters change over time. The above transformation does a weight averaging of the signal values measured over the entire time range. STFT is a transformation with the application of a window function. Spectral analysis is performed not on the entire signal, but on its shorter sections, which are cut by the moving measurement window. In this way, the frequencies that currently occur in the signal section are highlighted. The most frequently used window is a rectangular window with a length adjusted to changes in the value of signal parameters. In addition to the rectangular window, it is possible to use other function windows such as Hann, Hamming, Gauss, and Blackman. The length of the window used determines both the time and frequency resolution. The short window has good resolution over time, but has worse frequency domain. By extending the window, the frequency resolution is improved at the expense of lowering the time resolution [25,26].

The STFT result is a function of two variables of time and frequency. For the signal x(t), the transformation definition has the following form [25,26]:

$$STF{T}_x\left(t,\omega \right)={{\displaystyle \int }}_{-\infty }^{\infty }x\left(\tau \right)h\left(\tau -t\right)e^{-j\omega \tau }d\tau$$

(1)

For discrete signals, the STFT transform is given by the formula:

$$STF{T}_x\left(m,\omega \right)={\displaystyle \sum }_{n=-\infty }^{\infty }x\left(n\right)h\left(n-m\right)e^{-j\omega n}$$

(2)

where the temporal location is discretized and the m index corresponds to it. By definition, the transform remains continuous, although, in practice, the result is also discretized.

The STFT transformation generates results in a complex form, most often presented in the form of a spectrogram defined as the square of the module:

$$SPE{C}_x={\left|STF{T}_x\left(t,\omega \right)\right|}^2$$

(3)

3. Recorded Signals and Their Analyses

The analyzed sound samples were recorded, for safety reasons, at a distance of approx. 2 m from the cutting drum of the shearer. The 15 s samples of cutting coal and shale by the shearer were recorded. The sound of the scraper conveyor is also registered in the signal. The analysis of the acquired sound samples will allow examination of whether the simple digital sound recording system is sufficient for the purposes of the analysis, as a much cheaper alternative to the equipment required in other methods.

The recordings were registered at a sampling frequency of 48 kHz. One second samples of the recordings were extracted from the samples. Then, the samples were loaded into the Matlab R2020a and analysed using the implemented FFT algorithm. Figure 2 presents graphs of the recorded sounds.

In Figure 2, it can be seen that the graphs of the sound samples differ in amplitude. The sound of the conveyor belt has the lowest average amplitude; we can distinguish characteristic “peaks” in which the amplitude increases. The reason for this is the characteristic “hitting” of the conveyor chain against the conveyor trough. The sound of shale cutting should have a greater amplitude due to the greater hardness of the shales of about 15 MPa compared to the hardness of coal, which is on average 5 MPa. The reason may be that the cutting head sinks into the floor and thus the sound is damped by the accumulated spoil. Despite the fact that the sound of rock breaking is muffled by the accumulated material, what can be observed is a momentary increase in amplitude where the knives of the cutting head hit the rock. The sound of the cutting coal has the most uniform amplitude. However, the nonstationary nature of acquired signals is evident.

Using the built-in FFT algorithm in Matlab, the frequency spectra for the collected samples were plotted. The spectra of signals are shown in Figure 3.

Figure 3 shows the spectra of signals obtained after applying FFT. The spectrum of the conveyor signal has the smallest amplitudes. It can be compared to a random noise. At a later stage of coal and rock recognition research, this signal can be interpreted as a background noise and can be subtracted from the rock and coal signals to eliminate the noise generated by the scraper conveyor. The opposite trend is observed for shale and coal samples. The highest amplitude can be seen for the signal of shale cutting; with the frequency of 73 Hz, it has an amplitude of about 141. The coal cutting signal has the greatest amplitude of 50 for a frequency of approx. 85 Hz. The frequency spectrograms on the x-axis (Hz) were limited to the frequency of 2.5 kHz to ignore high-band noise.

Since noise generated by the conveyor can significantly blur signals caused by cutting the rock, sufficient signal processing will be required based on techniques of noise cancelling, which is planned in the next stage of the research.

The next step of the signal analysis was to plot three-dimensional spectrograms using the built-in STFT function algorithm. Hamming windowing was used in the STFT analysis. The Hamming window was selected for the analysis due to its most universal character, ensuring acceptable side-lobe suppression with relatively low bandwidth extension. Spectrograms corresponding to the collected signals were made, where the non-stationary nature of the signal is clearly visible. The result of the STFT calculation is a three-dimensional graph where the x-axis shows the frequency (Hz), the y-axis the signal amplitude, and the z-axis signal duration (s). The results are presented in a three-dimensional waterfall diagram. The vertical axis shows the value of the spectrogram as a function of time and frequency.

In Figure 4 and Figure 5, the STFT analysis for the scraper conveyor is plotted. The plotted signal can be compared to white noise. In the spectrogram in Figure 4, you can see that the power frequency is highest for 0–2.5 kHz.

The plot for the shale sample (Figure 6 and Figure 7) contains the signal with the highest amplitudes for frequencies 0–0.5 kHz. For this range, the mean amplitude is 80.

In the case of a coal sample (Figure 8 and Figure 9), the amplitude starts to increase significantly from the frequency of 1 kHz, and for the range of 0–500 Hz, the average amplitude value is 130. Despite the variability of the spectra as a function of time, the main components determining the cutting coal continue throughout the entire time interval of the recorded sample.

Different frequencies generated during coal and shale cutting can be used for further analysis of the signals in order to develop the neural network learning. A well-functioning neural network should be able to recognize with a very good accuracy which shale is being cut by the shearer’s head at a given moment. This will enable the formation of an automatic longwall shearer control system in the future.

The preliminary work has proven that there are visible differences between signals acquired from the scraper (background noise), shale, and coal cutting. In our further work, we are going to develop a model of the cutting process, as shown in Figure 10, in which sound signal acquired in the neighborhood of the operating cutting head will carry information of a drag of the cutting rock, while the output would be a signal carrying information about the response of the combine motor. A model of cutting coal will be trained using a machine learning approach. Then, signals acquired from the working system will be compared with the output of the model yielding residuals that will subsequently be classified. In case of significant differences between the measured and calculated output, the conclusion will be formulated that the combine is just cutting shale. The authors plan to demonstrate the operation of the system in the near future.

4. Conclusions

The analysis of the collected sound samples showed that frequency spectra of noise recorded during cutting coal and shale differ from each other. The low cost and easy installation of the acoustic sensor, compared to others, gives this approach a significant prospect for practical use. The development of a longwall shearer automatic control system may be one of the solutions that may revolutionize the operation of mining plants in the future. It will also allow a reduction in the costs of the crew, which will result in an increase in the profitability of production.

This introductory work allows the formulation of the following conclusions:

• Frequency spectra of noise recorded while a shearer head is cutting shale and coal differ significantly from each other, both in amplitudes of spectral dominant components, and in frequencies of their occurrence,
• Acoustic signals generated by the cutting process and operation of machinery and equipment located in the neighborhood are nonstationary.
• Background noise produced by neighboring systems can mask useful signals by worsening S/N ratio and, therefore, shall be eliminated by respective signal processing.
• The research made so far initially proves the hypothesis that it is possible to recognize the coal/shale border on the basis of acoustic signals.

Further research will be targeted on the development of respective systems for data acquisition, which will then allow the systematic collection of more data to be used for the development of a signal processing application, and the component for concluding about the coal/shale border.