1. Introduction
Coal mine video monitoring systems are the primary means of monitoring coal mine safety production, and the timely and accurate identification of abnormal behaviour in the video is of great importance in coal mine safety production. Due to the poor lighting conditions and dusty underground production, lack of clarity in underground monitoring videos is an important issue at present, and the process of enhancing subsurface images can easily cause halo expansion and image overexposure. Therefore, eliminating halo artefacts is essential for image enhancement and further analysis.
Traditional image-denoising methods are generally based on spatial-domain- and transform-domain-based image denoising. The Retinex model, based on Retinex theory, also has many improved versions, including the single-layer Retinex model [
1], a multi-scale Retinex model [
2], fusion-based enhancement methods [
3], illumination map estimation [
4], etc. There are also algorithms based on the variational Retinex model, such as the variational Retinex model as a quadratic optimisation problem [
5], the Retinex variational framework for the introduction of bright channels [
6], the variational Retinex model based on the L2 paradigm [
7], a hybrid L2-LP variational model with a brilliant track prior [
8], and the maximum entropy-based Retinex model. Based on the computational complexity of the variational approach, the disadvantage of the method is that it is time consuming to process the images.
In addition, Zhuang et al. [
9] proposed a new edge-preserving filtering Retinex algorithm that embeds gradient-domain-guided image filtering (GGF), prior information on reflections and illumination into a Retinex-based variational framework to improve image structure and reduce artefacts or noise. Oh and Hong [
10] proposed an adaptive image mapping method based on the Retinex model with a parametric non-linear mapping function. Only one luminance channel is used to estimate the reflectance component of the observed shimmering image. Li [
11] proposed a mining image enhancement method based on the cuckoo search (CS) algorithm, which is based on the HSV colour space and uses the cuckoo search (CS) algorithm in combination with a proposed new conversion function that exploits the advantages of the bilateral gamma adjustment (BIGA) function and the dual platform histogram equalisation (DPHE) function. By finding the optimal parameter values, the image contrast and brightness are globally enhanced, and the details of the images are enhanced.
In addition, Si [
12] proposed a hybrid algorithm (SSR-BF) based on the combination of single-scale Retinex (SSR) and bilateral filtering (BF) to improve the image quality of surveillance videos. Combining BF with SSR reduces noise and improves edge information in the image. The schematic diagram and pseudo-code of SSR-BF are designed, and through simulation, the parameters are set reasonably to ensure the enhancement effect. Xu et al. [
13] proposed a hierarchical feature mining network (HFMNET), which extracts illumination and edge features in different network layers, constructs a feature mining attention (FMA) module, and combines hierarchical supervised loss to mine key features in the appropriate network layers. Yang [
14] proposed an enhancement algorithm based on an optimised homomorphic filter and RGB colour correction and developed an optimised homomorphic filter to reduce the interference of impurities on the image. The interference of impurity particles on the picture is reduced, and a preliminary optimised image can be obtained. A multi-scale image enhancement algorithm is designed to enhance the image with colour, and an RGB correction algorithm is proposed to colour correct the vision to further improve the image quality considering the colour deviation of the image. Li [
15] proposed a multi-scale fusion framework for image enhancement combining Retinex and transmittance optimisation. The R, G and B channel greyscales are quantized to enhance image contrast, the colour constancy of Retinex is used to eliminate the adverse effects of scene illumination, and colour distortion, background light, backscatter and direct component transmittance are estimated. The defogged image is obtained by inverse resolution. Finally, the three input images and the corresponding weight maps are fused in a multi-scale framework to provide high-quality, sharpened results. Gao [
16] proposed a new RetinexDIP image enhancement method in which noise is considered a factor in image decomposition using deep learning generation strategies. The noise intervention makes the image more realistic, weakens the coupling between the three components, avoids overfitting and improves generalisation. However, these methods still need to be addressed and optimised for the problems of halos and overexposure during image enhancement.
In response to the above research, an improved algorithm based on Retinex is proposed, based on homomorphic filtering, fusing bilateral filtering and Retinex, with the addition of luminance processing to achieve image enhancement, which is expected to essentially reduce the effects of haloing in images and can result in more precise details and better processing results.
The improved algorithm (BF-MSRCR algorithm) consists mainly of the enhancement of the luminance component V and the adaptive stretching of the saturation component S so that the details of the image are better preserved during the enhancement process, without blurring the edges and without problems, such as overexposure and haloing. Compared with the previously proposed algorithm, this uses multiple scale operators for synthesis in the processing, and does not perform simple mean processing, which takes less time and is not very demanding for the enhancement equipment, and is not prone to the halo phenomenon in the strong shadow transition area, which can solve the problem of uneven illumination of the image under the mine. The common algorithm makes the image overexposed to halos.
2. Theoretical Foundations of Algorithms
2.1. HSV Model Analysis
HSV (hue, saturation, value) [
17] is created from the intuitive properties of a colour, which is also commonly referred to as the hexagonal cone model, as shown in
Figure 1. Each colour is represented using the three characteristics of hue, saturation and colour luminosity.
The hue, H, parameter indicates the image’s colour information, i.e., the position of the spectral colour in which it is located; the saturation, S, indicates the shade of the paint; and the value, V, indicates the brightness of the colour. This time, instead of using the traditional RGB mode for image processing, the image is converted in HSV space for enhancement, where specific attributes of the image are enhanced or diminished, which is more conducive to the processing of underground monitoring images.
The formula for converting from RGB space to HSV space is shown in Equations (
1) and (
2):
In the above equation, max and min represent the maximum and minimum values in RGB space.
The formula for converting from HSV space to RGB space is shown in Equations (
3) and (
4):
where floor means rounding down.
2.2. Retinex Model Analysis
Retinex [
10] is an image enhancement algorithm, which is often used on an experimental and analytical basis. The principle is that the colour of an object is determined by the reflectance of the object for long-, medium-, and short-wave light, so the algorithm can achieve good results in three aspects—edge enhancement, dynamic compression, and colour constancy—and can perform adaptive enhancement for different types of images. The primary expression for the Retinex model is shown in Equation (
5):
where
represents the original image, which is decomposed into an illuminance component and a reflection component; the
component expresses the smoothed information of the input image; and the
component expresses the original properties of the original image. Therefore, the key to using the Retinex model is to evaluate the illuminance component and calculate the reflection component
R from
L.
The single-scale Retinex algorithm (SSR) uses a logarithmic model when evaluating the reflectance component. The image is filtered using a centre surround function to approximate the illumination component, with the best centre surround function being a Gaussian kernel function [
18].
The single-scale Retinex algorithm better maintains the image’s brightness, reducing the loss of image detail caused by noise while improving information capture in dark areas. If the image being processed contains a large number of similar grey scale areas, the halo’s edges will blur.
The multi-scale Retinex algorithm (MSR) adds multiple scales to the SSR, which is universally divided into three scales, with the scale features still represented by
. Its calculation is shown in Equations (
6)–(
8):
The value of i can be taken as 1, 2 or 3, with i = 1 for low scale, i = 2 for medium scale, and i = 3 for large scale; is the weighting factor of the i-th scale when weighting is performed.
During the enhancement process of the MSR algorithm, the image may be distorted by the addition of noise, which distorts the colour of local details of the image and does not reveal the object’s true colour, making the overall visual effect worse.
To address this deficiency, the multi-scale Retinex algorithm with colour recovery factor (MSRCR) [
19] adds a colour recovery factor C to MSR to adjust the defects of colour distortion due to enhanced contrast in local areas of the image. The MSRCR algorithm uses the colour recovery factor C to adjust the proportional relationship between the three colour channels in the original image, thus bringing out the information in relatively darker areas and eliminating the defects of colour distortion in the image.
To date, MSR and MSRCR algorithms have commonly been used to solve filter-dependent problems, and they can effectively improve image contrast without relying on many filters, but they also increase the number of halo artefacts in the image, which is a significant drawback.
4. Experimental Verification
In order to verify the effectiveness of the improved Retinex algorithm (BF-MSRCR) for underground coal mine surveillance image enhancement, simulation experiments were carried out using Matlab programming for the same data set in the AMD R7, 16 G hardware environment, Windows 11 operating system. The images used in this paper are partly from publicly available underground coal mine images on the internet and partly from a real underground coal mine surveillance video. We extracted the video in frames, generated hundreds of coal mine surveillance images, and enhanced some of them. We selected a coal mine underground video and some underground images for training when testing the algorithm, and tested about 600 images in total.The specific images of the experiments are shown in
Figure 7, using the MSR algorithm and MSRCR algorithm for comparison experiments.
To make the image contrast enhancement effect more intuitive to represent, the method of grey level distribution is used in the verification, using the grey level histogram to verify that when the image contrast is high, its grey level distribution has a uniform and wide distribution.
The validation process uses the MSR algorithm, MSRCR algorithm and BF-MSRCR algorithm for comparison and validation of the image enhancement effect based on the generated greyscale histogram, and the validation results are shown in
Figure 8,
Figure 9 and
Figure 10.
To show more experimental results, we selected some more frame-separated intercepted images from the coal mine video for enhancement to verify the effectiveness of the algorithm. The experimental results are shown in
Figure 11.
According to the visual verification of the greyscale histogram, the MSR algorithm enhances the image, the blurred edges and significant light points are more apparent, and the MSRCR algorithm shows some improvement. However, the colour distortion of the image is still more severe, and the histogram distribution of the above two algorithms is more concentrated. However, the BF-MSRCR algorithm proposed in this paper has a specific improvement in the image’s edge sharpness and colour saturation, and the grey level distribution is wide and uniform. Therefore, the algorithm in this study has better contrast enhancement than the MSR algorithm and the MSRCR algorithm.
A series of existing image quality evaluation criteria were subsequently chosen to measure the image enhancement effect of different algorithms. There are three general types of objective evaluation of image quality: the complete reference, partial reference and no reference. In this study, the peak signal-to-noise ratio (PSNR) [
28], structural similarity (SSIM) [
29], standard deviation, mean gradient, mean value, and colour image information entropy [
30,
31] were used to measure the quality of the image, thus providing a more intuitive perception of the enhancement effect of the algorithm.
PSNR is used to measure the pixel quality of an image, and the larger the value, the better the quality of the image, as represented by the calculation formula shown in Equation (
17):
where
denotes the image size, and
and
denote the greyscale values of the image before and after enhancement.
SSIM was used to measure the similarity of an image, and its value is in the range of 0–1. The higher the value, the lower the distortion; standard deviation is used to measure the contrast of an image. The higher the value, the higher the contrast; the mean gradient is used to measure an image’s sharpness and texture variation. The higher the value, the more precise the image; the mean value is used to measure the brightness of an image. The higher the value, the higher the brightness, and the information entropy of an image contains the amount of information contained in the entropy of an image. The higher the entropy value, the more informative the image. It is calculated as shown in Equation (
18):
where
x and
y are images of the same size,
is the mean of
x,
is the variance of
x,
is the covariance of
x and
y, and
and
are constants.
Each calculation takes an window from the image, and then keeps sliding the window for calculation, and finally takes the average value as the global SSIM.
The validation process used the MSR algorithm, the MSRCR algorithm and the BF-MSRCR algorithm for comparative validation. It measures the effectiveness of the enhanced images according to the evaluation criteria mentioned above, and the validation results are shown in the following
Table 1 and
Table 2 (using downhole image 2 as an example).
As can be seen in
Table 1, the BF-MSRCR algorithm improves the peak signal-to-noise ratio, structural similarity and image information entropy by 8.28, 0.15 and 2.04, respectively, compared to the MSR algorithm; by 3.97, 0.02 and 1.33, respectively, compared to the MSRCR algorithm; by 6.45, 0.06, 0.93, respectively, compared to the homomorphic filtering algorithm; and by 4.92, 0.1, 1.65, respectively, compared to the Laplace enhancement algorithm.
As can be seen in
Table 2, the BF-MSRCR algorithm improved the standard deviation, mean gradient and mean value by 40.39, 7.38 and 52.92, respectively, compared to the MSR algorithm; by 34.33, 6.46 and 26.21, respectively, compared to the MSRCR algorithm; by 24.76, 4.72 and 42.34, respectively, compared to the homomorphic filtering algorithm; and by 22.18, 7.0 and 33.56, respectively, compared to the Laplace enhancement algorithm.
All evaluation criteria of the enhanced images improved, indicating that the quality of the images enhanced by the BF-MSRCR algorithm was optimised in all aspects.
Through objective verification of the indicators in
Table 1 and
Table 2, the image quality, brightness and contrast are shown to be enhanced by the BF-MSRCR algorithm are significantly improved. The amount of information in the image also increased, while the anti-distortion performance of the image improved in the presence of disturbing factors, such as low contrast, low brightness, uneven light distribution and high dust influence in the underground mine image.
5. Conclusions
In the presence of multiple disturbing factors in underground coal mines and facing the problem of a complex environment, an improved algorithm is proposed to transform different components by combining a bilateral filtering algorithm based on HSV space with a multiscale Retinex algorithm with colour recovery factors after homomorphic filtering, and the conclusions are shown below.
(1) The method uses homomorphic filtering to enhance each RGB component of the image, converts the image from RGB space to HSV space, keeps the hue H component unchanged, uses an algorithm of bilateral filtering and multi-scale Retinex fusion to enhance the luminance V component, gamma corrects its incident component, and performs histogram equalisation correction and Laplace operator enhancement for the reflection component. An adaptive nonlinear stretching process is performed for the saturation S.
(2) As verified by MATLAB simulation, the improved algorithm (BF-MSRCR) can effectively remove various noises in the image, significantly improve the brightness and contrast of the image. In addition, it does not cause a lot of haloes and edge distortion in the image, and the colour retention of the image is also more complete and high-definition.
This algorithm dramatically improved the enhancement of underground coal mine surveillance video images, supporting subsequent mine facility detection, personnel detection, and safety production. However, the algorithm used in this study is still a universal image enhancement method. In subsequent research work, we can consider using deep learning algorithms to train the surveillance graphics of underground mines and build a more adaptable image enhancement model for underground mines.