An Underwater Image Enhancement Algorithm Based on MSR Parameter Optimization

Abstract

The quality of underwater images is often affected by the absorption of light and the scattering and diffusion of floating objects. Therefore, underwater image enhancement algorithms have been widely studied. In this area, algorithms based on Multi-Scale Retinex (MSR) represent an important research direction. Although the visual quality of underwater images can be improved to some extent, the enhancement effect is not good due to the fact that the parameters of these algorithms cannot adapt to different underwater environments. To solve this problem, based on classical MSR, we propose an underwater image enhancement optimization (MSR-PO) algorithm which uses the non-reference image quality assessment (NR-IQA) index as the optimization index. First of all, in a large number of experiments, we choose the Natural Image Quality Evaluator (NIQE) as the NR-IQA index and determine the appropriate parameters in MSR as the optimization object. Then, we use the Gravitational Search Algorithm (GSA) to optimize the underwater image enhancement algorithm based on MSR and the NIQE index. The experimental results show that this algorithm has an excellent adaptive ability to environmental changes.

1. Introduction

Marine resources are abundant, but the marine environment is complicated, and underwater exploration by humans is difficult. Therefore, underwater robots [1,2] are widely used in the exploration of marine resources and for underwater target recognition [3,4,5,6,7]. However, due to the absorption of light and the scattering and diffusion of floating objects, underwater images often have distortion, color deviation, low contrast and low definition.

There are many existing non-underwater image enhancement algorithms, such as Histogram Equalization [8] (HE). HE transforms the original image into an output image with roughly the same number of pixels at most gray levels through pixel value transformation. This algorithm has a simple structure and low computation, but HE is not good at improving the local contrast of the image. In order to improve the deficiency of HE, Pizer et al. [9] proposed an Adaptive Histogram Equalization algorithm (AHE)—AHE redistributes the brightness value of the image by calculating the histogram of each significant area of the image, making it more suitable for improving the local contrast of the image. However, the AHE algorithm has other problems, such as amplified noise. In order to solve the shortcomings of the AHE algorithm, Zuiderveld [10] proposed a Contrast Limit Adaptive Histogram Equalization (CLAHE) algorithm. In addition, algorithms such as color-corrected white balance [11] and gray-scale edge assumption [12] have also been developed. However, using the existing image enhancement algorithms directly in underwater environments may cause the problem of limited enhancement or excessive enhancement. These algorithms can only deal with the contrast of the image to a certain extent, but they cannot solve the problem of the color shifting of underwater images; it is even possible for the algorithms to cause the image to have low local contrast and distortion.

Edwin H. Land proposed the Retinex theory [13]. The Retinex algorithm can balance the three aspects of edge enhancement, dynamic range compression and color constants in the image processing process, but when the light intensity is large, the problem of exposure will occur, resulting in the phenomenon of low local contrast in the image. Zhang et al. [14] proposed the extension of the multi-scale Retinex underwater image enhancement algorithm and the Multi-Scale Retinal with Color Recovery (MSRCR) algorithm [15] to the CIELAB color space, which successfully suppressed the halo phenomenon in the process of image enhancement; however, the algorithm has many parameters, so the calculation speed is slow. Lei Fei et al. extracted the color–grayscale ratio of RGB underwater images and carried out the tensile interception of Multi-Scale Retinex (MSR) [16] enhanced images; then, they used the grayscale ratio to restore underwater images, which improved the color distortion of underwater images and enhanced their clarity. However, in an image with a large local luminance difference, the MSR-enhanced image has a halo, resulting in the phenomenon that the local contrast is too low in the image. Since then, scholars have continued to improve the Retinex algorithm [17,18,19,20].

There are two other important research directions in this field: algorithms based on the Dark Channel Prior (DCP) algorithm and algorithms based on the Generative Adversarial Networks (GAN). He et al. proposed the DCP [21] algorithm, which has been widely used in the field of image enhancement [22,23] due to its simple defogging model and good defogging effect. Zhu et al. proposed unpaired image-to-image translation algorithm using cycle-consistent adversarial networks [24]. Image enhancement algorithm based on GaN is currently a very active research area.

Underwater image enhancement algorithm improves the stability of underwater robot in various application. Zhao et al. [25] applied their improved underwater image enhancement algorithm to Autonomous Underwater Vehicle (AUV). The experimental result shows that the AUV visual SLAM can achieve accurate positioning results by underwater image enhancement algorithm. Wei et al. [26] applied the improved Retinex algorithm to the Underwater Vehicle Manipulator System (UVMS) to track and capture underwater objects. The experimental result shows that the algorithm can improve the precision of AUV grasping.

In general, the existing underwater image enhancement algorithms use fixed parameters. Although they can improve the visual quality of underwater images to some extent, there are also problems of amplified noise and color distortion. At the same time, the existing MSR-based underwater image enhancement algorithms may be particularly effective when dealing with a specific underwater environment, and the existing underwater image enhancement algorithms are unstable when facing different underwater environments. In the traditional MSR algorithm, a fixed gaussian weighting factor is selected. If the image is taken from a cloudy water body, the enhanced image will have color bias and distortion. As shown in Figures 20c and 21c, the effect is unstable. The traditional DCP algorithm uses the water environment color A as the point with the brightness of the top 0.1% in the dark channel image. If the image is taken from the water with strong illumination, the image enhancement effect will be poor, and the color of the water environment will be left in the enhanced image. As shown in Figures 22d and 23d, the effect is unstable. Therefore, it is important to determine the appropriate parameters according to the changes of different scenarios.

At the same time, the actual computing conditions of underwater robots are limited, and it is difficult to meet the real-time requirements of image processing. Therefore, this paper attempts to reduce the computational complexity of the algorithm and thus to improve its ability to be used in engineering applications.

The structure of this article is as follows—Section 1 introduces the development of underwater image enhancement work; Section 2 briefly introduces the idea of this article and the purpose of each workflow; Section 3 briefly introduces relevant background knowledge; Section 4 introduces the selection method of the parameters and the no reference image quality assessment index; Section 5 introduces the main work content of this article; Section 6 conducts experiments on the algorithm proposed in this article and objectively analyzes its performance; and Section 7 presents the conclusion of the article.

2. Methodology

The existing underwater image enhancement algorithms use fixed parameters, which can improve the visual quality of underwater images to some extent, but the existing underwater image enhancement algorithms are unstable when facing different underwater environments. MSR-PO is based on the MSR algorithm, using non-reference image quality assessment indexes to optimize the parameters to achieve a stable underwater image enhancement effect and improve the efficiency of underwater exploration by underwater robots. Firstly, we find a suitable non-reference image quality assessment index for unreferenced images. Secondly, we use the traversal method to select and optimize the parameters. Finally, in order to improve the enhancement speed, we use the Gravitational Search Algorithm (GSA) [27] to replace the traversal method for parameter optimization and propose a strategy for optimization only in stable frames to increase the speed of the enhancement.

2.1. Selection of a Non-Reference Image Quality Assessment Index

We need to find a suitable non-reference image quality assessment index. There are many no reference image quality assessment indexes, and the most suitable one needs to be selected. In this paper, Gaussian noise of different degrees is added to the image to observe the changes of multiple classic no reference image quality assessment indexes. If the changes of a no reference image quality assessment index could correctly reflect the change trend of noise, and its dynamic range was large, the indicator would be selected and applied in this article.

2.2. Selection of Target Parameters to Be Optimized in MSR

We need to use the traversal method to find the appropriate algorithm parameters. The main method is to traverse the parameters in the MSR and observe the dynamic ranges of the no reference image quality assessment indexes. Generally speaking, the parameters corresponding to the indexes with large dynamic ranges can meet the requirements of the tasks in this paper. On this basis, if the indicator changes linearly, the optimal parameters could be directly calculated, and if the indicator changes nonlinearly, optimization algorithms would need to be used for optimization.

In specific applications, the computing power of an underwater robot is limited, the real-time requirement for image enhancement is high and parameter screening is required to reduce the amount of calculation. The main method used is to calculate a set of no reference image quality assessment index data after each parameter traversal and calculate the influence of the data variance comparison parameters. If the influence of a parameter increases with the variance, the parameter is retained; otherwise, it is deleted.

2.3. Improve Engineering Application Capabilities

The traversal method is stable but slow. This paper attempts to optimize the parameters by using the GSA optimization algorithm and conducts multiple sets of comparative experiments to ensure the stability of the GSA algorithm.

When enhancing video frames, considering that underwater robots require real-time image enhancement capabilities, this article attempts to propose a strategy that only optimizes the frame to increase the enhancement speed; the feasibility of this strategy is verified in this work.

The work flow chart of the algorithm in this paper is shown in Figure 1.

When the existing underwater image enhancement algorithm faces different underwater environments, the enhancement effect is unstable due to the limitation of the algorithm’s own fixed parameters. To solve these problems, we use the non-reference image quality evaluation index as the optimization index to optimize the parameters in the MSR algorithm and improve the enhancement speed. Based on this, we propose MSR-PO algorithm.

3. Background Knowledge

3.1. Retinex Algorithm Introduction

3.1.1. Single Scale Retinex (SSR)

The single-scale Retinex algorithm works by improving the center/surround Retinex. The basic idea of the center/surround Retinex algorithm is to estimate the brightness of the center pixel by assigning different weights to the pixels surrounding the target. The proportional relationships of these weights are determined by the surrounding function. Jobson proposed a single-scale Retinex algorithm (SSR) [28] based on this theory. The basic content of the algorithm is as follows. The algorithm assumes that the image is $I(x,y)$, the brightness image is $L(x,y)$, the reflected image is $R(x,y)$; then:

$$\begin{array}{c}\hfill I(x,y)=L(x,y)·R(x,y).\end{array}$$

(1)

The purpose of the algorithm is to obtain $R(x,y)$ from $I(x,y)$. The algorithm assumes that the image is smooth and approximates the low-frequency part of the image. The logarithmic domain processing makes the perception of the image more similar to the human eye’s perception of brightness. The single-scale Retinex can be expressed as

$$\begin{array}{c}\hfill ln\left[R(x,y)\right]=ln\frac{I(x,y)}{L(x,y)}=ln\left[I(x,y)\right]-ln[I(x,y)\ast G(x,y)].\end{array}$$

(2)

In Equation (2), $G(x,y)$ is a low-pass filter used to estimate the brightness map; subtracting the brightness map from the original image can restore the original color of the object. Jobson’s theory proves that the Gaussian convolution function can process the image locally to better enhance the image; thus, $G(x,y)$ is usually a Gaussian function, and the expression is

$$\begin{array}{c}\hfill G(x,y)=\lambda ·e^{\frac{-(x^2+y^2)}{c^2}}.\end{array}$$

(3)

In Equation (3), c is a scale constant. The smaller the c is, the larger the dynamic compression range would be. On the contrary, the larger the c is, the more severe the image sharpness would be. $\lambda$ is a constant matrix, and its value must satisfy Equation (4):

$$\begin{array}{c}\hfill \int \int G(x,y)dxdy=1.\end{array}$$

(4)

3.1.2. Multi-Scale Retinex Algorithm (MSR)

In view of the problem that the SSR algorithm has poor processing effects in terms of image detail enhancement and color fidelity, Joson et al. proposed a multi-scale Retinex algorithm (MSR).

The multi-scale Retinex algorithm is the weighted average of the single-scale Retinex algorithm. $r(x,y)$ is used to represent the final calculation result of the reflected image. The multi-scale Retinex algorithm can be expressed as

$$\begin{array}{c}\hfill ln\left[r(x,y)\right]=\sum _{i=1}^N{w}_i\left\{ln\left[I(x,y)\right]-ln[I(x,y)\ast G_i(x,y)]\right\}.\end{array}$$

(5)

In Equation (5), n represents the number of scales. In general, $n=3$ is a color image and $w_i$ is a Gaussian weighting coefficient. Assuming that the weights corresponding to each scale are equal, then $w_i=1/3$, and $G_i(x,y)$ is a Gaussian low-pass filter:

$$\begin{array}{c}\hfill G_i(x,y)=\lambda ·e^{\frac{-(x^2+y^2)}{{c_i}^2}}.\end{array}$$

(6)

In Equation (6), $c_i$ represents the scale.

3.2. No Reference Image Quality Assessment Index

Current underwater image quality assessment methods are divided into subjective assessment and objective assessment [29]. The subjective assessment method is mainly used for testers to observe the target image and make a subjective assessment and analysis. Generally, the Mean Opinion Score (MOS) or Differential Mean Opinion Score (DMOS) refer to the difference between the assessment scores of undistorted and distorted images by human eyes. However, in practical applications, subjective assessment is limited by many factors, and the uncertainty of subjective assessment makes it impossible to describe the method with a mathematical model; furthermore, it is difficult to achieve high-quality assessment. The objective assessment method is used to calculate the visual quality of the image through certain algorithms and mathematical methods. It includes Full Reference Image Quality Assessment (FR-IQA), Reduced Reference Image Quality Assessment (RR-IQA) and No Reference Image Quality Assessment (NR-IQA) [30]. The following is a brief introduction to four common and classic non-reference image quality assessment indexes.

3.2.1. Entropy

Entropy [31] appears in information theory to indicate the amount of information. It was not used to evaluate image quality until 1978. For the assessment of the quality of underwater images, the color information entropy assessment method is adopted. The RGB color space is selected; then, the information entropy is obtained based on the three color channels of the image, and the average value is finally obtained. The formula for information entropy is as follows:

$$\begin{array}{c}\hfill H=-\sum _0^{255}P(i){log}_{2}P(i).\end{array}$$

(7)

In Equation (7), i represents the gray value, and $P(i)$ represents the proportion of pixels with a gray value of i to all pixels in the image. Color image entropy is the average of the information entropy of the three channels R, G and B. The greater the information entropy of the image is, the more information the image has and the better the image quality is.

3.2.2. UCIQE

The Underwater Color Image Quality Evaluation (UCIQE) [32] is a new image quality evaluation index proposed by Yang Miao et al. The objective image quality evaluation index adopts the linear combination of chroma, saturation and brightness as the measurement method of image quality. The higher the UCIQE score is, the better the image restoration would be. The UCIQE is defined as

$$\begin{array}{c}\hfill UCIQE=c_1\times {\sigma }_c+c_2\times co{n}_l+c_3\times {\mu }_s.\end{array}$$

(8)

In Equation (8), ${\sigma }_c$ is the standard deviation of chroma,$co{n}_l$ is the brightness contrast, ${\mu }_s$ is the average value of saturation and $c_1$, $c_2$ and $c_3$ are the weighting coefficients. After a large number of underwater image experiments, the coefficients finally obtained under the underwater test data sets are $c_1=0.4680$, $c_2=0.2745$ and $c_3=0.2576$. Generally speaking, the higher the UCIQE index is, the better the quality of the underwater image would be; otherwise, poorer image quality would be obtained.

3.2.3. NIQE

The Natural Image Quality Evaluator (NIQE) [33] is an image quality evaluation index that was proposed by Mittal et al. in 2012. The algorithm is based on a probability model. In this algorithm, the image is first mean-removed normalized; then, the processed image is fitted by generalized Gaussian distribution and a multivariate Gaussian model to obtain the parameters. Finally, the distance between the parameters and the corresponding parameters in the image library is the score of the image. The NIQE algorithm calculates the distance between the distorted image and the original image in the image library. This distance is used to measure the image quality. If the distance is small, this means that the image is similar to the natural image in the image library, the degree of distortion is small and the image quality is good. Conversely, if the distance between the distorted image and the original image in the image library is large, it means the degree of distortion is large and the image quality is poor.

3.2.4. UIQM

Karen et al. [34] proposed an Underwater Image Quality Measurement (UIQM) based on the characteristics of underwater image degradation. The UIQM method uses the linear combination of the underwater image colorfulness measurement (UICM), the underwater image sharpness measurement (UISM) and the underwater image contrast measurement (UIConM) as the evaluation basis.

$$\begin{array}{c}\hfill UIQM=c_1\times UICM+c_2\times UISM+c_3\times UIConM.\end{array}$$

(9)

In Equation (9), $c_1$, $c_2$ and $c_3$ are the weighting factors of the measurement components in the linear combination. The method shows that the selection of weighting factors should be based on specific quality evaluation requirements. When the color quality requirements for underwater images are high, the value assigned to $c_1$ is large; when the image contrast and clarity quality requirements are high, the weight values assigned to UISM and UIConM need to be large, and the corresponding $c_2$ and $c_3$ values are large. In this paper, the weighting factors are $c_1=0.0282$, $c_2=0.2953$ and $c_3=3.5753$.

4. Selection of Parameters and Assessment Indexes

Whether the no reference image quality assessment index can be used for optimization depends on whether the index is able to correctly reflect the changing trend of noise and to have a large dynamic range. By adding different degrees of Gaussian noise to the image, this paper evaluates the image quality with four kinds of no reference image quality assessment indexes, including information entropy, UCIQE, NIQE and UIQM. The next step is to observe the change trends and dynamic ranges of these four indexes and choose a no reference image quality assessment index which can optimize the algorithm. Then, we use the index to traverse the parameters, observe the dynamic range of the NIQE index and select the parameters that can be optimized. In order to increase the speed of the calculation, the parameters need to be screened again, and the parameters with a larger influence must be selected.

4.1. Selection of Assessment Indexes

In this paper, we intend to optimize the parameters by using NR-IQA to achieve stable underwater image enhancement and improve the efficiency of underwater exploration by underwater robots. The selection of the no reference image quality assessment index mainly includes two aspects:

(1)

The change of the no reference image quality assessment index correctly reflects the change of the quality of underwater images;

(2)

The dynamic range of the non-reference image quality assessment index is large.

The main method used is to select four representative images (Figure 2) that are commonly used in the field of underwater image quality enhancement as experimental objects. Figure 2a is a greenish image of large targets. Figure 2b is a greenish image with light spot. Figure 2c is a bluish image of large targets. Figure 2d is a bluish image of small targets. These four images are selected in this paper to simulate different underwater environments. Gaussian noise is added to the images, and the variance in the Gaussian noise function is from 0 to 0.1 in steps of 0.002. With the increase of Gaussian noise in the image, we need to calculate four classic non-reference image quality assessment indexes and count their change trends to obtain four graphs as shown in Figure 3, where the X axis is the variance of the added Gaussian noise and the Y axis is the calculated four types of non-reference image quality assessment indexes. In Figure 3, the results of picture 1 are shown in Figure 2a, the results of picture 2 are shown in Figure 2b, the results of picture 3 are shown in Figure 2c, and the results of picture 4 are shown in Figure 2d.

Analyzing Figure 2 and Figure 3, we find that, with the increase of variance in Figure 3a, the UCIQE indicator shows an upward trend, indicating that the image quality is improving, but the UCIQE indicator ought to increase as the image quality becomes better, which is contrary to the requirement of this article. As the variance increases in Figure 3b,d, the UIQM indicator and information entropy show a trend of rising first and then decreasing, indicating that the image quality first improves and then becomess worse, so they are unable to correctly reflect the changing trends of the image quality, which is inconsistent with the requirements of this paper. As the variance increases in Figure 3c, the NIQE shows an increasing trend. The larger the NIQE is, the bigger the difference is, meaning that the image quality is getting worse. The NIQE indicator correctly reflects the trend of image quality change; the difference between the minimum and maximum of the NIQE index can reach about 13, indicating that its dynamic range changes greatly; thus, this article chooses the NIQE indicator as the optimization indicator.

In the specific calculation, NIQE first normalizes the image pixels, extracts the features of the salient image blocks, and then performs image quality calculation. By comparing the natural image and the distorted image, the mean $v_1$, $v_2$ and variance matrices ${\sigma }_1$, ${\sigma }_2$ of natural and distorted images are obtained, and finally the image quality is measured by calculating the distance between the natural image and distorted image fitting parameters; the calculation formula is as follows:

$$\begin{array}{c}\hfill D(v_1,v_2,{\sigma }_1,{\sigma }_2)=\sqrt{{(v_1-v_2)}^T{(\frac{{\sigma }_1+{\sigma }_2}{2})}^{-1}(v_1-v_2)}.\end{array}$$

(10)

Generally speaking, the smaller the NIQE value, the better the image quality.

4.2. Selection of Optimization Parameters

In this paper, we try to optimize the parameters by using no reference image assessment indexes to achieve stable underwater image enhancement. The changes of different parameters have different effects on the MSR algorithm results, meaning that the task presented in this section is to find the parameters that have the greatest influence on the MSR results.

There are many parameters in the MSR algorithm that are set by human experience. In this paper, two sets of parameters that have relatively large influences on the MSR algorithm results are obtained through preliminary screening. The first group is the scale parameter values sigma1, sigma2 and sigma3, and the second group is the Gaussian weighting factors W1, W2 and W3, and the pre-screening methods are the same as the methods in the section above; thus, the details will not be repeated here. At present, all MSR parameter values are artificially summarized and cannot be adapted to different environmental conditions. This paper uses no reference image quality assessment indexes to optimize the parameters, but the extent to which the changes in the above parameters could affect the quality index of the no reference image is not known.

Generally speaking, if the change of a parameter can lead to a large change in the no reference image quality assessment index, then the optimization effect of this parameter will be good, which is also true for the optimization index of our work in this section.

The process of selecting optimization parameters first traverses each parameter and observes the changes of NIQE indexes. In the traversal process, the range of parameter traversal must first be determined. The range of the traversal is mainly determined by the general values of the three parameters, and the values of the three scale parameters should at least be twice as large from the first to the third parameter. Based on this, the sigma1 traversal range is set to 10–40; the sigma2 traversal range is set to 30–125; and the sigma3 traversal range is set to 160–300. In this section, we randomly select two pictures in the sea urchin database of the National Fund of China (NFC) (Figure 4) and traverse their sigma1, sigma2 and sigma3. The results are shown in Figure 5.

In the graph of Figure 5, the abscissa is the traversal parameter, and the ordinate is the NIQE value. In Figure 5a,b, when traversing sigma1, the difference between the maximum value and the minimum value of the dynamic range of NIQE value is about 35%; in Figure 5c,d, when traversing sigma2, the difference between the maximum value and the minimum value of the dynamic range of NIQE is about 4–6%; and in Figure 5e,f, when traversing sigma3, the difference between the maximum value and the minimum value of the dynamic range of NIQE is about 1–2%.

The second set of weight parameters has the relationship of W1 + W2 + W3 = 1. The three parameters restrict each other. The third weight W3 can be expressed as 1 − W1 − W3, so two parameters are selected for simultaneous optimization. In this article, W1 and W2 are selected for simultaneous traversal, and the traversal range of both W1 and W2 is set to 0–1. This method traverses W1 and W2 in the two pictures in Figure 4 to observe the experimental results.

As shown in Figure 6, when traversing W1 and W2, the maximum value and the minimum value of the dynamic range of NIQE can differ by about 0.6.

According to the above analysis, the dynamic range in the graph becomes larger as the parameter values vary, so the five parameters can be optimized. There are multiple peaks and troughs in Figure 5 and Figure 6, indicating nonlinear changes, so the optimal values of parameters cannot be directly calculated, and an optimization algorithm is needed to obtain the optimal values of parameters.

From the above experiments, we can find that all five parameters can be optimized, and optimizing these five parameters will consume a large amount of time. This paper attempts to traverse each parameter and bring the results into the MSR algorithm for enhancement; five sets of no reference image quality assessment indexes could be obtained through the calculation, and the variance of these five sets of data could also be obtained. The greater the variance, the greater the influence of the parameter. The less influential parameters would not be optimized. Among them, the three parameters of the first group of scale parameters do not restrict each other, while the second group of Gaussian weighting factors restrict each other, and the mutually restricted parameters cannot exclude any of the factors. Therefore, this paper calculates the influence of the first group of parameters, as shown in Figure 7 and randomly selects four images from the sea urchin database of the China National Fund Commission for testing.

As shown in Figure 8, a histogram of variances with three Gaussian weighting factors for four pictures is listed. The three parameter variances are sorted as sigma1, sigma2 and sigma3 from largest to smallest. The small-scale parameter sigma1 has a greater impact on the image quality evaluation results than the mesoscale parameter sigma2 and the large-scale parameter sigma3. Therefore, sigma1 is selected for optimization in this paper.

It should be pointed out that the better the parameters, the more the indexes will be improved. With the improvement of underwater robot computing resources, we can optimize all parameters. This article only uses sigma1 as an optimization parameter and puts forward the idea of the algorithm in this paper. It is hoped that the stability can be improved at first and that the accuracy can be improved to some extent without limiting the number of optimized parameters, which could be increased later.

5. Our Algorithm

The three scales—sigma1, sigma2 and sigma3—of the existing MSR algorithm are derived from human experience. In theory, sigma3 depends on the image size while sigma1 depends on the size of target objects and the water turbidity. From the above experiments, it has been verified that sigma1, sigma2 and sigma3 have obvious impacts on image quality, and sigma1 has the greatest impact on image quality. This article makes adaptive adjustments to sigma1 of the MSR algorithm and optimizes and confirms the optimal value of sigma1 through the no reference image quality assessment index.

The existing MSR algorithm artificially sets the three Gaussian weighting factors to one-third, but this does not meet the data distribution characteristics of the underwater image color space and causes image distortion problems. It has been verified from the above experiments that W1, W2 and W3 have a great influence on image quality, and so this article makes adaptive adjustments to the Gaussian weighting factors of the MSR algorithm, and optimizes and confirms the optimal values of Gaussian weighting factors through no reference image quality assessment indexes.

In order to reduce the dimension of the optimization space and improve the optimization speed, the algorithm splits the original target (NIQE) and three variables (sigma1, w1, w2) into two steps for optimization. The first step is the optimization of the target (NIQE) and one variable (sigma1) to find the optimal sigma1. The second step is the optimization of the target (NIQE) and two variables (w1, w2) to find the optimal w1 and w2.

For practical reasons, we make an assumption: in continuous frames in dozens, the environment of the underwater Image is basically unchanged; that is, the size of the target object and the turbidity of the water body are unchanged. Then, we can adopt the strategy of optimizing only in stable frames. During this time, we do not need to repeatedly optimize the values of sigma1, w1 and w2; we can use the same parameters to perform the restoration work, which can increase the enhancement speed.

On this basis, the algorithm of this paper first obtains the acceleration of each frame image through the IMU in the underwater robot. Taking N video frames as a group, the frame with the smallest acceleration in a group of video frames is selected as the stable frame, and the optimal sigma1, first_W1 and first_W2 found by the GSA in the stable frame are used, and the remaining N-1 frame videos use sigma1, first_W1 and first_W2 to implement the image restoration.

Single image enhancement steps are shown in the workflow in Algorithm 1, and the video frame image enhancement steps are shown in Algorithm 2.

Algorithm 1 Single image enhancement steps

Input: Single image Output: Enhanced image  Step 1. Adaptive adjustment is made for sigma1 of the MSR algorithm. Through no reference image quality assessment indexes, the GSA algorithm is used to optimize and confirm the optimal sigma1 value.  Step 2. Adaptive adjustment is made for the gaussian weighting factor of the MSR algorithm, the optimal values of w1, w2, and w3 are optimized and confirmed through no reference image quality assessment indexes.  Step 3. The optimal sigma1, w1, w2, and w3 are applied to the MSR algorithm for underwater image enhancement.

Algorithm 2 Video frame image enhancement steps

Input: A set of video frame images Output: A set of enhanced video frame images  Step 1. The underwater robot calculates the frame with the smallest acceleration in a group of video frame images through the IMU, and sets the frame with the smallest acceleration as the stable frame.  Step 2. Adaptive adjustment is made for sigma1 of the MSR algorithm. Through no reference image quality assessment indexes, the GSA algorithm is used to optimize the optimal sigma1 value of the stable frame.  Step 3. The optimal sigma1 value of the stable frame is used as the sigma1 value of the group of video frames.  Step 4. The gaussian weighting factors optimized by the GSA algorithm for the first frame of each group are first_W1, first_W2, and the following N-1 frame videos use first_W1, first_W2 to realize the restoration of underwater images.

6. Experimental Analysis

6.1. Gsa Optimization Effect

Considering that the traversal method takes a long time, and the underwater environment is complex and changes greatly, in order to reduce the calculation time, we try to use the GSA optimization algorithm instead of the traversal method. Figure 9 is a time comparison diagram of the optimization of two Gaussian weighting factors in 16 video frame images using the traversal method and the GSA algorithm. From Figure 9, it can be found that the optimization efficiency of the GSA algorithm is far ahead in the two-parameter optimization problem, which is five times faster than the traversal method. The optimization algorithm is more efficient than the traversal method, but the accuracy is slightly reduced. This is well. known, and this article only makes a brief description of this fact here.

Four images (Figure 10) are randomly selected from the underwater sea urchin database and optimized to obtain Figure 11. The abscissa in Figure 11 is the number of iterations, and the ordinate represents the sum of R, G and B three-channel NIQE evaluation values. As the number of iterations increases, the score of image quality converges continuously. Figure 12 shows the optimal sigma1 found by the traversal method and the GSA algorithm. The optimal solutions found by these two methods differ in less than two scales, meaning that the GSA algorithm is feasible to use. In Figure 13, the optimization speed of the GSA algorithm is less than half of the traversal method; thus, when the underwater robot performs image enhancement, the GSA algorithm can improve the optimization speed.

6.2. Strategy and Time Analysis of Optimizing Only in Stable Frames

Figure 14 shows a set of 10 consecutive frames of video images collected by an underwater robot. The sequence of these video frames is from left to right and from top to bottom. The subsequent optimization experiments correspond to the order of these 10 pictures.

Figure 15 shows the iterative curves of the optimize_Score value for these 10 continuous frames of video images obtained by the GSA algorithm. The abscissa is the number of iterations, and the ordinate is the NIQE value after the RGB channel fusion of the image. As the number of iterations increases, the image quality scores continue to converge. Judging from the scores of these consecutive video frames, the overall difference is not large—the largest gap is only 0.35—and the overall curve completely converges within 20 epochs.

Figure 16 shows the optimal Gaussian weighting factors (best_W1 and best_W2) obtained by the traversal method and the optimized Gaussian weighting factor parameters (optimize_W1 and optimize_W2) corresponding to the GSA in Figure 15. In the Figure 16, the x-axis is the video frame order, and the y-axis is the Gaussian weighting factor value. From Figure 16, we can se that the best_W1 and optimize_W1 of the 10-frame video are concentrated between 0.1–0.2, the best_W2 and optimize_W2 are basically concentrated between 0.3–0.4, and the Gaussian weighting factors in the same scene are stable.

As can be seen in Figure 16, for w1 and w2, these two parameters show basically no change in the 10 frames of pictures, so the assumption in Section 5 can be confirmed.In continuous frames in dozens, the underwater image environment basically remains unchanged; that is, the size of the target object and the turbidity of the water body are unchanged. Then, we can adopt the strategy of optimization only in stable frames. During this time, we do not need to repeatedly optimize the values of sigma1, w1 and w2. We can use the same parameters to perform restoration work, which can increase the enhancement speed.

It should be noted that, through experiments, we find that not only are the 10 continuous frames as listed in this article are basically unchanged, but also in hundreds of consecutive frames, the image parameters remain basically unchanged; that is, these 10 frames can be replaced with hundreds of frames.

As shown in the Figure 17, the underwater robot used in the experiment is a new type of underwater robot launched by Aquabotix. As shown in the Figure 18, the robot operating system used in the experiment is configured with the GPU hardware platform of Nvidia Geforce GTX1060Ti. Figure 19 is a graph of the running time of the video frame Gaussian weighting factor optimization strategy, where the x-axis represents the number of video frames and the y-axis is the time for which the algorithm runs. The GSA algorithm needs 205 ms to optimize Gaussian factors, and the subsequent image color restoration is about 5 ms for each image.

In terms of time, a stable working time TT1 in this paper is shown in Equation (11):

$$\begin{array}{c}\hfill TT1=T1+10\ast T2,\end{array}$$

(11)

where TT1 is the consumed time of this algorithm in 10 consecutive frames, including one optimization time of T1, and the restoration time T2 of an image after obtaining optimization parameters 10 times. The number of shooting frames of the underwater camera is 30 FPS/s, then the time resource TT1 for this algorithm should be less than 333 ms.

By performing calculation with the GPU hardware platform of Nvidia Geforce GTX1060Ti, T1 in this article is 205 ms and T2 is 5 ms, so TT1 is 255 ms, and TT1 is less than 333 ms, which meets the calculation time requirement given in the project.

Since the optimization time T1 is longer than T2, and the image can be stable for hundreds of frames, according to our own needs, we can increase the number of frames from 10 to conserve more computing resources for further analysis and control. Due to the time limitation in engineering applications, in the MSR-OP algorithm, images with resolution of 400 × 300 can be restored and optimized in a group of 10 frames three times per second. If higher resolution is required, such as 800 × 600 images, 30 frames can be set as a group to meet the requirements of engineering applications, and the MSR-PO algorithm can be restored and optimized once per second.

6.3. Overall Algorithm Effect Analysis

In order to prove the applicability and superiority of the proposed enhancement algorithm, this paper uses the Contrast Limit Adaptive Histogram Equalization (CLAHE), the classic Multi-Scale Retinex with Color Restoration (MSRCR) algorithm and the Dark Channel Prior (DCP) algorithm for comparison. The underwater images processed by different algorithms are analyzed and compared from the two perspectives of subjective visual perception and an objective evaluation index.

In our research, hundreds of images are analyzed in the general underwater image database and the sea urchin database of the China National Fund Committee. Four groups of underwater image enhancement contrast experiments are randomly selected for demonstration here. The first (Figure 20a) underwater image is from the universal underwater database, and the next three underwater pictures (Figure 21a, Figure 22a and Figure 23a) are randomly selected from the sea urchin database of the National Fund of China.

Images (b–d) in Figure 20, Figure 21, Figure 22 and Figure 23 are the effect pictures of different algorithms. Image (b) in Figure 20, Figure 21, Figure 22 and Figure 23 is that of the CLAHE algorithm, image (c) in Figure 20, Figure 21, Figure 22 and Figure 23 is that of the MSRCR algorithm, image (d) in Figure 20, Figure 21, Figure 22 and Figure 23 is that of the Dark Channel Prior algorithm and image (e) in Figure 20, Figure 21, Figure 22 and Figure 23 shows the image generated by MSR-PO algorithm.

Analyzing the results of these three image enhancement algorithms through Figure 20, Figure 21, Figure 22 and Figure 23, it can be seen that the contrast and sharpness of the image processed by these three algorithms are improved to different extents. In scene 1 and scene 2, the CLAHE algorithm and the parameter-optimized MSR algorithm enhance the visual effect of the image very well, but in scene 3 and scene 4, the effect of the CLAHE algorithm is obviously not as good as MSR-PO algorithm. This is because the color channel of the CLAHE algorithm is limited, so the image still has the problem of color casting. In the four scenes, although the effect of the MSRCR algorithm can restore the details of the image well, the overall color of the image fades to white, and there is a slight distortion problem. In general, the MSRCR algorithm is visually inferior to the MSR-PO algorithm. In scene 3 and scene 4, the enhanced image of DCP algorithm is slightly green. This is because the DCP algorithm has a dark channel depth that is not obvious, so the image transmittance cannot be estimated, and the image contrast and color difference are not repaired well. Compared with the other two algorithms, MSR-PO algorithm performs better, and it can be widely used in different scenarios because its effect is more stable.

Table 1. Objective assessment index of underwater image quality.

Scene	Enhanced Algorithm	Entropy	UCIQE	NIQE
Scence 1	None	7.570	0.619	2.885
	CLAHE	7.864	0.650	3.045
	MSRCR	6.507	0.346	2.999
	DCP	7.549	0.6262	2.860
	MSR-PO	7.835	0.654	3.013
Scence 2	None	7.499	0.529	4.445
	CLAHE	7.875	0.592	4.105
	MSRCR	5.899	0.309	4.987
	DCP	7.5643	0.595	4.253
	MSR-PO	7.840	0.628	4.340
Scence 3	None	6.608	0.343	11.365
	CLAHE	7.329	0.438	9.660
	MSRCR	6.303	0.434	8.178
	DCP	7.019	0.382	10.560
	MSR-PO	7.642	0.641	7.341
Scence 4	None	6.595	0.340	12.426
	CLAHE	7.217	0.406	11.149
	MSRCR	6.413	0.421	9.731
	DCP	7.030	0.388	11.780
	MSR-PO	7.731	0.642	8.489

uses information entropy, UCIQE and NIQE to evaluate the images in Figure 20, Figure 21, Figure 22 and Figure 23. It is difficult to determine which algorithm has the best image enhancement effect only by observing Table 1. Therefore, the average index is used to prove the superiority of the MSR-PO algorithm. The final result is shown in

Table 2. Normalized indicator results.

Scene	Enhanced Algorithm	Entropy	UCIQE	NIQE	Average Quality Index
Scence 1	None	0.203	0.214	0.205	0.207
	CLAHE	0.211	0.224	0.195	0.210
	MSRCR	0.174	0.120	0.197	0.164
	DCP	0.202	0.216	0.207	0.208
	MSR-PO	0.210	0.226	0.196	0.211
Scence 2	None	0.204	0.200	0.198	0.201
	CLAHE	0.215	0.223	0.215	0.218
	MSRCR	0.161	0.116	0.177	0.151
	DCP	0.206	0.224	0.207	0.212
	MSR-PO	0.214	0.237	0.203	0.218
Scence 3	None	0.189	0.153	0.162	0.168
	CLAHE	0.210	0.195	0.190	0.198
	MSRCR	0.181	0.194	0.224	0.200
	DCP	0.201	0.171	0.174	0.182
	MSR-PO	0.219	0.287	0.250	0.252
Scence 4	None	0.189	0.155	0.169	0.171
	CLAHE	0.206	0.185	0.189	0.193
	MSRCR	0.183	0.191	0.216	0.197
	DCP	0.201	0.177	0.178	0.185
	MSR-PO	0.221	0.292	0.248	0.253

. An index evaluates the enhanced underwater images with different algorithms, and the best results in the table are bold fonts. After the normalization operation, the following can be found: the highest average quality index of the four scenes is the MSR-PO algorithm; furthermore, the performance of the MSR-PO algorithm is more stable.

7. Conclusions

The existing MSR-based underwater image enhancement algorithms use fixed parameters, and these algorithms are unstable when facing different underwater environments. Therefore, it is important to determine the appropriate parameters according to the changes of different scenarios.

MSR-PO chooses the appropriate NR-IQA method and optimizes the parameters using the MSR algorithm to achieve stable underwater image enhancement, making further underwater exploration possible. Secondly, we also consider the engineering application performance of this method, and we propose a way to restore the general parameters of short-time stable frames.

MSR-PO can complete the optimization and restoration of 30 frames within 1 s. At the same time, we reserve enough computing resources for the AUV’s own analysis and control. This indicates that the MSR-PO algorithm can be applied in engineering applications. By comparing different scenes, the experimental results show that MSR-PO algorithm’s subjective effect and objective quality scores are better than the existing classical MSR based algorithm and DCP algorithm. This shows that MSR-PO can be applied to different underwater environments and the effect is stable.