Application of Artificial Neural Network for Image Noise Level Estimation in the SVD domain

Abstract

The blind additive white Gaussian noise level estimation is an important and a challenging area of digital image processing with numerous applications including image denoising and image segmentation. In this paper, a novel block-based noise level estimation algorithm is proposed. The algorithm relies on the artificial neural network to perform a complex image patch analysis in the singular value decomposition (SVD) domain and to evaluate noise level estimates. The algorithm exhibits the capacity to adjust the effective singular value tail length with respect to the observed noise levels. The results of comparative analysis show that the proposed ANN-based algorithm outperforms the alternative single stage block-based noise level estimating algorithm in the SVD domain in terms of mean square error (MSE) and average error for all considered choices of block size. The most significant improvements in MSE levels are obtained at low noise levels. For some test images, such as “Car” and “Girlface”, at $\sigma =1$, these improvements can be as high as 99% and 98.5%, respectively. In addition, the proposed algorithm eliminates the error-prone manual parameter fine-tuning and automates the entire noise level estimation process.

1. Introduction

In this paper, the problem of noise level estimation is considered, whereby the principal objective is to ascertain the level of noise that is present in a noisy digital image. In general, the blind estimation of noise levels is an important area of digital image processing with numerous applications including image denoising [1,2,3,4,5,6], edge detection [7], super-resolution image reconstruction [8], image segmentation [9,10], and feature extraction [11,12]. Images are commonly degraded by noise during image acquisition, transmission, storage, and image processing [1,13]. Without any prior information, it is difficult to ascertain from a single image the extent to which the observed local variations are associated with the noise signal. It is even more challenging to produce a consistent noise level estimation performance for a wide range of noise levels, image contents, textures, contrast levels, and lighting variations. The blind noise level estimation from a single image is a very active research area and over the decades of research, numerous noise level estimation algorithms have been proposed. Generally, the noise level estimation algorithms can be classified into one of the following broad categories: block-based algorithms, filter-based algorithms, and transform domain algorithms. Each of the aforementioned approaches to noise level estimation has specific characteristics with associated advantages and drawbacks.

The block-based noise level estimation approach [14,15,16,17,18,19] commonly entails image tessellation in the spatial domain. The noise estimates are generally obtained as a result of some statistical analysis that is performed on smooth image blocks. It is assumed that these smooth image patches mostly contain the noise signal, rather than the underlying image signal. The fact that the statistical analysis is restricted to smooth image blocks represents an attempt to mitigate the influence of image texture on the noise level estimation performance. Generally, the performance of the block-based noise level estimation algorithms is affected by the choice of block size, the choice of method that is used to identify the homogenous image blocks, as well as the nature of the statistical analysis that is performed on the selected image blocks.

On the other hand, the filter-based methods [20,21,22,23,24,25] evaluate a noise level estimate on the “difference image”, obtained by subtracting the low-pass filtered image from the noisy input image. The principal assumption is that the difference image contains mostly noise. The ability to sufficiently suppress the underlying image signal depends on the characteristics of the low-pass filter as well as on the nature of the underlying image signal. It is particularly difficult to suppress the image signal that exhibits complex structures and a rich texture. Numerous methods have been proposed to overcome this problem. For example, in Reference [22], an adaptive edge detection is used to exclude the edges from the noise level estimation process. There are also hybrid algorithms [26,27,28] that attempt to combine the simplicity of the block-based approach and the accuracy and robustness of the filter-based approach to noise level estimation.

Transform-based methods perform the image analysis in a domain that offers a higher degree of separability between the noise signal and underlying image signal [29,30,31,32]. The choice of transformation and the associated domain of signal analysis are crucial factors in the performance of transform-based noise level estimation algorithms. The Principal Component Analysis (PCA) is becoming an increasingly relevant approach to patch-based noise level estimations [33,34]. The algorithm presented in Reference [33] does not require the existence of homogeneous areas in the input image to perform accurate noise level estimations. Thus, it can also be applied to textural images. In Reference [34], the authors propose an improvement to the PCA-based noise level estimation by selecting weak textured patches from a noisy image. The weak textured patches are selected based on their gradients and their statistics.

The noise level estimation in the SVD domain [35,36,37] is becoming an increasing popular area of research. In the seminal papers of References [35,36], it is shown that the noise level estimation in the SVD domain is comparable to the state-of-the-art algorithms. The algorithm presented in Reference [35] performs the singular value decomposition (SVD) on the entire image, whereas the algorithms presented in References [36,37] adopt the block-based approach to noise level estimation in the SVD domain. All three algorithms [35,36,37] perform the differential SVD analysis, whereby SVD decomposition is performed not only on the input image but also on the image that is obtained by corrupting the input image further by a known noise source. Thus, these algorithms are able to produce accurate noise level estimates for a range of different images contents. Nevertheless, in this paper, it is demonstrated that these algorithms exhibit a critical drawback. The noise level estimation performance of these algorithms significantly deteriorates at low noise levels. This is particularly true for algorithms in References [35,36]. This is attributed to the fact that the algorithms in References [35,36] perform a relatively simple signal analysis in the SVD domain. They are unable to adapt to the observable noise levels, and in addition, they require the user to adequately initialize the parameters of the algorithm. On the other hand, the algorithm noise level estimation algorithm in Reference [37] is a two-stage algorithm that has the ability to adapt the parameters of the algorithm according to the observed noise levels. The functions that govern the nature of the singular value tail length adaptation according to the observed noise levels needs to be specified for each specific block size. The added complexity, compared to the fixed singular value tail length algorithm in Reference [36], enables the adaptive algorithm in Reference [37] to have a more accurate noise level performance over a low noise level range.

In this paper, a novel block-based noise level estimation algorithm is proposed. The algorithm relies on the artificial neural network (ANN) to perform a more complex signal analysis in the SVD domain. The proposed noise level estimation algorithm has the capacity to adapt to the observed noise levels. It is capable of automatically estimating the parameters of the noise level estimation in the SVD domain, including the optimal singular value tail length for a given noise level, without the need for explicitly modeling or initializing any of these parameters. Thus, compared to the noise level estimation algorithms in References [36,37], the proposed algorithm greatly simplifies the implementation of the block-based noise level estimation in the SVD domain. Furthermore, the proposed algorithm eliminates the error-prone manual parameter fine-tuning and automates the entire noise level estimation process. The main contribution of this paper is the proposed ANN-based algorithm for noise level estimation in the SVD domain. The proposed algorithm simplifies and automates the block-based noise level estimation. The algorithm is capable of performing a complex image patch analysis in the SVD domain that facilitates a consistent noise level estimation performance across a wide range of noise levels. For the aforementioned reasons, the proposed algorithm constitutes a considerable improvement over the alternative block-based SVD domain algorithms in References [36,37]. In addition, the paper considers the effect of the block size choice on the noise level estimation performance, and the optimal choice of block-size is identified. The paper also studies the estimator variability and the effect of texture on the noise level estimation performance.

The remainder of the paper is organized as follows. The image-based [35] and the block-based [36,37] approaches to noise level estimation in the SVD domain are presented in Section 2. The proposed ANN-based algorithm for noise level estimation in the SVD domain is presented in Section 3. In Section 4, a comparative analysis of the proposed ANN-based noise level estimation algorithm with the block-based noise level estimation algorithm in SVD domain [36], the adaptive block-based noise level estimation algorithm [37], and the image-based noise level estimation algorithm in the SVD domain [35] is presented. The variability of the estimator is discussed. The three block-based noise level estimation algorithms are also applied to the problem of image denoising. Finally, Section 5 concludes the paper.

2. Noise Level Estimation in the SVD Domain

The singular value decomposition of some real rectangular $m\times n$ matrix A is defined as

$$\mathrm{A}=US{V}^T$$

(1)

Here, $U\in R^{mxm}$ is an orthogonal matrix. The columns of U denote the eigenvectors of $A{A}^T$. The matrix $V\in R^{nxn}$ is an orthogonal matrix. The rows of $V^T$ correspond to the eigenvectors of $A^{T}A$. Matrix S is a diagonal matrix containing a decreasing sequence of singular values s(1) ≥ s(2) … ≥ s(r) > 0, where r denotes the rank of matrix A. The SVD domain is used in many signal processing applications [38,39,40]. It is particularly suitable for noise level estimation as it enables the separation of the underlying image signal and the additive noise. The noisy image is the sum of the clean image (original) image and the additive noise. Both the original image and noise contribute to the singular value sequence of the noisy image. Let s(i) be some i^th value in the singular value sequence of the noisy image, such that i = 0,1,...,r. The singular value sequence of the noisy image is the sum of the singular value sequence due to the original image and the singular value sequence due to noise [35]. Thus, s(i) = s_s(i) + s_n(i). Here s_s denotes the singular value sequence due to the clean (original) image, whilst s_n denotes the singular value sequence associated with the noise. The underlying image signal mostly contributes to the first few values in the singular value sequence, whereas the noise signal mostly affects the tail end of the sequence. It is important to note that at low noise levels, the effective singular value tail length decreases as the underlying image signal becomes more dominant, and the number of singular values that are mostly affected by the noise signal is reduced.

The effect of Additive White Gaussian Noise (AWGN) on the singular value sequence is demonstrated on a 64 × 64 image block. The image block is corrupted by different levels of additive white Gaussian noise, and the singular value curves are evaluated. The singular value curves, the example image block, and the associated test image are presented in Figure 1. These results demonstrate that the additive white Gaussian noise mostly affects the tail end of the singular value sequence. Also, at low noise levels, the effective singular value tail length is reduced. In general, there are two main approaches to noise level estimation in the SVD domain, an image-based approach [35] and a block-based approach [36,37]. In the image-based approach, the singular value decomposition is performed on the entire image, whereas in the block-based approach, the singular value decomposition is performed on rectangular image patches. In the latter approach, multiple image blocks are used to obtain a noise level estimate for a single image. What follows is a brief description of the image-based [35] and the block-based algorithms [36,37] for the noise level estimation in the SVD domain.

2.1. Image-Based Noise Level Estimation in the SVD Domain

In the seminal paper for the noise level estimation in the SVD domain [35], an image-based noise level estimation algorithm is presented. The main feature of this algorithm is the differential SVD analysis. Each noise level estimate requires the singular value decomposition to be performed twice; once on the input noisy image and once on the image that is obtained by further degrading the input image by a known noise source. The algorithm in Reference [35] is shown to have a consistent performance across a range of image contents. The main steps of the algorithm are presented in Figure 2.

Figure 2 shows a noisy image A as an input into the noise level estimation algorithm. This image is corrupted by an unknown noise source $\sigma$. In the first step, the singular value decomposition is performed on the input image to obtain a sequence of singular values, $s_A\left(i\right).$ In the second step, the average of M trailing singular values $P_A\left(M\right)$ is evaluated as

$$P_A\left(M\right)=\frac{1}{M}{\displaystyle \sum }_{i=r-M+1}^r{s}_A\left(i\right)$$

(2)

Here, M denotes the singular value tail length. In Reference [35], it is proposed that a reliable noise level estimation in the SVD domain requires the singular value tail length to be restricted to $M\in$ [0.25r, 0.8r], where r denotes the matrix rank. The recommended singular value tail length is M = 0.75r. In the third step, the input image A is further degraded by a known noise source, e.g., a zero mean additive white Gaussian noise with the standard deviation ${\sigma }_B=20$ to obtain a new image B. This image is processed in a similar manner as the input image A. In the fourth step, the singular value decomposition is performed on the image B to obtain a sequence of singular values $s_B\left(i\right)$. In the subsequent step, the average of M trailing singular values is evaluated as

$$P_B\left(M\right)=\frac{1}{M}{\displaystyle \sum }_{i=r-M+1}^r{s}_B\left(i\right)$$

(3)

Finally, the noise level estimate $\widehat{\sigma }$ for the entire image is evaluated as

$$\widehat{\sigma }=\frac{\alpha {\sigma }_B^2}{2\left(P_B\left(M\right)-P_A\left(M\right)\right)}-\frac{P_B\left(M\right)-P_A\left(M\right)}{2\alpha }$$

(4)

Here, parameter $\alpha$ defines the slope of a linear function that describes the distribution of average P(M) values across a range of noise levels [35]. This parameter needs to be evaluated offline for each matrix rank and the singular value tail length. The procedure describing how to estimate parameter $\alpha$ is presented in Section 2.2.

2.2. Block-Based Noise Level Estimation in the SVD Domain

The block-based noise level estimation in the SVD domain is presented in Reference [36]. This algorithm applies the SVD analysis that is presented in Reference [35] on the selected image blocks rather than on the entire image. Thus, the two algorithms are intimately related. Essentially, the algorithm in Reference [36] is a block-based variant of the algorithm in Reference [35]. In Reference [36], it is recommended that the block-based noise level estimation algorithm that the singular value tail length, M, is initialized as M = 0.75 × r. For a given block size r × r and the singular value tail length M, parameter $\alpha$ is evaluated offline and stored in a lookup table. The input image A is tessellated into r × r blocks and, approximately, 40% of blocks are randomly selected. Subsequently, the individual blocks are processed via a sequence of steps that are presented in Figure 2, and a noise level estimate is obtained from each selected block. The noise level estimate for the entire image is found as the average value of all independent noise level estimates associated with the individual blocks. An important drawback of the noise level estimation in the SVD domain, as described in References [35,36], is the fact that the value of parameter $\alpha$ needs to be estimated offline.

Parameter $\alpha$ defines the slope of a linear function that describes the distribution of the average P(M) values across a range of noise levels. The value of this parameter depends on the matrix rank r and the singular value tail length M. Its estimation is a nontrivial task that entails a number of distinct steps. For a given noise level, the average P(M) value is evaluated over a set of N = 50 different pure AWGN images or image blocks. The average P(M) values are reported over a range of noise levels. Subsequently, the linear regression is applied to represent the average P(M) values as a function of noise level. The parameter $\alpha$ is equated with the resulting line gradient. The estimation of parameter $\alpha$ is demonstrated on five different choices of block size: 32 × 32, 48 × 48, 64 × 64, 96 × 96, and 128 × 128. For each considered choice of block size,

Table 1. The average P(M) values across a range of noise levels for different choices of block size.

Block Size	Noise Level, σ
	10	15	20	25	30	35	40	45	50	55
32 × 32	34.098	51.081	68.437	85.237	102.381	119.100	137.077	153.730	168.914	188.073
48 × 48	41.999	62.778	84.013	104.561	125.994	146.364	167.638	188.123	209.965	230.173
64 × 64	48.545	72.760	97.179	121.180	145.495	169.147	193.799	217.674	242.477	266.374
96 × 96	59.674	89.173	118.963	148.577	178.454	208.266	237.717	266.795	297.003	327.294
128 × 128	68.784	103.148	137.403	171.966	206.436	240.356	274.850	308.780	343.950	378.292

presents the average P(M) values over a range of noise levels.

In each case, the singular value tail length is fixed to M = 0.75 × r. For each block size, the distribution of the average P(M) values across a range of noise levels and the results of the least square fitting are presented in Figure 3. The estimated $\alpha$ values for each of the considered choices of block size are as follows: ${\alpha }_{32}=3.4052$, ${\alpha }_{48}=4.1867$, ${\alpha }_{64}=4.8394$, ${\alpha }_{96}=5.9391$, and ${\alpha }_{128}=6.8735$. Cleary, the parameter $\alpha$ value increases with the increasing block size.

2.3. Adaptive Block-Based Noise Level Estimation in the SVD Domain

The adaptive block-based noise level estimation in the SVD domain is presented in Reference [37]. Unlike, the previously discussed block-based algorithm [36], this algorithm has the ability to adjust the singular value tail length according to the observed noise levels. The algorithm is a two-stage algorithm. The first stage of the algorithm is essentially the noise level estimation algorithm in Reference [36]. The result of the first stage constitutes an initial noise level estimate. The initial noise level estimate is used to adjust the parameters of the algorithm and to perform a more accurate noise level estimation in the second stage of the algorithm. The length of the singular value tail and parameter $\alpha$ are changed according to the observed noise level estimates. In Reference [37], for different choices of block size, an empirically derived mathematical model is proposed to describe how the singular value tail length is to be adjusted according to the initial noise level estimates. Compared to the block-based noise level estimation algorithm [36], the adaptive block-based noise level estimation algorithm considerably improves the accuracy of noise level estimation. However, the complexity of the noise level estimation procedure is increased.

3. Proposed ANN-Based Algorithm for Noise Level Estimation in the SVD Domain

In this section, the proposed noise level estimation algorithm is presented. The algorithm uses an artificial neural network to perform the image block analysis in the SVD domain. The algorithm is designed with an intent to address some of the deficiencies and limitations of the block-based noise level estimation algorithms in References [36,37]. The principal limitation of the algorithm in Reference [36] is the fact that its performance significantly deteriorates at low noise levels. An inadequate singular value tail length is identified as the main reason for the inconsistent noise level estimation performance across noise levels. The algorithm in Reference [36] uses a fixed singular value tail length. However, one particular singular value tail length value is not appropriate for all noise levels [37]. The singular value tail length that is recommended in Reference [36], M = 0.75r, is more appropriate for higher noise levels. At low noise levels, the underlying image signal tends to dominate the noise signal in the SVD domain and the effective singular value tail length is smaller than the recommended value. When the singular value tail length is either overestimated or underestimated, the ability to generate the accurate noise level estimates is significantly impaired. Another identified drawback of the algorithm in Reference [36] is the fact that its implementation is not straight forward. It involves numerous distinct steps and requires the user to initialize the singular value tail length and to evaluate parameter α offline for each specific image block size and the singular value tail length. As illustrated in the previous section, the estimation of parameter α is a nontrivial task. When these parameters are not initialized precisely, the noise level estimation performance deteriorates. The algorithm in Reference [37] is an adaptive algorithm that can change the length of the singular value tail according to the initial noise level estimates. However, this algorithm increases the number of steps and the number of user-defined parameters even further. The algorithm is a two stages process, whereby the initial noise level estimates are used to adjust the parameters of the algorithm, including the length of the singular value tail. The mathematical model for the singular value tail length adjustment needs to be specified for each block-size. On the other hand, the proposed algorithm greatly simplifies the noise level estimation procedure. It is a single-stage algorithm that avoids the implementation of Equations (2)–(4) altogether. The algorithm does not require the user to specify the parameter $\alpha$ value nor does it require a defined mathematical model for singular value tail adjustment. It is a fully automated model that relies on the trained artificial neural network to perform a complex image analysis in the SVD domain and to define the complex input–output relationships between the differential singular value curves and the noise level.

3.1. Overview of the Proposed ANN-Based Noise Level Estimation Algorithm in the SVD Domain

The proposed algorithm uses an artificial neural network to perform image block analysis in the SVD domain. The output of the artificial neural network is a noise level estimate corresponding to one image block. The input feature vectors are derived from the singular value curves corresponding to the image block and the block that is obtain when the noisy block is further degraded by a known noise source. The proposed algorithm entails the following steps:

• Tessellate an input image A into r × r blocks.
• Randomly select 40% of the available blocks.
• Apply the singular value decomposition on each block to obtain the associated sequence of singular values $s_A\left(i\right)$, where i = 1, 2, …, r.
• Add the AWGN of a known standard deviation, e.g., ${\sigma }_B=20$, to the selected set of image blocks to obtain a new set of image blocks that are now associated with the image B.
• Apply the singular value decomposition on each block corresponding to image B to obtain the associated sequence of singular values $s_B\left(i\right)$, where i = 1, 2, …, r.
• From each selected block, extract the feature vector to form the artificial neural network input as

  $$I=s_B\left(i\right)-s_A\left(i\right)$$

  (5)

  where the size of the feature vector is defined by $i\in$ [0.2r, 1].
• A noise level estimate ${\widehat{\sigma }}_k$ is obtained for each of the K selected image blocks using the artificial neural network that has been trained to perform the noise level estimation in the SVD domain across a range of noise levels.
• Evaluate the noise level estimate for the entire image as the average value of K independent noise level estimates, where each independent estimate is associated with one distinct block.

3.2. Applying Artifical Neural Network for the Noise Level Estimation in the SVD domain

Artificial neural networks have been extensively used in various image processing problems [41,42]. In this paper, a multilayer perceptron is used to estimate the level of noise that is present in a noisy image block. A multilayer perceptron (MLP) belongs to a class of feedforward artificial neural networks. They are fully connected feed-forward network structures, whereby each neuron in one layer is connected w i j {\displaystyle w_{ij}} to every neuron in the following layer of the MLP architecture. MLPs are commonly applied to supervised learning problems where they are trained to model complex input/output relationships. Training entails adjustment of connection weights and biases in order to minimize the error function. The following MLP architecture is adopted in this paper. The MLP consists of one input layer, two hidden layers, and one output layer. The number of inputs in the input layer is a function of block size choice, $\lfloor 0.8r\rfloor$. Two hidden layers have 30 neurons each, and the output layer has one output. The network output denotes the noise level estimate. Hyperbolic tangent sigmoid is used as the neuron activation function. The MLP is trained to perform the noise level estimation in the SVD domain. Specifically, the Levenberg–Marquardt algorithm is used to perform the artificial neural network training, where the weight update at some i^th iteration is defined as

$$w_{i+1}=w_i-{\left[J^T\left(w_i\right)J\left(w_i\right)+\mu I\right]}^{-1}{J}^T\left(w_i\right)e\left(w_i\right)$$

(6)

where J, µ, and e denote the Jacobian matrix, a dumping parameter, and the error vector, respectively. The use of MLP greatly simplifies the implementation of the block-based noise level estimation algorithm. The length of the singular value tail does not need to be initialized by the user as it is automatically evaluated by MLP. Also, it eliminates the need for the user to define the parameter $\alpha$ values for varying singular value tail lengths. However, MLP does not simply model $\alpha$ and the Equation (4). It actually performs a much more complex image block analysis in the SVD domain. Adequately trained MLP is able to effectively adapt the singular value tail length, M, according to the observed noise levels. Thus, at low noise levels, the effective length of the singular value tail is reduced. Correspondingly, the parameter $\alpha$ is also adjusted as this parameter is a function of the singular value tail length.

Overall, the use of MLP enables the proposed noise level estimation algorithm to perform a more complex image analysis in the SVD domain compared to the algorithms in References [35,36,37]. The computational power of MLP enables the proposed algorithm to automatically adapt to the observed noise levels and to provide a consistent noise level estimation performance across a wide range of noise levels and image contents.

Although the proposed method uses a known noise source to further degrade the image patches during the process of noise level estimation, the MLP input is not based on the concatenation of two singular value curves but rather on their difference. Specifically, the MLP feature vector is formed as a difference between the two singular value sequence tails $I=s_B\left(i\right)-s_B\left(i\right)$, where $i\in$ [0.2 × r, 1]. The lower limit, 0.2 × r, denotes the singular value length of M = 0.8r, which is the upper limit of the singular value tail length for the reliable noise level estimation in the SVD domain [35]. The use of the proposed feature vector is justified by the fact that the expression for the noise level estimate from Equation (4) is a function of $P_B\left(M\right)-P_A\left(M\right)$, which in turn is a function of the proposed MLP input:

$$P_B\left(M\right)-P_A\left(M\right)=\frac{1}{M}{{\displaystyle \sum }}_{i=r-M+1}^r(s_B\left(i\right)-s_A\left(i\right))=\frac{1}{M}{{\displaystyle \sum }}_{j=1}^{M}I\left(j\right).$$

(7)

Here, the subscript j is used to index the feature vector elements. The proposed feature vector simplifies the MLP architecture that is required to perform the noise level estimation. For example, for the block size 64 × 64, the length of the feature vector is 51 rather than 128 which would be the case if the feature vector was formed by concatenating the two singular value curves.

4. Results and Discussion

In this section, the performance of the proposed ANN-based noise level estimation algorithm in the SVD domain is evaluated in terms of the average estimation error (AE), mean square error (MSE), and computational time. The objective of the experiments that follow is to compare the performance of the proposed algorithm with the alternative noise level estimation algorithms in the SVD domain. Specifically, the performance of the proposed algorithm is compared to the image-based noise level estimation algorithm in Reference [35] and the block-based noise level estimation algorithm in References [36,37]. The proposed algorithm is implemented for a range of different choices of block size. For each choice of block size, a specific multilayer perceptron is trained to perform noise level estimation across a range of noise levels. In each case, a subset of a test image dataset [43] is used. The training dataset is formed by exposing 50,000 image blocks to different levels of additive white Gaussian noise. For each image block, the noise level is selected randomly from a uniform distribution in the interval (0, 35). Subsequently, a feature vector is derived from each image block using Equation (5). The input and the target training datasets are defined as a set of 50000 feature vectors and a set of associated noise levels, respectively. All considered noise level estimation algorithms are implemented in MATLAB, and the experiments are performed on Intel i7-4510U 2.0 GHz CPU with 8 GB of RAM.

4.1. Mean Square Error and Average Noise Level Estimation Error

In the first experiment, the performance of the proposed noise level estimation algorithm is compared to the image-based noise level estimation algorithm in Reference [35] and the block-based algorithm in Reference [36]. The proposed algorithm and the block-based noise level estimation algorithm in Reference [36] are both implemented for different choices of block size. In this paper, the following choices of block size are considered: 32 × 32, 48 × 48, 64 × 64, 96 × 96, and 128 × 128. The comparative analysis is performed on a set of the first nine standard test images shown in Figure 4.

The intensity levels of all the test images are defined in a range 0–255. Different images at the same AWGN level might have different SNR ratios. The performance of each noise level estimation algorithm is evaluated across a range of noise levels. Specifically, the noise levels are varied from $\sigma =0.5$ to $\sigma =10$ in increments of $\Delta \sigma =0.5$ and from $\sigma =10$ to $\sigma =20$ in increments of $\Delta \sigma =2.$ For each noise level, the simulations are repeated 50 times and the mean square error and the average estimation error are evaluated. The noise level estimation mean square error is defined as

$$E_{MSE}\left(\sigma \right)=\frac{1}{L\times N}{{\displaystyle \sum }}_{n=1}^N{{\displaystyle \sum }}_{l=1}^L{\left({\sigma }_{n,l}-{\widehat{\sigma }}_{n,l}\right)}^2,$$

(8)

where $\widehat{\sigma }$ denotes the noise level estimate. Here, N and L denote a total number of test images and the number of simulations per noise level, respectively. On the other hand, the average estimation error is defined as

$$E_{av}=\frac{1}{L\times N}{{\displaystyle \sum }}_{n=1}^N{{\displaystyle \sum }}_{l=1}^L\left({\sigma }_{n,l}-{\widehat{\sigma }}_{n,l}\right).$$

(9)

In order to simplify the reporting of the results, abbreviations are used to identify a specific noise level estimation algorithm and the choice of block size. The abbreviation “SVD Image” denotes the image-based noise level estimation algorithm in Reference [35]. The proposed artificial neural network-based noise level estimation algorithm and the block-based algorithm in Reference [36] are abbreviated as “ANN” and “SVD”, respectively. In each case, after the abbreviation, a number is followed to indicate the choice of block size. Thus, the abbreviation “ANN 32” identifies the implementation of the proposed noise level estimation based on 32 × 32 blocks. The MSE and the average estimation error results across the selected test images and simulation trials are presented in Figure 5.

The mean square error and the average error results in Figure 5 show that the two block-based algorithms outperform the image-based noise level estimation algorithm over the range of considered noise levels. The difference in the performance between the block-based and the image-based approaches to noise level estimation is the highest at low noise levels. In consideration of the block-based approach to noise level estimation in the SVD domain, it is important to observe that the choice of block size has a significant effect on the noise level estimation performance. However, the two algorithms are not affected by the choice of block size to the same degree. Also, the optimal choice of block size is not the same for the proposed algorithm and the SVD algorithm [36].

The proposed algorithm offers the worst noise level estimation results for the block size 32 × 32. As the block size increases, its performance improves and MSE levels are reduced. The best performance is achieved using 96 × 96 blocks. Further increase in block size, i.e., to the block size 128 × 128, leads to deterioration in the MSE and average error levels. The block-based noise level estimation algorithm in Reference [36] is somewhat less affected by the choice of block size. At low noise levels, it attains the best performance results for the block size 32 × 32, whereas at higher noise levels, the optimal choice of block size is 64 × 64. In a direct comparison of the proposed ANN-based algorithm with the block-based noise level estimation algorithm in Reference [36], the results in Figure 5 show that the proposed algorithm offers a significantly better noise level estimation performance in terms of MSE and average error for all considered choices of block size. The most adequate comparative analysis between the two algorithms can be performed when their implementation is based on their respective optimal block size choices. Thus, the performances of ANN 96 and SVD 32 algorithms are examined in more detail.

At noise level $\sigma =1$, the proposed ANN 96 algorithm and SVD 32 obtain the mean square error values MSE = 1.30 and MSE = 4.27, respectively. Thus, the proposed ANN 96 algorithm reduces the MSE levels by 69.6% compared to the noise level estimation algorithm SVD 32 [36]. At noise level $\sigma =3$, the decrease in MSE level is 75.9%. At noise levels $\sigma =5$ and $\sigma =7$, the decrease in MSE levels are 51.9% and 25.6%, respectively. As the noise levels increase, the performance of the two algorithms become more similar. The average noise level estimation results follow the MSE levels. Both the MSE and average error results demonstrate that the proposed algorithm significantly improves the noise level estimation performance compared to the block-based algorithm in Reference [36]. These results demonstrate the effectiveness of the proposed method in providing a consistent noise level estimation performance across a range of noise levels.

Since, the MSE measure combines the estimator variability and the estimator bias, the MSE results are examined in more detail. In Figure 6, the MSE results associated with the three block-based algorithms are shown for specific test images. In order to increase the clarity of presentation, Figure 6 is restricted to displaying the results over the noise level range $\sigma \in \left[0.5,9.5\right]$, and the performance of each algorithm is presented with respect to their optimal choice of block size. Thus, the SVD domain method [36] is evaluated for the block size 32 × 32. The Adaptive algorithm in SVD domain (ASVD) [37] is evaluated for the block size 64 × 64. The proposed ANN-based method is evaluated for the block size 96 × 96, as determined by the results presented in Figure 5. The results in Figure 6 demonstrate that the proposed ANN-based algorithm is able to achieve lower MSE levels compared to the ASVD algorithm [37], for most test images. On the other hand, both the proposed algorithm and the ASVD [37] significantly outperform the SVD [36] algorithm. In some examples, the improvements in the MSE levels over the SVD algorithm [36] are particularly significant. Consider the noise level $\sigma =1$. On the test image “Car” the proposed ANN 96 algorithm attains the MSE levels of 0.016, whereas the SVD 32 attains the MSE levels of 1.648. Thus, the proposed algorithm reduces the MSE levels by 99%. On the test image “Girlface” and the noise level $\sigma =1,$ the proposed ANN 96 algorithm attains the MSE levels of 0.132, whereas the SVD 32 attains the MSE levels of 1.254. This corresponds to an improvement in MSE levels by 89.5%. Significant improvements in MSE levels are also achieved on other test images and noise levels. The block-based noise level estimation algorithms are strongly affected by the textural properties of the images. Consider the performance of the block-based noise level estimation algorithms on images Lena and Girlface. Clearly, Lena is a stronger textured image compared to the Girlface image. The background in the Girlface image is very uniform. Thus, the noise level estimation algorithms perform better on this image compared to Lena.

In general, the results in Figure 5 and Figure 6 demonstrated that the proposed algorithm outperforms the image-based algorithm in Reference [35] and the block-based algorithms in References [36,37]. The most significant improvements in the performance are associated with the low noise levels. These improvements in the noise level estimation performance can be attributed to the fact that the proposed algorithm is capable of performing a more complex image analysis in the SVD domain. The proposed algorithm is fully automated and does not require the error-prone manual parameter fine-tuning. Both the proposed algorithm and ASVD algorithm [37] attempt to improve the noise level estimation performance of the block-based algorithm in Reference [36]. However, they are conceptually opposites. Whereas, the ASVD algorithm [37] achieves the adaptivity and improvements in the noise level estimation performance through an increase in the complexity of the noise level estimation process, the proposed algorithm has taken the opposite approach, whereby the noise level estimation process is simplified. In ASVD [37], the noise level estimation entails two distinct stages, where the first stage approximates the SVD algorithm in Reference [36]. The initial noise level estimate obtained in the first stage is used to adjust the algorithm parameters. Specifically, the singular value tail length is adjusted according to some empirically derived function. The function is derived for various choices of block size. The parameter $\sigma$ is also adjusted. In addition, the algorithm uses the empirically derived threshold to decide if the algorithm is to enter the second stage. The second-stage noise level estimation is only performed when the initial noise level estimate is sufficiently low.

The proposed algorithm significantly simplifies the noise level estimation process. It is a single stage algorithm. It is even considerably simpler than the SVD algorithm [36]. The noise level estimates for a given image patch do not go through all the steps presented in Figure 2. Specifically, the implementations of Equations (2–4) are avoided altogether. The need to specify parameter $\alpha$ for a given block size and noise level is avoided as well. Instead, the difference between the singular value tails $I=s_B\left(i\right)-s_A\left(i\right)$ is used as an input to an artificial neural network that is trained to perform a more complex image analysis in the SVD domain.

It is the simplicity of the proposed algorithm combined with the capacity of the artificial neural network to describe the complex input–output relationships that enables the proposed algorithm to obtain a consistent performance across a range of noise levels. The proposed algorithm is versatile, easy to implement, fully automated, and does not require complex parameter initialization. For all of the aforementioned reasons, the proposed algorithm constitutes a considerable improvement over the two alternative block-based noise level estimation algorithms in References [36,37].

4.2. Estimator Variability

In the implementation of all considered block-based noise level estimation algorithms, including the proposed algorithm, a set of image patches is randomly selected from a set of available patches obtained through image tessellation. The selected patches are, thus, randomly distributed over the spatial domain. Each patch is processed independently. Each patch provides an independent source of noise level estimate for the entire image. The overall noise level estimate for the entire image is obtained by averaging the individual noise level estimates. In order to study the variability of noise level estimates over the different image patches, the following experiment is proposed. Five different test images are considered. Each image is corrupted by AWGN at $\sigma =3$. Each noisy image is tessellated into 96 × 96 blocks. All available blocks are used to produce the noise level estimate for the image. For each image, the distribution of noise level estimates across available image patches is shown in Figure 7.

Figure 7 shows how the image patch texture and the level of underlying image signal variability can affects the noise level estimation process. The more homogenous image patches tend to yield more accurate noise level estimates. The results show that different image patches corresponding to the same image and the same noise level can result in considerably different noise level estimates. In these examples, the noise estimates can differ by as much as $\Delta \sigma =2$. The averaging of the noise level estimates due to different image patches tends to produce a more reliable noise level estimate for the entire image.

Nevertheless, the random nature of image patch selection introduces the variability in the noise level estimation process. The following example demonstrates the extent of estimator variability. Five test images are corrupted by AWGN at $\sigma =3$. The proposed ANN 96 method is applied to estimate the noise level, and the MSE values are reported. For each image, the experiment is repeated 30 times and the distribution of noise level estimates is presented in Figure 8.

The results show that the noise level estimation is relatively consistent but nevertheless, some variance exists due to the random nature of image patch selection. A more sophisticated approach to image patch selection, whereby weakly textured patches are selected could reduce the estimator variability at the expense of the added computational complexity of the algorithm.

4.3. Highly Textural Images

The textural characteristics of the image can have a significant effect on the performance of noise level estimation algorithms. The proposed algorithms are evaluated on a subset of four images of a highly textural image dataset [44]. The highly textural test images are presented in Figure 9.

The performance of the proposed algorithm is compared to the two alternative block-based noise level estimation algorithms from References [36,37]. The results are presented in Figure 10.

The results show that the noise level estimation algorithms are able to cope with highly textural images. However, they are at their least accurate when the image has a stochastic texture as in Figure 10b, bumpy_0148. In this case all three algorithms overestimated the noise level.

4.4. Image Denoising

In order to demonstrate the importance of having a more accurate noise level estimation algorithm, the three noise level estimation algorithms are applied to the problem of image denoising. Five different test images are corrupted by additive White Gaussian noise at level $\sigma =2$. The three different block-based noise level estimation algorithms, ANN 96, ASVD 64, and SVD 32 are applied to estimate the noise level. Subsequently, a simple adaptive local noise-reduction filtering [45] is applied to denoise the images. The response of the filter at any point (x, y) is defined as

$$\widehat{f}\left(x,y\right)=g\left(x,y\right)-\frac{{\sigma }_{\eta }^2}{{\sigma }_L^2}\left[g\left(x,y\right)-m_L\right]\mathrm{for}{\sigma }_{\eta }^2\le {\sigma }_L^2$$

(10)

The filter operates on a local region defined by a 3 × 3 rectangular window $S_{x,y}$. Here, $g\left(x,y\right)$ denotes the noisy image at $\left(x,y\right)$. The parameters $m_L$ and ${\sigma }_L^2$ describe the local mean and local variance, respectively, of the pixels in $S_{x,y}$. The noise level estimation algorithms produce a noise level estimate for the entire image ${\sigma }_{\eta }$. In the implementation of the adaptive filter, if the term ${\sigma }_{\eta }^2>$ ${\sigma }_L^2$, the factor (${\sigma }_{\eta }^2/{\sigma }_L^2$ ) is set to 1. The noise level estimation results and the associated image denoising results for the five test images are shown in

Table 2. The image denoising performance associated with different noise level estimation algorithms.

Initial Noise Level		$\mathit{\sigma }$ = 2
Noise Estimation Method		SVD 32	ASVD 64	ANN 96
Lena	Noise estimate	$\sigma$ = 2.37	$\sigma$ = 2.77	$\sigma$ = 3.02
	Denoised image MSE	3.20	3.42	3.62
Town	Noise estimate	$\sigma$ = 2.58	$\sigma$ = 2.77	$\sigma$ = 3.26
	Denoised image MSE	3.26	3.33	3.66
Couple	Noise estimate	$\sigma$ = 2.64	$\sigma$ = 2.92	$\sigma$ = 3.48
	Denoised image MSE	3.73	3.94	4.56
Einstein	Noise estimate	$\sigma$ = 2.57	$\sigma$ = 3.07	$\sigma$ = 3.71
	Denoised image MSE	3.43	3.73	4.51
Girlface	Noise estimate	$\sigma$ = 2.08	$\sigma$ = 2.39	$\sigma$ = 2.70
	Denoised image MSE	2.73	2.76	2.87

.

The denoising results are reported in terms of the mean square error that is evaluated between the denoised image and the original (noise free) image. The results show that the proposed ANN 96 algorithm attains the most accurate noise level estimates across all test images. The ASVD 64 attains a somewhat less accurate noise level estimates, whilst the worst performance is attained by the SVD 32 algorithm. The image denoising performance reflects the accuracy of the noise level estimation. Thus, for all five images, the best denoising performance and the lowest denoised MSE values are associated with ANN 96. The worst denoising results are associate with the SVD 32 algorithm. The results clearly demonstrate that the improvements in the accuracy of noise level estimates can have a significant impact on the noise level estimation performance. For example, on image Einstein, the image denoising performance based on the noise level estimation algorithm ASVD 96 is associated with the MSE level 3.43, whilst the algorithm ASVD 96 is associated with denoising MSE = 4.51.

4.5. Computational Time

In

Table 3. The average computational times for the block-based noise level estimation algorithms.

Algorithm	SVD	SVD	ANN	SVD	ANN	SVD	ANN	SVD	ANN	SVD	ANN
Block size	Image	32 × 32	32 × 32	48 × 48	48 × 48	64 × 64	64 × 64	96 × 96	96 × 96	128 × 128	128 × 128
Time (s)	0.0811	0.0590	0.8906	0.0413	0.4146	0.0396	0.2417	0.0435	0.1549	0.0403	0.1073

, the computational time results for the proposed noise level estimation algorithm and the SVD algorithm [36] are presented for different choices of block size. The computational times of the image-based algorithm [35] are also evaluated. The reported results correspond to the average computational time obtained over a set of 50 images of size 512 × 512.

In a direct comparison of the two block-based algorithms for their respective optimal choices of block size, it can be observed that ANN 96 has a 2.6 times higher computational time compared to the SVD 32 algorithm [36]. This is the only identified drawback of the proposed algorithm. However, for most applications, the average computational time of 0.15 s per image is not very significant. In addition, there are many straight forward ways to mitigate this drawback. First, the complexity levels of the MLP architecture can be reduced. A reduction in the complexity of the MLP architecture could lead to a computational time reduction without any significant deterioration in the MSE levels. In addition, the proposed algorithm generates the noise level estimates based on 40% of the available blocks. It is possible that a smaller number of blocks could be used without a notable effect on the noise level estimation performance.

5. Conclusions

In this paper, a novel block-based noise level estimation algorithm is proposed. The proposed algorithm improves the noise level estimation performance and displays a number of advantageous characteristic over the alternative block-based noise level estimation algorithms in the SVD domain in References [36,37]. The proposed algorithms relies on the artificial neural network to perform an image patch analysis in the SVD domain and to evaluate the noise level estimates. The algorithm has the capacity to adjust the effective singular value tail length and the parameter α without explicitly modeling either parameter. In addition, the proposed algorithm eliminates the error-prone manual parameter fine-tuning and automates the noise level estimation process. The comparative analysis results demonstrate that the proposed algorithm outperforms the alternative SVD-domain-based noise level estimation algorithms and both the image-based SVD algorithm in Reference [35] and the block-based SVD algorithm in References [36,37]. Compared to the blocks-based algorithm in Reference [36], the performance improvements are obtained across a range of test images and for all considered choices of block size. For example, at noise level $\sigma =1$, for some images, such as “Car” and “Girlface”, these improvements can be as high as 99% and 89.5%, respectively. Compared to the adaptive block-based noise level estimation algorithm (ASVD) [37], the proposed algorithm considerably simplifies the noise level estimation process and also improves the levels of noise level estimation accuracy. Unlike ASVD [37], it is a single stage algorithm that does not require error-prone parameter initialization. The principal drawback of the proposed method is the added computational time. This drawback is justified by the large improvements in MSE and average error levels and the fact that the process of noise level estimation in the SVD domain is fully automated. Nevertheless, the computational time of the proposed algorithm can be reduced by optimizing the MLP architecture and the number of image blocks used in the SVD analysis. This will be the subject of further research. Since the proposed algorithm performs an averaging of the randomly spatially distributed local noise level estimates, the proposed method cannot be applied to the problem of spatially-varying noise level estimation [46,47].