No-Reference Blurred Image Quality Assessment by Structural Similarity Index

Abstract

No-reference (NR) image quality assessment (IQA) objectively measures the image quality consistently with subjective evaluations by using only the distorted image. In this paper, we focus on the problem of NR IQA for blurred images and propose a new no-reference structural similarity (NSSIM) metric based on re-blur theory and structural similarity index (SSIM). We extract blurriness features and define image blurriness by grayscale distribution. NSSIM scores an image quality by calculating image luminance, contrast, structure and blurriness. The proposed NSSIM metric can evaluate image quality immediately without prior training or learning. Experimental results on four popular datasets show that the proposed metric outperforms SSIM and well-matched to state-of-the-art NR IQA models. Furthermore, we apply NSSIM with known IQA approaches to blurred image restoration and demonstrate that NSSIM is statistically superior to peak signal-to-noise ratio (PSNR), SSIM and consistent with the state-of-the-art NR IQA models.

1. Introduction

Advances in digital techniques enable people to capture, store and send a large amount of digital images easily, which sharply accelerates the rate of information transfer. Images captured by cameras in the real world are usually subject to distortions during acquisition, compression, transmission, processing, and reproduction. The information that an image conveys is related to image quality, thus image quality is very significant for image acquisition and processing systems. If the quality of an image is bad, its processing result is usually bad as well. Therefore, image acquisition and processing systems in real applications need image quality assessment (IQA) to objectively and automatically identify and quantify these image quality degradations. In recent decades, many IQA methods have been proposed to solve this problem. IQA methods can be classified as full-reference (FR), no-reference (NR) and reduced-reference (RR) methods. FR methods, such as the mean squared error (MSE), the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) [1], require the original undistorted images as the references. In addition, RR methods need prior information about the original images, but either the original undistorted images or their prior information are rarely obtained in practice. NR methods can assess the quality of the distorted images only using themselves, thus are more suitable for actual applications. In this paper, we confine our work to NR methods and focus on the degradation of blur.

The popular FR metric SSIM [1] takes advantage of mathematical convenience and matching to known characteristics of the human visual system (HVS) but suffers from a lack of pristine reference images. Recent general NR models, such as [2,3,4], perform well when adapting with HVS, but lose computation convenience when training with distorted images in advance. In this paper, we propose an NR IQA method for blurred images. Our proposed metric, called no-reference structural similarity (NSSIM), is based on re-blur theory and SSIM, and can be regarded as an improvement of SSIM from FR to NR. The first part is the re-blur process, i.e., blurring the distorted images by a Gaussian low-pass filter. The second is to quantify the blurriness of the image by grayscale distribution. Finally we combine luminance, contrast, structure and blurriness to a quality score. Our NSSIM can be easily computed using the input image and its re-blurred image, and no previous training is needed. Experimental results from four popular datasets validate the better performance of NSSIM than the compared FR and NR methods. In addition, we apply NSSIM to evaluate the performance of the blurred image restoration. The results demonstrate that NSSIM performs as well as the state-of-the-art NR IQA metrics and can get better image quality evaluation than the existing FR IQA metrics PSNR and SSIM. The contribution of the paper is two-fold. One is that we propose a novel definition of image blurriness based on the grayscale distribution of the image, and verify its effectiveness of blurriness measurement and fitness with the subjective quality sense of a human. The other is that we extend the famous FR IQA metric SSIM to a no-reference manner, achieving state-of-the-art IQA performance without previous training.

The rest of this paper is organized as follows. In Section 2, we review previous works in IQA. In Section 3, we describe the blurriness definition and computation, and procedure of our model. In Section 4, we evaluate the performance of the proposed approach by comparing it with the state of the art and applying for blurred image restoration. Section 5 concludes the paper.

2. Related Works

FR IQA methods are usually used to get quantitative evaluation of image quality. For example, PSNR is a classical FR IQA that measures the difference between the maximum and minimum grayscale of the image, which is simple to achieve but cannot accurately simulate the HVS. Another popular FR metric SSIM [1] takes the advantage of mathematical convenience and matching to known characteristics of the HVS but suffers from a lack of pristine reference images. To avoid the requirement of the referred undistorted images, NR IQA algorithms are researched for applications when the referred undistorted images are unavailable. According to their capability, NR IQA algorithms can be divided into distortion-specific and holistic models. In the following, we survey NR IQA algorithms that target blur, compression, and several holistically operated models.

2.1. Distortion-Specific NR IQA Algorithms

Distortion-specific NR IQA algorithms assume that the distortion medium is known. Popular blur IQA algorithms model edge spreads and relate these spreads to perceived quality. Sang [5] proposes a blur IQA model by using singular value decomposition (SVD) to evaluate image structural similarity. Caviedes [6] computes sharpness using the average 2D kurtosis of the 8 × 8 DCT blocks and spatial edge extent information. Ferzli [2] evaluates image quality by Just Notice Blur (JNB). Joshi [7] presents an NR IQA method based on continuous wavelet transform. Similarly, the general approach to NR JPEG IQA is to measure edge strength at block boundaries and relates this strength as well as possibly some measure of image activity to perceived quality. Feng [8] measures the visual impact of ringing artifacts for JPEG images. Meesters [9] detects the low-amplitude edges that result from blocking and estimating the edge amplitudes. Wang [10] evaluates image quality by designing a computationally inexpensive and memory-efficient feature extraction method and estimating the activity of the image signal. JPEG2000 ringing artifacts in an image are normally modeled by measuring edge-spread using an edge-detection-based approach. For example, Sazzad [11] computes simple features in the spatial domain, Sheikh [12] assesses image quality by natural scene statistics (NSS) models, and Marziliano [13] calculates edge width by finding the start and end positions of the edge of each corresponding edge in the processed image.

2.2. Holistic NR IQA Algorithms

Holistic IQA algorithms are designed for measuring distortion of unknown type. Holistic models extract common features of various distortions or establish various models for different distortions. BIQI [14] assumes an image is subjected to a wavelet transform over three scales and three orientations using the Daubechies 9/7 wavelet basis [15], and assesses image quality with a two-step framework which estimates the presence of a set of distortions and evaluated the quality of the image along each of these distortions. BLIIND-II [3] is a multiscale but single-stage algorithm through machine learning that operates in the DCT domain, where a number of features, i.e., scale and orientation selective statistics, correlations across scales, spatial correlation and across orientation statistics, are computed from a natural scene statistics (NSS) model of block DCT coefficients. BRISQUE [4] details the statistical of locally normalized luminance coefficients in the spatial domain. Considering images are naturally multiscale, BRISQUE captures 36 features from two grayscales to identify image distortions and activate distortion-specific quality assessment. A support vector regressor (SVR) [16] is used to build a regression module to perceive quality score. NIQE [17] is founded on perceptually relevant spatial domain NSS features extracted from local image patches that capture the essential low-order statistics of natural images. NIQE takes the distance between the quality-aware NSS feature model and the MVG fit to the features extracted from the distorted image as quality score.

The up-to-date NR IQA algorithm NR-CSR [18] applies convolutional sparse representation (CSR) to simulate the entire image as a sum over a set of convolutions of coefficient maps, which has the same size as the image. NR-CSR uses a low-pass filter to obtain the sparse coefficient and calculating the gradient value to score the sharpness. Meanwhile, artificial neural network method has already been used by novel NR IQA algorithms. For example, Fan [19] proposed an NR IQA algorithm based on multi-expert convolutional neural networks (MCNN) and Li [20] proposed an IQA model for realistic blur image-based semantic feature aggregation (SFA).

It should be noticed that the state-of-the-art NR IQA algorithms like BRISQUE and BLIIND-II perform well at adapting HVS but lose computation convenience due to the training by distorted images in advance. Moreover, deep learning methods like MCNN suffer heavy time cost and high-level hardware requirements.

3. The Proposed IQA Metric

Our proposed NR IQA metric for blurred images aims to be applied without advance training using distorted/undistorted images. It can be regarded as an improvement of SSIM from FR to NR by using re-blur theory, thus we call it NSSIM. In this section, we describe the detail of NSSIM in four parts. Firstly, we revisit the FR structural similarity index (SSIM), and then introduce re-blur, which gives the twice-blurred image. The following part describes feature extraction, where we define image blurriness d from image grayscale histogram distribution. Finally, we compare luminance, contrast, structure and blurriness between the original image and its re-blurred image to land a quality score, i.e., our NSSIM.

3.1. Structural Similarity

The FR metric SSIM [1] compares luminance, contrast and structure of distorted image x and pristine reference image y, i.e.,

$$SSIM(x,y)={\left[l(x,y)\right]}^{\mathit{\alpha }}{\left[c(x,y)\right]}^{\mathit{\beta }}{\left[s(x,y)\right]}^{\mathit{\gamma }}$$

(1)

where $l(x,y)$, $c(x,y)$ and $s(x,y)$ respectively represent luminance, contrast and structure comparison functions. $\mathit{\alpha }>0$, $\mathit{\beta }>0$ and $\mathit{\gamma }>0$, which are parameters to adjust the relative weight of the three components.

Weber’s Law [21] indicates that the magnitude of a just-noticeable luminance change $▵I$ is approximately proportional to the background luminance I for a wide range of luminance values. Thus the luminance comparison function is defined as

$$l(x,y)=l\left({\mathit{\mu }}_x,{\mathit{\mu }}_y\right)=\frac{2{\mathit{\mu }}_x{\mathit{\mu }}_y+C_1}{{\mathit{\mu }}_x^2+{\mathit{\mu }}_y^2+C_1}$$

(2)

where ${\mathit{\mu }}_x=\frac{1}{N}{\sum }_{i=1}^N{x}_i$ and ${\mathit{\mu }}_y=\frac{1}{N}{\sum }_{i=1}^N{y}_i$ (N is the total number of image pixels. $x_i$ and $y_i$ are single pixels in x and y) represent the mean intensity of image x and image y respectively. $C_1$ is a positive constant to avoid instability when ${\mathit{\mu }}_x^2+{\mathit{\mu }}_y^2$ is very close to zero. $C_1={\left(K_{1}L\right)}^2$, $K_1\ll 1$ is a small constant, and L is the dynamic range of the pixel values (e.g., 255 for 8-bit grayscale images).

Similarly, the contrast comparison function is

$$c(x,y)=c\left({\mathit{\sigma }}_x,{\mathit{\sigma }}_y\right)=\frac{2{\mathit{\sigma }}_x{\mathit{\sigma }}_y+C_2}{{\mathit{\sigma }}_x^2+{\mathit{\sigma }}_y^2+C_2}$$

(3)

where $C_2={\left(K_{2}L\right)}^2$, $K_2\ll 1$, and ${\mathit{\sigma }}_x={\left(\frac{1}{N-1}{\sum }_{i=1}^N{\left(x_i-{\mathit{\mu }}_x\right)}^2\right)}^{1/2}$ and ${\mathit{\sigma }}_y={\left(\frac{1}{N-1}{\sum }_{i=1}^N{\left(y_i-{\mathit{\mu }}_y\right)}^2\right)}^{1/2}$ are standard deviations of x and y, respectively.

The definition of structure correlation function is

$$s(x,y)=s\left(\frac{x-{\mathit{\mu }}_x}{{\mathit{\sigma }}_x},\frac{y-{\mathit{\mu }}_y}{{\mathit{\sigma }}_y}\right)=\frac{{\mathit{\sigma }}_{xy}+C_3}{{\mathit{\sigma }}_x{\mathit{\sigma }}_y+C_3}$$

(4)

where $C_3$ is a small positive constant to avoid instability when ${\mathit{\sigma }}_x{\mathit{\sigma }}_y$ closes to zero, and ${\mathit{\sigma }}_{xy}=\frac{1}{N-1}{\sum }_{i=1}^N\left(x_i-{\mathit{\mu }}_x\right)\left(y_i-{\mathit{\mu }}_y\right)$.

3.2. Re-Blur

It is known that sharp images contain more high-frequency components, thus grayscale variations between adjacent pixels in sharp images are more distinct than those in blurred images. The re-blur theory [17] explains that the variation of quality of sharp images would be larger than that of blurred images after blur processing, which is also demonstrated in Figure 1. Considering the input image x, we can take re-blurred image y as the reference, and the image quality can be assessed by the quantity of high frequency components measured between x and y.

The re-blur procedure is shown in Figure 2. Considering Gaussian blur as the distortion type in this paper, we apply a Gaussian kernel $k_g$ (i.e., a Gaussian low-pass filter) to the distorted image x to obtain a re-blurred image y, which is formulated as

$$y=x*k_g$$

(5)

where * is the convolution operator, and the Gaussian kernel $k_g$ is sampled from a two-dimensional Gaussian function

$$G(u,v)=\frac{1}{2\pi {\sigma }^2}{e}^{-(u^2+v^2)/\left(2{\mathit{\sigma }}^2\right)}$$

(6)

It should be noted that $k_g$ is parameterized by the kernel size and the standard deviation $\sigma$. The impact of these parameters will be discussed in Section 4.

We decompose an $M\times N\times 3$ image into the high-frequency part $I^H$ and the low-frequency part $I^L$. $I^H$ represents the drastic part while $I^L$ represents the mild part. Thus the image grayscale variation would be sharper in drastic-change area. In order to focus on the main part that contributes to image quality and reduce the time cost, a down-sampling process for the inputted multi-dimensional image by a simple low-pass filter $f=max(1,round(min(M,N)/256\left)\right)$ is applied. If $f>1$, we define $f=\frac{1}{f\times f}\times lpf$, where $lpf={\left[\begin{array}{ccc}1& \cdots & 1\\ ⋮& \ddots & ⋮\\ 1& \cdots & 1\end{array}\right]}_{f\times f}$. Values outside the bounds of the image are computed by mirror-reflecting the image across the border. The filtered image should have the same size of the original image by restricting the points of filtering template in the original image. In addition, we convert both the multi-dimensional original image and the filtered, i.e., the re-blurred image into two-dimensional mode.

3.3. Feature Extraction

Considering the abundant information that images possess, the complexity of images can be represented based on their structure, noise and diversity [22] or based on fuzzy measures of entropy [23] or based on discrete wavelet transform decomposition [24], etc. In this paper, to represent the inputted image and the re-blurred image, we extract luminance, contrast, structure and blurriness of both the down-sampled 2D images, respectively. Then, we ameliorate the traditional structural similarity by combining the blurriness with luminance, contrast and structure in Figure 3. We would like to emphasize image blurriness in this subsection.

Image histogram reports grayscale distribution. As seen in Figure 4, we find that sharp images have a broader grayscale range. In contrast, grayscale distributions of blurred images are narrower and tend to approach the mean value according to their histograms. As is shown in Figure 4, grayscales close to the mean value $\mathit{\mu }$ of the blurred image (e.g., pixels whose grayscale close to the 132.92 in the right histogram in Figure 4) take the most proportion in the histogram. Thus we describe image blurriness by distributing different weights to different pixel values. We assign heavy weights to the pixels those close to image mean grayscale value and fewer weights to those away from that. Thus we define the image blurriness as

$$d=\sum _{g_i=0}^{L}p\left(g_i\right)w\left(g_i\right)$$

(7)

where d represents image blurriness, $g_i$ is gray value whose range varies from 0 to the dynamic range L (e.g., $L=255$ for 8-bit image), $p\left(g_i\right)$ is the proportion of $g_i$ on the whole image, and $w\left(g_i\right)$ represents the weight of $g_i$, which can be calculated as

$$w\left(g_i\right)=\left\{\begin{array}{c}\frac{g_i}{\mathit{\mu }},g_i<\mathit{\mu }\\ \frac{L-g_i}{L-\mathit{\mu }},g_i⩾\mathit{\mu }\end{array}\right.$$

(8)

According to the proposed image blurriness index, we test images in Figure 1 to verify the impact of re-blur process. Using SSIM and image blurriness as image quality indexes, the results are shown in

Table 1. Relationship between SSIM and image blurriness of Figure 1.

Image	SSIM	Blurriness
Sharp	1.0000	1016.6353
Blur	0.8442	1025.8226
Re-blur	0.7839	1034.1684

. It is shown that the quality of a sharp image has a more drastic decline than that of a blurred image. Then we analyze the image quality trend activated by re-blur, i.e., we blur the sharp image of Figure 1 with blur times varying from 0 to 11. Experimental results are shown in Figure 5. The proposed blurriness index has an approximate linear growth while the SSIM index gradually declined slower with the increase of blur times. This demonstrated that a high-quality image has a relatively small SSIM index when taking its blurred image as reference.

In order to verify the consistency between the proposed image blurriness metric and natural blur images, we selected five Gaussian blur images from each of the four datasets, including CSIQ [25], Live II [26], IVC [27] and TID2013 [28], shown in Figure 6.

Table 2. Blurriness of four sets of images shown in Figure 6, demonstrating the consistency between the proposed image blurriness metric and DMOS, i.e., the subjective scores given by the datasets.

CSIQ		LIVE II		IVC		TID2013
DMOS	Blurriness	DMOS	Blurriness	DMOS	Blurriness	DMOS	Blurriness
0.043	1899.433	0.562467	4047.479	4.4	2574.040	5.18919	2142.768
0.142	1916.220	0.963510	4061.419	3.3	2612.444	4.27222	2171.592
0.341	1939.591	1.450490	4073.350	2.6	2636.649	3.48649	2209.623
0.471	1976.500	2.510388	4092.591	1.9	2658.148	3.00000	2250.459
0.750	2044.500	3.541641	4107.935	1.4	2671.689	2.11111	2285.712

indicates that the proposed image blurriness metric has positive correlation with the difference mean opinion scores (DMOS), i.e., the subjective quality scores in CSIQ and LIVE II datasets, and negative correlation with DMOS those belong to IVC and TID2013 datasets. It can be concluded that our blurriness metric can precisely represent the blurriness of natural images and fit the subjective quality assessment of the HVS.

3.4. NSSIM Index

Similar to the definitions of luminance, contrast and structure in SSIM, we define blurriness comparison function as

$$h(x,y)=\frac{2d_x{d}_y+C_4}{d_x^2+d_y^2+C_4}$$

(9)

where $d_x$ and $d_y$ respectively represent blurriness of distorted image x and its re-blurred image y. $C_4$ is to avoid instability when $d_x^2+d_y^2$ closes to zero. Thus, we can calculate luminance, contrast, structure and blurriness to get a new metrics SSIM${}_r$ as

$$SSI{M}_r(x,y)={\left[l(x,y)\right]}^{\mathit{\alpha }}{\left[c(x,y)\right]}^{\mathit{\beta }}{\left[s(x,y)\right]}^{\mathit{\gamma }}{\left[h(x,y)\right]}^{\mathit{\lambda }}$$

(10)

where $\mathit{\lambda }$ is the exponent coefficient of $h(x,y)$. To better capture the blurriness in local areas of an image, we partition an image into $P\times P$ patches of the same size and compute the mean SSIM${}_r$ as

$$MSSI{M}_r(x,y)=\frac{1}{M}\sum _{i=1}^{M}SSI{M}_r\left(x^i,y^i\right)$$

(11)

where $M=P\times P$, and $x^i$ and $y^i$ are the i-th patches in x and y respectively. Finally, to make the quality score in accordance with the subjective impression, i.e., high-quality image gets high IQA score, we define our proposed NSSIM index as

$$NSSIM(x,y)=1-MSSI{M}_r(x,y)$$

(12)

4. Performance Evaluation

4.1. Datasets

We tested the proposed NSSIM on four popular datasets for IQA, including CSIQ [25], LIVE II [26], IVC [27] and TID2013 [28]. All datasets consist of several subsets of different distortion types. In this paper, we used Gaussian blur distortion for experiments. In particular, the CSIQ dataset contains 30 reference images and 150 Gaussian blur images, the LIVE II dataset contains 29 reference images and 145 Gaussian blur images, the IVC dataset contains four reference images and 20 Gaussian blur images, and the TID2013 dataset contains 25 reference images and 125 Gaussian blur images. All blur images in these datasets were used together with their DMOS as subjective quality scores. Some samples of Gaussian blur images for experiments are shown in Figure 6 together with their DMOS. It should be noted that DMOS scores provided by CSIQ [25] and LIVE II [26] have positive correlation with blurriness scores, while DMOS scores provided by IVC [27] and TID2013 [28] change in the opposite direction. As shown in Figure 6, the datasets used for performance evaluation contain blur images with different kinds of nature scenes which are generally distributed, thus are suitable for statistical analysis of evaluation results.

4.2. Indexes for Evaluation

In order to provide quantitative measurement of the performance of our proposed model, we follow the performance evaluation procedures employed in the video quality experts group (VQEG) [29]. We test the proposed IQA metrics using Spearman’s rank order correlation coefficient (SROCC), Pearson linear correlation coefficient (PLCC) and root mean square error (RMSE) as evaluation indexes. SROCC, PLCC and RMSE are respectively defined as

$$SROCC=1-\frac{6{\sum }_{i=1}^N{\left(R_{s_i}-R_{p_i}\right)}^2}{N\left(N^2-1\right)}$$

(13)

$$PLCC=\frac{{\sum }_{i=1}^N\left(s_i-\overline{s}\right)\left(p_i-\overline{p}\right)}{\sqrt{{\sum }_{i=1}^N{\left(s_i-\overline{s}\right)}^2{\left(p_i-\overline{p}\right)}^2}}$$

(14)

$$RMSE=\sqrt{\frac{1}{N}{\sum }_{i=1}^N{\left(p_i-s_i\right)}^2}$$

(15)

where N is the number of images, $s_i$ and $p_i$ represent the i-th scores given by subjective evaluation and objective evaluation, $\overline{s}$ and $\overline{p}$ represent mean subjective quality score and mean objective predicted score, and $R_{s_i}$ and $R_{p_i}$ represent the rank order number of $\overline{s}$ and $\overline{p}$, respectively. SROCC is a nonparametric measure of rank correlation to statistically assess how well the relationship between the subjective quality score and the objective predicted score can be monotonically described, while PLCC is a measure of the linear correlation between them. RMSE represents the square root of the quadratic mean of the differences between the subjective quality scores and the objective predicted scores, and is a frequently used measure. By using these three statistical measures, we can easily analyze the consistency between the subjective quality score and the objective predicted score, which indicates the capability of IQA methods.

4.3. Parameter Setting

In this subsection, the re-blur method including blur type and filter parameters are discussed. We also discuss the exponent coefficients of luminance, contrast, structural and blurriness, i.e., $\mathit{\alpha }$, $\mathit{\beta }$, $\mathit{\gamma }$ and $\mathit{\lambda }$ in Equation (10), to achieve the best performance of our proposed metric.

4.3.1. Filter Type and Parameter for Re-Blur

As mentioned in Section 3.2, we apply a Gaussian low-pass filter to get the re-blurred image of the input image. It should be noted that we compared three types of filters for re-blur, i.e., Gaussian blur, motion blur and defocus blur. The results on LIVE II dataset are illustrated in

Table 3. SROCC of three blur distortion types tested on LIVE II dataset. The best SROCC score is marked in boldface.

Type	Motion Blur	Gaussian Blur	Defocus Blur
SROCC	0.9031	0.9464	0.8848

. We can see that Gaussian blur leads to the highest SROCC. It is easy to understand such results, noting that the filter for re-blur is of the same type as the distortion of the image in LIVE II dataset. Thus we chose Gaussian filter for re-blur in the experiments. Furthermore, the size and deviation of Gaussian filter will impact running time. According to the experimental results shown in

Table 4. The impact of size and deviation of the Gaussian low-pass filter on running time (seconds), which is tested on LIVE II dataset. The minimum running time is marked in boldface.

Deviation\Size(Pixels)	5	7	9	11	13	15
0.5	2.2956	2.2740	2.2875	2.3679	2.3840	2.3569
1.0	2.2553	2.3779	2.2955	2.2769	2.3800	2.3440
1.5	2.2537	2.3232	2.2743	2.2495	2.3779	2.3457
2.0	2.2560	2.4236	2.2846	2.2967	2.4144	2.3379
2.5	2.2574	2.4542	2.3055	2.3547	2.4531	2.4553

, we take the Gaussian filter of 11 × 11 with deviation 1.5.

4.3.2. Patch Quantity

Since we partitioned an image into $P\times P$ patches when calculating NSSIM, the size and amount of the patches have impact on processing time. In

Table 5. Comparison of running time in LIVE II dataset with different patch quantity from 4 × 4 to 64 × 64. The minimum running time is marked in boldface.

Patch Quantity	4 × 4	8 × 8	12 × 12	16 × 16	20 × 20	32 × 32	64 × 64
Running Time (seconds)	3.4769	3.8733	3.6547	2.4373	4.5235	5.4144	18.3218

, we list the running time taken (in seconds) to compute each quality on an image of resolution 768 × 512 and 24-bit deep color from LIVE II dataset on a 2.6 GHz single-core PC with 4 GB of RAM. When $P=16$, each patch is 48 × 32 × 3 and it takes average 2.4373 seconds to evaluate, which is the least running time. Thus, we set $P=16$ in the experiments in this paper.

4.3.3. Exponent Coefficient of Blurriness Comparison Function

In this section, we evaluate the contribution of $\mathit{\lambda }$, which is the exponent coefficient of blurriness comparison function $h(x,y)$. The impact of $\mathit{\lambda }$ tested on LIVE II [26] dataset is shown in Figure 7 with other parameters the same as SSIM [1]. SROCC, PLCC and RMSE come to the best performance when $\mathit{\lambda }=1$, and when $\mathit{\lambda }=0$ our NSSIM degrades to the traditional SSIM [1]. To achieve the best performance and simplify the expression, we set $\mathit{\alpha }=\mathit{\beta }=\mathit{\gamma }=\mathit{\lambda }=1$, $C_1=0.01$, $C_2=0.03$, $C_3=C_2/2$, $C_4=C_2$.

4.4. Comparison with the State-of-the-Arts

We compared the performance of NSSIM against PSNR, original SSIM [1], and several state-of-the-art NR IQA models such as BRISQUE [4], BLIIND-II [3], MCNN [19], IQA-CWT [7], SFA [20], etc. In order to evaluate the statistically significant difference between the proposed metric and the existing IQA algorithms, we performed statistical analysis by paired-sample t-tests and reporting the p-values. The null hypothesis in our t-tests is that the pairwise difference between the proposed metric and others has a mean equal to zero, i.e., the differences in performances presented in the results are not statistically significant. p-values < 0.05 indicates the rejection of the null hypothesis at the 5% significance level, meaning that the differences are statistically significant. It should be noticed that results of some IQA algorithms, such as NR-CSR [18], MCNN [19], IQA-CWT [7], and SFA [20], are collected from the corresponding references, thus their p-values are not shown in the result tables.

As seen from

Table 6. Comparison with ten existing IQA algorithms on CSIQ dataset (150 Gaussian blur images). We take Spearman’s rank order correlation coefficient (SROCC), Pearson linear correlation coefficient (PLCC) and root mean square error (RMSE) as evaluation indexes. In each column, the best performance value is marked in boldface.

Algorithm	SROCC (p-Value)	PLCC (p-Value)	RMSE (p-Value)
PSNR	0.9366 (0.000068)	0.9249 (0.002634)	0.1090 (0.001652)
SSIM [1]	0.9089 (0.003128)	0.8861 (0.014146)	0.1328 (0.007789)
BIQI [14]	0.9182 (0.001193)	0.8974 (0.000621)	0.2851 (0.006433)
BRISQUE [4]	0.9033 (0.015126)	0.9279 (0.004072)	0.1068 (0.000032)
TIP [2]	0.8996 (0.007822)	0.9014 (0.018125)	0.2678 (0.005821)
NIQE [17]	0.8951 (0.005408)	0.9222 (0.009684)	0.1108 (0.000019)
BLIIND-II [3]	0.8915 (0.000362)	0.9011 (0.000067)	0.1243 (0.005118)
NR-CSR [18]	0.8822 (-)	0.8492 (-)	0.1513 (-)
MCNN [19]	0.8751 (-)	0.8882 (-)	0.1317 (-)
IQA-CWT [7]	0.8010 (-)	- (-)	0.1626 (-)
NSSIM	0.8546	0.8971	0.1266

, NSSIM achieves 0.8971 of PLCC and 0.1266 of RMSE on CSIQ [25] dataset, better than SSIM and MCNN metrics. We randomly sampled 100 images from CSIQ dataset by 10 times for the t-test, so that 10 samples of SROCC, PLCCs and RMSE were achieved for each algorithm in comparison. p-values in Table 6 show that t-tests reject the null hypothesis at 5% significance level, i.e., the alternative hypothesis is accepted that the pairwise difference between NSSIM and the other metrics does not have a mean equal to zero. This ascertains the differences of various metrics are the statistically significant.

Table 7. Comparison with ten existing IQA algorithms on LIVE II dataset (145 Gaussian blur images). We take Spearman’s rank order correlation coefficient (SROCC), Pearson linear correlation coefficient (PLCC) and root mean square error (RMSE) as evaluation indexes. In each column, the best performance value is marked in boldface.

Algorithm	SROCC (p-Values)	PLCC (p-Values)	RMSE (p-Values)
PSNR	0.7180 (0.000031)	0.6427 (0.000004)	1.6266 (0.000001)
SSIM [1]	0.8727 (0.005351)	0.8570 (0.000026)	1.8174 (0.000522)
BIQI [14]	0.9119 (0.004789)	0.7144 (0.000756)	2.0347 (0.000011)
BRISQUE [4]	0.9710 (0.010362)	0.9680 (0.004423)	1.5331 (0.001605)
TIP [2]	0.9011 (0.009542)	0.8811 (0.007826)	1.4768 (0.001412)
NIQE [17]	0.9721 (0.017612)	0.9561 (0.010149)	0.1884 (0.005590)
BLIIND-II [3]	0.9177 (0.001637)	0.9111 (0.000488)	0.8750 (0.007369)
MCNN [19]	0.9358 (-)	0.9459 (-)	6.4538 (-)
IQA-CWT [7]	0.9169 (-)	- (-)	7.8650 (-)
SFA [20]	0.9166 (-)	0.8305 (-)	0.7055 (-)
NSSIM	0.9464	0.9689	0.8669

shows that NSSIM achieves 0.9689 of PLCC which performs better than the other ten algorithms on LIVE II [26] dataset, and 0.9464 of SROCC holds the third position among the 11 metrics. A paired-sample t-test was also applied the same as we done on CSIQ [25] dataset. The p-values demonstrate the statistically significant improvement of the proposed metric in terms of PLCC.

Table 8. Comparison with eight existing IQA algorithms on IVC dataset (20 Gaussian blur images). We take Spearman’s rank order correlation coefficient (SROCC), Pearson linear correlation coefficient (PLCC) and root mean square error (RMSE) as evaluation indexes. In each column, the best performance value is marked in boldface.

Algorithm	SROCC (p-Values)	PLCC (p-Values)	RMSE (p-Values)
PSNR	0.7893 (0.000959)	0.8938 (0.001219)	0.5119 (0.005339)
SSIM [1]	0.8080 (0.000672)	0.7821 (0.000021)	0.8348 (0.000059)
BIQI [14]	0.8600 (0.015124)	0.6603 (0.000482)	1.0738 (0.000041)
BRISQUE [4]	0.8239 (0.005389)	0.9009 (0.009336)	0.4960 (0.010058)
TIP [2]	0.8847 (0.012782)	0.8847 (0.007216)	1.1021 (0.001250)
NIQE [17]	0.8638 (0.010644)	0.8994 (0.008812)	0.4990 (0.001229)
BLIIND-II [3]	0.8715 (0.000956)	0.8246 (0.000372)	0.6458 (0.006883)
NR-CSR [18]	0.9239 (-)	0.8775 (-)	0.5478 (-)
NSSIM	0.8886	0.9239	0.4367

indicates that NSSIM achieves 0.9239 of PLCC and 0.4367 of RMSE, which are the best among all nine algorithms tested on IVC [27] dataset. We also run a paired-sample t-test via randomly sampling 15 images from IVC dataset by 10 times. The p-values proves the statistically significant improvement of the proposed metric in terms of PLCC and RMSE.

As seen from

Table 9. Comparison with eight existing IQA algorithms on TID2013 dataset (125 Gaussian blur images). We take Spearman’s rank order correlation coefficient (SROCC), Pearson linear correlation coefficient (PLCC) and root mean square error (RMSE) as evaluation indexes. In each column, the best performance value is marked in boldface.

Algorithm	SROCC (p-Values)	PLCC (p-Values)	RMSE (p-Values)
PSNR	0.8406 (0.000027)	0.9609 (0.000003)	0.3448 (0.000203)
SSIM [1]	0.8646 (0.012710)	0.8737 (0.001305)	0.6071 (0.000518)
BIQI [14]	0.8065 (0.010284)	0.8352 (0.000208)	1.0273 (0.009078)
BRISQUE [4]	0.8505 (0.001421)	0.8630 (0.000932)	0.6287 (0.000455)
TIP [2]	0.8531 (0.002316)	0.8352 (0.005613)	1.5324 (0.000062)
NIQE [17]	0.8325 (0.009727)	0.8639 (0.000686)	0.6267 (0.000188)
BLIIND-II [3]	0.8555 (0.008715)	0.8577 (0.000384)	0.6415 (0.000512)
NR-CSR [18]	0.8520 (-)	0.8339 (-)	0.6476 (-)
NSSIM	0.8995	0.7357	1.0471

, the SROCC of NSSIM is 0.8995, which beats other FR or NR IQA algorithms on TID2013 [28] dataset. A paired-sample t-test was performed as we done on CSIQ and LIVE II dataset. The p-values give the demonstration of the robust superior SROCC performance of NSSIM than those of the other algorithms.

Furthermore,

Table 10. Average IQA performance (mean±standard deviation) on four datasets. We take Spearman’s rank order correlation coefficient (SROCC), Pearson linear correlation coefficient (PLCC) and root mean square error (RMSE) as evaluation indexes. In each column, the best performance value is marked in boldface.

Algorithm	SROCC (p-Values)	PLCC (p-Values)	RMSE (p-Values)
PSNR	0.8211 ± 0.0796 (0.319167)	0.8556 ± 0.1252 (0.835436)	0.6481 ± 0.5828 (0.929265)
SSIM [1]	0.8636 ± 0.0361 (0.356459)	0.8497 ± 0.0404 (0.650212)	0.8480 ± 0.6143 (0.493018)
BIQI [14]	0.8742 ± 0.0451 (0.526389)	0.7768 ± 0.0354 (0.335915)	1.1052 ± 0.3834 (0.164435)
BRISQUE [4]	0.8872 ± 0.0538 (0.738890)	0.9150 ± 0.0383 (0.385953)	0.6912 ± 0.5226 (0.766324)
TIP [2]	0.8847 ± 0.0193 (0.412921)	0.8756 ± 0.0246 (0.310203)	1.0948 ± 0.5053 (0.309981)
NIQE [17]	0.8909 ± 0.0519 (0.811307)	0.9104 ± 0.0336 (0.464863)	0.3562 ± 0.2133 (0.230560)
BLIIND-II [3]	0.8841 ± 0.0232 (0.296689)	0.8736 ± 0.0347 (0.882062)	0.5717 ± 0.2750 (0.741406)
NSSIM	0.8973 ± 0.0328	0.8814 ± 0.0879	0.6193 ± 0.3621

gives means and standard deviations of SROCC, PLCC, and RMSE of tested IQA algorithms on four datasets. Only IQA algorithms tested on all the four datasets are collected. It can be seen from Table 10 that our NSSIM metric achieves the highest mean SROCC and the third best of mean PLCC and RMSE. Meanwhile the t-test results shown in Table 10 led to the acceptance of the null hypothesis at the 5% significance level, indicating that the differences of average performance between NSSIM and the other metrics are not statistically significant on various datasets. This is understandable since our NSSIM cannot achieve significant improvement on every dataset in terms of all indexes. However, considering the differences of image complexity in different datasets, such as image size, contrast and diversity, the experimental results validate that the proposed NSSIM can be adapted to different image categories. Noticing that [3,4,17] all need prior training procedure, our NSSIM performs the best to maintain the balance of IQA and time-efficiency.

These test results also validate that NSSIM is a demonstration of the relationship between quantified image naturalness and perceptual image quality. NSSIM establishes a simple method to identify image quality without reference or prior training on human judgments of blurred images. Besides, compared with up-to-date NR IQA metrics NR-CSR [18], MCNN [19] and SFA [20], NSSIM is less time costly since the needless of training or learning procedure.

4.5. Consistency with Subjective DMOS Scores

We analyzed the consistency between NSSIM scores and the subjective DMOS scores on four datasets. Scatter plots of NSSIM and DMOS are shown in Figure 8. For CSIQ and LIVE II datasets, our NSSIM has negative correlation with DMOS because NSSIM has positive correlation with image blurriness while DMOS has negative correlation with image blurriness. For IVC and TID2013 datasets, NSSIM and DMOS both have positive correlation with image blurriness. The experimental results demonstrate that our NSSIM has good consistent to HVS, thus can be used for IQA effectively.

4.6. IQA for Blurred Image Restoration

The purpose of image restoration is to reduce or erase image degeneration during acquisition, compression, transmission, processing, and reproduction. IQA can be used to evaluate image restoration algorithm by assessing qualities of distorted image and restoration image. Sroubek [30] presented a deconvolution algorithm for decomposition and approximation of space-variant blur using the alternating direction method of multipliers. Kotera [31] proposed a blind deconvolution algorithm using the variational Bayesian approximation with the automatic relevance determination model on likelihood and image and blur priors. In this section, we use the proposed NSSIM to evaluate the performance of image restoration. Two groups of images including original image, blurred image and restorations are evaluated by the proposed IQA algorithm NSSIM and PSNR, SSIM and several state-of-the-art NR IQA algorithms. The experimental results are shown in Figure 9 and Figure 10 and

Table 11. IQA algorithms performance on Group 1 restorations. Original image is the reference (sharp) image given by LIVE II [26] dataset. NSSIM represents the quality score given by the proposed metric.

IQA Algorithm	Original Image	Blurred Image	Sroubek	Kotera
PSNR	\	23.4926	18.5741	16.7602
SSIM [1]	1.0000	0.7255	0.6571	0.5106
BIQI [14]	66.2057	34.4890	79.9299	79.7829
BRISQUE [4]	84.5863	112.1240	87.1534	84.8248
NIQE [17]	18.8170	19.7543	14.9457	14.9441
BLIIND-II [3]	73.5000	54.0000	93.0000	88.5000
SFA [20]	3.2221	2.4976	2.4577	2.6608
NSSIM (×${10}^{-4}$)	38.9533	0.0709	34.1906	39.0565

and

Table 12. IQA algorithms performance on Group 2 restorations. Original image is the reference (sharp) image given by IVC [27] dataset. NSSIM represents the quality score given by the proposed metric.

IQA Algorithm	Original Image	Blurred Image	Sroubek	Kotera
PSNR	\	23.3116	25.1542	25.0633
SSIM [1]	1.0000	0.8132	0.8904	0.8951
BIQI [14]	46.8936	23.1284	49.8244	49.2987
BRISQUE [4]	32.3916	41.5441	36.6743	35.2054
NIQE [17]	21.5543	27.3656	19.2333	19.1788
BLIIND-II [3]	45.8900	30.5677	51.4986	51.6889
SFA [20]	12.8285	6.2353	6.9934	11.0588
NSSIM (×${10}^{-4}$)	50.0684	16.7567	39.5872	42.2391

.

It can be seen from Figure 9 and Table 11 that PSNR and SSIM [1] fail to evaluate the quality of both restorations since the quality scores are smaller than that of the blurred image. BIQI [14] and NIQE [17] succeed to identity the restoration images but the difference between the two restorations is slight. While BRISQUE [4], BLIIND-II [3], SFA [20] and the proposed metrics NSSIM achieve better precision and the differences between two restorations are distinct. Moreover, the NSSIM-predicted score of blurred image is 0.0709 ×${10}^{-4}$, showing obvious variance between the blurred image and the original image. This demonstrates that NSSIM is extremely sensitive to blur. Similarly, Figure 10 and Table 12 also demonstrate that NSSIM is suitable for blur IQA.

5. Conclusions

IQA is important and useful for image acquisition and processing systems in many applications. In this paper, we focus on blurred IQA. We have proposed a novel NR IQA metric called NSSIM based on SSIM and re-blur theory. The proposed NSSIM takes the advantage of SSIM in mathematical convenience and expanded it from FR to NR. We blur the distorted image and take the re-blurred image as a reference. The definition of image blurriness is given by evaluating grayscale distribution. We score image quality by taking four parts of image features into consideration, including luminance, contrast, structural and blurriness. We discussed the impact of parameters of our algorithm on the performance. We tested the proposed NSSIM metric on four datasets. The experimental results show that NSSIM achieves promising performance and has good consistency of HVS. Compared to existing IQA models, NSSIM does not need reference or prior training or learning procedure, which makes it more time-efficient and convenient to apply. We also expanded the proposed metric to IQA for image restoration, which proves our metric is practically useful. We believe that NSSIM has a great potential to be applied in unconstrained environments.