Objective Prediction of Human Visual Acuity Using Image Quality Metrics

Featured Application

It is often difficult to subjectively determine subjects’ uncorrected visual acuity given their age, lack of cooperation with subjective measurement, etc. In other cases, it would be desirable to be able to predict this visual acuity prior to any type of ocular intervention. The proposed method allows such a determination to be objectively made by determining the degradation of the eye’s optical system from a set of natural images.

Abstract

This work addresses the objective prediction of human uncorrected decimal visual acuity, an unsolved challenge due to the contribution of both physical and neural factors. An alternative approach to assess the image quality of the human visual system can be addressed from the image and video processing perspective. Human tolerance to image degradation is quantified by mean opinion scores, and several image quality assessment algorithms are used to maintain, control, and improve the quality of processed images. The aberration map of the eye is used to obtain the degraded theoretical image from a set of natural images. The amount of distortion added by the eye to the natural image was quantified using different image processing metrics, and the correlation between the result of each metric and subjective visual acuity was assessed. The correlation obtained for a model based on a linear combination of the normalized mean square error metric and the feature similarity index metric was very good. It was concluded that the proposed method could be an objective way to determine subjects’ monocular and uncorrected decimal visual acuity with low uncertainty.

1. Introduction

The quality of the optical system, the quality of the retinal image, and subjective visual quality are three highly related concepts in the visual optics field. They are approached by objective functions, such as the modulation transfer function (MTF) of the system, the Zernike decomposition of the wavefront, or by subjective parameters, such as subjects’ decimal uncorrected visual acuity (VA). The relation among these three concepts is clear: if the quality of the human visual system (HVS) is poor, the quality of the resulting image will also be bad and, as a result, VA will be poor.

The retinal image is affected by aberrations of the system, scattering and diffraction of light, and retinal sampling. Nevertheless, the vision process is not simple because it results from the proper combination of physical, optical, physiological, neural, and psychological aspects. Thus, it is relatively normal to find people who indicate they “correctly view” a certain image because they recognize the structure of the object, but not its details. Therefore, they pose no need for refractive correction. Guirao and Williams [1] suggested that the visual quality metrics obtained on the retinal plane are more consistent with subjective measurements than those calculated on the pupil plane. Presently, the most widespread objective criterion to predict visual quality is the visual Strehl ratio (VSOTF). Cheng et al. [2] obtained a correlation between defocus and astigmatism, and 31 different visual quality metrics. They concluded that the VSOTF might be a good objective parameter. They also concluded that the value of the root mean square (RMS) wavefront error, or other parameters like the Strehl ratio (SR), are not reliable indicators of the subjective quality of the retinal image. Thus, they chose a metric that combined the point spread function (PSF) of the system with a spatial sensitivity function. Marsack et al. [3] demonstrated the need for single-value metrics other than RMS to assess the VA effects of low aberration levels. Later, Watson and Ahumada [4] proposed a model for VA that incorporates the set of ocular aberrations, optical and neural filtering, and neural noise.

An alternative approach to assess the image quality of the HVS can be addressed from the perspective of the image and video processing field. On the one hand, human tolerance to image degradation is quantified with mean opinion scores (MOS) [5]. On the other hand, a series of image quality assessment (IQA) algorithms are used to maintain, control, and improve the quality of processed images. Early works focused on comparing the degraded image to the initial one by using two objective parameters: peak signal-to-noise ratio (PSNR) and mean square error (MSE) or its square root (RMSE). MSE is the standard method applied in image comparisons because it is simple and fast but is not usually a good estimator of subjective perception because it does not consider HVS characteristics. It is also an unbounded metric, which makes it difficult to correlate with VA [6]. The PSNR metric is also based on pixel-by-pixel by comparing the reference image to the distorted image through MSE, and it is still one of the most popular ways to assess the quality difference between images [7]. Its disadvantages are that it is not a bounded metric, and it does not consider HVS properties. Thus, it does not correlate well with subjective tests. In the last few years, one of the most commonly used metrics has been the structural similarity index (SSIM) [8] because of the good correlation with MOS.

IQA objective methods can be classified into three types, full-reference, reduced-reference, and non-reference, depending on the use of a reference image, some of the information of that image, or no available reference image to make the comparison, respectively. In this paper, we focus on studying seven metrics that derive from full-reference algorithms to objectively determine subjects’ VA. They are MSE, PSNR, SSIM, the Multi-scale Structural Similarity Index (MSSSIM) [9], the peak signal-to-noise ratio based on the HVS (PSNR-HVS) [10], gradient magnitude similarity deviation (GMSD) [11] and the Feature Similarity Index (FSIM) [12]. These metrics were chosen because they are fast, easy to implement, and sufficiently confirmed.

We establish a relation between image quality metrics, hitherto restricted to the field of signal and image processing, and the quality of vision concept, quantified by monocular VA. As far as we know, no works use static image or video quality metrics to study the quality of human vision, and only two references that link both aspects appear in the literature. Iskander [13] discussed a possible relation between some image processing metrics and subjects’ visual quality. The study was performed for two sets of metrics: those based on comparing images and those based on the optical transfer function. The metric that best correlated with the evaluation of subjects’ ametropia was entropy. Later in [14], some of the authors adapted the MSSIM metric to the visual process (VMSSIM) with two subjects (one myopic and another hyperopic), both before and after being treated by LASIK surgery, to objectively predict their visual quality.

Several studies have been recently published on quantifying the quality of an image, but by combining several full reference metrics [15,16,17]. Their results reveal that these combinations seem to exceed individual metrics when predicting the quality of an image. We approached the problem of assessing HVS quality from the same point of view and proposed combining metrics to predict VA. Most psychophysical experiments are performed with relatively simple patterns, such as blobs, sinusoidal bars or grids, letters, etc. For example, the contrast sensitivity function is usually obtained from thresholds with global sinusoidal images. However, all these patterns are simpler than real-world images, which can be considered as a superposition of a larger number of simple patterns. VA measures in some ways the degradation of a subject’s optimal visual quality. The objective of this work is to relate digital image processing metrics, that use natural images, with VA, since there must be a relationship between them. Subjective VA tests measured using optotypes comprises both physical and neural factors. Natural images are necessary to quantify in some manner for human tolerance to image degradation because using optotypes as images (not natural images) and the wavefront aberration (only physical factors) would not be able to model neural or even subjective factors that in some manners are present in natural images evaluated with MOS.

The manuscript is structured as follows. Section 2 describes the procedure. First, an image database is defined from existing ones. Next, details of the subjects participating in the study and measurements are provided. Then, the calculation of the PSF and the image of the eye are explained. The method section ends by describing the metrics calculation and defining the fitting to subjective VA and conditions. The results appear in the third section and the conclusions are finally stated. The metrics are described in Appendix A.

2. Materials and Methods

This section describes the method proposed to obtain an objective evaluation of subjects’ VA based on the application of image processing metrics and the physical data of eyes. Figure 1 shows the flow chart of the whole process. For a distant object, a hyperopic eye is supposed to use crystalline lens accommodation to focus the image on the retina and thus, obtaining the maximum quality of vision. Such accommodation changes the value of the Zernike coefficient related to the $Z_2^0$ [18] following:

$$Z_2^0=\left\{\begin{array}{c}{R}_p{}^2\frac{\left(S-Ac\right)+\frac{C}{2}}{-4\sqrt{3}}; if S\ge Ac \\ R_p{}^2\frac{\frac{C}{2}}{-4\sqrt{3}}; if S<Ac\end{array}\right.$$

(1)

where S and C are the values, in diopters, of the sphere and cylinder of the studied eye. Thus, to obtain PSF in hyperopic subjects, the possible accommodation of the eye was considered through the adjustment of the average monocular accommodation of a subject as a function of age provided by Duane [19] to a third-degree polynomial.

2.1. Image Database

The image databases commonly used in image processing are available on the World Wide Web. They are composed of reference natural images, the corresponding degraded images, and the mean MOS and/or differential mean opinion score (DMOS) values for a significant number of subjects. We randomly chose a set of 49 different reference images that belong to three distinct image databases: 29 to the LIVE base from the image and video engineering laboratory of the University of Texas [20]; 10 from the IVC database of the research group of communication of images and video of the Research Institute on Communications and Cybernetics [21]; the last 10 images belong to the Toyama-MICT base [22].

2.2. Subjects

We studied 52 randomly selected eyes of 52 subjects of both sexes (50% women, 50% men) who had not suffered any eye disease or trauma. Their age range was wide (18 to 62 years old). The study did not present any invasive action. The tests to perform, their nature, and their purpose were explained to all the subjects. They agreed to undergo them and provided their consent. Experiments were conducted with the approval of the Ethics Committee of the University of Alicante and in accordance with the Declaration of Helsinki. Non-cycloplegic subjective test refractions and subjective monocular logMAR VA without correction for distant vision under photopic lighting conditions (85 cd/m²) were conducted by optometrists. Visionix VX-120 was used to capture corneal topographies by Placido’s ring-based technology, aberration maps of the eye, and tonometry, pachymetry, and anterior chamber data. All these measurements were taken three times per subject during sessions separated by a 24 h time interval. LogMAR VA values were converted to decimal VA for convenience reasons following the relation $V{A}_{decimal}={10}^{\left(-logMAR\right)}$. Pupillary diameters were not measured during the VA assessment. Photopic conditions establish natural pupil diameters ranging between 2 and 4 mm [23] and it is assumed that VA does not vary within that pupil diameter range.

In Figure 2, we represent the characteristics of the studied eyes. Figure 2A shows the refractive errors associated with each examined age group. The prevalence of nearsightedness over farsightedness, with a maximum spherical equivalent refraction of 0.75 D, is evident in most subjects. Figure 2B illustrates the average spherical equivalent of the eyes associated with each age range. Finally, Figure 2C depicts a histogram of the number of eyes with different subjective VA ranges.

2.3. The Point Spread Function and the Image of the Eye

The monochromatic PSF of the subject was computed from the wave aberration function $W\left(x,y\right)$ reconstructed from the Zernike coefficients measured with Visionix VX-120. In this calculation, the first three Zernike coefficients corresponding to the aberration components of piston, tilt X and tilt Y were not considered because they constitute translations and tilts of the reference system, which can be naturally compensated with eye movements. The PSF is obtained as the squared module of the Fourier transform of the generalized pupil function [24]:

$$PSF\left(x,y\right)={\left|FT\left(P\left(x,y\right)\right)\right|}^2$$

(2)

We considered the Stiles–Crawford effect due to the anatomical structure of photoreceptors [25], which can be modeled by an apodizing filter located at the entrance pupil [26]. Then, the generalized pupil function is given by [27]:

$$P\left(x,y\right)=e^{-0.116 R_p^2\left(x^2+y^2\right)}{e}^{-ikW\left(x,y\right)},$$

(3)

where $R_p$ is the pupillary radius, which was provided by the Visionix VX-120 system. In the work of Prakash et al. [28], it is shown that under photopic conditions the pooled pupil diameter is $4.07\pm 0.63$ mm so the Zernike expansion coefficients provided by Visionix were transformed into new ones [29,30] for pupil diameters above $4.07$ mm.

Finally, we considered that the eye’s image of an object could be obtained through the convolution of the function that represents the object $O\left(x,y\right)$ with the $PSF$ of the optical system. By applying the convolution theorem [24], this image can be obtained as the inverse Fourier transform of the product of the convoluted functions, i.e.,

$$I\left(x,y\right)={\mathcal{F}}^{-1}\left\{\mathcal{F}\left[O\left(x,y\right)\right]·\mathcal{F}\left[PSF\left(x,y\right)\right]\right\}$$

(4)

2.4. The Point Spread Function and the Image of the Eye

Having defined how to obtain the degraded eye’s image of an object, the values of the seven studied metrics were determined. We established the metrics result for each eye as the average of the metrics values for the 49 object images.

Subjects’ subjective VA and the results obtained from calculating the metrics and linear combinations of metrics were fitted with a monotonous logistic function that is commonly used to study image quality [15]. It comes as follows:

$$VA\left(Q\right)=a\left(\frac{1}{2}-\frac{1}{1+\exp \left[b\left(Q-c\right)\right]}\right)+dQ+e,$$

(5)

where $Q$ represents any of the used metrics or a linear combination $Q_L$, and (a, b, c, d, e) are the parameters to determine.

The first issue to address in the proposal of the linear combinations of full reference metrics is collinearity. Collinearity is a regression analysis problem; if predictors are in a linear combination, the influence of each one on the criterion cannot be distinguished by overlapping them with one another. In that case, the confidence intervals of the estimating coefficients are often wide, which indicates that the obtained estimates are imprecise and probably unstable. The difficulty of assessing the existence of collinearity lies in determining the maximum degree of the permissible relation between independent variables. No consensus on this issue has been reached. Neter et al. [31] considered a series of indicators to analyze the degree of multicollinearity among the regressors of a multivariate linear model. The simplest is the variance inflation factor (VIF) between two of the regressive variables, which is defined as:

$$VIF=\frac{1}{1-R^2},$$

(6)

where R² is the coefficient of determination between the two variables. According to these authors, if VIF is higher than 10, it can be concluded that the collinearity between the two selected variables is high and will affect multilinear fit by increasing the variance value.

Implementing regression fits requires compliance with a series of assumptions to reach conclusive results. These are the homoscedasticity, normality, and independence of residuals. The homoscedasticity hypothesis establishes that the variability of residuals is independent of the explanatory variables. Failure to comply with this condition may result in the fitted parameters varying according to sample size. Regarding normality, residuals should follow a normal distribution with a zero average. Last but not least, the Durbin–Watson (DW) [32] statistic can be used to verify the independence of residuals, $r_i:$

$$DW=\frac{{{\displaystyle \sum }}_{i=2}^n{\left(r_i-r_{i-1}\right)}^2}{{{\displaystyle \sum }}_{i=1}^n{r}_i^2};,$$

(7)

where n is the number of eyes. The null hypothesis (no statistical evidence that residuals are positively self-correlate) is rejected if the DW value is less than a lower critical value $D{W}_{\left(L,\alpha \right)}$, where $\alpha$ represents the significance level (in this work, $\alpha =0.05$). If DW is higher than an upper ${\mathrm{DW}}_{\left(\mathrm{U},\mathsf{\alpha }\right)}$ critical value, it is accepted that there is no correlation. In intermediate cases, the test is not conclusive. If the value $\left(4-DW\right)$ is less than ${\mathrm{DW}}_{\left(\mathrm{L},\mathsf{\alpha }\right)},$ there is statistical evidence that residuals negatively self-correlate. If the value $\left(4-DW\right)$ is higher than $D{W}_{\left(U,\alpha \right)},$ there is no statistical evidence for a negative self-autocorrelation [33].

Besides fulfilling the above assumptions, it is necessary to establish a criterion to select the metric model that best predicts VA. Wei et al. [34] introduced an objective index (WI) to evaluate the performance of an estimation model. In general, the more accurate the model is, the bigger R is and the smaller the mean of the squares of residuals. The WI index is defined as:

$$WI=\frac{R}{\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{r}_i^2},$$

(8)

Another descriptive statistic, which measures the dispersion of a dataset and can be used to compare models’ performance, is the quartile coefficient of dispersion (QCD) [35]. It is defined by:

$$\overline{QCD}=\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{\left(\frac{Q_3-Q_1}{Q_3+Q_1}\right)}_i,$$

(9)

where $Q_1$ and $Q_3$ are the first and third quartiles of the dataset, which consists of the results of each metric of every eye over the set of 49 images. The higher this coefficient is, the wider the data variability is. Therefore, a model with a low $\overline{QCD }\mathrm{is} \mathrm{expected}.$

3. Results and Discussion

The 49 mathematically degraded images that resulted from the convolution of the PSF of each eye with the original images were obtained. Subsequently, the values of all the metrics for each degraded image, their mean, and their standard deviation (SD) were computed.

Due to the major difference between the values of the metrics, and to compare the coefficients that resulted from the distinct fittings, the metrics results were normalized within the range [0, 1]. We called them nMSE, nPSNR, nGMSD, and nPSNR-HVS. Therefore, linear combinations of nMSE with the other metrics were chosen, i.e., $Q_L={\beta }_{1}nMSE+{\beta }_{2}Q$, with $Q$ being any of the other six herein used metrics. First, we evaluated collinearity with the VIF index (

Table 1. VIF values among metrics.

	nMSE	nPSNR	SSIM	nGMSD	MSSSIM	FSIM
nPSNR	7.64
SSIM	4.15	21.7
nGMSD	3.05	12.3	27.8
MSSIM	4.23	24.8	150	30.8
FSIM	5.93	35.8	76.7	15.2	57.4
nPSNR-HVS	7.78	63.8	27.9	14.6	31.7	57.2

). As shown, the VIF values were high, except for nMSE, which was related to the other metrics.

The generalized reduced gradient resolution algorithm was used to find the minimum value of the sum of the squares of the deviations between the value of subjective VA and the target value provided by the fitting to the logistic function (5). This fitting provided an optimal local solution. Figure 3A–G and Figure 4A–F show the subjective VA for distance vision versus the value obtained for all the metrics and the linear combination of metrics. The fitted functions are plotted in red, and the resulting parameters appear in

Table 2. Parameters obtained for the fittings of the normalized metrics and the linear combinations of nMSE with the other metrics to the logistic function (5).

	a	b	c	d	e	β1	β2
nMSE	−4.000	8.992	0.000	0.078	2.016
nPSNR	−4.364	−8.983	0.775	−4.522	4.211
SSIM	10.29	2.228	0.746	−3.054	3.255
nGMSD	0.002	0.000	−0.352	−2.501	2.580
MSSSIM	24.57	3.075	0.712	−15.19	11.55
FSIM	21.24	3.134	0.864	−12.47	11.78
nPSNR-HVS	−23.08	−3.773	0.729	−17.55	13.51
nMSE-nPSNR	64.78	0.021	45.16	−0.335	15.18	72.10	−12.75
nMSE-SSIM	0.173	155,1	−5.512	−0.223	−1.093	−0.772	−12.21
nMSE-nGMSD	−0,708	4.516	−4,870	0.213	2,418	−0.026	−12.78
nMSE-MSSSIM	37.89	17.24	−0,873	−1.326	17.27	0.111	−2.583
nMSE-FSIM	7.478	8.858	−3.770	−0.632	1.011	−0.944	−6.714
nMSE-nPSNR-HVS	41.54	0.282	3.670	−2.878	10.63	5.839	−1.391

. In all cases, the average amount of residuals was practically zero.

To establish the best fitting, we performed an analysis of variance (ANOVA) with the subjective VA and the values provided by the above fittings of metrics. The ANOVA results are found in

Table 3. ANOVA parameters for the above fittings. Coefficient of determination (R²), standard error of estimate (σ_est), F-number, and t statistic.

	R2	σest	F-Number	t Statistic
nMSE	0.9141	0.1563	532	23.1
nPSNR	0.9114	0.1588	514	22.7
SSIM	0.9218	0.1495	590	24.3
nGMSD	0.8891	0.1777	401	20.0
MSSSIM	0.9170	0.1540	553	23.5
FSIM	0.9266	0.1448	631	25.1
nPSNR-HVS	0.9152	0.1554	539	23.2
nMSE-nPSNR	0.9105	0.1600	509	22.6
nMSE-SSIM	0.9261	0.1454	626	25.0
nMSE-nGMSD	0.9178	0.1533	558	23.6
nMSE-MSSSIM	0.9228	0.1486	597	24.4
nMSE-FSIM	0.9309	0.1406	673	25.9
nMSE-nPSNR-HVS	0.9116	0.1589	516	22.7

. In all cases, regressions provided a very low probability (p < 0.001) of accepting the null hypothesis. The F-number was used to determine whether the high coefficient of determination values occurred by chance. The critical value for a 95% confidence level and a number of points (eyes; n = 52) was 4.03. As the F-number is much higher than the critical value, a significant relationship between the variables in the model must be accepted. Therefore, all the metrics are useful for predicting the subjective VA value. Regarding the t statistic, the metrics provided t values above 20. The critical value was 2.01 with an alpha value equaling 0.05 (probability of the null hypothesis). As the obtained values were higher than the critical value, it can be stated that fittings were statistically significant with a probability over 95%. All the metrics also provided high coefficients of determination, which means that the logistic Function (5) allowed the studied variables to be correlated as a high percentage.

The normality of the distribution of residuals was studied by applying three types of tests: Lilliefors, Anderson–Darling and Jarque–Bera with a significance level $\alpha =0.05$. The three tests indicated that the distributions of residuals could be accepted as normal. Only the obtained residuals using the nGMSD metric did not follow normal distribution.

Regarding the independence of residuals,

Table 4. Durbin–Watson, WI, and $\overline{QCD}$ indices (in red: those metrics or linear combinations that provide positive self-correlations in residuals).

	nMSE	nPSNR	SSIM	nGMSD	MSSSIM	FSIM	nPSNR-HVS	nMSE-nPSNR	nMSE-SSIM	nMSE-nGMSD	nMSE-MSSSIM	nMSE-FSIM	nMSE-nPSNR-HVS
DW	1.30	1.28	1.59	1.73	1.61	1.54	1.42	1.28	1.67	1.62	1.67	1.56	1.35
WI	40.7	39.4	44.7	31.1	42.0	47.7	41.7	38.8	47.4	42.4	45.2	50.8	39.3
$\overline{QCD}$	0.46	0.12	0.15	0.15	0.13	0.06	0.15	0.46	0.14	0.15	0.25	0.07	0.46

shows the computed DW statistics following (7) for the considered metrics. If the value of this statistic was two, the residuals were completely independent. For a sample size of 52 eyes and a significance level of $\alpha =0.05$, the critical values of the statistic were approximately $D{W}_{\left(L,\alpha \right)}=1.49$ and $D{W}_{\left(U,\alpha \right)}=1.60$. The obtained DW values indicated that no performed fit led to a negative self-correlation in residuals. The correlations of the decimal subjective VA with the nMSE, nPSNR, and PSNR-HVS metrics, and with the metrics obtained by the linear nMSE-nPSNR and nMSE-nPSNR-HVS combination, gave DW values below ${\mathrm{DW}}_{\left(L,\alpha \right)}$. Such values indicated statistically significant evidence at 95% and the error terms were positively self-correlated. In contrast, all the other used measures or combinations of measures gave DW values over ${\mathrm{DW}}_{\left(L,\alpha \right)}$, which indicates that residuals did not positively self-correlate, or the test was inconclusive (for SSIM and nMSE-FSIM).

The analysis of models’ performance was completed by calculating the WI and $\overline{QCD}$ indices, which are, respectively defined in (8) and (9), and also presented in Table 4. The higher the WI index, the more accurate the model. Conversely, the lower $\overline{QCD}$, the less spread the results of a metric, which indicates that the metric was relatively independent of the chosen set of images. Thus, its performance was better.

Of the metrics not ruled out by the DW evaluation, the results showed that the nMSE-FSIM metrics combination gave the best results. It can be accepted that residuals were normally distributed, the WI index was higher (higher coefficient of determination), and the index (QCD) had the second lowest value. It also indicated the smallest estimated error for all the metrics used in this paper.

To summarize, the model that best determined subjects’ VA from the image processing metrics based on a logistic function was the nMSE-FSIM combination:

$$VA=7.478\left(\frac{1}{2}-\frac{1}{1+exp\left[8.858\left(Q_L+3.770\right)\right]}\right)-0.632 Q_L+1.011,$$

(10)

with $Q_L=-0.944nMSE-6.714FSIM$ and an uncertainty of $\pm 0.14$.

4. Conclusions

It is often difficult to subjectively determine subjects’ uncorrected VA (their age, lack of cooperation with subjective measurement, etc.). In other cases, it would be desirable to predict this VA prior to any type of ocular intervention. The proposed method allows such a determination to be made objectively by determining the degradation of the eye’s optical system on a set of natural images.

A method is presented to make an objective assessment of subjects’ decimal VA, which can be determined with low uncertainty. The technique is based on using image quality metrics together with the determination of degraded images by the optical part of the HVS. This technique allows both the objective determination and quantification of visual quality. The evaluation of the correlation of VA with the results of different metrics using public domain images revealed that FSIM best performed of all the individual studied metrics. Furthermore, we propose a linear combination of these metrics that provides efficient and effective results, namely $Q_L=-0.944nMSE-6.714FSIM$.

It is worth noting that a common criticism of this type of mathematical prediction model is that correlation can be confused with chance. We believe that both numerical results and the justification of the hypothesis (image quality metrics can objectively determine subjects’ VA) more than suffice to confirm correlation and causality between the studied variables.

It would be interesting to conduct a more extensive study, such as using other grayscale metrics, including the color factor in images or combinations of metrics, to improve the results.