Recent Iris and Ocular Recognition Methods in High- and Low-Resolution Images: A Survey

Abstract

Among biometrics, iris and ocular recognition systems are the methods that recognize eye features in an image. Such iris and ocular regions must have a certain image resolution to achieve a high recognition performance; otherwise, the risk of performance degradation arises. This is even more critical in the case of iris recognition where detailed patterns are used. In cases where such low-resolution images are acquired and the acquisition apparatus and environment cannot be improved, recognition performance can be enhanced by obtaining high-resolution images with methods such as super-resolution reconstruction. However, previous survey papers have mainly summarized studies on high-resolution iris and ocular recognition, but do not provide detailed summaries of studies on low-resolution iris and ocular recognition. Therefore, we investigated high-resolution iris and ocular recognition methods and introduced in detail the low-resolution iris and ocular recognition methods and methods of solving the low-resolution problem. Furthermore, since existing survey papers have focused on and summarized studies on traditional handcrafted feature-based iris and ocular recognition, this survey paper also introduced the latest deep learning-based methods in detail.

1. Introduction

1.1. Background of Biometrics

Presently, methods are being researched and developed to apply biometrics, which are new-generation security measures that use individual physiognomic characteristics, to exceed the older generation security measures, such as passwords, that were previously used. Because earlier security measures, such as passwords, are usually created based on the user’s knowledge or memory, they are easily predicted. Furthermore, they are relatively less secure compared to the biometric methods of the new generation because they can be easily penetrated using brute force and probabilistic reasoning methods [1]. In contrast, because biometrics are implemented using individuals’ unique body parts, the difficulty of stealing and hacking attempts is increased, and the security risks are lower compared to traditional security measures. Security increases even more because replicating and forging body parts are much more difficult compared to attacking traditional security measures [2].

Biometrics employ a subject’s physiognomy as a security measure, usually using the hands, fingers, or face, which are convenient to use. First, the methods that use the palm or fingerprints are widely applied in general [3]. In recent years, studies have been conducted on the recognition method using a vein among the blood vessels located in the hand, such as the palm-vein or finger-vein [4,5,6]. The methods using a hand perform user recognition using a contact-based method with a sensor or camera, but traces remain on the contract surface, through which tracking or forgery may occur [7]. To solve the problems caused by the residual contact traces, contactless methods that use facial images have been considered. These recognition methods using the face commonly record an area (the entire face, an eye, etc.) to capture unique regional features. These methods include face recognition which uses the entire face [8]; ocular recognition which uses only the features of the eye [9,10]; ear recognition which uses various approaches of holistic, geometric, local methods; deep neural networks based on the features of ear region [11]; and iris recognition which segments and uses only the iris [12]. However, although there is no risk of leaving traces because the images are acquired and deployed without contact, unlike the hand-based methods, there are inherent constraints, e.g., the user must be positioned within a certain distance to capture the image. Furthermore, in face recognition, performance declines if facial features are deformed due to wounds, plastic surgery, and aging. Although ear recognition is less affected by these factors, a user should position his or her side view of face to the direction of camera in order to capture the ear image.

To address these issues, iris recognition, which uses the iris region of an eye located in the face, can be applied. The iris exhibits almost no change as the person ages, and because it is protected by the eyelid, deformations caused by external factors rarely occur. Furthermore, because the iris has unique and distinct features, it has the advantage of sufficiently identifying the owner when used in biometrics. However, iris recognition also has its drawbacks. It requires accurate segmentation of the iris region, and in the case of having a dark iris color, a near-infrared (NIR) camera is needed to capture the image. Moreover, it must support a sufficiently high image resolution to obtain a detailed iris pattern in small irises. In response to these problems, ocular recognition has been proposed that uses the iris in situ without accurate segmentation; this method recognizes the entire eye region, which is slightly larger, including the surrounding eyelid region. Ocular recognition has the advantage over iris recognition since it is less difficult to construct the acquisition environment because the constraints are relatively lower. Furthermore, when face or iris recognition is implemented, recognition can be performed immediately with the face or eye image without additional acquisition procedures. Therefore, it can be used as an auxiliary recognition system when face or iris recognition is applied.

1.2. Motivation

A biometrics system performs recognition based on the data input through the acquisition device, and for iris and ocular recognition, the data are commonly acquired as images. Furthermore, in general, security systems are applied in controlled environments and the iris and ocular images are obtained in high-resolution, to lower the risks that affect recognition performance. Therefore, studies on the high-resolution methods have mainly been to distinguish the features of each region more precisely and identify the subjects more rigorously. Conversely, if a low-resolution image is input, the performance of the recognition algorithm decreases dramatically. Because the number of pixels in the low-resolution image is small, only a small amount of information can be obtained, and it is difficult to restore the original information using a simple image interpolation method. That is, a high-resolution image means that it is captured by a camera sensor whose number of pixels is large (usually more than 1,000,000 pixels), whereas a low-resolution image represents that which has been captured by a camera sensor whose number of pixels is small (usually less than 1,000,000 pixels). For example, in the case of iris recognition, the captured iris image is usually regarded as a high-resolution one if the diameter of the iris is larger than 200 pixels in the captured image [13,14].

To solve the low-resolution problem, studies have been conducted on methods, such as super-resolution reconstruction (SR), to reconstruct a high-resolution image from a low-resolution image. Furthermore, general-purpose computing on graphics processing units (GPGPU) technology has been developed based on the improved parallel processing capability of graphic cards now used for general-purpose computing, rather than simply for graphics processing. Consequently, based on the technology developed for high-level parallel computing processes, which could not be handled by the conventional central processing unit (CPU) alone, operations are now possible that could not be processed due to the limitations of earlier devices in the conventional machine learning field. Accordingly, deep learning methods are being actively studied. Since these developments have been applied, convolutional neural networks (CNNs) have been studied for image processing and produced state-of-the-art methods [15,16,17]. These methods that show excellent performance have been also studied for application to SR purposes [18].

In the current survey papers, studies on high-resolution image-based iris and ocular recognition have been summarized, but studies on low-resolution image-based iris and ocular recognition have not been summarized in detail. Therefore, this paper aims to investigate high-resolution iris and ocular recognition to examine the existing methods, and then, in detail, introduce low-resolution iris and ocular recognition for solving the low-resolution problem. Furthermore, since such papers have focused on and summarized traditional handcrafted feature-based iris and ocular recognition methods, this survey paper will detail the latest deep learning-based trends.

1.3. The Scope of this Study

In this paper, we investigate the following points and propose a new approach to iris and ocular recognition in low-resolution images.

• The pros and cons of traditional iris and ocular recognition using high-resolution images are classified according to the approach taken, and the problems arising when a low-resolution image is input, are discussed.
• Iris and ocular recognition studies that have adopted the SR method to solve the low-resolution problem are classified according to the approach taken, and the pros and cons of each method are investigated.
• Iris and ocular recognition approaches that have applied the state-of-the-art deep learning SR methods have performed well.

In Section 2, we analyze the pros and cons of iris and ocular recognition in high-resolution images, which have been previously studied. Then, in Section 3, based on the approach taken, we analyze iris and ocular recognition methods that have solved the low-resolution problem by applying SR methods and summarize their pros and cons. Section 4 provides the conclusion of this survey paper.

2. Iris and Ocular Recognition Methods Using High-Resolution Images

The recognition methods used for the eye region are generally divided into two categories: first, those using the iris region; and second, those using the ocular region. Iris recognition is commonly implemented using high-resolution, good-quality images. Iris and ocular recognition use the entire eye region for recognition and employ high-resolution images obtained using NIR lighting and cameras like those in Figure 1 from the Institute of Automation, Chinese Academy of Science (CASIA) database.

Figure 1 shows that the entire face can be employed in the recognition system, or the iris region only can be immediately acquired and used. This may vary depending on the implementation environment. In general, for recognition, an expensive high-resolution camera is required, and as shown in Figure 1a, a high-resolution image of the whole face region is acquired in which the iris region, Figure 1b, is detected. Besides these, there is a method of acquiring only the iris region, Figure 1c, using a narrow-angle camera with a zoom function. Iris recognition from a high-resolution image focuses on accurate iris segmentation because the image already contains many recognizable features. Then, based on the result, it selects the methods to use for recognition. To explain this process, the next subsection introduces conventional high-resolution image-based iris recognition studies.

2.1. Iris Recognition Methods with High-Resolution Images

Among the biometric methods, iris recognition is mainly applied in environments that require high security and reliability. Every individual has a unique iris feature pattern, and since the iris barely changes due to aging or external factors, it is ideal for use in recognition. Historically, there have been many studies on implementing iris-based biometrics, and Daugman et al. [12] investigated and implemented a specific system. The dashed blue box on the left in Figure 2 represents the process flow of regular high-resolution image-based iris recognition systems.

First, in general, traditional high-resolution iris recognition—as shown in the dashed blue box in Figure 2—roughly detects the eye region from the input image using a method, such as the adaptive boosting (Adaboost) eye detector [19] (step (1)); by preprocessing, it then detects the iris, pupil, eyelid, and eyelash regions from the eye region [12] (step (2)); after preprocessing, to remove the eyelid and eyelash regions that impinge on the pupil and iris regions, the iris region is normalized to transform the Cartesian coordinates into polar coordinates (step (3)); based on the normalized iris region, a CNN or filters, such as the Gabor filter, extract the iris code (step (4)); the final step recognizes the iris through pattern matching using various dissimilarity measurements, such as the Hamming distance between the enrolled image’s iris code and that of the image being recognized (step (5)). In iris recognition using the traditional image processing method, the performance depends on how accurately the iris features are extracted. The performance is evaluated through the error equal rate (EER), an indicator widely used in biometrics. When actual recognition is performed, the EER shows the error rate at a point where the false acceptance rate (FAR) equals the false rejection rate (FRR). The FAR is the probability that a subject is incorrectly accepted, and the FRR is the probability that a user is incorrectly rejected as someone else. The studies on iris recognition in high-resolution images can be categorized as image processing, machine learning, and deep learning methods.

2.1.1. Image Processing Method

First, Thomas et al. [20] applied iris segmentation for image processing, which bases recognition on a random sample consensus (RANSAC). Then, the preprocessing for iris recognition is completed by transforming the iris region using a rubber-sheet model, which is commonly applied in most approaches for iris recognition. Afterward, the performance measurement, based on signals, termed the peak side-lobe ratio (PSLR) is used as the final recognition method. For this, the rubber-sheet image of the iris is transformed into a frequency signal region by the fast-Fourier transform (FFT). Then, the proposed method executes recognition through template matching using the PSLR value as the similarity measure for the enrolled and recognition images. Tahir et al. [21] proposed a novel method for initially detecting the iris region using a morphological filter to remove reflected light and a circle-shaped template for the horizontal and vertical axes. First, the proposed method of detecting the iris region uses a morphological filter to remove the pupil-reflected light, and then uses a circle-shaped template for the horizontal and vertical direction to find the iris region preliminarily. Next, image enhancement processes, such as histogram equalization, Gaussian filter, canny edge detector, and Hough transform, are applied to detect the iris region more accurately. Then, the refine-connect-extend-smooth (R-C-E-S) method is applied to detect the eyelid region, and a mask is created to remove it. This is applied to the iris region detected earlier, thereby obtaining only the complete iris region. After creating an iris code using the obtained iris region, the iris is recognized through template matching using the Hamming distance. Frucci et al. [22] studied iris recognition that uses visible spectrum images, unlike the aforementioned studies that used NIR. In the case of visible spectrum images, there are various changes (color, noise, etc.) in the iris features, unlike in NIR images. This renders it more difficult to implement the recognition system compared to the case of NIR images. In their study, a watershed transformation method is applied to binarize the image, after which a circle fitting algorithm is used to detect the limbus boundary. Based on this region, the canny edge filtering and circle fitting algorithms are used to find the pupil region. The iris region thus found is transformed into a rubber-sheet form to create an iris code, based on which the iris recognition is performed through template matching using the Hamming distance or cosine dissimilarity. Singh et al. [23] proposed an integer wavelet transform (IWT)-based iris recognition comparative to the discrete wavelet transform (DWT)-based iris recognition. It shows better performances than the DWT. Thumwarin et al. [24] proposed a dynamic radius matching method for iris recognition. The size of the iris region can vary due to the pupil. The pupil region becomes small, and the iris region becomes expanded if the illumination is too intense, and vice versa. That is why the author proposes the iris feature extraction method by using a dynamic radius.

The above studies [20,21,22] have proposed iris recognition systems using traditional image processing methods. The advantage of the traditional methods is that they do not need additional computing devices, such as a graphic processing unit (GPU). Conversely, if a predetermined level of input image quality is not guaranteed, the recognition performance may decline, which is a disadvantage. Another disadvantage is that the method of implementing the recognition process is complex. If an unexpected low-resolution image is input into such a system, the recognition performance may degrade sharply, requiring an appropriate system to be implemented when low-resolution images have to be used.

2.1.2. Machine Learning-Based Method

Considering the aforementioned drawbacks of the image processing methods, machine learning methods have been proposed. Salve et al. [25] proposed a recognition method that applies a support vector machine (SVM) and an artificial neural network (ANN) for iris recognition. First, the image preprocessing is virtually identical to the traditional image processing methods. The iris region is segmented using the canny edge detector and Hough transformation methods, and after transforming the segmented region into a rubber-sheet form, the 1D log-Gabor wavelet transform is applied to extract the iris code. The extracted iris code is input into the SVM and ANN-based classifier to perform iris recognition. Nalla et al. [26] proposed an iris recognition method that uses visible and NIR images simultaneously by applying a domain adaption (DA) method to propose an approach that exhibits an accurate performance irrespective of the image acquisition environment (sensor-specific or illuminator wavelength-specific). The input image is converted into a rubber-sheet form via preprocessing, and real-value features are produced using the log-Gabor filter. Then, cross-spectral iris recognition is performed using the DA-naïve-Bayes nearest neighbor (DA-NBNN) method, which applies the NBNN classifier to the DA method using the features. Moreover, they proposed an extended DA-NBNN (EDA-NBNN) method that combines spatial pyramid matching (SPM) to further improve performance. Adamović et al. [27] proposed stylometric features-based iris recognition using machine learning algorithms for classification. They used a Base64 encoder to generate iris templates instead of conventional Daugman’s methods like 1-D or 2-D Gabor filters.

Machine learning methods have the advantage that the recognition process can be implemented more finely, and, in a variety of cases, a high recognition rate can be achieved through training. However, the image must have a threshold or higher resolution to preprocess and transform it into a rubber-sheet model accurately, which is a distinct characteristic of iris recognition systems. Otherwise put, if a low-resolution image is entered, the performance may drop. Furthermore, another problem is the poor recognition performance for images from other environments which have not been learned.

2.1.3. Deep Learning Method

Studies have been conducted on iris recognition that uses deep learning, considering the aforementioned drawbacks of the machine learning-based methods. Gangwar et al. [28] proposed iris recognition that uses a CNN. It is similar to traditional image processing and machine learning approaches up to the process of transforming the iris region into a rubber-sheet form after segmenting it during preprocessing. Afterward, however, the long rectangular-shaped rubber-sheet is divided into halves, which are then vertically attached to each other to produce a square-shaped input image to use as an input of CNN. Instead of converting it into an iris code, the image is input as-is into the CNN, and the weighted filters of the CNN are used to extract the features, which are then used to perform iris recognition. The CNN architecture proposed in their study is called DeepIrisNet, and the features of the fully-connected (FC) layer before the SoftMax [29] layer, which is the last output layer, are used to calculate the similarity score between the enrolled and recognition images, based on which iris recognition technique is performed. Zhao et al. [30] have created a descriptor that creates spatial corresponding iris features based on a fully convolutional network (FCN) model and proposed an extended triplet loss (ETL) to train it. Furthermore, they have implemented a sub-network to obtain appropriate information to identify the iris region. The value obtained through this network is used as an important input in the proposed ETL. FeatNet is used as the main network, and MaskNet is used as the sub-network. To improve performance further over the method proposed by Zhao et al. [30], Wang et al. [31] proposed DRFNet by applying dilated convolution and residual connections. As previously seen, MaskNet is used to calculate the ETL and perform the training. Iris recognition is performed by inputting the segmented iris region image into the DRFNet and MaskNet, respectively. Minaee et al. [32] have used the visual geometry group (VGG)-16 model proposed by Simonyan et al. [16] to perform iris recognition using the eye image containing the iris as-is. Using the features of the iris image extracted using the VGG-16 model without additionally finetuning the iris image, a principal component analysis (PCA) is applied for dimension reduction, and then the multiclass SVM is used for iris recognition. Zhao et al. [33] proposed the deep learning-based iris recognition method using the capsule network architecture to enhance the performance. The capsule network for iris recognition shows the high recognition accuracy with learning part-whole relationships while increasing the robustness of the model. To enhance iris recognition accuracy on the noisy images captured from visible wavelengths, generative model-based data augmentation and exploiting three CNN methods are proposed by Lee et al. [34]. In the research by Wang et al. [35], they performed an iris recognition method using CNN on the cross-spectral iris images. They used a simple shallow CNN model, consisting of 3-convolutional layers, but it may cause performance degradation due to the shallow network. Therefore, they adopted the supervised discrete hashing algorithm to overcome this problem.

In the case of deep learning iris recognition, many methods extract features and perform recognition by constructing a rubber-sheet model by sufficient preprocessing, or by placing more weight on the deep learning model without segmentation and using the self-trained filtering ability. In this case, if a deep learning model is well implemented, it can demonstrate good recognition performance for more variations of images. However, if a low-resolution image is input, the weights of the model trained on high-resolution images may be derived as incorrect results. It is, therefore, also necessary to prepare it to use low-resolution images.

Table 1. Research into iris recognition with high-resolution images (* means that Daugman’s rubber-sheet model is used for iris region normalization); (Ref. and Illum. mean reference and illuminator, respectively).

Approach	Ref.	Illum.	Segmentation Methods	Recognition Methods	Performance	Databases	Pros	Cons
Image processing-based	[20]	NIR	RANSAC	PSLR *	Only graphs exist, and no accuracy is reported	WVU	Ellipse fitting algorithm with RANSAC enables accurate pupil detection	The remaining process is similar to the existing methods, and a specific performance table is not provided
	[21]	NIR	R-C-E-S [36]	Hamming distance *	Accuracy and EER of 96.48% and 1.76% (CASIA v1), 95.1% and 2.45% (CASIA v4), 93.6% and 3.2% (SDMULA)	CASIA (v1, v4 Lamp), SDUMLA-HMT	Can accurately detect pupil region unaffected by specular reflection using morphological filtering, and define iris ROI by canny edge detector and Hough transform	Requires a high-resolution image with detectable edges
	[22]	Visible	Watershed	Hamming distance, cosine distance *	Measured decidability of 2.0335 (UBIRISv1), 1.3850 (UBIRISv2)	UBIRIS v1 session 2, Subset of UBIRIS v2	Can define iris ROI region even in a noisy visible environment	Inconclusive that accurate iris ROI provides high recognition performance in noisy images, and implementation is complicated
	[23]	NIR	Handcrafted segmentation algorithm	Integer wavelet transform (IWT)	EER of 0.12%	UBIRIS v2	Not require additional computation devices (e.g., GPU)	High performance of iris segmentation is needed
	[24]	NIR		Dynamic radius matching	Accuracy of 94.89%	CASIA v1.0	Can be used when the size of the iris region is variable	This method is based on gray pixel values, so preprocessed image is required (e.g., removing specular reflection)
Machine learning-based	[25]	NIR	Canny edge detector and Hough transform	SVM or ANN *	SVM classification Accuracy of 94.6% (polynomial kernel), 95.9% (RBF kernel)	CASIA-iris-v4 Interval, Lamp, Syn, Thousand, and Twins	Performance is less affected by test data than by image processing method	The number of test images is small and the test is conducted in a closed-world scenario
	[26]	Visible + NIR	Handcrafted segmentation algorithm	EDA-NBNN *	Bi-spectral iris recognition (EER) of 3.97% (NIR), 6.56% (Visible)	IIIT-D CLI, ND Cross sensor 2012 iris, PolyU cross-spectral iris	Can use cross-spectral images from a learning-based feature	Performance is not much higher than single spectral recognition
	[27]	NIR	Handcrafted segmentation algorithm	OneR, J48, SMO, MultiboostAB, Random Forest, Support Vector Classification, Gradient Boosting	Accuracies of 0.9926~0.9997 (CASIA-iris-v4)	CASIA-iris-v4 MMU, IITD	Reduces the computation costs and increase the ability of discrimination by using the Base64 encoder for feature extraction	Preprocessing is still required, and it raises a security problem which comes from the small number of iris codes
Deep learning-based	[28]	NIR	Osiris	DeepIrisNet *	EER on two merged databases of 1.82%	ND-iris-0405, ND-CrossSensor-Iris-2013	Exhibits high accuracy based on deep features	Requires input image to be preprocessed
	[30]	NIR	Relative total variation-L (RTV-L) [37]	Triplet-loss with Two FCN *	EER of 0.99% (ND-iris-0405), 3.85% (CASIA-iris-v4 distance), 0.64% (IITD), 2.28% (WVU Non-ideal)	ND-iris-0405, CASIA-iris-v4 Distance, IITD iris, WVU Non-ideal	Extracts more sophisticated features using two CNNs	Performance is affected by Daugman’s rubber-sheet model
	[31]	NIR	Haar cascade eye detector	DRFNet with ETL *	EER of 1.30% (ND-iris-0405), 4.91% (CASIA-iris-v4 distance), 1.91% (WVU Non-ideal)	ND-iris-0405, CASIA-iris-v4 Distance, WVU	Uses more spatial features by dilated convolution	Requires converting iris image to Daugman’s rubber-sheet model by preprocessing
	[32]	NIR	No segmentation	VGG feature extraction-based iris recognition	Accuracy of 99.4%	CASIA-iris-v4, Thousand, IITD iris	Requires no iris segmentation	Uses a conventional VGG-16 model for feature extraction
	[33]	NIR	Handcrafted segmentation algorithm	Capsule network architectures	Accuracy of 99.37% and EER of 0.039% (JluV3.1), Accuracy of 98.88 and EER of 0.295% (JluV4), Accuracy of 93.87% and EER of 1.17% (CASIA-iris-V4 Lamp)	JluV3.1, JluV4, CASIA-iris-v4 Lamp	Outstanding performances have been shown	Preprocessing and sophisticated algorithm implementation are needed
	[34]	Visible		Three CNNs	EER of 8.58% (NICE-II), EER of 16.41% (MICHE), EER of 2.96% (CASIA-iris-v4 Distance)	NICE-II, MICHE, CASIA-iris-v4 Distance	Shows the better performances using the data augmentation based on the deep generative model to the noisy images	Requires more computational costs because that needs preprocessing, data augmentation, and three CNNs
	[35]	Visible +NIR		CNN and SDH	EER of 5.39%	PolyU cross-spectral iris	Achieves a more accurate performance using CNN while reducing the size of iris template by supervised discrete hashing	Preprocessing for input images and additional training of supervised discrete hashing parameters are required

below shows a summary of the aforementioned high-resolution image-based iris recognition studies.

2.2. Ocular Recognition Using High-Resolution Images

The dashed red box on the right side in Figure 2 shows the ocular recognition process with high-resolution images. In general, ocular recognition can be applied as a dual security device or an alternative to enhance iris recognition or cope with environments where it is difficult to obtain high-resolution images. Regarding implementation, it has the advantage that the maintenance and design costs are somewhat lower than those of other iris recognition systems. The reason is that the recognition image can be obtained more easily, compared to the conventional methods, because the ocular ROI can be extracted immediately from the face image or the general eye region. Furthermore, the ocular region contains various features, such as the pupil, iris, sclera, eyelid, and eyelash, and they affect the performance depending on how much these features are used for recognition. Another advantage is that these methods do not require accurate segmentation of the iris region, unlike the iris recognition methods. Put differently, because ocular recognition methods detect the approximate ocular region by preprocessing, they have the advantage that the algorithm’s complexity and processing time are reduced compared to those used for iris recognition [38]. Existing studies on ocular recognition in high-resolution images can be categorized as the image processing, machine learning, and deep learning approaches.

2.2.1. Image Processing-Based Method

Vyas et al. [39] have proposed an ocular recognition method using the eye images acquired in the NIR and visible environments simultaneously. This method uses only the periocular region, excluding the eye region, for the ocular region; segments the iris region separately; and obtains features from the periocular region and the iris region through the feature descriptor. Then, the features are concatenated for final recognition. Recognition is performed based on the local image description method by using both the periocular and iris regions obtained in cross-spectral illumination.

The advantage of these traditional image processing methods is that additional devices, such as GPUs, are not required. Conversely, however, there is a disadvantage since the recognition performance may diminish if a predetermined level of input image quality is not guaranteed when implemented. Furthermore, if a low-resolution image is inputted, the recognition performance may decline sharply, which means that an appropriate system must be implemented again in situations where low-resolution images must be used.

2.2.2. Machine Learning Method

After noticing that the conventional iris recognition methods have a weakness because the eyes must always remain open, Liu et al. [40] proposed an ocular recognition method, which performs recognition even with blinking eyes. In this method, after obtaining frontal images of the subject’s face, landmark points are detected to find the ocular region from the image. The landmark points found for the left and right eyes are used to extract the features of the surrounding ocular region. Here, geometric features are extracted using the histogram of gradient (HOG) method, and features are extracted using the local binary pattern (LBP) method and the weighted LBP. Finally, recognition is performed based on the SVM using the extracted features.

The machine learning methods have the advantage that a higher recognition rate can be achieved compared to the image processing methods through training for various cases. However, if a low-resolution image is entered, the performance may decline. Another problem is that the recognition performance may drop for images of other environments that have not been trained.

2.2.3. Deep Learning Method

Lee et al. [41] studied ocular recognition using deep learning. In this method, the ocular ROI is selected using a rough pupil detection method, and this ROI is used to extract the features needed for recognition. Here, features are extracted using a deep residual CNN. Reddy et al. [42] have proposed OcularNet composed of the residual connection-based PatchNet. This method localizes the landmarks of the eye region, based on which six patch ROIs are chosen. These selected patches are extracted using PatchNet, the proposed model, and different PatchNets are used for each of the six regions rather than using just one PatchNet for all. Patterns are matched using the Euclidean distances drawn from the six feature sets and using the enrolled images and verification images as inputs. Reddy et al. [43] proposed robust subject-invariant feature learning utilizing an autoencoder for recognition using ocular images acquired in the visible spectrum. They proposed ocular recognition when only the autoencoder is used, when using the proposed encoded feature loss, and when combining various methods, as well as when using the autoencoder alone by assigning a sparsity constraint by KL divergence. To perform ocular recognition using the autoencoder in the visible spectrum, they proposed using the encoded feature loss and KL divergence sparsity constraint, which allows subject-invariant learning of the ocular region’s features. The study by Vizoni et al. [44] found that, after extracting feature sets of the recognition and enrolled images using CNNs, the difference vector of the two is used for matching based on the SVM. Zanlorensi et al. [45] proposed a recognition method that obtains the ocular region and then the iris region to extract feature sets of the ocular and iris ROIs using CNNs and uses them to calculate the cosine distance for recognition.

Table 2. Research of ocular recognition from high-resolution images (respectively, MBGC and FRGC denote the Multiple Biometrics Grand Challenge and Face Recognition Grand Challenge v2.0) (Ref. and Illum. denote reference and illuminator).

Approach	Ref.	Illum.	Regions	Feature Extraction	Recognition Methods	Performance	Databases	Pros	Cons
Image processing-based	[39]	NIR, visible	Periocular, iris, ocular	Statistical or transform-based feature descriptor	Same or cross-spectral matching	EER of 4.87% (visible to visible match), 6.36% (NIR to NIR match), 16.92% (visible to NIR match)	Cross-Eyed	Implements cross-spectral periocular, iris, and ocular recognition	Cannot achieve high performance because features are handcrafted
Machine learning-based	[40]	Visible	Ocular, skin texture, eyelids	ULBP, WT-LBP, geometric features, the probabilities of eyelids single- or double-fold, and combinations of these methods	SVM	F1 score of 0.9969 (“lights” subset), 0.868 (“Yale face database), 0.8694 (MBGC database), 0.8108 (FRGC 2.0 database)	“lights” subset of CMU PIE, Yale Face, MBGC, FRGC v2.0	Offers high performance by multiple combinations of feature extraction and SVM	The combination of specific algorithms is required
Deep learning-based	[41]	NIR	Ocular	Deep residual CNN	Feature extraction by CNN and Euclidean distance matching	EER of 2.1625% (CASIA-iris-Distance), 1.595% (CASIA-iris-Lamp), 1.331% (CASIA-iris-Thousand)	CASIA-iris-v4 Distance, Lamp, Thousand	Achieve a high accuracy without accurate segmentation by deep residual CNN	High-resolution input is required and huge dataset is needed for training the model
	[42]	NIR	Ocular	PatchNet based on landmark points	Euclidean distance matching	EER of 16.04% (Cross-Eyed), 10.22% (UBIRIS-I), 10.41% (UBIRIS-II)	UBIRIS-I, UBIRIS-II, Cross-Eyed	Exploits various ocular features by dividing region patches and CNNs	Each patch requires a CNN, which increases memory and computational costs
	[43]	Visible	Ocular	Autoencoder with EFL and KL divergence	Matches cosine similarity or Hamming distance	EER of 14.46% (VISOB database with Resnet-50 fine-tuned on UBIRIS-II, UBIPr, MICHE)	UBIRIS-II, UBIPr, MICHE, VISOB	Demonstrates high accuracy by EFL loss and modified autoencoder	Input size is limited and conversion to grayscale is required
	[44]	Visible	Ocular	CNN (ResNet50, VGG16, VGG19, etc.)	Cosine distance or Euclidean distance matching	EER of 3.18% (SVM), 12.74% (cosine distance), 15.25% (Euclidean distance)	UBIPr	High recognition accuracy and reliability using the pairwise approach	Performance enhancement is limited by conventional CNNs
	[45]	NIR, visible	Periocular, Iris	CNN, feature fusion	Cosine distance matching	EER on cross-spectral and iris-periocular fusion by Resnet-50 of 0.49% (PolyU), 1.4% (Cross-Eyed)	PolyU, Cross-Eyed	Many experiments are done in various cross-spectral environments, which confirm good accuracy	Insufficient analysis of experimental results

below provides a summary of the aforementioned high-resolution ocular image recognition studies.

2.3. Analysis and Discussions

In iris and ocular recognition with high-resolution images, the images used are in high-resolution, irrespective of the implementation method. Therefore, it is most important whether the method of extracting features from the large amount of information contained in the image can be implemented accurately to demonstrate good recognition performance. First, studies on traditional image-processing iris recognition focus on accurate segmentation of the iris region. This is because rich and high-quality iris features can be obtained by accurate segmentation. Afterward, to create an iris code for recognition, Daugman’s rubber-sheet model is applied for preprocessing to transform the iris region into a polar coordinate region. Then, it is converted into an iris code through an appropriate kernel, and a measurement, such as the Hamming distance, is used [20,21,22]. Going further, machine learning or deep learning iris recognition methods are implemented to replace the segmentation and the recognition processes [25,26,28,30,31,32]. Conversely, for ocular recognition in high-resolution images, the highly accurate segmentation of the eye region is not needed, unlike for iris recognition. In the traditional image processing used to implement ocular recognition, the distribution of the features is statistically analyzed [39], or information (e.g., edges) is extracted [40]. According to the analysis results, the following problems may arise in high-resolution image-based iris and ocular recognition; although, many advantages should also be considered.

–

It may be difficult to implement a system to acquire high-resolution iris and ocular images. Camera devices and lenses for capturing high-resolution images are generally expensive and require mandatory subject cooperation to capture images. If the subject moves, a low-quality image will be obtained due to motion blurring, despite using an expensive high-quality device.

–

In general, iris colors differ among individuals and light irises show more detail than dark ones. When attempting to identify a dark iris, the recognition performance may be poor. Therefore, most iris recognition systems are implemented in environments using NIR lighting. This increases the system’s implementation cost and also its size.

–

Because high-resolution images are used, a computer with high computing power is required when the system is implemented. Compared with low-resolution, high-resolution captures more information and requires greater computation for processing. This creates the possibility of performance degradation in low-resolution mobile and embedded systems.

–

Since ocular image recognition systems do not require accurate eye region segmentation, they mitigate the complexity of conventional iris recognition systems. In another sense, however, this means that important iris features cannot be properly exploited, resulting in lower recognition performance compared to iris recognition systems. Therefore, a method that can better extract the features of the iris and the surrounding eye region is required to solve this problem.

–

To supplement the drawbacks of iris and ocular recognition, their information may be combined and used, as shown in the dashed purple box in Figure 2. To this end, as shown in Figure 2, PCA and linear discriminant analysis (LDA) feature-level fusion methods are used with feature sets respectively extracted from the iris and ocular recognition systems. Score-level fusion methods based on the weighted SUM, weighted PRODUCT, and SVM could also be used with the matching scores obtained from each recognition system. In general, compared to using iris or ocular recognition alone, multimodal biometrics combined with them produce a higher recognition performance. Conventional multimodal biometrics, such as face + fingerprint and iris + fingerprint, are less convenient for subjects because biometric information is input twice. However, iris + ocular recognition is highly convenient because both types of biometric information can be obtained simultaneously from a single high-resolution camera. When using a low-resolution camera, it is difficult to obtain the iris + ocular information simultaneously from a single image. Therefore, the constraint arises that either a high-resolution camera or two aligned cameras (one each for the iris image and the ocular image) must be used simultaneously. In the future, additional research should be conducted to overcome this constraint.

3. Iris and Ocular Recognition Methods with Low-Resolution Images

For iris and ocular recognition, low-resolution images may be input in a variety of situations, such as at long distances, using a low-resolution camera, and with weak subject constraints. In such cases, recognition is performed by reconstructing a super-resolution image from a low-resolution image, as shown in the upper box in the dashed purple box in Figure 2. However, because it is difficult to obtain a pair of high-resolution and low-resolution iris images in an actual environment, most studies have used an image processing method to create a low-resolution image based on the high-resolution image, as shown in Figure 3, and performed super-resolution reconstruction based on the pair. That is, Figure 3 shows the examples of pairs of low-resolution iris images of the 2nd and 4th columns (which are produced from the corresponding high-resolution iris images of the 1st and 3rd columns by traditional image processing method [46]) and corresponding high-resolution iris images. These pairs are the experimental images used for implementing and testing the algorithm of super-resolution reconstruction.

Low-resolution images are reconstructed using traditional image processing or state-of-the-art deep learning using a database of high-resolution and low-resolution pairs. To compare and evaluate the super-resolution reconstructions quantitatively, the signal-to-noise ratio (SNR) [47], peak-signal-to-noise ratio (PSNR) [48], and structural similarity index measure (SSIM) [49] are used. SNR and PSNR measure the enhancement quality based on the mean squared error (MSE) between the original high-resolution and reconstructed images, and Equations (1)–(3) show the mathematical formulas for MSE, SNR, and PSNR, respectively.

$$\mathrm{MSE}=\frac{1}{mn}{\displaystyle \sum }_{i=0}^{m-1}{\displaystyle \sum }_{j=0}^{n-1}{\left[I_o\left(i, j\right)-I_r\left(i, j\right)\right]}^2$$

(1)

$$\mathrm{SNR}=10{log}_{10}\left(\frac{\frac{{{\displaystyle \sum }}_{i=0}^{m-1}{{\displaystyle \sum }}_{j=0}^{n-1}{\left[I_o\left(i, j\right)\right]}^2}{mn}}{\mathrm{MSE}}\right)$$

(2)

$$\mathrm{PSNR}=10{log}_{10}\left(\frac{{255}^2}{\mathrm{MSE}}\right)$$

(3)

I_o is an original high-resolution image and I_r is the low-resolution reconstruction. m and n denote the width and height of the image, respectively. Equation (4) shows the mathematical formula of SSIM.

$$\mathrm{SSIM}=\frac{\left(2{\mu }_r{\mu }_o+S1\right)\left(2{\sigma }_{ro}+S2\right)}{\left({\mu }_r{}^2+{\mu }_o{}^2+S1\right)\left({\sigma }_r{}^2+{\sigma }_o{}^2+S2\right)}$$

(4)

μ_o and σ_o denote the mean and standard deviation of the pixel values of an original high-resolution image, respectively. μ_r and σ_r show the mean and standard deviation of the pixel values of the reconstructed image from the low-resolution image, respectively, and σ_ro is the covariance of the two images. S1 and S2 are positive constant values that make the denominator non-zero. In the following Section 3.1 and Section 3.2, iris recognition and ocular recognition in a low-resolution image are respectively explained.

3.1. Iris Recognition Methods with Low-Resolution Images

Existing iris recognition using low-resolution images can be categorized as the image processing methods, machine learning methods, and deep learning methods, which are introduced in more detail in Section 3.1.1, Section 3.1.2 and Section 3.1.3, respectively.

3.1.1. Image Processing Method

Liu et al. [50] implemented heterogeneous iris recognition through a code-level approach. Heterogeneous iris recognition refers to cases where cross-quality or cross-condition afflict the enrolled and the acquired images. In cross-quality situations, recognition is attempted using a high-resolution image against a low-resolution image, or in cross-condition situations of images acquired in different environments, the performance drops. In this study, therefore, we extract the features of the iris through a Markov network-applied model and perform recognition through matching at the code level. This method still requires a Daugman’s rubber-sheet model and has the drawback that the recognition performance may degrade if a predetermined level of input image quality is not guaranteed. Deshpande et al. [51] proposed iris super-resolution and recognition methods using multi-frame images. They use a handcrafted-based method approach. First, they select the best frame in the multi-frame images, and then aligning is performed. Then, patches are selected, and they provide Gaussian process regression (GPR) and enhanced iterated back projection (EIBP) for super-resolution. Finally, these reconstructed high-resolution iris images are classified by using a neural network classifier. Alonso-Fernandez et al. [52] proposed PCA-based iris super-resolution methods using eigen-patch. In this method, they learn the features of each patch and generate the eigen-patch hallucination to apply re-projection for reconstructing the high-resolution image. Furthermore, in order to further enhance the performance of reconstructing high-resolution images, they adopt a PCA-transformation, and matcher fusion is applied [53]. Deshpande et al. [54] proposed Papoulis-Gerchberg (PG) and projection onto convex sets (POCS) methods-based iris feature super-resolution. This method analyzes the image features, and enhancements are performed for more high-quality iris features. Jillela et al. [55] conducted iris recognition experiments using a principal component transform (PCT)-based information fusion on the low-resolution iris videos. This uses eigenvectors provided by PCT, and this method shows better recognition performance compared to related image processing-based methods.

3.1.2. Machine Learning Method

Liu et al. [56] proposed the code-level information fusion based on the modified Markov network model for low-resolution iris images. Using the Markov network model, they transform all the iris images into iris code. Then, they match between these transformed iris codes using Hamming distance matching. Alonso-Fernandez et al. [57] proposed a more sophisticated method that uses learning-based methods rather than the previous method [53]. In this method, a multi-layer locality-constrained iterative neighbor embedding method is applied to achieve the goal of reconstructing high-resolution from the low-resolution image.

3.1.3. Deep Learning Method

Zhang et al. [58] proposed optimized ordinal measures (OMs) features and a CNN-based method. Firstly, detection and cropping ROI region processes are performed for iris recognition on mobile devices. These preprocessed images are passed to the proposed CNN model. It is a simple architecture, but they exploit the pairwise CNN model. It provides a correlation between two irises. Moreover, produced image features are calculated using ordinal feature selection and fuse with these pairwise features. Finally, this method shows the high performance on mobile devices. Ribeiro et al. [59] applied an SR method to reconstruct the image to prevent a decline in recognition performance when a low-resolution image is entered for iris recognition. To reconstruct a high-resolution image from the input low-resolution iris image, they use a stacked autoencoder and a CNN with three convolutional layers, similar to the SRCNN model, which was previously studied for SR. Then, they compare the iris recognition performance. Among various learning methods, the fine-tuned CNN showed the best performance in the experiments. Because iris recognition is applied in many low-resolution environments (mobile environments, etc.), Ribeiro et al. [60] conducted experiments for the SR methods that reconstructed high-resolution images from low-resolution images to prepare for such cases. To determine how well images are reconstructed when applying SR using the texture of the iris only and applying SR using the natural iris image, they experiment with each deep learning model and each database. Furthermore, they conduct experiments to check how well images are reconstructed at different low-resolutions. Methods, such as weighted adaptive Hough and ellipsopolar transform (WAHET), quadratic spline wavelet (QSW), and wavelengths complex GaborFilterbanks (CG), are used for recognition. Then, SSIM, PSNR, and the Hamming distance are used to evaluate the EER-based SR performance. Kashihara et al. [61] proposed a modified SR method based on the SRGAN, which was previously studied to research SR for iris recognition by CNNs. To improve SR performance, using a deep CNN (DCNN), they apply content loss to the image reconstructed by SRGAN. This is for the prediction and to measure the accuracy of using the DCNN for the image obtained from the super-resolution reconstruction, based on which loss of SRGAN is affected to reflect in the reconstruction performance. To train the mapping function between low-resolution and high-resolution iris images, in the training stage, Ribeiro et al. [62] combined the original iris image and an artificial low-resolution image, which is created by reducing the original image to a lower resolution and converting it back into a higher resolution using interpolation. They experiment with it using various CNN models (VDCNN, SRGAN, DCSCN, etc.), and several databases, such as the describable texture dataset (DTD), CASIA-iris-v3 Interval, and visible spectrum smart-phone iris database (VSSIRIS). Guo et al. [63] proposed an IrisDnet model to reconstruct a high-resolution image from a low-resolution iris image. After reconstructing the high-resolution image from the low-resolution image, it is input into the model along with the original high-resolution image, based on which the adversarial loss and identity-preserving loss are calculated to train the model. Mostofa et al. [64] proposed cross-spectral and cross-resolution iris recognition based on conditional GAN (cGAN). They conduct the various cases of experiments including high-resolution NIR image versus low-resolution visible image, and all of these combinations.

Table 3. Research into iris recognition with low-resolution images (Ref. and Illum. denote reference and illumination).

Approach	Ref.	Illum.	SR and Feature Extraction Method	Recognition Method	Performance	Databases	Pros	Cons
Image processing-based	[50]	NIR	Modified Markov networks	Code-level feature matching	EER is plotted only in the figure. 98.74% (GAR@FARs = 10−3) 95.94% (GAR@FARs = 10−4)	Q-FIRE, Notre Dame database	Resolution-independent recognition by modified Markov network	Still requires Daugman’s rubber-sheet model
	[51]	NIR	GPR, EIBP	Neural network classifier	Accuracy of 96.14	CASIA-iris-database	Reduces the image acquisition difficulty by using multi-frame images	Implementation is difficult and it needs much preprocessing
	[52]	NIR	Eigen-path hallucination	1-D Gabor filter	EER of 0.66% (downscale ×1/6, patch size ×1/32)	CASIA-iris-Interval v3	Not require a huge datasets	Not much restores the image of very low-resolution
	[53]	NIR	Eigen-transformation	Log-Gabor filter, SIFT	EER of under 6% (Log-Gabor filter), under 8% (SIFT), under 5% (Log-Gabor+SIFT)	CASIA-iris-Interval v3	Adopts preprocessing and eigen-transformation compared to [52]	It still shows the low performance when reconstructing high-resolution images on very low-resolution
	[54]	NIR	Papoulis-Gerchberg (PG) and projection on to convex sets (POCS)	Gray Level Co-occurrence Matrix (GLCM)	Not reports as specific measurements	CASIA-iris-database	Simple and it shows faster processing	It just enhances the image edges and features on the polar iris images not low-resolution
	[55]	NIR	Principal components transform	Template matching	EER of 1.76%	Multi-Biometric Grand Challenge (MBGC)	Image level fusion and principal components transform show the better performance	Requires the image sequences (e.g., videos), and preprocessing
Machine learning-based	[56]	NIR	Markov network model	Hamming distance matching	EER of close to 0.9% (at FAR@GAR = 10−2, code-level fusion)	Q-FIRE	Robust to the image noises because image features are transformed into iris code	Cannot exactly know whether the images are correctly restored, and specific measurement results are not reported. Moreover, preprocessing is still required
	[57]	NIR	Multi-layer Locality-Constrained Iterative neighbor embedding (M-LINE)	Log-Gabor filter, SIFT	EER of under 4% (Log-Gabor filter), under 3.6% fusion of Log-Gabor filter and SIFT)	CASIA-iris-Interval v3	Obtains high performances using learning-based method compared to previous method [53]	Bigger size of images is not tested. It is good if the experiments are performed using bigger high-resolution and smaller low-resolution images
Deep learning-based	[58]	NIR	Pairwise CNN	Ordinal measures features, pairwise features	EER of 0.64%, 0.69%, and 1.05% (20–20 cm, 20–25 cm, 20–30 cm)	Newly composed database on the mobile device	Obtains the high performance using ordinal features and pairwise features	Needs preprocessing of detection for eye region and converting polar image
	[59]	NIR	CNN with three convolutional layers	Normalized Hamming distance	EER (downscale ×1/4, 1/8, 1/16) of 0.68%, 1.41%, 11.46%, respectively	CASIA-iris-v3 Interval	Obtains higher accuracy by stacked autoencoder	Shows low accuracy in cases of very low downscaling rates
	[60]	NIR	VDCNN, SRCNN, and SRGAN	WAHET, QSW, CG	EER (downscale ×1/2, ×1/16) of 3.78%, 32.03% (VDCNN), 3.84%, 30.17% (SRCNN), 4.27%, 38.41% (SRGAN) on CASIA-iris-Interval database, respectively	CASIA-iris-v3 Interval, Lamp (v4), UBIRIS v2, Notre Dame, etc.	Various experiments are conducted according to downscaling rates, architectures, and databases	Does not demonstrate high SR performance in cases of very low downscaling rates
	[61]	Visible	Fast-SRGAN (SRGAN custom for iris SR)	DCNN	ANOVA significant difference (F(9,40) = 39.47; p < 0.01).	UBIRIS v1	Deep learning-based classifier shows good accuracy	Entirely restored images are not shown and the experiment with very low downscaling rates is not conducted
	[62]	Visible, NIR	VDCNN, DCSCN, SRGAN	Log-Gabor filter and SIFT fusion	EER of 5.37%, 8.86%, 5.52% (VDCNN, DCSCN, SRGAN), respectively	CASIA-iris-v3 Interval, DTD, VSSIRIS	Repeated image input according to upscaling factor simplifies implementation	SIFT shows low performance at every scaling factor
	[63]	NIR	IrisDnet	LightCNN29 and SIFT	EER (downscale ×2, ×4, ×8) 1.25%, 1.70%, 5.70% (CASIA-iris-V1), 3.75%, 4.17%. 8.09% (Thousand), respectively	CASIA-iris-V1, Thousand	Obtains a good accuracy by LightCNN26 and IrisDnet	Structurally, two additional CNNs are used, which leads to increased computation
	[64]	Visible + NIR	cpGAN	Euclidean distance matching	EER of 1.28%, 1.31% (on the cross-resolution and cross-spectral)	PolyU, WVU databases	Supports iris recognition on the cross-spectral and cross-resolution based on conditional GAN	Still requires iris segmentation to whole processing and needs high computational costs

provides a summary of the aforementioned studies on low-resolution image-based iris recognition.

3.2. Ocular Recognition Methods with Low-Resolution Images

Only the following deep learning methods exist as ocular recognition methods for low-resolution images.

Deep Learning Method

The recognition method proposed by Ipe et al. [57] uses a very deep SR (VDSR) method to reconstruct a super-resolution image from a low-quality ocular image obtained under unconstrained conditions. Recognition is performed using AlexNet and SVM classification. Tapia et al. [58] proposed periocular recognition in a selfie image, from which the user’s entire facial region is obtained. An ESISR model is proposed to improve performance while using less memory by reducing the total number of weights in the network and improving the SR performance. After converting the SR-applied RGB image into the YCrCb color region, only the Y channel is used for feature extraction. Furthermore, they proposed a novel perceptual loss function to balance the information when a high-resolution image is reconstructed from a low-resolution image.

Table 4. Research into ocular recognition with low-resolution images (Ref. and Illum. denote reference and illuminator).

Approach	Ref.	Illum.	SR and Feature Extraction Method	Recognition Method	Performance	Databases	Pros	Cons
Deep learning-based	[65]	Visible	VDSR, AlexNet	SVM classifier	Rank-1 accuracy 91.47%	UBIRIS v2	Implements the end-to-end method by connecting VDSR, AlexNet, and SVM classifiers	Only the SVM classification performance is presented
	[66]	Visible	DCSCN, WDSR-A, SRGAN, ESISR	FaceNet, VGG-FACE	EER of 8.90% (ESISR ×3) 9.90% (ESISR ×4), respectively	Selfie database by Samsung device, Set-5E, MOBIO	Shows good recognition accuracy with images in noisy mobile environments, with low computational load	Only focuses on noise improvement without conducting experiments in a low-resolution image environment

below shows a summary of the aforementioned studies on low-resolution image-based ocular recognition.

3.3. Analysis and Discussions

In low-resolution iris and ocular recognition, the resulting performance varies depending on how well the features of the original image can be reconstructed. However, in SR studies on reconstructing high-resolution images from low-resolution images, traditional image processing methods have rarely been studied, except for [50], because they have certain limitations. Recently, however, due to the active development of GPGPU, deep learning approaches have been studied, and many researchers have applied them to SR and achieved high performance. Such deep learning SR studies have been actively applied to iris and ocular recognition. Through experimental results, recently studied deep learning SR has demonstrated significantly higher performance compared to conventional image processing SR methods. Often, low-resolution images are produced for iris and ocular recognition in situations where recognition is attempted using a low-resolution camera on a mobile device or at a distance. However, because there is no database of such low-resolution images, many studies have conducted experiments by arbitrarily downscaling high-resolution images. When recognition is performed using reconstruction based on low-resolution images, the following situations will be encountered and must be resolved to implement a high-performance recognition system.

–

Basically, even if low-resolution images are downscaled from high-resolution images, much information is lost, causing difficulty in reconstructing high-resolution images using only the low-resolution images’ information. Therefore, a database containing both low and high-resolution images is needed when implementing a reconstruction method.

–

Deep learning SR research requires a database consisting of many images because the deep learning models must be trained. However, because of privacy concerns due to the characteristics of biometric images, many images, especially low-resolution images, are difficult to obtain. Consequently, artificial augmentation methods are used but may lead to overfitting in training. Therefore, alternative methods must be studied to solve this problem.

–

Most SR method studies have often used conventional methods such as PSNR or SSIM to evaluate the quality of reconstructed images. However, when these are used, whether an image has been properly reconstructed is difficult to evaluate because it cannot be known whether the image has been reconstructed. Therefore, for accurate recognition performance, methods should be researched to determine the degree to which an image has been reconstructed.

4. Conclusions

In this study, we investigated high-resolution image-based iris and ocular recognition methods and reviewed studies that applied SR methods to solve the problems arising when low-resolution images are used for recognition. Furthermore, we analyzed each study and examined its problems. Some of the important impediments that increase the difficulty in implementing iris recognition systems are (i) accurate extraction of the iris region segmentation and extracting unique features from the iris features to distinguish each person. (ii) To solve these problems, the ocular recognition biometric methods that use the entire eye region were researched. (iii) Here, images were also acquired and used in ocular recognition, and low-resolution images will probably be obtained depending on the environment, which may cause problems with system accuracy.

Deep learning-based SR methods, which have been actively studied in recent years in environments where low-resolution images are acquired, can achieve a similar performance to high-resolution image systems. This can reduce the difficulty of implementing a biometrics system when applying it in a secure environment, thus raising the security level in environments equipped with the mobile devices commonly used in everyday life.

The results of these studies are increasingly applied to everyday life, such as remote driver identification in deluxe cars and the supervision of VIP members in hotels and resorts based on iris and ocular identification at a distance. In addition, intelligent surveillance systems can be used to search for missing children, the aged with dementia, and criminals. To guarantee system reliability in these applications, algorithms should be researched that are robust to occlusion, camera diversity, and the subjects’ positioning. In addition, a lightweight algorithm should be researched for adoption in low processing-power embedded systems in car and surveillance camera environments.