Multi-planar Full-Field Blur Correction Algorithm for Infrared Microscopy ^†

Abstract

The present work proposes a method for the 3-D full-field focusing for microscopic infrared (IR) imagery. It is based on the partial analysis of Point Spread Function (PSF) for a confined volumetric universe of vision in a microscopic IR system. The ability of the algorithm to compensate for localized blur is demonstrated using two different real MWIR microscopic video sequences, which were captured from two microscopic living organisms using a Janos-Sofradir MWIR microscopy setup. The performance of the proposed algorithm is assessed on real and simulated infrared data by computing the root mean-square error and the roughness-laplacian pattern, which was specifically developed for the present work.

1. Introduction

IR imaging systems are widely used to register useful information in the framework of scientific, medical, industrial and defense applications, among others [1,2]. In spite of the large number of advances achieved in IR transducers to fabricate better focal-plane-arrays (FPA) and the ones accomplished in new read-out integrated circuit (ROIC) technology [3], the raw IR image quality still evidences a very low signal to noise ratio (SNR), and it is subjected to degrading agents that can, to a lesser or greater extent amount [4,5,6,7], cause the loss of information, or even more serious, the appearance of false information where there is none.

Given the problem, on the one hand, several authors have been proposed correction methods aimed to reduce the Fixed Pattern Noise (FPN) [8,9,10,11,12,13,14,15], and optic Blurriness, however, only as a global nuisance in the field of view. On the other hand, the optic exploration studies of dynamic exothermic processes that come from biological as inert organisms, where the image system lacks the ability to observe in a fully focused way the Field of View (FOV) at any given moment, the need arises to process through Algorithm the various depths of field to obtain an image in full focus.

As a global approach, our proposed method collects the capturing experience at the microscope in order to solve simultaneously the intrinsic several ROIs defocus of a 3D sample at the Microscope. The developed method operates by PSF deconvolution in each region of interest (ROI) of the full Field of View (FOV) [16]. The defocus level assessment and evaluation is made by using standardized different indexes, the RMSE and Roughness Laplacian Pattern (RLP) metrics will be considered.

The recorded corrupted video sequences is processed for correction considering different levels of noise, camera movement, movements of bodies within the scene with a fixed camera and combinations of the previous ones.

Finally the method is validated over real microscopic IR video-sequences, demonstrating the method’s operatively over materials, biological and even medical thermal imagery.

2. Materials and Methods

The built-in IR microscope unit is composed of a MWIR camera (Sofradir model EC-IRE 320M) with a HgCdTe FPA transducer that has a spectral response between 3.7 and 4.8 micrometers. The FPA is composed by an array of 320 × 256 IR detectors, with a 14-bit analog/digital converter. The FPA can operate up to 320 frames per second. The optical system (IR objective) is integrated by an array of lenses from Janos Technology, allowing a 4X magnification. According to our experiments, such a microscope permits to integrate IR exothermal process with images contained in a 1.99 × 1.49 mm scene area, with a noise-equivalent temperature difference (NETD) of 10 mK.

At the volumetric 3d-space of vision of the IR microscope, the integration of the 2D-PSF from a spatial ROI for several depths of field, as shown in Figure 1, makes possible to generate the hyper-PSF for such ROI.

The observation model for the${\left(i,j\right)}^{th}$ detector in the array generates the measured signal (detector response) $Y(i,j)$ given by the follow model:

$$Y\left(i,j\right)=A(i,j)·X_p\left(i,j\right)+B(i,j)$$

(1)

where $A(i,j,n)$ and $B(i,j,n)$ are the gain and offset of the ${\left(i,j\right)}^{th}$ detector and $X_p(i,j)$ is the irradiance collected by the ${\left(i,j\right)}^{th}$ detector during the integration time.

The mathematical foundation for the estimation of the IR microscope PSF used in this work is based on the experimental method proposed in [6]. There, it is assumed that the optical system PSF isotropic and separable, so it can be computed from the combination of the estimated line PSFs in the x and y axes separately, $h\left(i\right)$ and $h\left(j\right)$, respectively. Each line PSF can be estimated as the derivative of a sharp transition step function in the desired direction of the scene as follow

$$h\left(i\right)=\frac{g^{’}\left(i\right)}{B},\hspace{1em}h\left(j\right)=\frac{g^{’}\left(j\right)}{B}$$

(2)

where $B$ is the intensity value of the scene background and $g^{’}\left(i\right)$, $g^{’}\left(j\right)$ are the step’s derivative with respect to x and y direction respectively. These results are obtained by also assuming that the acquisition setup is a linear and shift-invariant system. Finally, and assuming that the PSF varies smoothly in all the other directions indeed not separable, the PSF $h\left(i,j\right)$ can be found by a 2D Gaussian fit enforcing the values of $h\left(i\right)$ to $h\left(j\right)$.

This procedure was automatically extra poled to the full FOV in order to estimate the whole distortion of the magnifier lens’s optic.

The resultant PSF’s corresponding to all deep of field by each ROI are subjected to a deconvolution algorithm, and processed to find the sharpest image, and assign it to a correspondent deep of field and thus, to a z axis position, mapping by deep estimation the sample surface.

3. Results

The resultant application of the method over an index finger pad microscopic IR video sequence is showed in Figure 3.

The first raw MWIR microscopy imagery sample is shown in Figure 4. Note that the raw image is highly corrupted by both NU noise and blur. The proposed algorithm is able to simultaneously compensate for both issues. The naked-eye evaluation of the result shows a significant improvement in spite of the severity of the NU noise and blur. Moreover, the dead and saturated pixels shown in Figure 4 are correctly compensated in the scene by the algorithm. Overall, the proposed algorithm provides smaller RLP values than the raw frame. A lowest RLP of 0.697 is achieved using the proposed algorithm, while the raw frame RLP value is 0.871.

3. Discussion

The new algorithm combines a well-known NUC method based on constant statistics and estimation-based PSF deconvolution method in a full field FOV de-focus correction. The method was subjected to images with low and high levels of NU noise and different localized deep of field through the FOV, evidencing its effectiveness. The uses of this method are broadly extra poled to another sciences areas, on order to get arise us to a microscopy lossless exploration.