Illumination Calibration for Computational Ghost Imaging

Abstract

We propose a fast calibration method to compensate the non-uniform illumination in computational ghost imaging. Inspired by a similar procedure to calibrate pixel response differences for detector arrays in conventional digital cameras, the proposed method acquires one image of an all-white paper to determine the non-uniformity of the illumination, and uses the information to calibrate any further reconstructed images under the same illumination. The numerical and experimental results are in a good agreement, and the experimental results showed that the root mean square error of the reconstructed image was reduced by 79.94% after the calibration.

1. Introduction

Over the past two decades, ghost imaging has been one of the rapidly developing computational imaging schemes [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21]. Ghost imaging reconstructs images by illuminating an object with a series of varying light intensity distributions and associating the knowledge of these distributions and the corresponding total light intensity measured with a bucket detector [2]. In a standard pseudothermal two-detector ghost imaging scheme [2,3,4,5], the light intensity distributions are usually obtained by a scanning single-pixel detector or a detector array. With the development of the micro-optical electromechanical system, computational ghost imaging (CGI) was proposed [7]. In CGI, the intensity distributions are generated by illuminating a spatial light modulator (SLM) with programmable masks on it. The image is then reconstructed by correlating the calculated intensity distributions of the masks at the object plane with their corresponding light intensities measured by a single-pixel detector. CGI significantly simplifies ghost imaging systems and reduces acquisition time by calculating the intensity distributions numerically rather than measuring them experimentally. However, difference inevitably exists between the numerical calculations and experimental measurements, and one major cause of such difference is the non-uniformity of illumination sources [20,21,22,23,24].

To enhance the quality of a degraded image with non-uniform illumination, the Retinex [22,23] algorithm is commonly used. It is well known that an image is the pixelwise multiplication of the illumination component and reflectance component. The Retinex algorithm uses estimating illumination methods to extract uneven illumination components from images and then normalizes them. An effective method for estimating the illumination component is the key to calibrate non-uniform illumination based on Retinex theory [24]. The Retinex algorithm posits that the illumination component is the smoothed version of the degraded image. Several techniques have already been reported in the literature [24,25], such as some illumination estimation algorithms based on a filtering strategy [24,26,27,28], PDE-based Retinex methods (the illumination is obtained by solving a partial differential equation) [29,30], and variational model-based Retinex methods [25,31]. The above algorithm is also called retrospective calibration, which is a posteriori calibration applied after the acquisition. However, their calibration accuracy is unsatisfactory [24], because the illumination component is estimated.

At present, the imaging performance of CGI is nowhere near image sensors using detector arrays, especially silicon-based charge-coupled devices and complementary metal oxide semiconductors [32,33]. Beside the fact that the research and manufacture of detector arrays are intensively invested in due to the global market demands, sophisticated calibration procedures, such as dark current noise suppression and pixel-response non-uniformity compensation, are applied before the image sensors are put into actual use [34,35,36,37,38,39]. Therefore, it would be beneficial to investigate the same concept to improve the image quality of CGI.

In this work, we propose a calibration method to compensate for the non-uniformity of illumination in CGI. The proposed calibration acquires the knowledge of illumination non-uniformity by reconstructing an image of an all-white paper. The knowledge is then used to calibrate further reconstructed images and to improve their image qualities. Theoretical analysis and experimental results indicated that the proposed method is feasible. Specifically, the root mean square error (RMSE) of the experimentally reconstructed image was reduced by 79.94%, from 0.2618 to 0.0525, after the non-uniformity was compensated for using the proposed method.

2. Theory

2.1. The Principle of CGI

The scheme of CGI is performed as shown in Figure 1. Beams from the laser source are modulated by a SLM, which is controlled by the computer to generate a series of binary patterns and provide structed illumination. An imaging lens projects the patterns onto the object, which forms the conjugation between the SLM and the object. A collection lens and a bucket detector are used to collect the measured signal. The signal is then transferred to computer for reconstruction by using a highspeed analogue-to-digital convertor.

In CGI, the measured signal $S_i$ is the illuminating light intensity distribution $I_L$ modulated by the SLM mask pattern $P_i$ and transmitted or reflected by the pixelated object $I_o$; i.e.,

$$S_i=K·{\displaystyle \iint }{I}_L\left(x,y\right)·P_i\left(x,y\right)·I_o\left(x,y\right)dxdy,$$

(1)

where K is a scaling constant, and x and y refer to the spatial coordinates in the transverse plane. After many measurements, the reconstructed image $I_r$ can be calculated using the knowledge of $S_i$ and $P_i$ [7,9]. If the patterns form an orthonormal basis, then an N pixelated object can be completely sampled with N measurements. The reconstructed image can be obtained by using [13,14,15,16,17,18,19]

$$I_r={{\displaystyle \sum }}_{i=1}^N{S}_i·P_i.$$

(2)

2.2. The Calibration of Non-uniform Illumination in CGI

In most CGI works, illuminating light intensity $I_L$ is uniform distribution, considered as a constant by presumption, and is not included in Equation (1). However, in this work, we were to address the non-uniformity of illumination intensity distribution $I_L\left(x,y\right)$, and therefore it needed to be considered.

If the illumination was uniform, the reconstructed image $I_r$ would be strictly proportional to the object $I_o$. Here, by substituting Equation (1) into Equation (2), it is demonstrated that $I_r$ is not a scaled $I_o$, but rather a scaled dot product of $I_o$ and $I_L$; i.e.,

$$I_r={{\displaystyle \sum }}_{i=1}^N\{(K·I_L\left(x,y\right){\displaystyle \iint }{P}_i\left(x,y\right)·I_o\left(x,y\right)dxdy)·P_i\}=K\prime ·I_L·I_o,$$

(3)

where $K\prime$ is another scaling constant to ensure that Equation (3) stands. If the illuminating intensity distribution $I_L$ was obtained, then an authentic image $I_c$ of the object could be yielded as

$$I_c=I_r/I_L=K\prime ·I_o.$$

(4)

To retrieve the non-uniform illuminating intensity distribution $I_L$, the easiest way is to set the object $I_o$ with a constant reflectivity R, such as a sheet of white paper. It is worth mentioning that a white paper is not perfectly uniform in its reflectivity in general. However, due to the quasi-Lambertian nature of the sheet of paper, combined with the fact that the half angular size of the paper with respect to the detection point is small (7° in the experiment), the sheet of white paper was considered as of a constant reflectivity R in this work. Consequently, the image $I_{WP}$ of the white paper is proportional to $I_L$ as

$$I_{WP}={{\displaystyle \sum }}_{i=1}^N\left\{\left(K·I_L\left(x,y\right){\displaystyle \iint }{P}_i\left(x,y\right)·R·Ones\left(x,y\right)dxdy\right)·P_i\right\}=K″·I_L,$$

(5)

where R and K″ are also scaling constants and Ones is an all-one matrix. The authentic image I_c can be obtained by

$$I_c=I_r/I_{WP}=\left(K^{\prime }/K″\right)·I_o.$$

(6)

It is worth mentioning that all scaling constants, $K$, $K\prime$, and $K″$, are irrelevant to the reconstructions and their image quality evaluation because unity normalization will be performed on all reconstructions.

3. Simulation and Experiment

3.1. Numerical Simulation Results

Numerical simulation was performed to demonstrate the proposed calibration methods; its procedure is shown in Figure 1. The illumination light distribution $I_L$ was set to be a Gaussian function, which presented an expanded laser beam illumination, as

$$I_L\left(x,y\right)=\alpha \frac{1}{\sqrt{2\pi \sigma }}{e}^{{\left(x-{\mu }_1\right)}^2+{\left(y-{\mu }_2\right)}^2}+\beta ,$$

(7)

where ${\mu }_1$ and ${\mu }_2$ are the mathematical expectation in the x and y dimension, and $\sigma$ is variance. $\alpha$ and $\beta$ are the coefficients for adjusting the relative value.

In the simulation, the parameters for Gaussian illumination were ${\mu }_1=30$, ${\mu }_2=30$, $\sigma =2000$, $\alpha =0.0089$, and $\beta =0.1$, and it was decentered in position, as shown in Figure 2a. An alphabet with 128 × 128 pixel resolution was used as the object, as shown in Figure 2b. Hadamard patterns were used for modulation purpose [14,15,16,17,18,19]. Detector noise was added to the measured signals, and the averaged signal-to-noise ratio (SNR) of the measured signal was 46 dB. The uncalibrated reconstruction is shown in Figure 2c, exhibiting the influence of non-uniform illumination, such as missing letters at the bottom-right corner. After being calibrated by the proposed method, image non-uniformity was significantly suppressed and the missing letters were recovered, as shown in Figure 2d, though with some noise. Since the proposed method is to calibrate the global error caused by non-uniform illumination, we chose RMSE as the main evaluation indicator. The root mean square error (RMSE) was reduced by 39.21%, from 0.2548 to 0.0999.

3.2. Experimental Results

An experiment was performed, as shown in Figure 3, using the same CGI system setup as in numerical simulation. Beams from the laser source (Viasho VA-I_LNS-532, 532 ± 0.1 nm, 200 mw) were expanded, and then modulated by a digital micromirror device (DMD, Texas Instruments V-7000, 1024 × 768, operating at 2 kHz). A camera lens (Nikon AF Nikkor, f = 35 mm, F = 1.8 G) imaged the DMD patterns onto the object. A single-pixel detector (Thorlabs PDA100A-EC, 320–1100 nm, operating at 20 dB) and a highspeed analogue-to-digital convertor (ADC, PicoScope 6404D, operating at 100 MS/s sampling rate and 500 MHz bandwidth) were used to measure and transfer the intensity signals to the computer for reconstruction.

To calibrate the non-uniformity of the illumination, a white paper was set as an object and its image, $I_{WP}$, was obtained using Equation (5), which would be used as the estimated illumination distribution $I_L$. Hadamard patterns [14,15,16,17,18,19] with 128 $\times$ 128 pixel resolution and differential measurement [11] were used for the image reconstruction. As shown in Figure 4a, the laser-sourced illumination had a Gaussian distribution, laser speckles, and other non-uniformity.

Under the same illumination condition and parameters configuration, a standard CGI experiment was performed with an alphabet object. The reconstructed image $I_r$ was influenced by the non-uniform illumination. As a result, some letters could not be distinguished, and some letters had speckles on them, as shown in Figure 4b. To eliminate the non-uniform illumination influence on the reconstructed image, the proposed calibration method was performed using Equation (6) with the measured $I_{WP}$. The quality of the calibrated image $I_c$, as shown in Figure 4c, was significantly improved, and all letters became distinguishable. The RMSEs, calculated with the ground truth, of the images before and after calibration were 0.2518 and 0.0525 respectively, indicating a 79.94% improvement by the proposed calibration method. Both the ground truth and the reconstructed images were normalized and aligned in a manner such that they had the same dynamic range and the same field of view. To demonstrate the whole procedure, the intensities (the normalized greyscale values) of the same line in Figure 4a–d are illustrated in Figure 4e. The intensity of the reconstructed image $I_r$ (red line) was enveloped by the non-uniform illumination $I_L$ (black dashed line). The intensity of the calibrated image $I_c$ (green line) is in good agreement with the ground truth (blue dotted line).

It is worth mentioning that the experiment contained two major non-uniform illumination scenarios, which were the global non-uniform due to the laser Gaussian distribution and the local one caused by laser speckles, such as those on letters “H” and “I” in Figure 4b.

For comparison, a retrospective calibration method was applied to calibrate the non-uniform illumination of Figure 4b. The retrospective calibration method performs calibration by estimating the non-uniform illumination with the assumption that the illumination distribution is smooth [24]. However, such an assumption is invalid for local non-uniformity, such as laser speckles. On the contrary, the calibration method proposed here calibrates both global and local non-uniform illumination. The comparison presented in Figure 5 shows that the laser speckles on letters “H” and “I” were not eliminated after retrospective Gaussian filtering calibration. The RMSE improved from 0.0689 to 0.0525 by using the proposed calibration method rather than the existing retrospective one.

4. Discussions

Interestingly, there are existing works [20,21] to eliminate the influence of non-uniform illumination in traditional ghost imaging schemes, i.e., ghost imaging with signal and reference paths. However, due to the nature of traditional ghost imaging, these works had to use charge-coupled devices to record the non-uniformity of the illumination by accumulating many frames of speckle patterns, which jeopardizes the real-time performance of a ghost imaging system.

It is worth noting that the noise became obvious at the corner areas after the calibration. That is because the SNR at these areas was low due to the weak illumination intensities, and the proposed method only reduced the non-uniformity of the reconstructed image caused by the illumination but would not improve the SNR of the image. The SNR was calculated using the following equation:

$$\mathrm{SNR}=\left(<I_f>-<I_b>\right)/\left(\frac{{\sigma }_f+{\sigma }_b}{2}\right),$$

(8)

where $<I_f>$ and $<I_b>$ are the average intensities of the image feature and background, respectively (here calculated from the data within the white part of the letter and the black part around the letter). ${\sigma }_f$ and ${\sigma }_b$ are the standard deviations of the intensities in the feature and the background, respectively [40].

The image-quality improvement yielded in the proposed calibration was fundamentally due to a global dynamic range normalization of the reconstructed image, which was achieved by a pixelwise division in Equation (6). Importantly, the proposed illumination-calibrated method cannot improve the SNR of a local area in the reconstructed image. However, the calibrated images are more suitable for global observation and analysis.

5. Conclusions

In this work, an illumination calibration procedure was proposed to address the non-uniform illumination problem in computational ghost imaging. Without any extra device, the proposed procedure acquires one image of an all-white paper to determine the non-uniformity of the illumination and uses the acquired information to calibrate any further reconstructed images under the same illumination condition. Numerical and experimental results demonstrated that, without the proposed calibration, certain areas in the reconstructed images became indistinguishable and image information was missing due to the non-uniform illumination. The missed information can be recovered after the proposed calibration; the quality of the reconstructed images was significantly improved by approximately 80%. The proposed calibration method can be applied to other ghost imaging techniques.