A Single-Pixel High-Precision Imaging Technique Based on a Discrete Zernike Transform for High-Efficiency Image Reconstructions

Abstract

Single-pixel imaging (SPI) has attracted increasing attention in recent years because of its advantages in imaging systems. However, a low reconstruction quality and a long reconstruction time have hindered the development of SPI. Hence, in this study, we propose a Zernike SPI (ZSPI) technique to reduce the number of illumination patterns and reconstruction time whilst retaining robustness. First, the ZSPI technique was theoretically demonstrated. Phase-shifting Zernike moment projections were used to illuminate the target and an inverse Zernike transform was used to reconstruct the desired image. In order to prove the feasibility, numerical simulations were carried out with different sample ratios (SRs) ranging from 0.1 to 0.3; an acceptable reconstruction appeared at approximately 0.1. This result indicated that ZSPI could obtain satisfactory reconstruction results at low SRs. Further simulation and physical experiments were compared with different reconstruction algorithms, including noniterative, linear iterative, and nonlinear iterative methods under speckle modulation patterns at a sample of 0.1 in terms of different targets. The results revealed that ZSPI had a higher imaging quality and required less imaging time, particularly for low-frequency targets. The method presented in this study has advantages for the high-efficiency imaging of low-frequency targets, which can provide a new solution for the SPI method.

1. Introduction

Single-pixel imaging (SPI) [1,2] is different from traditional array detector imaging, and its image information acquisition method and acquisition efficiency are different. SPI has advantages in terms of the hardware complexity and industrial cost. In addition, this technique has potential application prospects for the simplification and integration of future imaging systems, particularly in infrared and terahertz bands. That is, the array sensor is immature, and three-dimensional (3D) imaging and spectral imaging require a high resolution and sensitivity. The advent of SPI technology brings new solutions to these problems. Although detector array technology has a superior performance in the visible band, SPI (such as polarimetric imaging [3,4,5], holographic imaging [6,7], multispectral imaging [8,9], X-ray imaging [10,11,12], and THz imaging [13,14,15,16,17,18]) is more suitable for unconventional imaging and has an accurate time and depth resolution. In addition, SPI can meet the application requirements of different fields, including imaging through scattering media [19,20], remote sensing [21], compressive radar [22], optical encryption [23], bioluminescence microscopic imaging [24,25,26], gas detection [27], and 3D imaging [28,29,30].

SPI has certain advantages and characteristics compared with traditional imaging technology and has achieved fruitful research results. However, the problems of imaging quality and sampling efficiency have not been completely solved, thereby greatly restricting the practical application of SPI technology [31,32]. How to improve the imaging quality and reduce the sampling time has become a leading topic discussed by researchers. The spatial distribution of the projection pattern and the running time of the reconstruction algorithm are two main factors that restrict SPI systems, and determine the image quality and efficiency [33].

In SPI technology, random patterns and orthogonal basis patterns are generally selected as the projection pattern. The use of random speckles [34], random binary patterns [35], or other random patterns as the projection patterns often does not have orthogonality. Moreover, the use of a spatial light modulator makes it difficult to solve the problem of SPI in the sampling or reconstruction of a long time period. Compressed sensing reduces the measurement time to an extent in the case of under-sampling. However, it increases the computation time of the reconstruction algorithm accordingly, and the quality of the image reconstruction depends on the algorithm robustness [36,37]. At present, an increasing number of researchers have used the orthogonal basis pattern rather than the random pattern as the projection pattern to improve the quality and efficiency of SPI. In accordance with the different projection patterns, SPI techniques such as wavelet transform [38,39], discrete cosine transform (DCT) [40,41], Fourier transform [42,43,44], and Hadamard transform [27,28,43,45,46,47,48] have been produced.

Zernike polynomials are widely used in the field of optical engineering owing to their orthogonal and complete characteristics in the unit circle [49]. The description method provided by Zernike polynomials has achieved great success in the application of optical system designs and analyses [50], adaptive optics [51], atmospheric optics [52], optical testing [53], wavefront shaping [54], wavefront sensing [55], interferometry [56], aberration characteristics and the correction of human eyes [57], and other fields. Considering the orthogonal and complete characteristics of Zernike polynomials, Zernike polynomials can be introduced into the SPI method.

In this work, a Zernike single-pixel imaging (ZSPI) method based on a discrete Zernike transform was proposed. The ZSPI method was theoretically demonstrated. Phase-shifting Zernike moment projections were used to illuminate the target, and an inverse Zernike transform was used to reconstruct the desired image. ZSPI could obtain satisfactory reconstruction results at low sample ratios. Further simulation and physical experiment results revealed that the Zernike single-pixel imaging had a higher imaging quality and required less imaging time.

2. Theory and Methods

The proposed ZSPI technique was based on the theorem of a Zernike transform. This technique employs Zernike polynomials as structured light patterns to illuminate the scene, and uses a detector that has no spatial resolution to collect the resulting light. A two-step phase-shifting Zernike illumination pattern based on a discrete Zernike transform was employed for the Zernike moment acquisition, which could obtain the Zernike coefficients and eliminate random noise. The final target was reconstructed by an inverse discrete Zernike transform.

2.1. Zernike Basis Pattern

The Zernike polynomial is a common mathematical description in optics such as defining any function of pupil optical systems, wavefront phases, or wave aberrations [58]. In this study, we attempted to sample the image through the Zernike basis in SPI.

Zernike polynomials are a product of angular functions and radial polynomials [52] as shown in Equation (1). Zernike basis patterns are orthogonal in the unit circle. Therefore, they are defined with polar coordinates as follows:

$$\begin{array}{c}\left\{\begin{array}{c}{\mathrm{Z}}_{\mathrm{eve}nj}=\sqrt{n+1}{R}_n^m\left(r\right)\sqrt{2}\cos m\theta \\ {\mathrm{Z}}_{oddj}=\sqrt{n+1}{R}_n^m\left(r\right)\sqrt{2}\sin m\theta \end{array}\right.\begin{array}{cc}& m\ne 0\end{array}\\ Z_j=\sqrt{n+1}{R}_n^0\left(r\right)\begin{array}{cc}& \begin{array}{cc}\begin{array}{cc}& m=0\end{array}& \end{array}\end{array}\end{array}$$

(1)

where $\left(\mathrm{r},\theta \right)$ is the polar coordinate in the unit circle, $n$ is the order of the Zernike polynomial, and $m$ represents the angular frequencies, which must meet $m\le n$; $n-\left|m\right|$ is even. $Z_j$ denotes the Zernike polynomial of mode order $j$ [52].

$R_n^m\left(r\right)$ represents the radial polynomial and is deduced from the Jacobi polynomials. It is defined as follows:

$$R_n^m\left(r\right)={\displaystyle \sum _{s=0}^{\left(n-m\right)/2}\frac{{\left(-1\right)}^s\left(n-s\right)!}{s!\left[\left(n+m\right)/2-s\right]!\left[\left(n-m\right)/2-s\right]!}}{r}^{n-2s}$$

(2)

Similar to Fourier patterns, which have been used as representative SPI techniques for illumination by a deterministic orthogonal model rather than random patterns, the Zernike basis pattern is a matrix of grayscale values. Figure 1 shows the Zernike basis patterns of different mode numbers. It is well-known that Hadamard basis patterns only have horizontal and vertical features. Fourier basis patterns have horizontal, vertical, and oblique features whilst Zernike basis pattern have all direction features; the higher the order, the greater the directional resolution. In the experiment, the Zernike base pattern of multiple gray levels was generated for the projection by upsampling and dithering binarization to achieve high-speed and high-quality SPI technology.

2.2. Zernike Transform and Inverse Zernike Transform

The proposed method was based on direct and inverse discrete Zernike transforms. For a continuous signal, the Zernike moment of order $\left(n,m\right)$ was defined as the double integrals inside the unit circle, which is expressed as follows [59]:

$${\mathrm{A}}_{nm}=\frac{n+1}{\pi }{\displaystyle {\iint }_{D}I(x,y)Z_{nm}^{*}(x,y)dxdy}$$

(3)

where $I\left(x,y\right)$ represents a two-dimensional (2D) image function and D is the integration range of the unit circle $\left(x^2+y^2\le 1\right)$. $Z_{nm}^{}\left(x,y\right)$ is a Zernike function and * stands for the conjugate.

For a discrete signal, we were required to discretize a digital image by summation rather than integration to calculate its Zernike moment. Assuming that $I\left(x_i,y_j\right)$ was a function defined from the identity element, Equation (4) could be used to approximate the following:

$$\widetilde{{\displaystyle A_{nm}}}=\frac{n+1}{\pi }{\displaystyle \sum _i^N{\displaystyle \sum _j^{N}I\left(x_i,y_j\right)w_{nm}\left(x_i,y_j\right)}}$$

(4)

where $x_i=\left(2i-N-1\right)/N$ and $y_j=\left(2j-N-1\right)/N$.

i and j were taken such that $\left(x,y\right)\in D$, and

$$w_{nm}\left(x_i,y_j\right)={\displaystyle {\int }_{x_i-\frac{\Delta }{2}}^{x_i+\frac{\Delta }{2}}{\displaystyle {\int }_{y_i-\frac{\Delta }{2}}^{y_i+\frac{\Delta }{2}}{Z}_{nm}^{}\left(x,y\right)}}dxdy\approx {\Delta }^2{Z}_{nm}^{}\left(x_i,y_j\right)$$

(5)

where $\Delta =\left(2/N\right)$ represents the pixel width. $w_{nm}\left(x_i,y_j\right)$ could be numerically calculated, and the most commonly used formula was as follows:

$$w_{nm}\left(x_i,y_j\right)\approx {\Delta }^2{Z}_{nm}^{}\left(x_i,y_j\right)$$

(6)

From the above derivation, the precision of the Zernike moment was affected by geometric and numerical integration errors. The geometric error was caused by the fact that the total area covered by all square pixels involved in the Zernike moment calculation was not an exact unit circle from Equation (4) [59]. The numerical integration was caused by the approximate formula of Equation (6). Although the above errors could be reduced by a few techniques, they could never be eliminated as long as the Zernike moments were computed in Cartesian coordinates [59].

Therefore, image $I\left(x_i,y_j\right)$ could be approximated as follows:

$$I^{’}\left(x,y\right)={\displaystyle \sum _n^{}{\displaystyle \sum _m^{}{A}_{nm}{Z}_{nm}\left(x_i,y_j\right)}}$$

(7)

In accordance with the theoretical analysis, a 2D image could be regarded as a linear superposition of a series of Zernike base patterns and the Zernike coefficient was the weight of the corresponding Zernike base patterns. For a 2D image, the Zernike forward transform could decompose the 2D image into different Zernike base patterns and corresponding weights. The inverse Zernike transform combined different Zernike base patterns into 2D images in accordance with the weight.

2.3. Principle of ZSPI

ZSPI is based on a Zernike transform and an inverse Zernike transform. ZSPI provides a solution when an image consists of a weighted sum of the Zernike basis pattern with different mode ordering. Therefore, image reconstruction by the Zernike moment is the process of obtaining the weight of the image corresponding with each Zernike basis pattern. Zernike basis patterns were generated by using the method of Section 2.1. These patterns were projected onto the target and a bucket detector was used to detect the reflected light intensity of the target.

We assumed that the target was reflective and the reflection intensity of the target in the direction of the bucket detector measurement was $\mathrm{Ref}\left(x,y\right)$. In this case, the image was proportional to the intensity reflection distribution. That is, $I\left(x,y\right)\propto \mathrm{Ref}\left(x,y\right)$.

Therefore, the intensity of the reflected light of the target illuminated by the Zernike basis pattern lighting $E_{\varphi }\left(n,m\right)$ could be given as follows:

$$E_{\varphi }\left(n,m\right)={\displaystyle {\iint }_D\mathrm{Ref}\left(x,y\right)}\cdot Z_j{}^{*}\left(x,y\right)dxdy$$

(8)

where D represents the projected region of the Zernike basis pattern. The response value of the bucket detector was as follows:

$$T_{\varphi }\left(x,y\right)=T_n+\epsilon \cdot E_{\varphi }\left(n,m\right)$$

(9)

where $T_n$ is the DC component caused by the background illumination and $\epsilon$ is a factor related to the magnification of the bucket detector and the position relationship between the object and bucket detector.

Two measurements were needed to obtain the Zernike moment corresponding with each pixel in the target image. We illuminated the scene with two Zernike basis patterns of mode ordering $j$ with a phase-shift of $\pi$. They were denoted as Z₁ and Z₂.

$$\left\{\begin{array}{c}{Z}_1\left(x,y\right)=Z_j{}^{*}\left(x,y\right)\\ Z_2\left(x,y\right)=-Z_j{}^{*}\left(x,y\right)\end{array}\right.$$

(10)

The bucket detectors received light signals from the target, and the signal acquisition and analog/digital conversion were conducted by the signal acquisition device. The computer recorded the response value T of the detector. The response value of the bucket detector was denoted as T.

$$\left\{\begin{array}{c}{T}_1\left(x,y\right)=T_n+\beta \cdot E_{\varphi 1}\left(n,m\right)\\ T_2\left(x,y\right)=T_n+\beta \cdot E_{\varphi 2}\left(n,m\right)\end{array}\right.$$

(11)

In accordance with the two-step phase-shift algorithm, the Zernike forward transformation integral formula of the reflection intensity $\mathrm{Ref}\left(x,y\right)$ could be obtained as follows:

$$\begin{array}{l}{T}_1\left(x,y\right)-T_2\left(x,y\right)=\beta \cdot \left(E_{\varphi 1}\left(n,m\right)-E_{\varphi 2}\left(n,m\right)\right)\\ =\beta \cdot \left({\displaystyle {\iint }_D\mathrm{Ref}\left(x,y\right)}\cdot Z_j{}^{*}\left(x,y\right)dxdy-{\displaystyle {\iint }_D\mathrm{Ref}\left(x,y\right)}\cdot (-Z_j{}^{*}\left(x,y\right))dxdy\right)\\ =2\beta \cdot {\displaystyle {\iint }_D\mathrm{Ref}\left(x,y\right)}\cdot Z_j{}^{*}\left(x,y\right)dxdy\\ =2\beta \cdot Z\left\{\mathrm{Ref}\left(x,y\right)\right\}\end{array}$$

(12)

where $Z\left\{\right\}$ represents the Zernike transformation. The distribution relation between the object image and the reflection intensity of the object was $I\left(x,y\right)\propto \mathrm{Ref}\left(x,y\right)$; that is, the proportional relation was satisfied. Hence, the Zernike moment in the process of SPI could be obtained from the following formula:

$$A\left(x,y\right)=T_1\left(x,y\right)-T_2\left(x,y\right)$$

(13)

3. Results and Discussion

3.1. Numerical Simulations

In this section, an image named “cameraman” (128 by 128 pixels) was utilized as the target scene and simulated by ZSPI with different sampling ratios (SRs), which were taken from the Zernike basis pattern. SR was the ratio between the actual and total measurement numbers on behalf of the capture efficiency. The peak signal-to-noise ratio (PSNR) and root mean square error (RMSE) were used to quantitatively evaluate the quality of the target reconstructions.

$$PSNR=10{\log }_{10}\left(D^2/MSE\right)$$

(14)

$$RMSE=\sqrt{MSE}$$

(15)

where MSE denotes the mean square error.

All the simulations were performed using MATLAB R2020a (R2020a, MathWorks, Nedick, MA, USA, 2020) on a personal computer (PC, Intel Core i7 CPU, 2.90 GHz, 16 G RAM, 64 bit, Windows 10). The target scene was reconstructed by ZSPI of different SRs ranging from 0.1 to 0.3. Figure 2 shows the results. From Figure 2, the reconstructions of SR below 0.1 had a degree of ring call in the image; an acceptable reconstruction appeared at above 0.1, which could be set as the quality benchmark. Figure 3 shows the distributions of the light intensity sequence and measurement numbers. From Figure 3, it can be seen that the large intensity values were concentrated in front of the horizontal axis of the coordinate axis. This finding indicated that the Zernike reconstruction could obtain the desired result by using the first few projections of the Zernike pattern.

In order to quantitatively evaluate the quality of the reconstructions, the normalized RMSE and PSNR were determined. Figure 4 shows the results of the normalized RMSE and PSNR of the reconstructions of different SRs. The most obvious result was that the SR of 0.14 could obtain a smaller RMSE and a larger PSNR with under-sampling, which could be considered to be the appropriate SR of ZSPI. This finding indicated again that the intensity values of the Zernike pattern were concentrated in the front part by the Zernike reconstruction.

In this study, we selected the noniterative methods of differential ghost imaging (DGI), linear iterative methods of conjugate gradient descent (CGD), nonlinear iterative methods of discrete cosine transform (DCT), and compressive sensing based total variation (TV) algorithms for a comparison to further verify the imaging performance of the ZSPI system [60]. Figure 5 shows the performance of ZSPI with various algorithms at the same SR of 0.1, including the DGI, CGD, DCT, and TV algorithms from speckle modulation patterns based on the simulated data. At the same low sampling rate, the image quality of the different reconstruction algorithms was compared. As shown in Figure 5, ZSPI had a higher reconstruction quality than several other algorithms. The PSNR and RMSE were used to quantitatively compare the imaging quality.

To quantitatively compare the quality of the target reconstructions, the RMSE and PSNR of different reconstruction methods were determined. The results of the performance of ZSPI, DGI, CGD, DCT, and TV are shown in Figure 6a. From these results, it could be seen that the PSNR of ZSPI was remarkably higher than that of the other reconstruction methods. Moreover, the RMSE was remarkably lower than that of the other reconstruction methods, which proved that ZSPI had a better imaging quality at low SRs. Furthermore, to investigate the efficiency of the image reconstruction, the reconstruction times were measured for DGI, CGD, DCT, TV, and ZSPI. The results are shown in Figure 6b. The reconstruction times of DGI, CGD, DCT, TV, and ZSPI were 0.16868, 11.25305, 151.19757, 72.42121, and 3.29553, respectively. Although there were five reconstruction methods for comparison, the data of DGI are not contained in Figure 6b because its reconstruction times were too small to be exhibited. However, the DGI algorithms from the speckle modulation patterns had the largest PSNR and the lowest RMSE. Therefore, we observed that ZSPI required less running reconstruction time for a small-scale reconstruction.

3.2. Experiments

Figure 7 shows the physical experiment setup of ZSPI, which included an LED light source (400–760 nm @ 20 W), a digital micromirror device (DMD, ViALUX V-7001), a bucket detector (Thorlabs PDA36A), a data acquisition card (Gage CSE22G8, Vitrek), and a computer. The computer was used to generate the illumination patterns, which were sent to the DMD. The DMD projected these patterns onto the object. The bucket detector was used to receive the light reflected by the object whilst converting the intensity into the corresponding voltage, which was recorded by the data acquisition card. The resolution of the DMD was 1024 × 768 and the refresh rate was 22 kHz. The “cameraman” and the emblem of Beijing Information Science and Technology University were selected as the imaging targets. The reconstruction size was 128 by 128 pixels.

Two groups of experiments based on the above experimental setup were performed to demonstrate the reconstruction performance of ZSPI. In these experiments, ZSPI was compared with various algorithms at the same sample of 0.1, including DGI, CGD, DCT, and TV algorithms from speckle modulation patterns. Figure 8 shows the physical experimental results. Intuitively, ZSPI had a better imaging quality. The structural similarity index measure (SSIM) was calculated for these experimental results to quantitatively evaluate the experimental performance. Figure 9 shows the SSIM performance. The most evident result was that ZSPI had a maximum value of the SSIM for both imaging targets compared with other algorithms from speckle modulation patterns. Therefore, it could be concluded that ZSPI had a better imaging quality at a low SR. Furthermore, from Figure 9, the values of the SSIM of ZSPI of the “cameraman” and emblem were 0.2794 and 0.3169, respectively. It was indicated that ZSPI had the advantage of reconstructing a low-frequency target. Figure 9 also shows the reconstruction time of DGI, CGD, DCT, TV, and ZSPI. As shown in the figure, for the same SR, ZSPI corresponded with the shortest running time, which was the same as the numerical simulation results.

4. Conclusions

In this study, a ZSPI technique was proposed to reduce the number of illumination patterns and reconstruction time whilst retaining its robustness. In order to prove the feasibility, numerical simulations were carried out with different SRs ranging from 0.1 to 0.3; an acceptable reconstruction appeared at approximately 0.1. This result indicated that ZSPI could obtain satisfactory reconstruction results at low SRs. Further simulation and physical experiments were compared with different reconstruction algorithms such as DGI, CGD, DCT, and TV under speckle modulation patterns at a sample of 0.1 in terms of different targets. The results proved that ZSPI had a higher imaging quality and required less imaging time, particularly for low-frequency targets. The proposed method had an evident advantage in reconstructing high-quality pictures with relatively few illumination patterns and a lower reconstruction time, which could be used as a reference for SPI methods such as low-frequency target fast SPI. In addition, the proposed method could be used as a rotated object classification method in microcosmic or remote sensing due to the rotation invariant of the Zernike moments.