Three-Dimensional Imaging via Time-Correlated Single-Photon Counting

Abstract

Three-dimensional (3D) imaging under the condition of weak light and low signal-to-noise ratio is a challenging task. In this paper, a 3D imaging scheme based on time-correlated single-photon counting technology is proposed and demonstrated. The 3D imaging scheme, which is composed of a pulsed laser, a scanning mirror, single-photon detectors, and a time-correlated single-photon counting module, employs time-correlated single-photon counting technology for 3D LiDAR (Light Detection and Ranging). Aided by the range-gated technology, experiments show that the proposed scheme can image the object when the signal-to-noise ratio is decreased to −13 dB and improve the structural similarity index of imaging results by 10 times. Then we prove the proposed scheme can image the object in three dimensions with a lateral imaging resolution of 512 × 512 and an axial resolution of 4.2 mm in 6.7 s. At last, a high-resolution 3D reconstruction of an object is also achieved by using the photometric stereo algorithm.

1. Introduction

In recent years, single-photon counting lidar has been widely used in three-dimensional (3D) imaging under weak light conditions because of its single-photon sensitivity and picosecond time resolution. This technology has many significant applications, such as long-distance imaging, underwater target detection [1]. The core idea of this technology is time-correlated single-photon counting (TCSPC). TCSPC is the most effective way to measure the temporal structure of weak optical signals, which is based on the accurate measurement of photon arrival times and time difference [2,3]. TCSPC was mainly used in the detection of fluorescence lifetime in the early stage [4,5,6,7,8,9,10,11,12]. After that, TCSPC began to be used in the field of laser ranging [13,14,15,16,17,18] and gradually developed to the direction of laser long-distance 3D imaging based on time-of-flight (TOF) algorithm [19,20,21,22,23,24,25,26]. Recently, Li et al. achieved the imaging of targets 45 kilometers away [27]. Kirmani et al. [28] and Xialin Liu et al. [29] detected the first photon in TCSPC system to image the object.

In the general TCSPC system, the image result is greatly affected by environmental noise because of the ultra-high sensitivity of the single-photon detectors (SPD). Up to now, most of the research teams used the range-gated technology to decrease the impact of noise [20,24,27]. Besides, Kong et al. used two Geiger-mode avalanche photodiodes and an AND gate to reduce false alarms caused by environmental noise [30]. Huang et al. come up with a new method to decrease environmental noise by using correlative photons and spatial correlations [31].

In this paper, we combine SPD, high repetition rate laser pulses illumination and TCSPC technology to design a new imaging scheme. The new scheme has the capability of de-noise, high-resolution imaging, accurate analysis of the photon flight time and 3D imaging. Three experiments are completed to demonstrate the power of the new imaging scheme. The first is the de-noise experiment which proves that our scheme can image the object when the signal-to-noise ratio (SNR) is decreased to −13 dB and improve the structural similarity index (SSIM) [32] of imaging results by 10 times. The second proves that the scheme has a lateral imaging resolution of $512\times 512$ and an axial resolution of 4.2 mm. The third is 3D imaging by photometric stereo algorithm which uses multiple detectors to reconstruct the 3D image of the object. Besides, the new imaging scheme has the potential to realize turbulence-free imaging [33], non-of-sight imaging technology [34] and imaging in a complicated environment.

2. Theory Analysis

In our scheme, the core idea is to use the TCSPC technology to get the information of the number and arrival time of each signal photon. Aided by the range-gated technology, the environmental noise can be significantly reduced. Furthermore, with these information the TOF algorithm and photometric stereo algorithm can be used to reconstruct the 3D model of the object.

2.1. TCSPC

TCSPC technology, which is mainly used to detect the low intensity and high repetition frequency pulsed laser, is developed on the basis of the photon-counting technology. The realization process is as follows: pulse laser emits a pulse, and a synchronization signal is generated at the same time to trigger the START of the TCSPC module to start timing, as shown in Figure 1a. Each photon detected by the SPD triggers the STOP of the TCSPC module to stop timing. Then the TCSPC module can record the arrival time of each photon reference to the emitted laser pulse. At last, after statistical processing, the histogram of the number of photons VS the arrival time can be obtained, as shown in Figure 1b.

2.2. Detection Probability of Signal Photon Events

As described in [28,35], supposing the object point $(x,y)$ is illuminated by a pulse $S\left(t\right)$ and the echo pulse reflected the SPD is $S\left(t-\frac{2z_{x,y}}{c}\right)$. Then the theoretical number of photons detected by SPD is

$$r_{x,y}\left(t\right)={\alpha }_{x,y}S\left(t-\frac{2z_{x,y}}{c}\right)+b,$$

(1)

where $z_{x,y}$ represents the flight distance of the laser from the target point, c represents the speed of light, ${\alpha }_{x,y}$ represents the reflectivity of target and b represents the number of background photons. Considering that the quantum efficiency of SPD is $\eta$ and the dark counts are d, the actual number of photons ${\lambda }_{x,y}$ detected by the SPD is

$${\lambda }_{x,y}\left(t\right)=\eta r_{x,y}\left(t\right)+d=\eta {\alpha }_{x,y}S\left(t-\frac{2z_{x,y}}{c}\right)+(\eta b+d).$$

(2)

Let the pulse laser period be $T_r,$ then the average number of photons received by the SPD in a period is

$$\begin{array}{cc}\hfill {\lambda }_{x,y}={\lambda }_{x,y}\left(t\right)·T_r=\eta {\alpha }_{x,y}{\int }_0^{T_r}S\left(t\right)dt+(\eta b+d)T_r& =\eta {\alpha }_{x,y}S+B.\hfill \end{array}$$

(3)

According to the Poisson probability model [36], for each pulse emitted by the laser, the number of photons k received by the detector satisfies the following distribution:

$$P_{x,y}(n=k)=\frac{{\lambda }_{x,y}^k}{k!}{e}^{-{\lambda }_{x,y}}.$$

(4)

When the SPD does not record any photon, that is $k=0$, the probability can be denoted by $P_{x,y}(n=0)=e^{-{\lambda }_{x,y}}$.

2.3. Imaging Scheme Based on TCSPC

In our scheme, the dead time of SPD is longer than the pulse width of laser. Therefore, the number $N_{x,y}$ of photons received by the SPD after each point scanned by the scanning mirror is

$$N_{x,y}=\frac{1}{\omega T_r}\left(1-e^{-{\lambda }_{x,y}}\right),$$

(5)

where $T_r$ denotes the pulse laser period and $\omega$ denotes the scanning frequency of scanning mirror. The pixel value of target point $(x,y)$ is represented by $N_{x,y}$. When the whole object is scanned by the scanning mirror, the two-dimensional image of the target can be obtained by rearranging the pixel values. Besides, because the flight time of each photon and the number of photons can be obtained accurately by the TCSPC module, the depth information of the object can be calculated by the TOF algorithm. Lastly, multiple SPDs are used to detect objects from different angles. By combining the photometric stereo algorithm, we can also get the 3D model of the object.

3. Experimental Setup and Results

To verify the 3D imaging scheme based on TCSPC proposed in this paper, three groups of experiments are performed to confirm the de-noise ability, high-resolution imaging ability, 3D imaging based on the photometric stereo algorithm. We use a pulsed laser with a wavelength of 532 nm as light source, which generated 30 ps duration pulses at a repetition rate of 10 MHz. The SPD module is the PDM series manufactured by Micron Photon Devices. The dead time of SPD is 80 ns and the quantum efficiency at 532 nm is about 45%. Besides, HydraHarp 400, whose time resolution down to 1 ps and maximum count rate of up to 12 Mcps per channel, is used as the TCSPC module and a field-programmable gate array (FPGA) is used as the main controller.

3.1. Noise Reduction Experiment Based on Range-Gated Technology

The experimental setup is depicted in Figure 2. FPGA controls the laser to emit a pulse and simultaneously generates a synchronous signal input to the HydraHarp 400 to trigger the START signal to start timing. After each fixed number of pulses produced by the laser (1024 pulses in our experiment), FPGA controls the scanning mirror to rotate once. Then SPD records the corresponding photon signal. Since the pulse width (30 ps) of the laser is smaller than the dead time of SPD (80 ns), the SPD can only respond to one photon at most for each pulse. So the probability that SPD detects the signal light is as shown in Equation (5). After the entire object is scanned by the scanning mirror, the computer can get the image of the object by rearranging the photon signals of HydraHarp 400 according to the preset scanning mode of scanning mirror.

The experimental results are shown in Figure 3. During the experiment, the number of noise photons in the environment is kept at around 10,000 counts per second (cps), and the number of signal photons is decreased from 3000 cps to 500 cps in turn. The experimental results are compared between the imaging results with and without the range-gated technology. As shown in Figure 3, the experimental image quality with gating is obviously better than that without gating. When the number of signal photons decreased to 500 cps, that is SNR is −13 dB, the experimental results with gating can still distinguish the object while the image without gating cannot. Besides, the SSIM, defined as Equation (6), is used to evaluate the qualities of the reconstructed images.

$$SSIM(x,y)=\frac{\left(2{\mu }_x{\mu }_y+C_1\right)\left(2{\sigma }_{xy}+C_2\right)}{\left({\mu }_x^2+{\mu }_y^2+C_1\right)\left({\sigma }_x^2+{\sigma }_y^2+C_2\right)},$$

(6)

where ${\mu }_x,{\mu }_y,{\sigma }_x,{\sigma }_y$ and ${\sigma }_{xy}$ are the local means, standard deviations, and cross-covariance for images $x,y$. Furthermore, $C_1={(0.01\times L)}^2$, $C_2={(0.03\times L)}^2$, where L is the specified dynamic range value (10 bits in this experiment). As shown in Figure 4, the SSIM of reconstructed images with gating is 10 times higher than that without gating.

3.2. High-Resolution Imaging Experiments for Complex Objects

In this part, two sets of experiments are set to verify the high resolution including the lateral resolution and the axial resolution of the scheme. Firstly, the lateral resolution of our scheme can be calculated by

$$R=1.22\frac{\lambda z}{D_0}\approx 0.5\mathrm{mm},$$

(7)

where $\lambda$ is the wavelength of the laser, $D_0$ is beam waist diameter of laser, z is the distance from the target to laser. To further prove the lateral resolution, we choose a dinosaur model with a size of $8\mathrm{cm}\times 6\mathrm{cm}\times 2\mathrm{cm}$ as the imaging object in this experiment. The experimental process is roughly the same as that of the experiment in Section 3.1. Figure 5 shows the original object and the experimental imaging result, in which the image resolution of the object is $512\times 512$, the image depth is 8 bit and the time for one frame is about 6.7 s.

Then to ensure the axial resolution, a whiteboard ($5\mathrm{cm}\times 5\mathrm{cm}$) is placed on the stepper motor (TSA150-B, repeat positioning accuracy less than 7 um). The stepper motor moves ten times and moves 10 mm each time. The distance that the whiteboard moves is evaluated by the TOF algorithm, defines as:

$$D=\frac{1}{2}c\times \mathsf{\Delta }t,$$

(8)

where D is the distance, c is the speed of light and $\Delta t$ is the time interval. In this experiment, the theoretical total time jitter mainly consists of the pulse width of the laser $(t_{\mathrm{pulse}}\sim 30\mathrm{ps}),$ the time jitter of TCSPC module $(t_{\mathrm{TCSPC}}\sim 12\mathrm{ps})$ and the time jitter of $SPD\left(t_{\mathrm{SPD}}\sim 50\mathrm{ps}\right),$ as the function of

$$T_{\sigma }=\sqrt{t_{\mathrm{pulse}}^2+t_{\mathrm{TCSPC}}^2+t_{\mathrm{SPD}}^2}\approx 60\mathrm{ps}.$$

(9)

As shown in Figure 6a, the total time jitter measured by the experiment is 90 ps. Therefore the distance error $D_{\sigma }$ of our scheme is $D_{\sigma }=1/2\times c\times T_{\sigma }\approx 13.5\mathrm{mm}$. Because we take the average value of 10 points near the peak position as the measurement value each time, the actual measurement error ${\overline{D}}_{\sigma }$ can be decreased to 4.2 mm, as the function:

$${\overline{D}}_{\sigma }=D_{\sigma }/\sqrt{N}.$$

(10)

Figure 6b shows the measurement results and Figure 6c shows the measurement errors which are less than ±1.5 mm.

3.3. 3D Imaging Experiment Based on Photometric Stereo Algorithm

The photometric stereo algorithm is a method to reconstruct the 3D model of an object from multiple images. It uses a set of cameras to take pictures of objects from different directions and then obtains the 3D information by calculating the reflectivity and surface normal vector [37]. We define the direction of the light source is $\overrightarrow{L}$, the normal unit vector of a point on the surface of the object is $\overrightarrow{N}$, the reflectivity is $\rho$ and the intensity of the pixel on the corresponding image is I. Then we can get the Lambert model

$$\overrightarrow{I}=\rho \overrightarrow{N}·\overrightarrow{L},$$

(11)

where

$$\overrightarrow{N}={\left(N_x,N_y,N_z\right)}^{\mathrm{T}}={\left(-\frac{p}{\sqrt{p^2+q^2+1}},-\frac{q}{\sqrt{p^2+q^2+1}},\frac{1}{\sqrt{p^2+q^2+1}}\right)}^{\mathrm{T}},$$

(12)

$\mathrm{T}$ represents the matrix transpose operator, p and q respectively represent the two gradient directions on the surface of the object. Assuming that there are three known non-coplanar light directions, then Equation (11) can be obtained. Because $\overrightarrow{L}$ is a full rank matrix, the normal vector can be expressed as $\overrightarrow{N}=\frac{1}{\rho }I·{\overrightarrow{L}}^{-1}$. Then we can get the depth information by integrating the normal vector of the object surface,

$$\left\{\begin{array}{c}{I}_1=\rho \left(L_{11}{N}_x+L_{12}{N}_y+L_{13}{N}_z\right)\hfill \\ I_2=\rho \left(L_{21}{N}_x+L_{22}{N}_y+L_{23}{N}_z\right)\hfill \\ I_3=\rho \left(L_{31}{N}_x+L_{32}{N}_y+L_{33}{N}_z\right)\hfill \end{array}\right.$$

(13)

Figure 7 shows the experimental setup. Three SPDs are used in this experiment to detect the 3D object (a rabbit model with a size of $4\mathrm{cm}\times 3\mathrm{cm}\times 7\mathrm{cm}$) from different directions, which need to be calibrated in advance. The experimental data is processed by the photometric stereo algorithm to obtain a 3D image of the object. As shown in Figure 8, the method can reconstruct the 3D model of the object well.

4. Conclusions

In summary, a 3D imaging scheme based on TCSPC technology has been proposed and demonstrated. In the scheme, we adopt the range-gated technology to reduce environmental noise and improve the SSIM of imaging results by 10 times. At the same time, the TOF algorithm and photometric stereo algorithm are used for three-dimensional imaging of objects. Experiments show that the proposed scheme can realize imaging with a lateral resolution of $512\times 512$ and an axial resolution of 4.2 mm. Finally, the proposed imaging scheme has the potential to be widely used in imaging in complicated environments, especially in remote, haze and underwater.