Design for Ultrafast Raster Photography with a Large Amount of Spatio-Temporal Information

Abstract

Due to the lack of theoretical research on the amount of spatio-temporal information in high-speed photography technologies, obtaining an optimized system with the best amount of spatio-temporal information remains a challenge, resulting in insufficient effective information and observation accuracy for ultrafast events. This paper presents an ultrafast raster imaging (URI) system with a large amount of spatio-temporal information based on the all-optical raster principle in single-shot. Specifically, we derive the optimal equation of spatial resolution and the expression for the maximum amount of spatio-temporal information that can achieve excellent performance for a URI system. It serves as a general guideline for obtaining a large amount of information design in the URI system. Compared with the existing URI systems, the advanced URI system exhibits an improvement of nearly one order of magnitude in the amount of spatio-temporal information and more than twofold in spatial resolution. It shows great potential for capturing intricate and non-repetitive ultrafast events on the femtosecond time scale.

1. Introduction

High-speed photography, which extends temporal resolution beyond that of the human eye, is a time-amplified technique for blurring-free observations of transient processes. It is an indispensable tool for exploring fundamental mechanisms in physics, chemistry, and biology and has been widely applied in industries, energy, and medicine [1,2,3,4,5]. It is strongly established that imaging techniques with different temporal resolutions are needed to study transient processes with different characteristic times. Traditional film-based cameras [6] and high-speed framing cameras based on digital memory [7,8] can capture movies with a frame interval time as short as 10 ps [7]. However, these methods are insufficient for observing transient dynamics on the atomic time scale (1 ps∼10 fs), primarily due to the limitations of mechanical mechanisms and the electronic readout speed of detectors. Fortunately, the generation and development of ultrashort pulse lasers have propelled the temporal resolution of imaging based on pump-probe technology into the attosecond region [9,10,11,12,13], enabling the effective observation of repeated ultrafast events. To capture non-repetitive processes, various single-shot ultrafast imaging techniques have emerged, boasting frame rates of up to a trillion frames per second (Tfps). These include ultrafast compressed photography (CUP) [14,15,16,17,18,19] with passive detection and active detection-based photography [20,21], which leverage spatial multiplexing encoding [22] and division techniques such as space and are based on space division [23,24,25,26], angle [27,28,29], wavelength [30,31,32,33,34,35,36,37,38,39], spatial frequency [40,41,42], and polarization [43]. Despite significant advances in ultrafast photography, a comprehensive theoretical study of the amount of spatio-temporal information of these systems is still lacking. As a result, obtaining an optimized system with a large amount of spatio-temporal information remains a challenge. Therefore, it is necessary to study the amount of spatio-temporal information, design the imaging system with optimal performance, obtain more effective information on ultrafast events, and improve the observation accuracy.

In this paper, we propose an active illumination ultrafast raster imaging system with a large amount of spatio-temporal information based on the all-optical raster principle in single-shot. The URI system is designed based on the Nyquist sampling theorem and the frequency-time mapping method. Specifically, as shown in Figure 1, the object (transient scenes) is illuminated with a linearly chirped laser pulse. After sampling (e.g., using a microlens array) and imaging, spectrally time-coded raster images are formed, and they pass through a frequency-spatial mapping device (a 4f optical system and a diffraction grating placed in the Fourier plane. Here, spectrally time-coded raster images are focused by the first lens on the Fourier plane. On this plane, light of different frequencies is separated by diffraction grating at specific angles, with each frequency corresponding to a particular diffraction angle. Subsequently, the second lens re-images these diffracted beams on the detection plane), resulting in a spatially dispersed raster image on the detection plane. Here, raster images of different wavelengths are located at different positions on the detector. After that, we extract each single-wavelength raster image with system calibration and reconstruct the object by performing the Fourier transform algorithm based on the Nyquist sampling theorem. In our previous work, the URI system [43] obtained 12-frame images with an intrinsic spatial resolution of 10-line pairs per millimeter (lp/mm). The frame rate and the temporal resolution are 2 Tfps and 460 fs, respectively. However, there is still a lot of room for optimizing the main parameters of spatial resolution, temporal resolution, frame rate, and frame number. Therefore, here, we derive the optimal equation of spatial resolution and the expression for the best amount of spatio-temporal information that can achieve excellent performance for a URI system with an improved amount of spatio-temporal information by nearly one order of magnitude greater than the previous work [43]. This serves as a general guideline for obtaining a large amount of information design in the URI system, which is renowned for its high spatial–temporal resolution and high frame rate and exhibits great potential for capturing intricate and non-repetitive ultrafast events on the femtosecond time scale.

2. The Amount of Information in High-Speed Photographic Systems

For high-speed photography systems, the key goal is to obtain more effective information on the dynamic processes. The equation for the amount of spatio-temporal information in an imaging system can be expressed as [44,45].

$$\left\{\begin{array}{l}I=I_{\mathrm{R}}{I}_{\mathrm{T}}=S{R}^2\ln \kappa f{g}^{2/3}\\ I_{\mathrm{R}}=S{R}^2\ln \kappa \\ I_{\mathrm{T}}=f{g}^{2/3}\end{array}\right.$$

(1)

where I represents the total amount of spatio-temporal information, which refers to the number of bits transmitted and recorded with a high-speed camera per unit of time (bit/s). It covers both time and space information: the amount of temporal information, denoted as I_T, and the amount of temporal information, denoted as I_R, so it is an important indicator for comprehensively evaluating the performance of high-speed cameras.

The frame rate f, which is the reciprocal of the frame interval time τ_f, is a fundamental parameter that characterizes a camera’s temporal resolution capability. The modified temporal qualify factor g^2/3 can be determined as the ratio of the frame interval time τ_f to the effective exposure time τ. In order to avoid information blurring caused by overlapping between adjacent frames, the optimization condition is set to g^2/3 = 1.

The space information amount I_R is the product of the spatial bandwidth product (S·R²) of a single frame and the information capacity (ln κ) of a single pixel. Here, κ denotes the number of information levels and usually indicates the signal-to-noise ratio. S and R correspond to the area and spatial resolution of the image, respectively. I is usually used to evaluate the quality of a high-speed camera, which reflects not only the ability of the camera to record spatio-temporal information but also the level of design and manufacturing of the high-speed camera.

In summary, it is crucial to design an imaging system with a large amount of spatio-temporal information. The maximum amount of spatio-temporal information I_max should be equal to the product of the optimal amount of temporal information I_Topt and the optimal amount of temporal information I_Ropt. Typically, we design imaging systems with optimal spatial resolution and the optimal effective frame rate to achieve this goal.

3. Analysis and Discussion

3.1. Characterization of the Spatial Resolution and Frame Number

Spatial resolution R is a key parameter in high-speed photography. For a URI system, the spatial resolution (intrinsic spatial resolution) is mainly decided by the pitch h of sampling points of the raster image. The sampling and framing of the URI are depicted in Figure 2. The red squares represent the raster sampling points after sampling the 2D image, and the size of sampling points is δ. Framing the raster image along the time direction K, where K refers to the spectral dispersion direction of the probe pulse on the detection plane, thus, raster images of different wavelengths (colors) are located at different positions on the detector. According to the linear time–wavelength mapping relationship of linear chirped laser pulses, squares of different colors represent sampling points at different times of the raster image. L is the maximum length of spatially dispersive raster sampling points along the time direction K, and d₂ is the distance between two adjacent spatially dispersed sampling points of the raster image. The geometric relationship between the parameters is given by

(2h + x) h = (d₁ + d₂) L

(2)

Substituting equation (h + x)/h = d₁/d₂ into Equation (2), and after simplification, we obtain the relationship h² = d₂ L, Considering the framing of the raster image sampling points by sampling points, the value of L should be the product of the size of sampling points δ and the frame number n, i.e., L = (n + 1) δ. When d₂ = δ, the relationship between the pitch h of the sampling point and the frame number n can be mathematically formulated

N = h²/δ² − 1

(3)

Theoretically, R is inversely proportional to the h of sampling points, that is, R = 1/2h. Combining Equation (3), it can be expressed as

$$R=\frac{1}{2}\frac{1}{\sqrt{(n+1)}\delta }$$

(4)

In URI, the size of the sampling point of raster image δ is mainly modulated by the entrance pupil of an objective of the imaging system, and the common size of sampling points is around 10 μm. Here, we consider the relationship between spatial resolution and the frame number in the case of δ = 10 μm, 12 μm, and 14 μm, as illustrated in Figure 3. It is evident that there exists a trade-off between spatial resolution and frame number, that is, we can obtain more frames at the cost of an obvious decline in spatial resolution. But, on the brighter side, reducing the size of the sampling point of the raster image can potentially improve either spatial resolution or the frame number.

3.2. Frame Rate and Spatial Resolution

Frame rate f is another critical parameter in high-speed photography. To effectively capture ultrafast events on the atomic time scale, frame rate should be performed at the femtosecond level. For a URI system, the frame rate is determined by the duration T of a probe laser pulse and the size of the sampling point of the raster image. The size of the sampling point can be adjusted with the entrance pupil of the objective of the optical system. By combining Equation (4) with the relationship Tf = L/δ = n, the relationship between the spatial resolution and the frame rate is given by

$$f=\frac{1}{T}(\frac{1}{4R^2{\delta }^2}-1)$$

(5)

Figure 4 illustrates the relationship between frame rate and spatial resolution. Here, we consider the cases T = 10 ps and δ = 10 μm, 12 μm, and 14 μm. Obviously, the frame rate is inversely proportional to the square of the spatial resolution. Notably, for a given pitch h of sampling points, by reducing the size δ of the sampling point, the frame rate of a URI system can be effectively improved.

3.3. Optimal Temporal Resolution and the Spatial Resolution Equation for a URI System

The temporal resolution of the URI system mainly relies on the duration of the original Fourier-transform limit femtosecond pulse and the linearly chirped pulse. The equation $\tau =\sqrt{t_{0}T}$ indicates an optimal temporal resolution [46], where t₀ is the Fourier-limited duration of the original femtosecond pulse, and the time window T is the FWHM (full width at half maximum) of the chirped laser pulse. Therefore, a shorter Fourier-limited duration femtosecond pulse source can further improve the temporal resolution of a URI system, and the FWHM of the chirped pulse should cover the duration of the ultra-fast process. Here, to ensure the optimal temporal degree of freedom (DOF) and avoid information blurring caused by overlapping between the adjacent frames, the modification factor $g^{2/3}$ is set to 1. That is, the frame interval (Δt) equals the effective exposure time τ. Therefore, the optimal frame number is denoted by $n_{\max }=T/\tau =\sqrt{T/t_0}$, which should be a constant for a given light source system and the time window of the observation.

For a high-speed imaging system, Equation (1) indicates that improving the spatial resolution can effectively improve the amount of spatio-temporal information. In URI, in a static situation, spatial resolution is time independent. However, at the femtosecond scale, the frame number $n=\sqrt{T/t_0}$ should be given based on the case of optimal temporal resolution and then substituting it into Equation (4), so, we can obtain the time-dependent spatial equation

$$R=\frac{1}{2}\frac{1}{\sqrt{(\sqrt{T/t_0}+1)}\delta }$$

(6)

According to the previous analysis and discussion, we know that reducing the size of the sampling point of a raster image is the key to improving the spatial resolution and frame rate. As we known, the size of the sampling point of a raster image should be equal to a shifting size (Δx) of adjacent frames, i.e., δ = Δx = (f_l/Λ), where Λ and f_l are the grating period and the focal length of the lens in the 4f optical system, respectively, and Δλ is the corresponding wavelength difference, which should be greater than or equal to the spectral resolution of URI system. The spectral resolution (δλ) can be denoted by δλ = λ₀Λ/S′, where S′ is the probe beam width on the grating (given by S′ = f_l d/f_m, where d and f_m are the diameter and focal length of each lens in the microlens array, respectively). Therefore, the spectral resolution δλ can be expressed as δλ = λ₀Λf_m/f_l d. Considering that Δλ ≥ δλ, we can obtain δ ≥ λ₀f_m/d. Meanwhile, since one microlens corresponds to one sampling point, the size of the sampling point must be greater than or equal to the diffraction limit of the microlens, that is, δ ≥ 1.22λ₀f_m/d, where λ₀ is the center wavelength of the probe laser pulses. Obviously, when δ_min = 1.22λ₀f_m/d, it follows that Δλ ≥ δλ, and thus, we obtain the optimally time-dependent spatial resolution

$$R_{opt}=\frac{1}{2}\frac{d}{1.22\sqrt{(\sqrt{T/t_0}+1)}{\lambda }_0{f}_{\mathrm{m}}}$$

(7)

Figure 5 shows the relationship between the time window and the spatial resolution in the case of δ = δ_min for previous URI systems. When the designed time window is T = 6 ps, it follows that n = 13, f = 2.2 × 10¹² fps, and the spatial resolution is approximately improved to 26 lp/mm. Therefore, compared with existing URI systems, the spatial resolution of the optimized URI system is significantly improved by 2.6 times. In addition, a higher spatial resolution can be obtained using a shorter central wavelength of the probe laser pulse.

3.4. The Maximum Amount of Information of a URI System

According to Equation (1) above, combining the expressions of the spatial resolution and frame rate, the amount of spatio-temporal information in a URI system is given by

$$I=\frac{S}{4T}\frac{n}{(n+1){\delta }^2}\ln \kappa g^{2/3}$$

(8)

where S = a × b is the area of the image, where a and b are the length and width, respectively. By substituting the optimal parameters into Equation (8), we obtain the expression for the maximum amount of spatio-temporal information

$$I_{\max }\approx \frac{S}{2T}\frac{\sqrt{{T/t}_0}}{(\sqrt{{T/t}_0}+1){\lambda }_0{}^2}{(\frac{d}{f_{\mathrm{m}}})}^2$$

(9)

For a given light source system of a URI, Equation (9) provides the expression for designing the optimal URI. Here, the time window should be designed to cover the duration of the ultrafast process. Next, we design a suitable relative aperture (d/f_m) of a microlens based on the current manufacturing level of the microlens array. Finally, we design the optimal sampling points pitch h of the URI system. As shown in Figure 6, after optimizing the existing URI system, namely, Best-URI, the maximum amount of spatio-temporal information will reach 1.4 × 10¹⁷ bit/s. Meanwhile, it is evident from Figure 7 that the maximum amount of spatio-temporal information of Best-URI is close to one order of magnitude greater than that of 2.1 × 10¹⁶ bit/s in the existing URI system.

In this study, we clearly demonstrated that by optimizing parameters such as the pitch of sampling points, the size of the sampling point, and the frame rate, we obtain Best-URI with the maximum amount of information on the femtosecond time scale. However, as a single-shot imaging technique in an active detection mode, further improving the system’s temporal resolution (i.e., effective frame rate) remains a challenge due to the constraints imposed by the time-frequency uncertainty principle of the probe laser pulse. To address this issue, we plan to combine polarization time-encoding techniques with raster principles to circumvent the time–frequency uncertainty constraints of the probe pulses, thereby further improving the URI system’s temporal resolution and obtaining a larger amount of spatio-temporal information.

4. Conclusions

This paper presents a formula regarding the amount of spatio-temporal information based on the all-optical raster principle to realize an ultrafast imaging system with a large amount of spatio-temporal information. We derived the spatial resolution expression of the system by considering the pitch of sampling points, the size of the sampling point, and the frame number, as well as the relationship between spatial resolution and the frame rate. Furthermore, we also obtained the optimal time-dependent spatial resolution and the maximum amount of information in the system operating at the femtosecond time scale. Compared with existing URI systems, an optimized URI system designed based on the maximum amount of information exhibits an improvement of nearly one order of magnitude in the amount of spatio-temporal information and more than twofold in spatial resolution. The optimized URI system, renowned for its large amount of spatio-temporal information and high spatio-temporal resolution, exhibits greater potential for capturing intricate and non-repetitive ultrafast events on the femtosecond time scale.