The Effect of Intensity Fluctuations on Sequential X-ray Photon Correlation Spectroscopy at the X-ray Free Electron Laser Facilities

Abstract

How materials evolve at thermal equilibrium and under external excitations at small length and time scales is crucial to the understanding and control of material properties. X-ray photon correlation spectroscopy (XPCS) at X-ray free electron laser (XFEL) facilities can in principle capture dynamics of materials that are substantially faster than a millisecond. However, the analysis and interpretation of XPCS data is hindered by the strongly fluctuating X-ray intensity from XFELs. Here we examine the impact of pulse-to-pulse intensity fluctuations on sequential XPCS analysis. We show that the conventional XPCS analysis can still faithfully capture the characteristic time scales, but with substantial decrease in the signal-to-noise ratio of the $g_2$ function and increase in the uncertainties of the extracted time constants. We also demonstrate protocols for improving the signal-to-noise ratio and reducing the uncertainties.

1. Introduction

The evolution of solid-state materials at equilibrium and under external perturbations is crucial to understanding and controlling critical electronic properties in a wide range of materials, such as ferroelectrics [1], quasi-two-dimensional materials [2], and superconductors [3]. Numerous studies have focused on the relationships between the crystal and electronic structures [4,5]. Historically, such insights have primarily been gained in terms of average atomic positions and electronic densities of the material, while the importance of meso- and nanoscale heterogeneities ubiquitous in realistic functional and quantum materials has come to light more recently. For example, grain boundaries in polycrystalline metals have a major impact on the macroscopic optical constants and electron relaxation times [6], and can affect the dissipation of energy over sub-nanosecond time scales when the material is subject to intense optical excitation [7]. In a class of ferroelectric materials known as “relaxors”, polar nano regions (PNRs)—nano-size ferroelectric domains—are thought to be responsible for the observed 1000-fold increase in dielectric permittivity [8,9,10] with a frequency dependence spanning from Hz to MHz and beyond. In the realm of quantum materials, the domain walls in otherwise static charge-density-wave materials can be depinned by applying mild electric fields [11], leading to stochastic electric spikes in the MHz regime. These examples reveal how meso-/nanoscale inhomogeneities and their dynamics over fast and ultrafast time scales affect the macroscopic properties of materials. Thus novel experimental probes and analysis methods with matching spatial and temporal sensitivities are particularly valuable in understanding the role and dynamics of heterogeneous materials.

X-ray photon correlation spectroscopy (XPCS) has recently emerged as a tool for capturing the dynamics of structural evolution in materials and molecular assemblies [12,13,14]. Coherent X-ray photons scattered by the material produce a speckle pattern in the far field, which is the Fourier transform of all the diffracting features within the beam footprint. The real-space motion of these diffracting features results in changes in the fine structure of the speckle pattern, resulting in a speckle decorrelation with time. XPCS quantifies the characteristic time scale of speckle decorrelation and sheds light on the material dynamics responsible for it. Until recently, the temporal resolution of XPCS practiced at synchrotron storage ring X-ray sources was on the order of seconds and milliseconds. Since many of the critical processes may take place on substantially faster time scales, improving the time resolution of XPCS will enable means of observing these dynamics.

The advent of X-ray free electron lasers (XFELs) in the last decade has made available highly coherent X-ray pulses with a duration of tens to hundreds of femtoseconds, making it possible to explore faster material dynamics. Compared to synchrotron-based measurements, XPCS experiments at XFELs face unique challenges including pulse-to-pulse fluctuations in the pointing of X-rays and most prominently 100% variations of the incident XFEL intensity. Originating mostly from the properties of the XFEL source itself, whether and how these fluctuations affect XFEL-based XPCS experiments is under intense investigation [15,16,17,18].

At present, two modes of XPCS studies have been discussed and demonstrated at XFELs: (1) the sequential mode, where one XPCS speckle pattern is collected per X-ray pulse [19,20], similar to synchrotron-based XPCS experiments; (2) the split-pulse mode [16,21,22,23,24,25], where one ultrafast X-ray pulse is split into two, and subsequently recombined with a relative delay from a few picoseconds up to nanoseconds. In the latter mode, each image collected from the X-ray detector is the sum of speckle intensities from two pulses. The time resolution of the sequential mode is limited by the XFEL repetition period, while that of the split-pulse mode is given by the delay. To date, substantially fewer XFEL XPCS studies have been carried out in the sequential mode because the repetition period was 120 Hz at Linac Coherent Light Source (LCLS) and tens of Hz elsewhere, giving a relatively modest time separation of ∼8 milliseconds or more between adjacent XFEL pulses.

The construction of European XFEL [26] and the upgraded Linac Coherent Light Source-II (LCLS-II) have made possible XFEL sources with repetition periods shorter than microseconds. In parallel, faster X-ray detectors with frame rates commensurate with the pulse repetition rate have become available at both XFEL and synchrotron facilities [27,28]. These developments have made possible the first sequential XPCS study in the microsecond range [29], and will enable more XPCS studies of key material dynamics in the upcoming decades, filling a time gap between synchrotron-based and split-pulse XPCS studies. Notably, this regime covers the time scale where domain motions become prominent in a variety of important material systems, including the aforementioned charge-density-wave materials and the ferroelectric relaxors.

As such, it is critical to understand the impact of XFEL fluctuations on sequential XPCS analysis to inform future experiments. Previous XPCS studies at XFELs have been carried out in model soft materials [19,20]. Carnis et al. [19] address the issue of fluctuations in the incident beam by utilizing integrated speckle patters summed over 100 consecutive XFEL pulses. Though this strategy dampened the effect of intensity fluctuations in the data, it also reduced the time resolution. In comparison, Lehmkühler et al. [20] demonstrated pulse-to-pulse sequential XPCS analysis and discussed the role of the XFEL pointing instability, but not intensity changes.

Here we discuss the effect of highly fluctuating XFEL intensities on the sequential XPCS analysis and develop strategies to optimize these analysis. Specifically, this paper will address the following two questions:

• To what degree can sequential XPCS analysis capture the dynamics of a material from relatively low intensity speckle patterns in the presence of significant pulse-to-pulse intensity variations typical of XFELs?
• What data analysis strategies can be developed to improve the determination of the time constants from such data sets?

We go about answering these questions via simulations of XFEL small-angle scattering experiments from diffusing particles, thus providing well-defined dynamics that can be compared to the analysis results of simulated XPCS experiments. We test various scenarios, including the cases of low intensity speckle and fluctuating or non-fluctuating X-ray source intensity, to evaluate their impact on sequential XPCS analysis. Our conclusion is that XPCS analysis of data sets with high pulse-to-pulse intensity fluctuations can indeed capture the characteristic time scale of an evolving system, and that certain changes to the analysis can be invoked to decrease the uncertainty of this quantity. We end with a discussion of future prospects for applying the sequential XPCS method at MHz-XFELs to measuring the dynamics of solid state materials via the analysis of wide-angle scattering speckle patterns.

2. Materials and Methods

2.1. Simulating the Speckle Pattern

To generate a series of speckle patterns for testing XPCS analysis under different conditions, we utilized a two-dimensional (2D) simulation of particle diffusion on a lattice. We generated representations of the real space arrangements of 1000 particles that evolve in time over the course of 2000 frames. The total number of frames is chosen to be on the same order as what can be obtained in a realistic XPCS experiment at European XFEL and LCLS-II at the moment and in the near future. Compared to synchrotron-based XPCS experiments with substantially more frames, the smaller frame number is bounded by hardware constraints, including the number of continuous XFEL pulses in a pulse train, and by the number of images that can be stored in the cache of the detector. The individual time steps are $200\times 200$-pixel frames composed of single pixel particles in an otherwise zero-valued array. The frame-to-frame time evolution of the position of these particles obeys a random walk in 2D with a jump length of $a=1$ pixel and a jump rate for each particle of $\Gamma =0.1$ frame. Each image is a snapshot of the 2D random walk as a function of time t with equal duration between images. The particle diffusivity is $D=(1/4)\Gamma a^2=0.025$ pixels${}^2$ per frame. This provides a known answer for comparing with the correlation time $\tau =1/\left(D{Q}^2\right)$ expected from the XPCS speckle pattern analysis, which depends on the wavenumber Q. We note that while larger area detectors are being used at XFELs and synchrotron facilities, usually only a small region of interest is relevant in an XPCS analysis. Thus we use a relatively small number of pixels in our simulations. Also as the rest of our analysis will be based on the average photons per pixel, our main conclusion will not depend on the total number of pixels used in the simulations.

The speckle intensity patterns from each real-space image is generated by calculating the square of the Fourier transform

$$\begin{array}{c}\hfill I_{t,NF}(q_x,q_y)={\left|\mathcal{F}\left(R_t(x,y)\right)\right|}^2.\end{array}$$

(1)

In this expression, $R_t(x,y)$ is the distribution of particles in the t-th image frame expressed in real space $(x,y)$, and $I_{t,NF}(q_x,q_y)$ denotes the corresponding speckle intensity distribution expressed in reciprocal space coordinates $(q_x,q_y)$ via a Fourier transformation operation ($\mathcal{F}$). The subscript $NF$ indicates that this speckle intensity field is continuous and “noise-free”. In Figure 1a, we show the 100-th “noise-free” speckle pattern from the entire time series, and a zoomed-in view of the speckle pattern is displayed in Figure 1b. The determination of ${\tau }_{NF}\left(Q\right)$ from the simulated XPCS dataset is detailed in Section 2.2.

To simulate low-intensity and variable-incident-intensity speckle patterns from the noise-free dataset, we follow a two-step procedure. First, we multiply the continuous intensity field of each noise-free speckle pattern by a scaling factor $\alpha$, resulting in a scaled intensity field that is also continuous-valued. Second, Poisson statistics are applied to produce a discrete value (photon count) at each pixel of the noisy speckle pattern. This assumes an ideal photon-counting detector, where the detector does not have miscounting errors due to background (pedestal) subtraction, or spurious electronic noise. The notation we adopt hereafter to refer to simulations and analysis that include Poisson counting statistics is to use the subscript “$SIM$” (as in $I_{t,SIM}$). An example speckle pattern for which Poisson counting statistics have been simulated is shown in Figure 1c. This pattern was generated with an $\alpha =0.001$ giving an average intensity of ∼1 photons/pixel in the zoomed-in view. It is evident that the number of photons at each pixel in Figure 1c is not exactly proportional to the intensity in Figure 1b due to the Poisson statistics.

2.2. Calculating the Time Correlations

For the purposes of calculating the time-dependent correlation of speckle patterns in this work, we adopt the formulation as detailed by Sutton et al. [30]. The two-time correlation between speckle patterns taken at $t_1$ and $t_2$ is calculated via

$$\begin{array}{c}\hfill TT(t_1,t_2)=\frac{\langle I_{t_1}{I}_{t_2}\rangle }{\langle I_{t_1}\rangle \langle I_{t_2}\rangle },\end{array}$$

(2)

where the $\langle \rangle$ denotes an average over the equivalent Q points in reciprocal space. In our simulation of 2D diffusion, the Q average is taken over the annular regions of interest (ROI) within a narrow range of $Q=[2\pi /(Na\left)\right]R$ where $N=200$ is the number of pixels along each direction in real space, and R is the radius of the ROI in pixels.

The $g_2$ function is further calculated as

$$\begin{array}{c}\hfill g_2(\Delta t)=\overline{TT(t_1,t_1+\Delta t)}=1+\beta {\left|f(Q,\Delta t)\right|}^2,\end{array}$$

(3)

where the over line denotes the average over all the pairs of $t_1$ and $t_2$ separated by the duration $\Delta t$ and $\beta$ is the speckle visibility. $f(Q,\Delta t)$ is the normalized intermediate scattering function (ISF). For dynamics described by diffusion, an appropriate model for the ISF is [19]

$$\begin{array}{c}\hfill f(Q,\Delta t)=exp(-\Delta t/\tau ).,\end{array}$$

(4)

with $\tau =1/\left(D{Q}^2\right)$. These expressions indicate that the characteristic time constant $\tau$ and the speckle visibility parameter $\beta$ can be determined via a two-parameter least-squares fit to the data points of the $\Delta t$-dependent $g_2$ function obtainable from experimental data at a given Q, as is commonly done in XPCS analysis [14].

To establish baseline values of $\tau$ and $\beta$ for comparison with later simulations, $g_{2,NF}$ was extracted from the noise-free set of speckle patterns. Fitting to Equation (4) yields a characteristic time scale for a given annular ROI. The $g_{2,NF}$ and corresponding fits for the three representative ROIs in Figure 1a are displayed in Figure 1d. All three fits give ${\beta }_{NF}\sim 1$. In the rest of this paper, $\beta$ is fixed to 1 for all the fits. The extracted ${\tau }_{NF}\left(Q\right)$ values are close to the values calculated from the analytical formula. For example, at $Q=2\pi R/\left(Na\right)$ with $R=30$ (gray ROI), our fit gives ${\tau }_{NF}=45.05\pm 0.23$ frames (a unit that stands in for $\Delta t$ in a real experimental data set), while the calculated value $\tau =1/\left(D{Q}^2\right)=N^2/({\pi }^2{R}^2\Gamma )=45.08$ frames. For $R=20$ and $R=50$, the fitted ${\tau }_{NF}$’s are $100.08\pm 0.65$ and $18.49\pm 0.08$ frames respectively. We have confirmed the $\tau \propto 1/Q^2\propto 1/R^2$ dependence over a range of R from 20 to 70 pixels. For the rest of this paper, we choose the $R=30$ ROI as the starting point for simulating the effect of weak and fluctuating speckle intensities.

3. Results

X-ray pulses generated at XFELs have stochastically fluctuating intensities, and a relatively broad bandwidth in energy [31,32]. To achieve the longitudinal X-ray coherence needed for wide-angle XPCS experiments, a monochromator is often used, which tends to further increase the shot-to-shot intensity fluctuations [15] and fraction of near-zero intensity pulses. Thus, an XFEL-based XPCS data set measured with a monochromatic beam in practice contains a large fraction of extremely weak speckle patterns (with only a few photons over the entire ROI), and a large intensity difference between the strongest and weakest speckle patterns. Our simulations aim to recreate these aspects of an XFEL-XPCS data set to assess and minimize their impact on data analysis, first by considering the effect of uniformly scaled low intensities on XPCS analysis, then by also incorporating XFEL-like intensity fluctuations of the incident beam.

3.1. Simulating the Effect of Weak Speckle Intensities

To understand the effect of low intensity data, we start by simulating the scenario where the intensity of all the speckle patterns in the time series is uniformly reduced, without adding the incident beam fluctuations. This is accomplished by applying a fixed $\alpha$ for the entire 2000 images, and creating instances of the speckle patterns consistent with Poisson counting statistics. Several different $\alpha$ values were applied and resulted in a series of data simulations with the mean intensities over the ROI spanning a range from 0.005–10 average photons/pixel, or equivalently, from 2 to almost 3800 photons over the entire ROI. These simulations are relevant here because they establish a baseline of the performance of sequential XPCS analysis in the low intensity limit in the absence of substantial incident beam intensity fluctuations, mimicking the situation encountered at synchrotron facilities.

In Figure 2, we show our XPCS analysis of simulated speckle datasets with Poisson counting statistics applied to different uniformly scaled intensities compared with the noise-free case. The $g_{2,SIM}$ function and the characteristic time scales ${\tau }_{SIM}$ were determined using the same protocol as for the “noise free” dataset. In Figure 2a we display all the $g_{2,SIM}$ using colored markers, whereas the black dashed line corresponds to $g_{2,NF}$. The fitted ${\tau }_{SIM}$’s are displayed in Figure 2b. For intensities over 1 photon/pixel, the XPCS analysis produces reliable estimates of ${\tau }_{SIM}$. Within this regime, uniformly scaling the speckle intensities for all the images does not affect the ability of standard sequential XPCS analysis methods to determine $\tau$. With 0.02 average photons/pixel, which amounts ∼8 photons within the entire ROI, the confidence interval of the fitted ${\tau }_{SIM}$ does not encompass ${\tau }_{NF}$. Further reducing the average intensity, the point scatter of $g_{2,SIM}$ increases significantly. At the same time, a significant increase in the error bars of ${\tau }_{SIM}$ is observed, owing to the wide scatter in the $g_{2,SIM}$ values at this low intensity. For non-fluctuating incident intensities, the signal to noise ratio (SNR) of $g_2-1$ can be evaluated as [33]

$$r_{SN}\equiv (g_2-1)/{\sigma }_{g_2}=\beta \overline{\langle I_t\rangle }{\left(n_{pix}{n}_{pair}\right)}^{1/2},$$

(5)

where $n_{pix}$ is the number of pixels in the ROI, and $n_{pair}$ is the number of two-time pairs contributing to $g_2$, and $\overline{\langle I_t\rangle }$ is the time- and pixel-averaged number of photons per pixel. Figure 2c shows the SNR extracted from the simulations as a function of $\overline{\langle I_t\rangle }$, estimated using ${\sigma }_{g_2}$ evaluated from the first 30 non-zero time differences, normalized to the noise-free value. We compare this to the formula in Equation (5), with $\beta =1$, $n_{pix}=380$ for the $R=30$ ROI, and $n_{pair}\sim 2000$ for small time differences (red dashed line in Figure 2c). The extracted SNR is indeed proportional relative to $\overline{\langle I_t\rangle }$ with values in agreement with Equation (5). At $\overline{\langle I_t\rangle }=0.005$ photons/pixel, the extracted SNR is as low as $r_{SN}\sim 2$, while an SNR over 50 is desired in a typical XPCS experiment [34]. As we will show in Section 3.3, the two-time correlation with weak pulses will contribute greatly to the noise in the $g_2$ calculation. It is thus desired to develop a strategy by which to analyze sequential XPCS data differently, especially when considering the case of XFEL data where many shots with near-zero intensity are expected.

3.2. Simulating the Effect of Highly Fluctuating Speckle Intensities

Building on the above discussion, we further investigate the application of sequential XPCS analysis in the context of highly fluctuating speckle intensities combined with low mean intensities. The results are summarized in Figure 3. We consider two representative scenarios of distributions of incident pulse intensities that can be encountered in XFEL-XPCS experiments.

The first is when the most probable XFEL pulse and speckle pattern intensity is nonzero, as would be the case using the pink-beam energy spectrum of the x-ray pulses with no monochromator. This was modeled by generating a series of fluctuating $\alpha$ values that follow a Gamma distribution and that are used to scale the series of noise-free speckle intensity patterns. We use the probability density function of the Gamma distribution

$$\begin{array}{c}\hfill f(\langle I_t\rangle ,\lambda )=\frac{{\lambda }^2\langle I_t\rangle e^{-\lambda \langle I_t\rangle }}{\Gamma \left(2\right)},\end{array}$$

(6)

where $\Gamma$ is the Gamma function, and $\langle I_t\rangle$ is the average number of photons within the ROI. The time-average $\overline{\langle I_t\rangle }=2/\lambda$. Subsequently, the Poissonian shot noise was simulated in each differently scaled pattern. The fluctuating ROI-averaged $\langle I_t\rangle$ (photons/pixel) for this series of patterns is plotted in Figure 3a, and the intensity histogram is shown in Figure 3b. The time-averaged intensity is $\overline{\langle I_t\rangle }=0.04$ photons/pixel.

The second scenario we simulated represents the case of a narrow-bandwidth monochromator. The histogram of X-ray pulse and speckle intensities follow an exponential decay, with the most probable speckle intensity being zero [15]. In this case, the probability density function of $\langle I_t\rangle$ takes the form

$$\begin{array}{c}\hfill f(\langle I_t\rangle ,\lambda )=\lambda e^{-\lambda \langle I_t\rangle },\end{array}$$

(7)

with the time-average $\overline{\langle I_t\rangle }=1/\lambda$. We show the intensity for each pulse and the histogram for this scenario in Figure 3c,d. The time-averaged intensity is also set to $\overline{\langle I_t\rangle }=0.04$ photons/pixel.

The calculated $g_{2,SIM}$ from the two scenarios are displayed in Figure 3e. We have excluded all completely dark images in the calculation of $g_{2,SIM}$, since the two-time correlation is ill-defined in those situations. By examining Figure 3e, we see that the calculated $g_{2,SIM}$ still roughly track $g_{2,NF}$, with a significant scatter. The SNR for the Gamma distributed XFEL intensity is $r_{SN}\sim 11$, and for the exponential decay distribution, $r_{SN}\sim 6$. These numbers are significantly smaller than the SNR of ∼35 calculated from Equation (5) for a constant $\overline{\langle I_t\rangle }=0.04$ photons/pixel.

After performing fits to both $g_{2,SIM}$ sets, we see that the uncertainties of the fitted time scales increase significantly. The relaxation times obtained are ${\tau }_{SIM}=43.77\pm 3.11$ (Gamma distribution) and ${\tau }_{SIM}=49.88\pm 3.96$ (exponential distribution), compared with ${\tau }_{NF}=45.05\pm 0.23$. While the time-averaged intensity is comparable, the uncertainty of the fitted ${\tau }_{SIM}$ for the exponential distribution is larger, likely due to the larger fraction of low intensity XFEL pulses and a larger intensity difference between the strongest and weakest speckle patterns. Moreover, for the Gamma distribution case, ${\tau }_{NF}$ is on the edge of the confidence interval of the fitted ${\tau }_{SIM}$. Similar to our XPCS analysis of the weak speckles, it is evident new XPCS analysis strategies are needed to reduce the uncertainties in determining characteristic physical constants from XPCS.

3.3. Reducing the Noise in the XPCS Analysis Based on XFEL-Like Datasets

Our simulations of the XFEL-based XPCS highlight the need to increase the accuracy in determining the characteristic time scales. This requires reducing the noise in the calculation of $g_2$. Thus, it is essential to directly identify how weak shots lead to increased noise in $g_2$ in the presence of highly fluctuating X-ray intensities, and design intelligent approaches to treat these weak shots accordingly. We note that for a given time separation $\Delta t$, $g_2(\Delta t)$ is the mean value of pair-wise two-time correlations $TT(t_1,t_1+\Delta t)$. Thus $g_2(\Delta t)$ has contributions from pairs of strong speckle patterns, pairs of weak speckle patterns (as discussed in Section 3.1), and from pairs with very different intensities. We explore the distribution of $TT(t_1,t_2)$ for different combinations of $\langle I_{t_1}\rangle$ and $\langle I_{t_2}\rangle$. For $\Delta t=1$, the histograms of $TT(t_1,t_2=t_1+1)$ are shown in Figure 4, generated using the exponential distribution scenario in Figure 3c,d due to its larger dynamic range. In both panels, the vertical axes are the normalized distribution functions for the two-time correlation. The “noise free” $g_{2,NF}(\Delta t=1)$ is marked using the black dashed vertical line. The uncertainty in the calculated $g_2(\Delta t=1)$ is reflected in the width of histogram. In Figure 4a, the width of the histogram for the simulated $TT(t_1,t_2=t_1+1)$ is over three times that of the noise free case, and is consistent with the larger point scatter in $g_{2,SIM}$. In Figure 4b we show the partial histograms of $TT(t_1,t_2=t_1+1)$ between two strong (green solid line), one strong and one weak (magenta solid line), and two weak (blue shaded region) speckles respectively. The width of the histogram from a pair of strong speckles is slightly larger than that of the “noise free” case, while the width of the histogram from a strong and a weak speckle is comparable to the width of $TT(t_1,t_2=t_1+1)$ including all the pairs (red solid line in Figure 4a). This means the large degree of point scatter observed in $g_2$ is dominated by pairs consisting of strong and weak shots. As for the $TT(t_1,t_2=t_1+1)$ between two weak shots, the histogram is dominated by the contribution from $TT(t_1,t_2=t_1+1)=0$. With an average intensity of 0.05 photons/pixel, which is ∼20 photons distributed over the 380-pixel ROI, there is a large probability that $\langle I_{t_1}{I}_{t_2}\rangle =0$, while neither image is dark. Thus, to reduce the noise, we need to remove the two-time correlation where one or both speckle patterns are weak.

Hereafter we discuss possible protocols for reducing the uncertainties in determining $g_{2,SIM}$ and improving the accuracy of extracting the characteristic physical constants. In synchrotron-based XPCS analysis, the multi-tau algorithm is often used by averaging over logarithmically increasing time differences, which reduces the computing time and improves the SNR [35]. However, the use of multi-tau methods also leads to reduced number of $\Delta t$ available for fitting $g_2$. In our case, a much smaller number of continuous time steps will be available compared to typical, slower synchrotron experiments today. As such, we will use all the possible $\Delta t$ in our analysis instead of the multi-tau methods.

From the histogram analysis in Figure 4, a straightforward approach is to put a lower threshold on the speckle intensity. Only $TT(t_1,t_2)$ where both speckle intensities are “strong” will be counted towards $g_{2,SIM}$. However, having too high a threshold will greatly reduce the fraction of qualifying “strong” speckle pairs, and increase the uncertainty in the $g_2$ calculation. In Figure 4, the definitions of “strong” and “weak” are determined from the simulations with fixed intensity scalings in Figure 2, where we know an average of 0.02 photons/pixel or lower will lead to a noisy $g_2$. This procedure is not practical for many, if not most, XPCS experiments. The key objective is to develop a self-sufficient approach of determining the appropriate lower threshold. We propose and demonstrate the following protocol:

• Apply a low (but positive) $\langle I_t\rangle$ threshold to the two-time correlation. Calculate $g_2$ only using speckle images with the average photon number above this threshold;
• Fit the time scale $\tau$ from the calculated $g_2$ using Equation (3);
• Increase the intensity threshold and repeat the first two steps;
• Track the fitted uncertainties ${\sigma }_{\tau }$ as a function of intensity threshold. ${\sigma }_{\tau }$ will initially decrease due to reduced noise in the calculated $g_2$;
• ${\sigma }_{\tau }$ will eventually increase upon further increasing the intensity threshold. This is where $g_2$ gets noisy again, as there are too few qualifying speckle pairs;
• We choose the “turning point” in ${\sigma }_{\tau }$ as the threshold for qualifying shots from which to calculate $g_2$ and thus minimize the uncertainties in physical constants fitted from $g_2$.

In Figure 5b,c, we show the ${\tau }_{SIM}$ and its standard deviation extracted from the fits to Equation (3), as a function of the applied $\langle I_t\rangle$ threshold in units of average photons/pixel. It is evident that ${\sigma }_{\tau }$ reaches a minimum at around ∼0.02 photons/pixel (Figure 5c). The calculated $g_{2,SIM}$ corresponding to this threshold is plotted in Figure 5a using the blue circles, with reduced noise compared to that calculated from all the speckle images (red line). Also shown in Figure 5b is ${\tau }_{NF}$ obtained from the noise free dataset (dashed horizontal line). At the photon threshold we choose, the fitted ${\tau }_{SIM}$ has the smallest uncertainties, with ${\tau }_{NF}$ residing inside the confidence interval. This illustrates the success of our proposed protocol for XFEL-based XPCS datasets with strong pulse-to-pulse intensity variations.

In Figure 5e, we show the SNR $r_{SN}$ calculated from the scatter in $g_{2,SIM}$ at small $\delta t$ as a function of the threshold. Consistent with the improved fit, the SNR increases by a factor over 2 at the optimized threshold. Meanwhile, this improvement accompanies a reduction in the fraction of two-time pairs contributing to the $g_2$ (Figure 5d). This is due to the large fraction of weak XFEL shots in the exponential decay distribution.

Figure 6 shows the same procedure applied to the XPCS data generated using the Gamma-distributed intensity fluctuations. As discussed previously, this simulation has a smaller difference between the strongest and weakest pulses. We get the 0.02 photons/pixel as the chosen threshold using the same protocol (Figure 6d). The fitted ${\tau }_{SIM}$ using both thresholds include ${\tau }_{NF}$ (Figure 6b). Also, the SNR is improved (Figure 6e). In Figure 6a, we show the “cleaned up” $g_2$ using the 0.02 photons/pixel. The noise in the calculated $g_2$ is smaller for $\Delta t<\sim$50, while the noise reduction is less appreciable at larger time differences.

Further reducing the dynamic range between the strongest and weakest speckle patterns, we test our procedure on the “all-weak” speckle patterns generated in Figure 2 for simulating the synchrotron-based scenario. The shot-to-shot difference in the average photon numbers per pixel arises completely from the Poisson statistics at each pixel. Our procedure fails completely for the fixed $\alpha$ with 0.005 photons/pixel where the fits for obtaining $\tau$ do not converge for most thresholds. We apply the procedure for the case of 0.01 photons/pixel (yellow open circle in Figure 2a). The analysis is displayed in Figure 7. The fitted ${\sigma }_{\tau }$ minimizes around a threshold of 0.0055 photons/pixel. We reduce the uncertainty of our fit by ∼10% after applying the threshold. This corresponds to only keeping ∼50% of all the two time pairs for $\Delta t=1$ frame (Figure 7d). Perhaps unsurprisingly, the thresholded $g_2$ is improved only slightly.

Thus to summarize, our proposed protocol works best in the presence of highly fluctuating speckle intensities as in the case of XFEL-based XPCS, and may even improve the accuracy of fitted time scales for XPCS analysis of weak speckle patterns with small, but nonzero intensity difference. This protocol successfully rejects the “weak” speckle patterns, but the trade-off is a smaller number of available two-time pairs for a given $\Delta t$, which is the other important factor that determines the SNR.

While the above approach using a single $\langle I_t\rangle$ threshold to reject weak speckle patterns is clearly successful at improving the signal-to-noise in $g_2$ and the accuracy of fitted $\tau$, we have also considered an approach using a weighted average of all patterns. We propose that weighting be applied to the average of the two-time correlations used to determine $g_2$,

$$\begin{array}{c}\hfill g_2(\Delta t)=\frac{\sum w_{t_1,t_1+\Delta t}TT(t_1,t_1+\Delta t)}{\sum w_{t_1,t_1+\Delta t}},\end{array}$$

(8)

where the weighting factors $w_{t_1,t_2}=1/Var\left(TT(t_1,t_2)\right)$ are chosen to be the reciprocal of the variance of $TT(t_1,t_2)$. As an approximation, we assume $Var\left(TT(t_1,t_2)\right)$ is dominated by the variance of the numerator, $Var\left(\langle I_{t_1}{I}_{t_2}\rangle \right)$. Due to the dominant effect of Poisson statistics in the low-intensity speckle patterns considered in this work, we assume $Var\left(I_t\right)=\langle I_t\rangle$. This gives

$$\begin{array}{ccc}\hfill w_{t_1,t_2}& =& \frac{n_{pix}\langle I_{t_1}\rangle \langle I_{t_2}\rangle }{\langle I_{t_1}\rangle +\langle I_{t_2}\rangle +1},\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill g_2(\Delta t)& =& \frac{\sum \langle I_{t_1}{I}_{t_1+\Delta t}\rangle /(\langle I_{t_1}\rangle +\langle I_{t_2}\rangle +1)}{\sum \langle I_{t_1}\rangle \langle I_{t_1+\Delta t}\rangle /(\langle I_{t_1}\rangle +\langle I_{t_2}\rangle +1)}.\hfill \end{array}$$

(10)

For the weakest speckle patterns, $\langle I_{t_1}\rangle \to 0$, $w_{t_1,t_2}\to 0$. Also, a pair of weak speckle patterns contributes less to the $g_2$ than the combination of one strong and one weak speckle, consistent with our observations in Figure 4. This weighting factor de-emphasizes the effect of weak shots in the determination of $g_2$ and avoids the hard threshold we used in the aforementioned protocol. We have tested the feasibility of Equations (9) and (10), and the results are shown in Figure 8 for the three scenarios we discussed in Figure 5, Figure 6 and Figure 7. We see that the weighted $g_2$ is cleaner than those derived using the protocol with a hard threshold. For comparison with the protocol discussed above, we calculate the SNR from the simulated $g_2$’s and the uncertainty ${\sigma }_{\tau }$ of the fitted ${\tau }_{SIM}$’s from the thresholding vs. the weighting approaches, for different distributions of the incident X-ray intensity. The histograms of $r_{SN}$ and ${\tau }_{SIM}$ are displayed in Figure 9. It appears that the approach using the weighted $g_2$ consistently gives further improvement to the uncertainty of the fitted time scales.

4. Discussion

Our simulations so far have focused on the XFEL-based XPCS analysis in the forward-scattering geometry with a high-energy-resolution monochromator. For solid-state materials, the XPCS measurements are usually carried out in the wide-range XPCS geometry to access scattering in the vicinity of Bragg reflections originating from the crystal symmetries, though such experimental efforts are still nacent at XFELs.

An issue often encountered in wide-angle XPCS datasets is a reduction in the contrast $\beta$ arising from effects related to partial coherence, ranging from a mismatch between speckle size and pixel size to a small coherence volume compared with the scattering volume. The latter effect comes into play especially in wide-angle XPCS experiments of bulk crystals, when the x-ray penetrating scattering path length in the material outruns the longitudinal coherence criterion, even at x-ray sources such as XFELs that deliver fully transversely coherent beam. This reduction in $\beta$ reduces the SNR of $g_2-1$, making it even more critical to apply strategies to improve the SNR. As such, the protocol presented here should be beneficial to wide-angle XPCS studies at XFELs. This can be tested as experimental data becomes available from European XFEL and LCLS-II.

The protocols developed here may also benefit the split-pulse XPCS studies. An important part of analyzing the speckle visibility from these datasets is to address the intensity difference between the two split pulses [16], for example, by correlating the intensity ratio with the average number of photons/pixel. Investigating other statistical features from each image such as the variance in this paper might inspire novel and more efficient analysis of the split-pulse XPCS datasets.

5. Conclusions

By simulating the effects of pulse-to-pulse intensity fluctuations on XPCS data sets, we show that standard XPCS analysis can successfully be used to extract correlation times $\tau$. Furthermore we develop strategies for improving the $g_2$ calculation and reducing the uncertainties in determining $\tau$. Our work complements previous analysis on the impact of the spatial instability of XFELs, and provides an avenue towards analyzing upcoming sequential XPCS experiments at high repetition-rate XFEL facilities around the world.