Optimizing Single-Shot Coherent Power-Spectrum Scattering Imaging Adaptively by Tuning Feedback Coefficient for Practical Exposure Conditions

Abstract

With only one piece of the power-spectrum pattern, the single-shot coherent power-spectrum imaging can provide a clear object image for real-time applications even if the object is hidden by opaque scattering media, in which the feedback coefficient β value for the reconstruction with least retrievals and fastest speed has to be pre-estimated through time-consuming iterative loops. Here we report a method for estimating the optimal β value from the captured raw power-spectrum images adaptively to optimize the single-shot coherent power-spectrum imaging for practical exposure conditions. The results demonstrate that, based on exposure level analysis of the captured raw power-spectrum images even of underexposure, moderate exposure, and overexposure cases, the β value could be quickly determined with a compact expression for the algorithm to achieve clear reconstruction output efficiently. The proposed method helps to push ahead of the coherent diffractive imaging devices for real-time imaging through turbid mediums in Artificial Intelligence (AI), driving assistance, and flight assistance applications.

1. Introduction

Achieving clear images of the objects embedded in turbid media has always been a challenge for biomedicine, nanotechnology and underwater imaging [1,2,3,4,5,6,7], as the scattering medium usually causes irregular scattering interference beams [8]. Though non-destructive coherent imaging methods [9,10] with angle scanning and additional holographic reference light beam can help to solve the problem somewhat, the practical complex operations still hinder their real-time imaging applications. Tang et al. proposed a single-shot coherent power-spectrum imaging method for imaging objects hidden by opaque scattering media [11] without angle scanning and additional holographic reference light, which may be a promising method for real-time scattering imaging.

Tang’ s method could reconstruct object images hidden in scattering media with only a piece of the single-shot coherent power-spectrum pattern, which is a good attempt in real-time scattering imaging. However, for the practical captured power-spectrum pattern, the object images cannot always be successfully reconstructed quickly. The main factor causing this result is the setting of experimental conditions, including the roughness of the scattering layer, the accuracy of the power-spectrum pattern, the exposure time of the acquisition spectrum, and so on. In terms of algorithm, the feedback coefficient β is an important value, which can reduce the iteration error and accelerate the convergence of the algorithm. Many researchers set the feedback coefficients as an empirical constant, but the value of β will greatly change the process of object reconstruction, its selection is closely related to the experimental conditions, current literature have not studied whether the feedback coefficients need to be adjusted when the experimental conditions are changed. The most significant variable affecting the scattering imaging process among the conditions is the exposure time. None of the above literature has studied the reconstruction effect with the power spectrum acquired with different exposure times, so no certain relationship between the feedback coefficient and exposure time has been uncovered, including Tang’ s work, which is only a feasibility experimental verification. The fact is, the feedback coefficient β value for the hybrid input-output (HIO) reconstruction algorithm has to be carefully selected from a wide range of 0.5–1.0 which Fienup suggested [12], and even go through 0.1–2.0, so as to achieve the optimum reconstruction, which seriously reduces the reconstruction efficiency. Actually, even under the same experimental conditions, the exposure levels of various captured power-spectrum patterns will be different for various camera’s exposure time settings, which then makes the information distribution of the patterns different. Therefore, the fixed single feedback coefficient β value cannot efficiently serve for the single-shot power-spectrum imaging method to reconstruct object images from the power-spectrum patterns with different exposures efficiently [13,14,15].

Herein, we find that the optimum β value is closely related to the exposure degree of the captured power-spectrum patterns, which is not always limited in the range of 0.5–1.0. Furthermore, we analyze the relationship between the β values and the exposure conditions, which will be described in detail as an equation in Section 4, so as to improve the single-shot coherent power-spectrum imaging method for different exposure conditions by selecting suitable β values adaptively. We found that, for the underexposure, moderate exposure and overexposure cases, the optimum β value could be quickly selected from various narrow ranges, which promotes the rapid and accurate object reconstruction.

2. Inconsiderateness in Previous Single-Shot Coherent Power-Spectrum Imaging Methods

By applying Tang’s single-shot power-spectrum imaging method for imaging a target hidden in an opaque scattering medium, with a fixed β value, we found that the reconstruction performance of various power-spectrum patterns captured with different exposure levels is different which usually occurs in practice, as shown in Figure 1. At the same time, we also show the power spectrum collected without a scattering layer in Figure 1(g1,g2) for comparison.

The images shown in the upper part of Figure 1(a1–f1), are the captured power-spectrum patterns of the character ‘5’ on a United States Air Force (USAF) 1951 resolution test chart, with the different exposure times of the camera. In the experiments, we made the illuminating laser power to be the same. In the reconstructions, we used the fixed β = 0.9, and the results are shown in the lower part of Figure 1, i.e., Figure 1(a2–f2), which are considered the best output for various patterns.

In order to check the effect of the value of the parameter β on the reconstruction output, we went through the feedback coefficient β in the range of 0.1–1.5 with intervals of 0.1 for various input patterns, to get the best reconstruction outputs, as shown in

Table 1. Reconstructing objects hidden in the scattering media by traversing β values. Y denotes reconfigurability; N denotes non-reconfigurability.

Captured Power-Spectrum Pattern	(a1)	(b1)	(c1)	(d1)	(e1)	(f1)
β = 0.1	N	N	N	N	N	N
β = 0.2	Y	N	N	N	N	N
β = 0.3	Y	Y	N	N	N	N
β = 0.4	Y	Y	N	N	N	N
β = 0.5	N	Y	N	N	N	N
β = 0.6	Y	Y	N	N	N	N
β = 0.7	Y	Y	Y	Y	N	N
β = 0.8	Y	Y	Y	Y	N	N
β = 0.9	N	Y	Y	Y	N	N
β = 1.0	N	N	Y	Y	Y	Y
β = 1.1	N	N	N	N	Y	Y
β = 1.2	N	N	N	N	Y	Y
β = 1.3	N	N	N	N	Y	Y
β = 1.4	N	N	N	N	Y	N
β = 1.5	N	N	N	N	N	N

, Y denotes reconfigurability, N denotes non-reconfigurability. Y or N is determined by the following criteria with the normalized root-mean-square error (NRMS) [16] expressed in Equation (1), the threshold was set to NRMS = 0.03. We record the NRMS of each reconstructed image with the power spectrum under the same exposure condition at different β values as $N[{\beta }_1,{\beta }_2,\cdots {\beta }_n]$, then calculate ${\Delta }_i=N[{\beta }_{i+2}]-N[{\beta }_i],$ $i\in [1,n-2]$, and find out the max value of ${\Delta }_i$. After this, we use the average of $N[{\beta }_{i+2}]$ and $N[{\beta }_i]$ corresponding to the maximal ${\Delta }_i$ as the threshold of the successful reconstruction.

$$NRMS={\left[\frac{{\displaystyle \sum _{x\in r}|g_k^{\prime }(x){|}^2}}{{\displaystyle \sum _x|g_k^{\prime }(x){|}^2}}\right]}^{1/2}$$

(1)

The power-spectrum pattern captured at different exposure levels could reconstruct objects with suitable feedback coefficient β values. The suitable β values for different exposure level patterns are distinguished from each other. It should be pointed out that, when the input power-spectrum pattern size is 480 × 480, each reconstruction in Figure 1(a1–f1) took 13 min with a normal laptop (ASUS N75S, Intel (R) Core i7-2670QMCPU, 16G RAM) by going through the β value range of 0.1–1.5. Most of the time was spent was going through the wide β value range to find out the suitable value, which is much longer for larger size input power-spectrum pattern, hence, it hinders the real-time imaging applications. However, if the suitable β value can be pre-estimated properly, the reconstruction time is much shorter, which is only about 8 s for each successful reconstruction in Table 1. Therefore, pre-estimating the suitable β value for object image reconstruction is significant for the single-shot coherent power-spectrum scattering imaging in practical applications.

3. Exposure Level Estimation and Adaptive Selection of Suitable β Value

3.1. Exposure Level Estimation

We define four internals for the practical 8-bit image pixel value in the range of 0–255, i.e., $a_1$ (0–10), $a_2$ (11–150), $a_3$ (151–245), and $a_4$ (246–255). The figures shown in Figure 2a,c,e are taken by the camera with different exposure conditions for the same scene. From Figure 2a,e, we can intuitively find the loss of information in brighter or darker areas, while the information in Figure 2c is relatively complete and all objects in the figure can be seen clearly. In order to show how the pixel value distribution changes with exposure time, the interval proportion among the total pixel number is calculated to estimate the exposure level, Figure 2b,d,f are, respectively, the histograms of Figure 2a,c,e, and the estimation criterion is expressed as following Equation (2).

$$y=\{\begin{array}{c}-1\hspace{0.17em}\hspace{0.17em},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}_1>0.1\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\\ 0\hspace{0.17em}\hspace{0.17em},\hspace{0.17em}{a}_1\le 0,a_2+a_3>0.8\hspace{0.17em}and\hspace{0.17em}{a}_4\le 0.1\\ 1\hspace{0.17em}\hspace{0.17em},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{a}_4>0.1\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\end{array}$$

(2)

where $a_1$, $a_2$, $a_3$ and $a_4$ are in range of 0~1, $y=-1$ means underexposure; y = 0 means moderate exposure; and $y=1$ means overexposure.

In practice, the exposure time of the camera and the different illumination will lead to many differences in the captured power-spectrum patterns. For the underexposure case, the corresponding gray histogram contains a large number of pixels with small gray value, as shown in Figure 2e,f. On the contrary, for the overexposure case, the number of high gray level pixels is be very large, and even many pixels are overflowed when limited to 255 in a large image area, as shown in Figure 2a,b. When the exposure is moderate, most of the pixels have a gray value around interval $a_2$ and $a_3$ [17], as shown in Figure 2c,d.

3.2. Adaptive Selection of Suitable β Value

In the reconstruction algorithm, our constraint is that the pixels must be non-negative, for pixels that violate the constraint, we use Equation (3) [18] to replace them. Among them, γ is the set of points which violates the object-domain constraints, β is a constant, which prevents the algorithm from falling into the local optimum points.

$$g_{k+1}(x)=\{\begin{array}{l}{g^{\prime }}_k(x),x\notin \gamma \\ g_k(x)-\beta {g^{\prime }}_k(x),x\in \gamma \end{array}$$

(3)

So, we change the points which violate the object-domain constraints, which use the second line of Equation (3). Suppose the $kth$ input $g_k(x)$ is a 3 × 3 matrix A. The output $g_k^{\prime }(x)$ is another 3 × 3 matrix B, hence, the part of violates the object-domain constraints (the pixel value must be non-negative) can be expressed as:

$$g_{k+1}(x)=\left[\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\\ a_{31}& a_{32}& a_{33}\end{array}\right]-\beta \left[\begin{array}{ccc}{b}_{11}& b_{12}& b_{13}\\ b_{21}& b_{22}& b_{23}\\ b_{31}& b_{32}& b_{33}\end{array}\right] =\left[\begin{array}{ccc}{a}_{11}+\beta \left|b_{11}\right|& a_{12}+\beta \left|b_{12}\right|& a_{13}+\beta \left|b_{13}\right|\\ a_{21}+\beta \left|b_{21}\right|& a_{22}+\beta \left|b_{22}\right|& a_{31}+\beta \left|b_{32}\right|\\ a_{31}+\beta \left|b_{31}\right|& a_{32}+\beta \left|b_{32}\right|& a_{33}+\beta \left|b_{33}\right|\end{array}\right]$$

(4)

Among them, ‘||’ is an absolute value symbol, each element of the matrix in Equation (4) is an independent positive proportion function. As a positive proportional coefficient, β determines the change of the whole iteration process. As β becomes larger, the multiple change of pixel value makes too much difference between the $(k+1)th$ and $kth$ inputs, which will lead to the instability of the whole system and fails to reconstruct the object image finally. Therefore, the β value should be selected carefully, a single β value cannot make all coherent power-spectrum patterns reconstruct clear objects, and there is a corresponding β interval for input $g_k(x)$.

We validate the above conclusions by simulation. According to Equation (4), using the ‘cameraman’ image with a pixel size of 256 × 256 as the object to be reconstructed. The value of β traverses between 0.1–2 intervals and 0.01 intervals, and 100 iterations. According to the result of reconstruction, the object with β value in the range of 0.1–1.3 is very clear (the reconstruction effect can refer to Figure 3c. The recovery images appear blurred when β is bigger than 1.4, as shown in Figure 3e. When β is selected up to 2, no matter how many iterations are performed, the object images cannot be reconstructed successfully. The simulation results also show that β reconstruct a clear object not only in a the range of 0.5–1.0 proposed by Fienup. For the image of the ‘cameraman’, β is in the range of 0.1–1.3.

According to the above analysis, it is summarized as follows: in the single-shot coherent power-spectrum scattering imaging experiment, the exposure time of the camera determines the exposure degree of the collected power-spectrum. The exposure degree is also a reflection of the overall pixel value. The larger the exposure degree, the set of violations of the constraints needs to be adjusted by setting a larger β, making adjusted input is closer to the real object. So we choose to adjust the feedback coefficient β to solve the problem of acquiring single coherent power-spectrum scattering imaging with different exposure. We use the above conclusion to reconstruct the experimental power-spectrum (with the increase in the exposure level, the β value also needs to increase, accordingly). According to the β value set by prior knowledge, the reconstructed effect is shown in Figure 4a–c with a randomly selected power-spectrum pattern, Figure 4d–f are reconstructed by Figure 4a–c by setting different β values.

4. Experiment Results and Discussion

The experiment setup is shown in Figure 5, in which the coherent plane-wave beam is generated by a He-Ne laser (λ = 632.8 nm), and an opaque scattering layer S is placed between the object and the Fourier lens L, the exposure level is controlled by adjusting exposure time of the camera.

With character ‘5’ on a USAF 1951 resolution test chart as the object, we collected 200 pieces of coherent power-spectrum patterns with exposure time ranged from 100 μs to 20,000 μs to cover underexposure, moderate exposure, and overexposure. We used NRMS to evaluate the termination condition and reconstructed effect of the iterative algorithm. For each power-spectrum pattern, the suitable β value for successful reconstruction is selected by going through 0–2 with interval 0.01 to get the minimum NRMS value. The suitable β values for the 200 power-spectrum patterns are shown in Figure 6.

It can be seen from Figure 6, that the optimal β values for various exposure level input patterns are different, with an increasing trend in 0.2–1.3 for underexposure, moderate exposure, and overexposure cases. According to the statistics, the optimal β values for underexposure, moderate exposure, and overexposure cases lay in the range of 0.3–0.8, 0.8–1.1, and 1.0–1.25 respectively, which could be expressed by a fitting expression, as shown in Equation (5).

$$\beta =b+0.01\times {\displaystyle \sum _{i=1}^{i=4}R(a_i)\times M(a_i)}$$

(5)

where, $R(a_i)$ represents the pixel number proportion of interval $a_i$, $M(a_i)$ represents the median of interval $a_i$, and b represents the bias set [−1:0] by the exposure state, b = 0 for underexposure or moderate exposure, otherwise, b set [−1:−0.7].

According to Equation (5), three power-spectrum patterns, in underexposure, moderate exposure, and overexposure states, respectively, are input for the reconstructions, and the corresponding results are shown in Figure 7. Obviously, for different exposure level cases, the reconstructions with corresponding different β values are clear. Furthermore, the time consumption for every reconstruction is no more than 9 s.

The other objects are also used for the experiments, as shown in Figure 8. For the practical power-spectrum patterns with different exposure levels, by choosing carefully the β value according to the given optimal β interval according to Equation (5), the images of the objects hidden by the scattering medium can be reconstructed well, which confirms the feasibility of the adaptive selection method of β value for the algorithm.

5. Conclusions

In summary, we analyze the influence of the exposure level of the power-spectrum pattern on the coherent power-spectrum scattering imaging method and propose a method of adjusting the feedback coefficient β adaptively. As a negative feedback coefficient for the reconstruction algorithm, β is used to reduce the error between the iteration output and the expected image, and finally, make the process work stably. The simulation results and the experimental results show that, for different exposures cases of captured power-spectrum patterns, the β values for the reconstruction algorithm should be changed, accordingly so as to achieve the optimum reconstructed output. For the three exposure cases of power-spectrum pattern capture, i.e., underexposure, moderate exposure, and overexposure, the β value should be selected in the range of 0.3–0.8, 0.6–1.1, and 1.0–1.25, respectively. Anyway, the β value selection guides will be helpful for the coherent diffraction imaging (CDI) with the hybrid input and output (HIO) algorithm, which could avoid the inappropriate β value for achieving successful object image reconstructions, and enhance real-time performance of coherent diffraction imaging. This method promotes the real-time imaging of single-shot coherent scattering imaging, and could also be used in fast variable-aperture Fourier ptychography scattering imaging [19].