Binary Computer-Generated Holograms by Simulated-Annealing Binary Search

Abstract

The binary computer-generated hologram (BCGH) has attracted much attention recently because it can address the high-speed binary spatial light modulator (SLM), such as a digital micromirror device (DMD) SLM. In this paper, our concern is the development of an algorithm to produce high-quality BCGHs. In particular, simulated annealing (SA) is an efficient algorithm used to produce a phase-only computer-generated hologram. In the study of SA for the production of a BCGH, we found some inherent shortcomings of SA, and the quality of the produced BCGHs is limited. Accordingly, we have modified SA and propose the simulated-annealing binary search (SABS) algorithm. We have also proposed a method to quickly determine the parameters for SABS. In the comparison with SA, the mean square error of the SABS BCGHs decreases by 32% on average. Therefore, the SABS is a promising technique for a high-quality holographic display by DMD.

1. Introduction

The computer-generated hologram (CGH) has been widely applied for two-dimensional (2D) display and three-dimensional (3D) display [1]. Usually, a phase-modulation spatial light modulator (SLM), such as a liquid-crystal-on-silicon (LCoS) SLM, is applied for the display of the CGHs [2,3,4]. Alternatively, a holographic display can also be realized by using a digital micromirror device (DMD) [5]. The DMD is a binary-modulation SLM, and thus it must be addressed by binary computer-generated holograms (BCGHs). Although the display quality of a BCGH is usually worse than that of a phase-only CGH, the BCGH-based holographic display is still attractive because the modulation speed of a binary-modulation SLM is much faster than that of a LCoS SLM. This feature advances various holographic display techniques, including the time multiplexing [6,7,8], the space multiplexing [9,10], the angular multiplexing [9,11,12], the color-sequence display [13], and the intensity-accumulation display [14,15,16].

Considering the production of a Fresnel BCGH for free-space 3D display, we can produce the complex field of the object light and then binarize the field by direct sign-thresholding (DST) [17,18,19,20]. This method is fast, but the produced hologram always contains serious quantization errors. Error diffusion (ER) is an efficient method used to redistribute the quantization error to high frequency region, and thus can improve the image quality [21,22,23,24]. However, the best parameters for ER BCGHs must be found by trial and error. For the DST BCGHs, the display quality can be improved by an intensity-accumulation display [14,15,16], i.e., multiple holograms of the same scene are quickly displayed to average the random noise (speckle). A good display quality can be achieved by using several tens of BCGHs for a scene. Hence, the holographic system also needs several tens of larger storage space and transmission bandwidth. This is an apparent shortcoming for a portable holographic display system. To produce a single BCGH with good quality, iterative algorithms are usually applied. The iterative Fourier transform algorithm (IFTA) is one of the most widely applied methods [25,26,27]. It applies constraints on both the hologram plane and the image plane, and iteratively calculates the light fields on the two planes to optimize the resulting BCGH. However, the computing time of IFTA is long, and its performance is limited [16]. Pixelwise iterative algorithms, such as direct binary search (DBS), can produce a BCGH with a relatively high quality [28,29,30,31]. DBS evaluates the contribution of the light emitted from every pixel of the BCGH and determines the on/off state of every pixel. The quality of the DBS BCGH is the best among the discussed methods, while its computing time is also the longest. In this paper, our concern is another pixelwise iterative algorithm, the simulated annealing (SA) method. SA has been applied to the generation of a phase-only CGH [32], but, to the best of our knowledge, it has never been applied to the generation of a BCGH. While the computing complexity of SA is roughly the same as that of DBS, the performance of SA should be better than that of DBS. Nevertheless, in the study of SA, we found some inherent problems of SA; thus, we propose a new method: the simulated-annealing binary search (SABS).

The remainder of this paper is organized as follows. In Section 2, we introduce the principle of DBS and SA. Because there is a severe shortcoming in SA, we also introduce our proposed simulated-annealing binary search (SABS). In Section 3, we discuss the methodology for the determination of the parameters in SABS. Simulation results and discussions are provided in Section 4, followed by concluding remarks in Section 5.

2. Method

2.1. Direct Binary Search

The flowchart of the DBS is shown in Figure 1, while the steps in red are bypassed. First, an initial binary hologram (typically a binary random function) $H_b(m,n)$ with size $M\times N$ is produced. The variables $(m,n)$ are the indices of the hologram pixels. The hologram is then reconstructed by the scalar diffraction formula [33] to obtain the reconstructed field. The mean square error (MSE) of the reconstructed image $I_r(m^{\prime },n^{\prime })$ with size $M^{\prime }\times N^{\prime }$ is measured as

$$MSE=\frac{1}{M^{\prime }{N}^{\prime }}{\displaystyle \sum _{m^{\prime }}{\displaystyle \sum _{n^{\prime }}{\left. [O(m^{\prime },n^{\prime })-I_r(m^{\prime },n^{\prime })\right]}^2,}}$$

(1)

and is regarded as the initial cost function $MS{E}_0$. Here, the variables $(m^{\prime },n^{\prime })$ are the indices of the reconstructed image, and $O(m^{\prime },n^{\prime })$ is the target image. In the next step, the hologram pixels are evaluated one by one by a raster-scan order. For every pixel, the pixel value is changed according to

$$H_b(m,n)\leftarrow \left\{\begin{array}{c}1\hspace{1em}\mathrm{if}\hspace{0.33em}{H}_b(m,n)=0\\ 0\hspace{1em}\mathrm{if}\hspace{0.33em}{H}_b(m,n)=1\end{array}\right.,$$

(2)

where “$\leftarrow$” means that the value on the left side is replaced by the value on the right side. The updated binary hologram is reconstructed again to measure the MSE ($MS{E}_{mn}$) of the new reconstructed image. Subsequently, the difference in $MS{E}_{mn}$ and $MS{E}_0$ is calculated by

$$\mathsf{\Delta }E=MS{E}_{mn}-MS{E}_0.$$

(3)

If $\mathsf{\Delta }E$ is larger than zero, the change in pixel value is negative and thus the change is undone. Otherwise, the change is accepted, and the cost function $MS{E}_0$ is updated by $MS{E}_{mn}$. In one round of evaluation, every pixel of the hologram is evaluated once in the same manner. The algorithm terminates after finishing the preset rounds. Usually, after three to five rounds of evaluations, the binary hologram will converge, i.e., nearly all of the pixel values are unchanged in one round of evaluation.

2.2. Simulated Annealing

Simulated annealing (SA) is an optimization algorithm and has been applied to the production of a phase-only Fourier CGH [32]. Here, we applied SA to the production of a Fresnel BCGH. Because the procedures of SA are similar to those of DBS, the flowchart of the SA is also shown in Figure 1. For simplification, here, we only explain the steps that are not involved in the DBS. In step (a), an initial temperature ($T_0$) is given, which will be used in step (b). In addition, the pixels are not evaluated in the raster-scan order. Instead, the pixels are randomly selected in SA. The pixel-selection sequence is also generated in step (a). The next step is pixel evaluation [(Equation (3)], and the SA also accepts the pixel-value change if $\mathsf{\Delta }E\le 0$. However, if $\mathsf{\Delta }E>0$, SA moves to step (b), which will accept the negative pixel-value change according to the probability

$$P=\exp (-\mathsf{\Delta }E/T),$$

(4)

where $T$ is the current temperature of the status. In step (c), the status of the calculation is evaluated. In the conventional SA, if the $\mathsf{\Delta }E$ of any pixel is less than the preset value, e.g., 0.05 of the initial $\mathsf{\Delta }E$ at the same temperature, then the calculation is regarded to be stable, and the temperature decreases according to the annealing function

$$T=\frac{T_0}{1+t},$$

(5)

where $t\in 0,\hspace{0.17em}1,\hspace{0.17em}2\dots$ is the count of temperature changes. This step avoids too fast convergence of the calculation, and thus it is possible to find a BCGH with better quality. However, this algorithm never converges on occasion. In the subsection, we will address this issue in detail.

2.3. Stimulated Annealing Binary Search

The SA can directly produce a BCGH. However, we found some shortcomings of SA, and thus its performance is unstable, which also motivated us to propose the SABS method. The flowchart of the proposed simulated annealing binary search (SABS) method is also shown in Figure 1. It is basically the same as SA, except for some modification in steps (a) and (c). In step (a), conventional SA produces a random sequence of pixels for evaluation. In other words, one pixel is probably evaluated several times, but another pixel was probably never evaluated in the whole calculation. For this reason, the sequence is modified to be “pseudo random”: every pixel must be evaluated once in a round of evaluation with a random sequence, and the random sequence of every evaluation round is different.

The second modification is in step (c). The condition of stability in conventional SA depends on the initial $\mathsf{\Delta }E$ of the first evaluated pixel in that round. If the initial $\mathsf{\Delta }E$ is very small, it is nearly impossible to satisfy the condition of stability. When this situation occurs, the temperature will be kept high, and the probability for accepting the negative pixel-value change never decreases. An example of this situation is shown in Figure 2 (the detail of the calculation will be explained in the next section). In Figure 2a, the temperature count for SA is frozen at $t=1561$, and the corresponding probability is shown in Figure 2b. The curve of $P$ is plotted using the average of every 1000 pixel-evaluations to clearly show the trajectory. In Figure 2b, it is noted that the probability for SA does not decrease but increases slightly as the temperature has been frozen. The reason for this is that $\mathsf{\Delta }E$ gradually decreases with the calculation, which results in a slightly increased p [Equation (4)]. Consequently, the hologram always contains pixels with negative pixel values even at the end of the calculation. To avoid the temperature-frozen problem, we made two modifications. First, we directly change the temperature after evaluating $q$ pixels. In addition, we set $T=0$, i.e., $P=0$, at the last round of the evaluation. In other words, the last round of evaluation bypasses the annealing processing (step (b) and (c) in Figure 1) for converging the calculation. Because the method is not the same as the conventional SA, we call the proposed method the simulated-annealing binary search (SABS) method. As a comparison, the $t$ and $P$ (excluded last round) curves of the SABS are also shown in Figure 2a,b, respectively. The problem of being temperature-frozen in the SA was solved in SABS.

3. Determination of the Parameters in SABS

Although the procedure of SABS was revealed in the last section, the use of SABS is not straightforward because the parameters $T_0$ and $q$ must be properly set to control the probability $P$. However, $P$ is a function of $\mathsf{\Delta }E$ [Equation (4)], which means that the algorithm is sensitive to not only the environmental conditions (wavelength, object distance, etc.), but also the object function. In other words, the parameters must be found by trial and error for every hologram, which is time-consuming and impractical for display applications. For this reason, we developed a methodology to determine the parameters quickly. The discussion comes with some simulations using the following setup. The hologram size, as well as the size of the object (i.e., the target image), is $512\times 512$ pixels, with a pixel pitch of $7.6\hspace{0.17em}\mathsf{\mu }\mathrm{m}$. The distance between the hologram and the object is $300\hspace{0.17em}\mathrm{mm}$, and the wavelength is $0.635\hspace{0.17em}\mathsf{\mu }\mathrm{m}$. The algorithms can be applied to produce either on-axis holograms or the off-axis holograms. For producing an off-axis hologram, a plane wave with a $1.6°$ off-set angle was applied as the reference light. In the evaluation of every pixel value, the method of look-up table was applied to speed up the computing [30]. The reconstruction of the resulting BCGH was implemented by the angular spectrum method.

To begin with, the number of rounds of evaluations was set according to the tradeoff between the hologram quality and the computing time. In the demonstration, we adopted eight rounds of evaluations, i.e., seven rounds of the modified SA and the last round of binary search. The parameter $q$ is to be determined. It should be noted that the probability [Equations (4) and (5)] is a function of $T_0$, $\mathsf{\Delta }E$, and $t$. Therefore, the range of $t$ can be flexible (e.g., $t=0~6000$), whereas the probability curve is kept the same by adjusting $T_0$. From this point of view, we set $q=350$ and the corresponding maximum $t$ is 5243 ($512\times 512\times 7/350$). Finally, a suitable $T_0$ was determined. The probability curves for two different $T_0$’s are shown in Figure 3a. The MSE curves corresponding to the two probability curves are shown in Figure 3b. In this example, $T_0=1000$ is not a good choice because its probability curve is still high at the end of the evaluation, which results in a larger MSE. The probability curve can be lowered by using a smaller $T_0$ for the next trail. As a rule of thumb, the probability at the end of evaluation is better in the range of 0.01~0.02. The curve for $T_0=333$ in Figure 3a is the resulting good probability curve. Usually, the suitable $T_0$ can be found within some trials.

The above procedure to find the suitable parameters is effective. However, if this procedure must be carried out every time a new hologram is produced, the time cost will be substantial. Fortunately, it is possible to use the same parameters for producing other BCGHs with the same setup. The key point is still the control of the probability, which is a function of $\mathsf{\Delta }E$. It is noted that MSE is proportional to the squared intensity of the target image, so as $\mathsf{\Delta }E$. This can be expressed as $\mathsf{\Delta }E\propto {\overline{O}}^2$, where $\overline{O}$ is the average of the target image. Because $\mathsf{\Delta }E$ is the numerator in the argument of the probability function [Equation(4)], we can let the denominator be proportional to ${\overline{O}}^2$ in order to eliminate the influence of different target images. Explicitly, we only need to use a modified initial temperature ${\tilde{T}}_0$,

$${\tilde{T}}_0=\frac{{\overline{O}}^2}{{\overline{O}}_0^2}{T}_0,$$

(6)

where ${\overline{O}}_0$ is the average of the target image for finding the parameter $T_0$. In this way, the probability curves corresponding to different target images will be nearly the same.

4. Results and Discussions

We compared the performance of the DBS, the SA, and the proposed SABS with both the off-axis scheme and the on-axis scheme, respectively. The MSE curves versus the number of evaluations are shown in Figure 4. The resulting BCGHs and the corresponding reconstructed images are shown in Figure 5 (off-axis) and Figure 6 (on-axis), while the performance metrics, including the MSE, the structural similarity (SSIM) index [34], and the diffraction efficiency, are listed in

Table 1. Performances of different BCGH methods.

	Metrics	DBS	SA	SABS
off-axis BCGH	MSE	0.0334	0.0349	0.0290
	SSIM	0.222	0.209	0.246
	diffraction efficiency	3.51%	1.34%	1.73%
on-axis BCGH	MSE	0.0158	0.0144	0.0118
	SSIM	0.329	0.334	0.380
	diffraction efficiency	32.2%	19.9%	17.8%

. Here, the diffraction efficiency is defined as the ratio of the light energy in the region of interest (ROI) and the total incident light energy. According to the definition, the maximum diffraction efficiency of the BCGHs cannot exceed 50% because the binary-amplitude CGH will typically absorb 50% of the incident light energy.

For the off-axis BCGHs [Figure 4a], the MSE for SA drops quickly in the forepart, but then does not change much. At the end of the evaluation, SA is even worse than that of the DBS. The reason for this is that the conventional SA always retains negative-influence pixels as the temperature is frozen. In contrast, both DBS and SABS can continuously improve the quality up until the end of the evaluations. At the end of the calculation, the quality of the SABS BCGH is the best among the three methods. It should be noted that the diffraction efficiency of the off-axis holograms is low. This is the reason for why the speckle noise is apparent in the reconstructed images (Figure 5d–f). The diffraction efficiency of the DBS BCGH is higher than that of the SA BCGH and the SABS BCGH. It is reasoned that there is a trade-off between the diffraction efficiency and the image quality for the BCGHs. As the SABS focuses on the improvement in the quality, it sacrifices the diffraction efficiency.

For the on-axis BCGHs (Figure 4b), the difference between the three methods is clearer, and the SABS is always better than DBS and SA in all calculation stages. Similarly, the diffraction efficiency of the DBS BCGH is the highest. It is noted that all of the discussed methods are optimization-based algorithms. In the production of the on-axis BCGHs, both the zeroth-order light and the twin image are in the region of interest, and thus are taken into account in the optimization based on the MSE. For this reason, both the image quality and the diffraction efficiency in the on-axis scheme are better than those in the off-axis scheme.

The parameter $T_0$ was found by using the target image in Figure 5g. By using the modification method in Equation (6), we produced more BCGHs of different target images. The reconstructed images of the BCGHs are shown in Figure 7. The quality of the reconstructed images of SABS BCGHs are also the best, which proves the ability of the control of the probability curve.

5. Conclusions

In this paper, we have proposed and investigated the simulated-annealing binary search (SABS) for the production of binary computer-generated holograms. Being similar to the simulated annealing (SA), SABS allows for negative pixel-value changes in the early stage of the calculation. This enables SABS to find a better solution than DBS. On the other hand, SABS uses a fixed temperature change rate, which avoids the problem of being temperature-frozen in SA. Finally, SABS bypasses the step of accepting negative pixel-value changes in the last round of the evaluation, which minimizes the error of the produced BCGHs. The determination of the initial temperature is a time-consuming procedure. We have also proven that, as an ideal initial temperature has been found from a target image, the initial temperatures for other target images can be easily found without additional simulation trials. On average, the MSE of the SABS holograms decreases by 38% and 32%, respectively, in comparison with the DBS holograms and the SA holograms. It should be noted that the number of evaluation rounds is large in our simulation, and thus the calculation of all methods converges. For a smaller number of evaluation rounds, the difference between DBS and SABS should be larger, as shown in Figure 4b. The only shortcoming of the SABS BCGHs is that its diffraction efficiency is lower than that of the DBS BCGHs, which is believed to be the cost in achieving the extremely low MSE. Finally, we only discussed the production of binary-amplitude CGHs in this paper. According to our preliminary study, the SABS can be also applied to produce binary-phase CGHs. This will be investigated in the future.