Parallelized and Cascadable Optical Logic Operations by Few-Layer Diffractive Optical Neural Network

Abstract

Optical computing has gained much attention due to its high speed, low energy consumption, and the fact that it is naturally parallelizable and multiplexable, etc. Single-bit optical logic gates based on a four-hidden-layer diffractive optical neural network (DONN) have been demonstrated with paired apertures. Here, we show a parallel-logic operation strategy based on two-hidden-layer DONN, showcasing their efficiency by multiple-bit (up to 16-bit) optical logic (e.g., NAND) operations. In addition, we demonstrate how NAND-DONN units can be utilized to achieve NOR and AND operations by flipping and cascading the DONN.

1. Introduction

Optical computing has gained significant attention for its remarkable features, such as high speed, low energy consumption and latency, innate parallelizability and multiplexing abilities. Recently, a variety of on-chip photonic accelerators and specific-task computation units based on silicon photonic integrated circuits have been proposed [1,2,3]. Despite the fact that on-chip photonic neural networks are robust and have a small footprint, the 3D free-space diffractive optical neural network (DONN) has also become a major platform for photonic artificial intelligence (AI). DONN represents a deep neural network based on free-space optical propagation, diffraction, and scattering [4,5,6,7,8,9,10,11]. It essentially comprises a series of diffractive thin layers that can modulate the wavefront in a pixel-wise fashion to construct a deeply connected network whose preliminary form is an ordinary neural network [4]. Mathematically, the optical field diffraction across the layers are similar to a deep neural network’s connections. Note that the phase modulation coefficients represent the trainable weights of the DONN. To this end, various DONNs have been successfully applied in image classification [4,5,6,7], quantitative phase imaging [8], holographic display [10], structured beam manipulation and recognition [12,13,14], multichannel interfering [15], on-chip optical neural networks [16,17], single-bit optical logic operations [9], etc.

As a matter of fact, a variety of optical techniques have been applied to make optic logic gates, such as photonic crystals [18,19], plasmonic waveguides [20], silicon waveguides [21,22,23], metasurfaces [24].In addition to these techniques, single-pixel imaging [25] and semiconductor optical amplifiers [26] are examples. The study of optical logic gates is of importance and relevance because optical logic gate operations are regarded as the most fundamental issue regarding all-optical information processing systems. For instance, it is often desirable to use one set of optical signals to address and/or control another set of optical information [27]. In parallel with the DONN development towards AI inference, DONN-based optical logic gates have been attracting much attention [9,28,29,30,31]. In practical implementations, the logic values are usually encoded at the input and output planes of the DONN using two spatially separated apertures (e.g., at a relative separation $d_a$), and the logic operation is defined/judged by the relative power between them. In this regard, a multiple-bit input, high-density integration, and parallel operation are naturally expected and demanding, and represent the most effective technique for computation capacity scaling.

Here, we show a scheme based on DONN with two hidden layers (see Figure 1) and explore the possibility of a multiple-bit logic operation and a high-density integration in both the transverse and longitudinal directions. We note that the selection of the number of hidden layers is a tradeoff between the overall operation accuracy and the total transmission efficiency. Using one hidden layer is actually possible and could offer a higher transmission [28], but the design degree of freedom may be insufficient, which is not favorable for achieving a complex function. Using over two hidden layers can increase the “0” and “1” output contrast and make the multi-bit parallelization more stable. However, it inevitably reduces the total light intensity in each layer and the output plane. With such a two-hidden-layer DONN, we successfully demonstrate a simultaneous logic operation with N-parallel (N up to 4 × 4 = 16) inputs (e.g., aperture twins). Furthermore, we present a specific case for sequentially connecting such DONNs for various logic operations. More specifically, we demonstrate how to build a DONN for a NAND (not and) gate. As the basic building unit, such a NAND DONN can be effectively utilized to construct logic gates of NOR (not or) and AND (and) as well.

2. System and Method

Figure 1a schematically shows the designed DONN, which consists of two diffractive layers that have pixelwise modulation (phase or amplitude) units (each layer with a 200 × 200 unit). The input layer, the two hidden layers, and the output layer are arranged in a designated alignment and put at different locations along the optical axis with an equal spacing $Z$. In the input layer, the parallel optical logic gates are composed of identical sub-logic gates (for short, let us call this a “logic pixel”) that are spatially arranged in an array with a pixel-to-pixel gap $d_p$. More specifically, each logic pixel consists of four apertures with a size $L$ and an equal separation of $L+d_a$. Note that two apertures in a column represent one input bit (see Figure 1b): the upper aperture “on” (e.g., with a relatively strong intensity) and lower aperture “off” (with a relatively weak intensity) represent input state “1”, and the opposite situation holds for input “0”. The two side-by-side aperture twins in a nearby column refer to two input bits (see Figure 1c for the four states of 2-bit combinations) whose outgoing optical fields shall be processed deliberately by the two hidden layers and finally projected to specified apertures in the output plane. Additionally, the output states are set before by specifying target regions. Here, the target regions refer to two apertures identical to those in the input plane. They form a new pair of apertures (e.g., “detection area”), with a simultaneous upper aperture $A_u$ on and lower aperture $A_l$ off representing output “1”, and the opposite holding for output “0”, respectively. We stress that both the output region and the strategy for defining the output states are somehow arbitrary, requiring, however, the corresponding training of the hidden layer in order to match the definition.

The DONN training process is essentially based on the forward propagation model of the Rayleigh−Sommerfeld diffraction theory. The point-to-point optical field $U(x,y,z)$ is connected by:

$$U(x,y,z_{l+1})=U(x,y,z_l)\otimes g(x,y,Z)$$

(1)

with the layer-to-layer propagator:

$$g(x,y,z)=\frac{z}{r^2}(\frac{1}{2\pi r}+\frac{1}{j\lambda })\exp (\frac{j2\pi r}{\lambda }),$$

(2)

Over the hidden-layer, the optical field is modulated by:

$$U_o(x,y,z_l)=U_i(x,y,z_l)w(x,y,z_l),$$

(3)

In Equations (1)–(3), $\lambda$ is the operating wavelength, $r=\sqrt{x^2+y^2+z^2}$, $z_l$ marks the layer position on the optical axis, and $w(x,y,z_l)$ represents the neuron weighting factor. Here, we assume that the optical power remains constant across the hidden layers but experiences a phase modulation:

$$w(x,y,z_l)=\exp (j{\varphi }_l),$$

(4)

Obviously, the training of the DONN involves a back-propagation process that minimizes the objective function by finding the optimized phase modulation ${\varphi }_l$ ($l=2$ or $3$), which then explicitly defines the DONN for specific computation tasks.

The objective function in our case can be expressed as:

$$L={\displaystyle \sum _{i=1}^P{l}_i^{(b)}},$$

(5)

which sums up all the logic bits from $i=1$ to $P$, and $l_i^{(b)}$ measures the loss function of the $i$-th logic pixel:

$$l_i^{(b)}=l_{\mathrm{expect}}^{(b)}+\alpha l_{\mathrm{penaty}}^{(b)}+\beta l_{\mathrm{uphi}}^{(b)}+\gamma l_{\mathrm{uint}}^{(b)},$$

(6)

The first term on the right-hand side of Equation (6) drives the system towards a high light intensity $I(x,y)$ in the correct output area, which corresponds to the expected output $O=1$ or $O=0$, according to the logic truth table:

$$\begin{array}{l}{l}_{\mathrm{expect}}^{(b)}=[1-I(x,y|x,y\in A_u]O+[1-I(x,y|x,y\in A_l)](1-O)\hfill \end{array},$$

(7)

The second term on the right-hand side of Equation (6) penalizes the system output with a high light intensity in the wrong area corresponding to the unexpected output $O=1$ or $O=0$:

$$l_{\mathrm{penaty}}^{(b)}=I(x,y|x,y\in A_u)(1-O)+I(x,y|x,y\in A_l)O,$$

(8)

The third and fourth terms are standard deviations of the quantity around certain output apertures. They are used to maintain the phase and intensity uniformity inside apertures $A_u$ and $A_l$, respectively:

$$l_{\mathrm{uphi}}^{(b)}=\mathrm{std}[\varphi (x,y|x,y\in A_u)]O+\mathrm{std}[\varphi (x,y|x,y\in A_l)](1-O),$$

(9)

$$l_{\mathrm{uint}}^{(b)}=\mathrm{std}[I(x,y|x,y\in A_u)]O+\mathrm{std}[I(x,y|x,y\in A_l)](1-O),$$

(10)

Through this work, we have set the factors $\alpha =0.5$, $\beta =0.2$, and $\gamma =0.2$.

3. Results

3.1. 2-Hidden-Layer DONN for Various Logic Operations

Figure 2 demonstrates the operation of a typical parallel logic NAND gate. In this specific case, the parameters used in the design and calculation are: $Z=40\lambda$, $S=100\lambda$, $L=4\lambda$, $d_p=10\lambda$, and $d_a=2\lambda$. Figure 2a clearly shows the light field intensity distribution on the input plane (upper panels) and the output plane (lower panels) for $\lambda =600$ nm. The panels in the columns from the left to the right represent the logic operations ‘1110 NAND 0011 = 1101′, ‘1111 NAND 1001 = 0110′, ‘0111 NAND 1101 = 1010′, and ‘0101NAND 0100 = 1011′, respectively. This system actually has four “logic pixels”, and there are essentially 256 possible combinations. In other words, the parallel NAND has 256 input states. Note that for the DONN training, one must digitalize the aperture in the numerical calculation. If we assume unit power for each digital area in the aperture, the input power for each pixel gate is 128 for 8 × 8 digital areas inside. The numbers beside the aperture in the lower column of Figure 2a show that the received power amounts to around 65% (literally with a total input power amount of 128 × 4).

Figure 2b shows all 256 results of the 4-bit parallel NAND logic operation. For comparison purposes, we have put the theoretical (expected) output intensity below the horizontal line marking zero intensity. The real output intensity distribution, as shown in the lower panels of Figure 2a, are summarized above the horizontal line of zero intensity. One can see that the “1” and “0” output intensities of the four logic pixels are in accordance with what is expected, suggesting that a parallel NAND operation with a 100% accuracy is achieved.

3.2. Effect of Pixel–Pixel Distance and Densely Integrated Logic Pixels

It is evident that if the logic pixels are adequately spaced, the parallel operation would exhibit a discernible signal-to-noise ratio. However, for densely integrated logic pixels, the mutual interaction among them is not negligible, and the diffractive optical fields would introduce a strong crosstalk between the logic pixels, possibly deteriorating their performance and functionality. To explore the integration density limit of the logic pixels, we have studied the configuration of $3\times 3$ logic pixels (similar to the case schematically shown in Figure 1a) with varying pixel gaps ranging from $d_p=0$ to $d_p=20\lambda$, while keeping the other parameters the same as in Figure 2. To evaluate the degree of the crosstalk between the logic pixels, we keep the input states of the center logic pixel as (0,0), (0,1), (1,0), and (1,1), respectively, and for each input of the center logic pixel, we iterate all input combinations of the surrounding logic pixels, which amounts to $4^8=65,536$ cases. That means that we have performed a total of 65,536 × 4 calculations. Figure 3a–c show the output of the center logic pixel for pixel gaps $d_p=3\lambda$, $4\lambda$, and $10\lambda$, respectively. In the case of a small gap $d_p=3\lambda$, the parallelization operation obviously does not work for the center pixel state ‘10’ (see Figure 3a). However, it works well for $d_p=4\lambda$ (see Figure 3b), and the intensity levels are separated more dramatically for a more increased $d_p$ (see Figure 3c). The logic pixel crosstalk can be measured by this intensity separation. Figure 3d shows that the crosstalk decreases dramatically for an increased pixel-pixel gap. Therefore, we could mark the safely working regime as $d_p\ge 4\lambda$. It is worth noting that according to the Rayleigh−Sommerfeld diffraction theory, a point in the diffraction plane takes almost all of its power from the Huygens secondary sources in a square of lateral size $a=4\sqrt{\lambda Z}=4\sqrt{40{\lambda }^2}\approx 25.3\lambda$. Our results suggest that it is feasible to have a dense integration far below this estimation. We remark that for $Z\ne 40\lambda$, one would expect a different threshold $d_c$.

3.3. 16-Bit NAND Gate Operation and Transformation to NOR Gate

The above results are valid for the case with four “logic pixels”, namely four bits. We have also examined several cases with more bits, e.g., $4\times 4=16$, as shown in Figure 3d. Note that in this case there are a total of $2^{32}$ input possibilities, which prevents complete testing for all states. However, we have randomly tested 40,000 input configurations and confirmed the results. Figure 4 shows an example of $4\times 4=16$ bits with the input defined in Figure 4a. Here, $d_p=10\lambda$, and the remaining settings remain similar to the case in Figure 2. One can see that in the output plane, as illustrated in Figure 4b, the logic NAND operation clearly reflects the truth table (see

Table 1. Parallel NAND operation analysis for the configuration shown in Figure 4. Note that the relative output light intensities in two specific areas of one particular logic pixel are used for the output logic judgment. The upper (lower)-area domination case means that the output is “1” (“0”).

Logic-Pixel Index	Input 1	Input 2	Expected Output	Output Light Intensity	Correctness Check
(1, 1)	1	1	0	(3.75, 75.66)	√
(1, 2)	0	1	1	(47.98, 20.82)	√
(1, 3)	0	1	1	(49.64, 20.54)	√
(1, 4)	1	0	1	(50.89, 19.23)	√
(2, 1)	1	0	1	(50.03, 21.13)	√
(2, 2)	1	0	1	(49.27, 20.42)	√
(2, 3)	1	0	1	(48.69, 21.48)	√
(2, 4)	1	0	1	(50.39, 18.90)	√
(3, 1)	1	0	1	(50.58, 22.36)	√
(3, 2)	1	1	0	(4.08, 77.43)	√
(3, 3)	1	1	0	(4.24, 79.30)	√
(3, 4)	0	1	1	(49.43, 19.82)	√
(4, 1)	1	1	0	(3.75, 75.49)	√
(4, 2)	0	1	1	(48.65, 20.99)	√
(4, 3)	1	1	0	(3.76, 73.37)	√
(4, 4)	0	0	1	(78.85, 1.12)	√

).

In view of the logic pixel definition, one can clearly see that flipping the two apertures in a logic pixel could switch the logic operation from NAND to NOR, as schematically shown in Figure 5a. Figure 5b,c show the case of a NOR operation of ‘1010001011010110 NOR 0011000110001100 = 0100110000100001’. The other test cases are summarized in

Table 2. Parallel NOR operation analysis for the configuration shown in Figure 5.

Logic-Pixel Index	Input 1	Input 2	Expected Output	Output Light Intensity	Correctness Check
(1, 1)	1	0	0	(19.13, 52.4)	√
(1, 2)	0	0	1	(72.56, 3.892)	√
(1, 3)	1	1	0	(1.095, 78.6)	√
(1, 4)	0	1	0	(17.39, 50.0)	√
(2, 1)	0	0	1	(77.92, 3.837)	√
(2, 2)	0	0	1	(78.85, 4.345)	√
(2, 3)	1	0	0	(20.98, 47.3)	√
(2, 4)	0	1	0	(19.82, 8.529)	√
(3, 1)	1	1	0	(1.465, 77.6)	√
(3, 2)	1	1	0	(21.13, 49.23)	√
(3, 3)	0	0	1	(73.56, 4.645)	√
(3, 4)	1	1	0	(20.71, 48.6)	√
(4, 1)	0	1	0	(17.64, 50.5)	√
(4, 2)	1	1	0	(1.769, 75.9)	√
(4, 3)	1	0	0	(20.93, 50.6)	√
(4, 4)	0	0	1	(75.20, 3.716)	√

.

3.4. Cascaded DONNs for AND Logic Operation

In this section, we proceed to discuss the possibility of cascaded optical computation by using the DONN devised in Section 3.3 in a sequential fashion. Figure 6 schematically shows the actual optical configuration using two NAND DONNs. The bottom inset shows that the optical logic gates can be sequentially connected to form an AND gate.

After implementing this with the previous NAND DONN, we obtain the results shown in Figure 7b for a specific input (1001100111010000 AND 0011000100000101 = 0001000100000000) defined in Figure 7a. The correctness of the AND gate operation is checked, as shown in

Table 3. Parallel AND operation analysis for the configuration shown in Figure 7.

Logic-Pixel Index	Input 1	Input 2	Expected Output	Output Light Intensity	Correctness Check
(1, 1)	1	0	0	(10.10, 18.4)	√
(1, 2)	0	0	0	(1.436, 41.8)	√
(1, 3)	0	1	0	(10.83, 18.0)	√
(1, 4)	1	1	1	(34.14, 3.70)	√
(2, 1)	1	0	0	(9.116, 18.61)	√
(2, 2)	0	0	0	(1.574, 43.7)	√
(2, 3)	0	0	0	(1.625, 44.1)	√
(2, 4)	1	1	1	(32.45, 3.213)	√
(3, 1)	1	0	0	(8.832, 17.86)	√
(3, 2)	1	0	0	(8.880, 18.2)	√
(3, 3)	0	0	0	(1.591, 44.15)	√
(3, 4)	1	0	0	(9.656, 15.69)	√
(4, 1)	0	0	0	(1.156, 40.7)	√
(4, 2)	0	1	0	(8.645, 16.53)	√
(4, 3)	0	0	0	(1.367, 43.2)	√
(4, 4)	0	1	0	(9.613, 13.71)	√

.

4. Discussion

Basically, all our designs only involve phase modulation in each hidden layer. Experimentally, there are several approaches to fabricating diffractive layers and integrating them for the DONN. For example, Goi et al. utilized galvo-dithered two-photon nanolithography to fabricate a nanoscale single layer with a lateral resolution of around 100 nm and an axial resolution of around 10 nm for near-infrared optical inference [7]. Luo et al. designed and fabricated a metasurface with a feature cell size of around 200 nm for multi-channel optical computation [6]. Our design requires a phase layer pixel with a size of around 300 nm. We believe that it is possible to construct the DONN proposed here with both nano-printing and top-down metasurface fabrication approaches. On the other hand, it is possible to use vaccination training strategy or jointly trained hybrid optical-electronic neural networks to accommodate the fabrication and layer-alignment inaccuracy [32].

5. Conclusions

In conclusion, we have demonstrated the design and usage of DONN for parallel logic operations. Specific examples of NAND DONN are designed to process combinations of a pair of 4-bit binary numbers. An extension for a dense integration for up to 16 bits is presented, and the mutual crosstalk among the parallel bits is examined. It is shown that for a DONN layer spacing of $Z=40\lambda$ working at $\lambda =600$ nm, the closest valid pixel gap could be $d_p\ge d_c\approx 4\lambda$. Furthermore, we demonstrate ways to build an AND and NOR operation based on the fundamental NAND DONN. These results can be verified by experiments with a carefully designed metasurface and may find applications in optical signal processing, image processing, and security.