Liquid-Crystal-on-Silicon for Augmented Reality Displays

Abstract

In this paper, we review liquid-crystal-on-silicon (LCoS) technology and focus on its new application in emerging augmented reality (AR) displays. In the first part, the LCoS working principles of three commonly adopted LC modes—vertical alignment and twist nematic for amplitude modulation, and homogeneous alignment for phase modulation—are introduced and their pros and cons evaluated. In the second part, the fringing field effect is analyzed, and a novel pretilt angle patterning method for suppressing the effect is presented. Moreover, we illustrate how to integrate the LCoS panel in an AR display system. Both currently available intensity modulators and under-developing holographic displays are covered, with special emphases on achieving high image quality, such as a fast response time and high-resolution. The rapidly increasing application of LCoS in AR head-mounted displays and head-up displays is foreseeable.

1. Introduction

Dating from the 1970s, Hughes research labs creatively combined liquid crystal (LC) with semiconductor substrates, enabling optically addressed liquid crystal light valves [1,2], which paved the way for the invention of liquid-crystal-on-silicon (LCoS) display technology [3,4,5]. Through decades of extensive efforts, LCoS technology has proven its success in a wide range of fields like displays [6,7,8,9,10,11,12,13], adaptive optics [14,15], metrology [16,17], telecommunications [18], quantum physics [19,20], etc. The emergence of LCoS is jointly contributed by the application of liquid crystal (LC) electro-optic response behaviors and the development of the silicon complementary metal oxide semiconductor (CMOS) backplane technique. The structure of a typical reflective LCoS panel is shown in Figure 1: an LC layer is sandwiched between an indium tin oxide (ITO)-coated glass and a CMOS backplane. Two thin polyimide (PI) alignment layers determine the LC alignment direction. The Aluminum (Al) electrodes on the Silicon CMOS backplane are pixelated to provide independent voltage control. When an incident linearly polarized light traversing through the LC layer twice, its polarization state or phase is modulated depending on the LC alignment mode.

Recently, LCoS has been widely implemented in augmented reality (AR) displays because of its high luminance (>30,000 nits), compactness, high-resolution density (>4000 pixels per inch), and potential of holography generation [12,21,22,23]. However, the cutting edge LCoS still needs improvement to meet the stringent AR requirements. In this paper, we review the LCoS technology intended for AR displays. In Section 2, we briefly describe the basic light modulation principles of LCoS. In Section 3, we address the fringing field effect issue impairing the LCoS display performance, and propose a new device structure to mitigate this problem. High-resolution and small-pitch LCoS panels are especially desirable for augmented reality displays, but how to suppress the fringing field effect remains a technical challenge. In the last section, the LCoS-integrated AR head-mounted display (HMD) and head-up display (HUD) systems are demonstrated. The key parameters determining the display performance are discussed in detail.

2. LCoS Working Principles

2.1. Birefringence Effect of Liquid Crystals

Figure 2 illustrates the working mechanism of an LC phase retarder. Here, we assume the LC molecules are uniformly aligned along the x-axis with a tilt angle θ (Figure 2a). The uniaxial LC exhibits an extraordinary refractive index n_e along its optical axis and an ordinary refractive index n_o along the two orthogonal axes. Thus, the corresponding birefringence is defined as Δn = n_e − n_o. In Figure 2b, when an incident light passes through the LC layer along the z-axis, the x-polarized light experiences a tilt-angle-dependent refractive index n_x (θ), while the y-polarized light experiences a constant index n_y = n_o. The accumulated phase retardation between the two linear polarizations is

$$\Gamma =[n_x-n_y]\cdot \frac{2\pi }{\lambda }\cdot d.$$

(1)

Here d is the cell gap (LC layer thickness), and n_x (θ) follows

$$\frac{1}{n_x{\left(\theta \right)}^2}=\frac{{\cos }^2\theta }{n_e{}^2}+\frac{{\sin }^2\theta }{n_o{}^2}.$$

(2)

In a reflective display, the incident light passes through the LC layer twice. As a result, the effective cell gap is 2d and Γ is doubled.

When an external voltage is applied, the LC molecules (with Δε > 0) would reorient toward the electric field direction which, in turn, modifies the phase and polarization of the outgoing light. Depending on the surface alignment and the LC material properties, three LC modes are widely adopted: vertical alignment (VA) and mixed-mode twist nematic (MTN) mainly for intensity modulation, as well as the homogeneous alignment for phase modulation.

2.2. Vertical Alignment Mode

In the VA mode [24], the initial LC directors are nearly perpendicular to the substrates (pretilt angle α ~ 88°), as Figure 3 shows. For intensity modulation, the VA cell is sandwiched between two crossed polarizers, whose absorption axes are ±45° with respect to the x-axis. In the voltage-off state (V = 0), the incident light propagating along the z-direction experiences a negligible phase retardation. That means that the linearly polarized incident light remains unchanged and cannot pass through the crossed analyzer. As the applied voltage exceeds a threshold, the LC molecules with a negative dielectric anisotropy (Δε < 0) would be reoriented perpendicular to the longitudinal electric field. Because the surface anchoring force decreases as the distance from the alignment layer increases, the LC molecules keep the pretilt angle near the substrates and get fully reoriented in the central layer (Figure 3b). A pretilt angle α ≠ 90° gives a preference of x-z plane reorientation. Balancing between the competitive surface anchoring force, electric force and elastic restoring force, the LC tilt angle can change from ~88° towards 0° as the voltage increases. The associated phase retardation can be tuned from 0 to π, thus, the output light intensity can be progressively changed from dark to bright. The VA mode produces an excellent dark state and high contrast ratio (CR > 5000:1) regardless of the cell gap, wavelength, and operation temperature. However, the fringing field effect on the VA mode is more severe, which will be addressed in detail in the next section.

In principle, a VA cell can also be used for phase-only modulation, if the incident light polarization is parallel to the LC directors (in the voltage-on state). However, the negative-Δε LC materials usually exhibit a higher viscosity and a smaller |Δε| than the corresponding positive LCs because the dipoles are in the lateral positions. As a result, homogeneous cells using a positive-Δε LC are preferred for phase-only spatial light modulators.

2.3. Mixed-Mode Twist Nematic (MTN) Mode

In a twisted-nematic (TN) mode [25,26], the LCs form twisted structures by the surface alignment and the doped chiral agent. For example, in a 90° TN cell, the top and the bottom LCs are aligned orthogonally with a twist angle ϕ = 90°, as illustrated in Figure 4a. Such a molecular alignment can be achieved by mechanical rubbing on polyimide (PI) [15,27] or by the photo-alignment technique [28,29]. For intensity modulation, the LC cell is sandwiched between two crossed polarizers. The polarization axis of the front polarizer has an in-plane angle β with respect to the front LC alignment. When β = 0°, the polarization rotation effect takes place, which guides the incident x-linearly polarized light to y-linear polarization. The Gooch–Tarry first minimum condition $d\Delta n/\lambda =\sqrt{3}/2$ [30] determines the thinnest cell gap for a given LC birefringence to achieve maximum transmittance. When a longitudinal electric field is applied, the LC molecules (Δε > 0) are reoriented toward the vertical direction and a dark state is produced (Figure 4b). TN cells work well as a normally-white intensity modulator. It can also be used for phase-only modulation [31], but the voltage should be controlled below the optical threshold [32], leading to a slow response time [33]. For intensity modulation, the response time of the TN mode and VA mode could be comparable, depending on the employed LC material and the cell gap. A major drawback of TN mode is the compromised contrast ratio (CR ~1000:1).

In a reflective LCoS, the doubled optical path leads to a smaller dΔn requirement. However, when β = 0° the normalized optical efficiency is only 70% [6]. Two methods can be used to boost the optical efficiency: (1) to increase β to about 20°, and (2) to decrease the twist angle to 80° or 70° [34]. At β ≈ 20°, both the polarization rotation effect and birefringence effect coexist. This is called the mixed-mode twisted nematic (MTN) mode [6]. The optical efficiency increases to ~88%. Decreasing the twist angle can enhance the optical efficiency to nearly 100%, but the contrast ratio is decreased dramatically. Therefore, 90° MTN a favored choice, especially for field sequential color (FSC) displays, because of its merits in high contrast ratio, low operation voltage and fast response time. By removing the spatial color filters, FSC operation could triple the optical efficiency and resolution density.

2.4. Homogeneous Alignment

In a homogeneous (HG) cell (Figure 5), the LC alignment is parallel to the x-axis with a small pretilt angle (α ~ 2°) [29]. Using a positive Δε LC material, its electro-optic properties are opposite to the VA mode: phase retardation is maximized at the voltage-off state and decreases as the applied voltage increases. Due to the narrow viewing angle and the staggered RGB dark state, the HG cell is rarely used for broadband intensity modulation but is a favored choice for a reflective phase modulator. The viewing angle can be enlarged by a compensation film and by the mirror image effect in reflective LCoS [35]. The unmatched RGB phase retardation is not a big issue for phase modulation. The major advantage of the HG mode is its fast response time. To achieve the 2π phase range, the LC layer is usually twice thicker than its counterpart intensity modulator, as a result, its response time (τ is proportional to d²) would be 4x slower [33]. Thanks to the wide selection of positive LC materials, high Δε for low operation voltage and low rotational viscosity γ₁ for fast response time can be obtained more easily than those negative-Δε LCs for the VA mode [36].

2.5. LC Material Optimization Strategies

In electro-optic modulation, a general requirement applies to all of the above-mentioned modes: dΔn/λ equals a certain value. Regarding electrical properties, the CMOS circuitry limits the operation voltage to <6 V for small-pitch panels. Such a low voltage can be achieved by increasing the dielectric anisotropy of LC materials, which could be obtained by applying lateral and terminal polar groups (such as F, CN, NCS, CF₃ or OCF₃) [37,38,39], polar CF₂O bridges [40], and polar heterocycles [41,42]. However, these components often lead to a compromised nematic temperature range, lower solubility and higher viscosity γ₁ [38]. Breaking this trade-off, the difluoromethoxy bridge (-CF₂O) [43] and isothiocyanato group (-NCS) [44,45] could improve Δε while retaining a reasonably low γ₁. Whereas the former breaks the molecular π-π conjugation resulting in a low Δn, and the latter gives rise to the annoying charge accumulation effect, which degrades the voltage holding ratio. Here comes the second consideration on electrical properties—the resistivity which determines the voltage holding ratio. Although LCoS devices demand a lower LC resistivity (>10¹² Ω·cm) than the active matrix displays (>10¹³ Ω·cm) [46,47], it still limits the LC component selection. Compounds with a large Δε would require multiple dipoles, which may lead to a compromised γ₁.

From the aspect of device configuration, a thinner cell gap helps gain fast response time with the adoption of a slightly higher Δn LC. To be noticed, the toughest color to achieve the 2π phase modulation is red because of the longest wavelength and the lowest birefringence. Various terphenyls and tolanes have been developed for high Δn LCs [48,49]. Although many precise modifications of the molecular chemical structures have been mentioned [50,51,52], such as shifting the location of the mesomorphic core, introducing the lateral substituents into different positions and the various terminals, the high γ₁ of terphenyls and the poor UV stability of tolanes remains unsolved [53]. Those are the properties not intended to be compromised. Additionally, the voltage shielding effect by alignment layer reduces the effective voltage applied to the LC layer in thin cells [54,55]. In summary, a delicate balance should be reached between high birefringence, high dielectric anisotropy, low melting point, low viscosity, high resistivity, and good UV stability according to the device configuration. The state-of-the-art LC material development reaches a submillisecond response time for VA [56,57,58] and MTN [46,59] LCoS at 40 °C, and a ~2 ms response time for 2π phase modulation [46,47]. More than that, these optimization strategies and the developed materials could be applied to other LC phase modulation-based devices such as tunable lenses, scanners, tunable filters, and optically addressed spatial light modulators [60,61,62,63,64], which may be incorporated in AR systems in other ways.

3. Fringing Field Effect and Novel Solution

To optimize the optical performance of LCoS systematically, several research groups have devoted efforts in device modeling [65,66,67]. However, until recently, the fringing field effect (FFE) still remains the major issue to be tackled with [68,69,70,71,72,73]. In order to display greyscale images, different pixels could have different voltages. The unequal voltages on adjacent pixels give rises to FFE. As Figure 6 depicts, when a bright pixel is adjacent to two dark pixels, a dark split region may appear in the bright pixel area, and light leakage arises at the edge of the dark pixel, leading to display luminance loss and reduced contrast ratio. The severity of FFE depends on the display mode and the device configuration. Generally speaking, VA has the strongest FFE, resulting in a ~30% luminance loss; 90° MTN is the least sensitive to FFE; while the HG mode is a favored choice for phase modulation. Figure 6 shows the FFE of VA mode with different pixel sizes and cell gap. Because of the small inter-pixel gap, LCs on the pixel edge experience strong horizontal electric field, giving rise to phase distortion. This phenomenon is not obvious in direct-view transmissive LCDs because the inter-pixel region is covered by a large-area black matrix (>5 μm). The asymmetry of phase retardation/reflection is determined by the alignment direction of the pretilt angle. In Figure 6a, the same resolution density is compared, which is determined by pixel pitch p. With a thinner cell gap d, the FFE-affected region is compressed to a narrower edge. Applying this thin cell strategy [69,70], the cell gap in VA LCoS is optimized to d < 2 μm [73].

FFE is especially pronounced in small-pitch LCoS devices [69], which is highly desirable in augmented reality applications. Because of the reduced pixel pitch and cell gap (p/d) ratio, the unwanted horizontal electric field is strengthened [70]. This can be seen from Figure 6b: when the resolution density is doubled (p changes from 16 μm to 8 μm), the absolute FFE width w is relatively stable, while the ratio of the FFE-affected region increases dramatically. Several methods have been proposed to reduce FFE. In 2002, Fan-Chiang et al. explored how sloped electrodes help reduce FFE with quantitative exploration [68]. An optimal electrode slope of 1 was demonstrated for 80° MTN cell and 45° TN cell, while the VA and HG modes call for further improvement. In the same year, a finger-on-plane type common electrode was designed to eliminate the pixel-splitting phenomenon [74]. Whereas the diffraction effect and relatively low optical efficiency hinder its widespread applications [69]. In 2005, Gu et al. minimized FFE by a double-sided electrode design, where the requirements of dual TFTs and pixel registration brings manufacturing challenges [75]. Later, Fan-Chiang et al. proposed to adopt circularly polarized light in the VA mode for FFE suppression [69]. Though an impressively high optical efficiency and fast dynamic response were demonstrated, hardly can this method be integrated into the preferred normally black reflective VA mode. In other words, the contrast ratio is compromised. In 2015, Li et al. adopted non-rectangular pixel electrodes for pixel edge FFE compensation [76]. This design works well on large pixels while the degree of structural modification is constrained on small-pitch configurations. To satisfy the urgent need of FFE reduction on small-pitch LCoS devices, here, we propose a novel design of patterned pretilt angle control [77,78,79]. The formed inhomogeneous LC director distribution could oppose the FFE induced by neighboring pixels.

3.1. Pretilt Angle Pattern Determining Method

The basic concept of our design is to find an optimized pretilt angle pattern for minimizing FFE. The initial step is to identify the phase retardation profile as a function of location in the pixel region. Since FFE is most pronounced when the largest voltage difference occurs between adjacent pixel electrodes, two worst scenarios are used in our evaluation and optimization. These are (1) maximum operation voltage applied to the central pixel electrode while minimum operation voltage applied to the adjacent pixel electrodes; and (2) minimum operation voltage for the central pixel while the maximum voltage for the neighboring ones.

After basic evaluation, there are two determining methods to optimize the pretilt angle pattern. The first is to control the phase retardation error over the whole pixel region within a predetermined percentage level. As a general determining approach, the lack of a convergent solution makes the optimization complicated, and the iterative algorism results in heavy computation loads. Therefore, here we introduce the second determining method. As shown in Figure 6b, based on the initially identified phase retardation profile, we divide the pixel region into two zones with an FFE border width w. From Figure 7, we can see that the pretilt angle of the inner zone and outer zone is optimized to be α_i and α_o, respectively. The target of the optimization is to reduce the overall phase retardation error. With this simpler determining approach, pretilt angle patterns with an FFE-opposing feature can be designed.

3.2. Exemplary Optimization Results

Here we use an exemplary optimized structure to demonstrate the performance of the proposed design. The basic HG LCoS parameters are set as follows: cell gap d = 1.5 μm, pixel length l = 6.2 μm and pixel gap g = 0.2 μm, which corresponds to pitch length p = l + g = 6.4 μm. The LC parameters are Δn = 0.16 at 550 nm, Δε = 3.1, γ₁ = 118 mPa·s, and (splay, twist, bend) elastic constants (K₁₁, K₂₂, K₃₃) = (13.2, 6.5, 18.3) pN at 25 °C (from Merck). As illustrated in Figure 8a, by simulating the phase retardation profile with an even pretilt angle, the FFE-affected outer zone and the unaffected inner zone are differentiated with w = 1 μm. Applying the second determining method, the optimized inner zone pretilt angle α_i = 88° and outer zone pretilt angle α_o = 85° are obtained. Figure 8b shows the results with our modified pretilt angle pattern design: the electric field distortion at pixel edge is much suppressed and the FFE is effectively reduced.

One potential concern of our design is the phase retardation uniformity when an equal voltage is applied to adjacent pixel electrodes. Therefore, we plot the phase retardation curves with our special pretilt angle design. From Figure 9a we can see, in the large pretilt angle region (α > 80°), that neither phase retardation nor reflectance is sensitive to the pretilt angle. In Figure 9b, generally flat spatial profiles are obtained in our design. The spatial fluctuation at all-pixels-dark state is only ΔΓ = Γ (α = 85°) − Γ (α = 88°) = 0.01π for phase retardation and R (α = 85°) − R (α = 88°) = 0.03% for reflectance, suggesting that a high contrast ratio could be maintained. At all-pixels-bright state, the reflectance fluctuation is within 0.002%. In other words, the side effect of our proposed design is negligible compared to the gain in FFE reduction.

3.3. Fabrication Method

The pretilt angle pattern can be realized by a photoalignment layer, which provides inhomogeneous anchoring energy distribution. Two methods have been proposed to realize the spatial variant pretilt angle control by Kwok’s group. In 2011, the stack of a horizontal alignment polymer on a vertical alignment layer was reported [80]. By controlling the dosage of UV light exposure, the degree of polymerization can be adjusted spatially. After rinsing away the unexposed area, the ratio of polymerized homogeneous alignment domain and exposed vertical alignment domain coordinately determine the local LC pretilt angle. In 2013, Fan et al. reported a new alignment material which enables direct UV patterning on a pretilt angle [81]. Based on the above-mentioned methods, a two-step manufacturing process can generate FFE-opposing features (Figure 10): (1) A uniform pretilt angle α₁ is produced on the polyimide (PI) alignment layer on ITO glass by uniform UV irradiation; (2) A second photomask for selective UV exposure is added to differentiate the pretilt angle α_i in inner zone and α_o in outer zone. If the alignment layer is replaced by silicon oxide [82], an e-beam treatment can be adopted for the patterning.

For infrared spatial light modulators, the patterned pretilt angle and anchoring energy can also be achieved on the polymer-stabilized liquid crystal (PSLC) [83,84]. First, an LC-monomer mixed precursor is prepared and filled in the LCoS panel. Second, a photomask with gradient optical density is required to generate spatially variant UV light intensity. The UV light irradiated on the LC layer is more intense in the center region than at the pixel edge, therefore, in the center, it generates a higher degree of polymerization. Through this spatially variant polymerization process, an inhomogeneous pretilt angle and anchoring energy distribution can be formed, as Figure 11 illustrates.

4. LCoS for Augmented Reality Displays

4.1. Augmented Reality Head-Mounted Displays

So far, LCoS is the dominant solution for augmented reality (AR) head-mounted display (HMD) because of its compact size (<1.5 inches), high optical efficiency (>90%), high fill factor (>90%), and high resolution (4K2K achieved). In 2013, Kress et al. reviewed the HMD optical systems focusing on the combiner designs [85]. Here, we just use Figure 12 to schematically show how an LCoS functions in an AR-HMD optical system (Google Glass’s design [86]). A polarizing beam splitter (PBS) is engaged as crossed polarizers for the reflective display system. When the light emitted from an LED source enters the PBS (purple line), the s-polarized light (red line) is reflected away so that only the p-polarized light (blue line) can reach the LCoS panel, indicating that only half of the incident light could be modulated. To enhance the display brightness, polarization conversion optical elements have been developed [87] and can be inserted between the LED source and the PBS together with collimation lenses. Passing through the LCoS, the unmodulated p-polarized light (thin blue line) transmits back to the LED source, while the π-phase retarded s-polarized light (red) gets reflected by the PBS and directed to human eyes by a partially-reflective mirror (PRM) and a focusing mirror (FM). As the ambient light transmits PRM as well (green line), an AR experience with an LCoS-generated display and real world can be achieved.

Current 3D displays are mainly based on binocular disparity: two eyes receive different images representing slightly different viewing points. However, the mismatch of other focus cues gives rise to visual discomfort [88,89] and distorted depth [90,91,92]. As a promising one-stop solution to provide correct focus cues with aberration correction [93], phase-only LCoS-based AR-HMD has been demonstrated [12,13,21]. Different from the intensity modulation configuration, the incident linearly polarized light on phase-only LCoS should be parallel to the LC directors. Therefore, PBS could be eliminated in phase-only AR systems, enabling an eyeglasses size form factor [12]. In contrast to the hardware lightness, the computational burden on phase encoding is heavy. In the past decades, extensive efforts have been devoted to developing high-quality real-time holography algorithms [94,95,96,97,98]. Though prototypes have been demonstrated [12], the commercialization of high-quality holographic AR-HMD may still need time.

4.1.1. Response Time

The LCoS response time greatly affects the AR viewing experience. To avoid the flickering effect [99], a refresh rate of 60 Hz is the minimum requirement, while >144 Hz is highly favorable. However, the relatively slow LC response time could impair grey level accuracy at high frame rates, as Figure 13 illustrates. Taking 120 Hz as an example, the frame time is 8.3 ms. The green curve shows the applied voltage at the two consecutive frames. To switch from dark to bright state, the voltage is raised from the minimum to the maximum value, as the green curve denotes. The LC molecules respond gradually so that the reflectance increases from 0 to 100% during the first frame (0~8.3 ms) (blue curve). As a result, the average reflectance in the first frame is much lower than the expected 100%, as the red dashed lines mark. Similarly, if the voltage is released at the second frame, it only generates a relatively poor dark state. The relatively slow response time (tens of milliseconds) of conventional LC materials cannot satisfy the >60 Hz refresh requirement (>180 Hz color field rate). To overcome the slow response issue in the above-mentioned analog driving scheme, a digital driving scheme, say, pulse width modulation (PWM), is widely adopted in an LCoS intensity modulator [100]. By applying binary voltages with various pulse widths within one frame, the LC molecules respond to the root-mean-square (RMS) voltages and human eyes perceive the average reflectance. Although digital driving lays less burden on LC response time and provides better grey level accuracy, stronger FFE may appear with small grey-level changes. In 2000, Worley et al. designed a bit-splitting method to overcome the FFE in digital driving by sub-frame rate increasing [101]. The LC response time in digital driving scheme needs to be carefully selected: it should be less than the frame time, but longer than the pulse width to avoid a perceivable light intensity fluctuation [102]. In theory, digital driving is not suitable for phase-only LCoS because phase information cannot be averaged with an RMS value. In practice, phase flickering has been suppressed to an acceptable level [103]. Jointly with its merits of fast response time, high repeatability and good phase vs. gray level linearity, digital driving is widely adopted in commercial phase-only LCoS devices [104]. In general, fast-response LC materials help improve the performance of both analog-driven and digital-driven LCoS devices. Therefore, extensive efforts have been made to obtain submillisecond response LCs [46,57,58,59].

4.1.2. Resolution

To cover the ~100° field of view (FoV) with human eye acuity 1 arcminute, ideally, a 6K6K resolution is required for each eye [105]. As a reference, the resolution of most commercially available products is ≤4K2K, which is still insufficient. Increasing the hardware resolution brings several challenges: (1) If the panel size remains the same, then increasing resolution density implies a smaller pixel size, which would result in a stronger FFE [69]. The limit of inter-pixel gap control may lead to a decreased fill factor; (2) the gate lines in an active matrix driving scheme are opened one by one. This scanning update method makes the product of panel resolution and each line’s signal time (gate opening + data addressing time) a constant, i.e., the frame time. Keeping the same frame rate, an insufficient charging issue appears if the resolution is increased much; (3) the computational burden increases with the display resolution, which may decrease the source input frame rate.

Hardware-focused solutions have been proposed to address some of the above issues. In 2004, Kanazawa et al. proposed to integrate four LCDs with a pixel-offset method to achieve 8K4K performance [106]. Besides, in foveated displays, an additional panel provides ultrahigh resolution at the central region (FoV ~ 20°) [107]. For the stacked methods, an optical system to accommodate the reflective LCoS without bulkiness needs to be carefully designed. In 2008, JVC demonstrated an 8K4K LCoS projection display, but a smaller pixel pitch is still desired [108].

To detour the hardware issues, people attempt to shift the burden from the spatial domain to the time domain. The field-sequential color (FSC) method is commonly adopted in LCoS projection displays [109,110]. That is how the 4K2K resolution is achieved. FSC display is to blink the RGB backlight in time sequence. In other words, each image frame is divided into three monochrome subframes. In this way, the resolution density could be tripled, but the trade-off is a tripled system refresh rate. For example, a 60-Hz display frame rate with FSC design needs 180 Hz color field rate. Recently, the image shifting and overlaying design were also demonstrated in AR displays [105,111]. By generating two light fields in each frame, the perceived resolution is enhanced. Similar to an FSC display, the required field rate is doubled.

In real-time holographic displays with phase-only LCoS, the required resolution is different from 6K6K as in intensity modulation. Taking phase grating as an example, the FoV correlates to the maximum diffraction angle, which is inversely proportional to the pixel pitch. On the other hand, the angular resolution is inversely proportional to the panel size according to the theory of Fourier transform. In the path light projected to human eyes, the applied optical system adds on factors to the inversely proportional relationships, so that the FoV and angular resolution could be increased or reduced while keeping the same ratio. To obtain a larger FoV with finer angular resolution, AR displays call for higher spatial resolution LCoS panels. Fitting in the same panel size for compact form factor, the pixel pitch decreases as the resolution increases. The state-of-the-art pixel pitch has reached 2.5~3 μm. Ideally, AR displays call for a 1-μm pitch. Challenges of fabricating such a small pitch come from both the electrical and optical sides. The limited space in the pixel area could accommodate very few transistors and storage capacitors, resulting in a low applicable voltage on the LC layer. The fringing field effect and low fill factor impair the throughput optical power. Presently, the relatively large pixel size and limited resolution provided by commercial LCoS panels cannot provide a satisfactory 3D image quality. An upgrade in hardware or new optical system design is highly desirable.

4.2. Augmented Reality Head-Up Displays

Head-up display (HUD) was first developed for military aircraft. Nowadays, it is extended to automobile driving assistance. By projecting a virtual image onto the windshield, the driver does not need to head down to watch for the information displayed on the cockpit cluster. In so doing, the distraction could be minimized. A schematic of today’s HUD is shown in Figure 14. By using a double mirror (M1 and M2) reflection and the windshield as the combiner, the displayed information can be integrated with the front scenery. In some cost-effective designs, an additional combiner is employed so that the HUD box works with multiple vehicle models. Current HUD systems are employing transmissive TFT-LCD panels, providing a limited FoV and a relatively short virtual image distance (VID). The typical FoV = 5° × 3° and VID = 2 m still need to be improved keeping the compact package size [112]. Multiple studies suggest a preferred HUD information placement at 5° eccentric to the right and a FoV no larger than 20° × 10° for fast recognition [113,114]. Display registration with the real-world objects is also highly recommended, requiring a variable VID with a wide range [112,113,115,116]. The LCoS-integrated system could realize holographic AR-HUD with a larger FoV (>10° × 5°) and a variable VID (>10 m) [93]. From the LCoS device aspect, the requirements in AR-HUD are somewhat different from that in AR-HMD: (1) the longer VID (>2 m) and the smaller FoV help relieve the burden on resolution; (2) the driving conditions with an intense daylight demand a much higher display luminance for keeping the high ambient contrast ratio [117]; (3) the LCoS device is expected to work in extreme environments with a wide temperature range [59,118]. Among them, the thermal management challenge related to high luminance is a major concern.

5. Conclusions

After several decades of intensive efforts, LCoS has finally reached several key milestones: (1) 4K2K resolution panels are commercially available and 8K4K prototypes have been demonstrated [108]; (2) presently, a pixel pitch as small as 2.5~3 μm has been developed, while 1 μm is desirable for AR displays; (3) new LC materials with a submillisecond response time for intensity modulation [46,58,59] and ~2 ms response time for phase modulation at 40 °C [46,47] are available, enabling high-frame-rate LCoS with both analog and digital driving; (4) new methods are proposed to suppress the fringing field effect from a large-pixel design to the accommodation of small pixels [78]. Because of the abovementioned achievements and the merits of high fill factor, compact size and light weight, LCoS microdisplay has been integrated into several commercial AR headsets as an intensity modulator [86]. Multi-focal plane [13] and holographic [12] AR-HMDs, and AR-HUDs [93] with phase-only LCoS are under intensive development. More AR displays incorporating upgraded LCoS panels are expected to emerge in the near distant future.