Human Eye Contrast Sensitivity to Vehicle Displays under Strong Ambient Light

Abstract

Vehicle displays are becoming more integrated into our daily lives. Achieving a premium driving experience demands the display panel to have high-resolution density and sufficient brightness, particularly when exposed to intense ambient light, as direct sunlight can obscure the displayed images. Combining Barten’s model and diffraction theory, the performance of both infotainment displays and head-up displays (HUDs) is evaluated. For infotainment displays, over 800 nit brightness is essential for the driver to discern 55 pixel-per-degree (PPD) patterns under direct sunlight. For HUDs, a delicate balance between resolution density, brightness, transparency, and image quality must be exercised. By slightly reducing the resolution density to 50 PPD, the transparent micro-LED panel can concurrently achieve a reduced background image blur, low required display brightness (~4000 nits), and high background transmittance (~90%).

1. Introduction

Different from indoor displays, vehicle displays often encounter strong ambient light in the daytime [1]. Two types of displays are commonly integrated into a vehicle: infotainment displays and transparent head-up displays (HUDs). An infotainment display combines both information and entertainment features in a single screen [2]. It is commonly integrated with a dashboard to function as a digital instrument cluster, such as a speedometer, and to display global positioning system (GPS) images. Among different vehicle displays, liquid crystal displays (LCDs) and organic light-emitting diode (OLED) displays are two currently dominating technologies. LCD has merits in high-temperature tolerance, long lifetime, and low-cost [3], while tandem OLED [4,5] is a strong contender due to its wide color gamut, good flexibility, and high contrast ratio to avoid the postcard effect at night [6]. At Display Week 2023, leading vehicle display companies such as BOE demonstrated a 21.6” curved mini-LED (mLED) backlit LCD prototype with 786 dimming zones and Tianma showcased a 9.94” OLED trifold screen. However, for traditional LCDs, the image content may be washed out by strong ambient light due to the relatively low CR (<5000:1) and limited peak brightness, while OLED materials are still facing thermal instability [7]. An mLED backlit LCD can achieve a comparable CR to OLED [8], while further reducing the halo effect [2,9,10] and increasing the optical efficiency from LCD is still challenging.

Micro-LED (μLED) and mLED (without LCD) are emerging as next-generation infotainment displays due to their ultra-high peak brightness, excellent dark state, long lifetime, good flexibility, wide color gamut, and ultrafast response time [11,12,13,14]. Especially for μLEDs, the small aperture ratio (usually ~1%; the remaining 99% of the area is filled with black matrices) can further reduce the ambient light reflectance from the bottom metal electrode. As a result, μLED will be a strong contender for automotive displays once its manufacturing processes become more mature.

On the other hand, a HUD allows users to view digital information without changing their natural line of sight [15]. It is playing an increasingly important role in automotive displays because the drivers can receive information directly from the windshield. To deliver a clear display image with plain environmental conditions, HUD should exhibit high transparency (>70%), ultra-high brightness, high-resolution density (~60 pixels per degree (PPD)), and low diffraction [16]. There are two main types of HUDs: projection [17] and flat panel [18]. The former has several demerits in terms of high optical loss, small eyebox, and strong color dispersion. For the flat-panel type, the light sources are directly embedded under the windshield. Recently, the direct view µLED has been gaining momentum owing to its high brightness, small mesa size, very wide temperature range, and exceedingly long lifetime [19,20]. At Display Week 2023, PlayNitride demonstrated a 9.38” 114 PPI μLED panel with 65% transparency. AUO also introduced a 5000 nit 13.5” μLED prototype with 163 PPI and 55% transparency.

However, very few studies investigate the human recognizability of vehicle display panels under strong ambient light. In this paper, we systematically analyze LCD and μLED under strong ambient light. Firstly, we analyze the performance of different infotainment displays under different ambient lighting conditions based on Barten’s model. Then, we validate the calculated data with visual tests. Afterward, a transparent display model is built based on diffraction theory and photopic function to balance the transmittance of environmental light and required display brightness. Finally, multi-scale structure similarity is analyzed by considering high-order diffraction to determine the image quality and optimize the aperture ratio.

2. Infotainment Display under Different Ambient Light

2.1. Contrast Sensitivity and Barten’s Model

Contrast sensitivity (CS) is a measure of the human eye’s ability to distinguish sinusoidal patterns at each spatial frequency, and the contrast sensitivity function (CSF) is defined as the reciprocal of the minimum contrast of a sinusoidal grating that can be detected by the human eye at a specific spatial frequency [21,22], as illustrated in Figure 1.

The CSF is correlated with several factors, such as field of view (FoV), resolution density of the display panel, spatial frequency, brightness, and ambient light intensity [23,24,25]. In 1999, Barten developed a simplified physical model for the CSF of the human eye using massive visual tests [26]. Based on empirically fitting the psychophysical data, Barten’s model has the following form:

$$CSF\left(f\right)=af{e}^{-bf}{\left(1+c{e}^{bf}\right)}^{0.5},$$

(1)

where

$$a=\frac{\left[540{\left(1+\frac{0.7}{L}\right)}^{-0.2}\right]}{[1+\frac{12{\left(1+\frac{f}{3}\right)}^{-2}}{w}]},b=0.3{\left(1+\frac{100}{L}\right)}^{0.15},c=0.06.$$

In Equation (1), f is the spatial frequency of the stimulus, w is the FoV of the stimulus in degrees, and L is the average luminance of the stimulus in cd/m². The reciprocal of the contrast (RoC) under a specific ambient light can be expressed as:

$$RoC=\frac{1}{Contrast}=\frac{L_{max}+L_{min}+2R\times I_a/\pi }{L_{max}-L_{min }},$$

(2)

where I_a (unit: lux) is the ambient light illuminance and R is the surface reflectance. The human eye recognition threshold can be calculated using the intersection of Equations (1) and (2).

2.2. LCD and μLED Infotainment Displays under Different Ambient Conditions

For simplicity, we consider two display panels with the same resolution density: (1) a 400 nit LCD panel with a 0.5 nit light leakage at a dark state and (2) a 1000 nit μLED with a perfect dark state. In a perfect dark room, the calculated RoC is 1.002 for the LCD panel and 1 for the μLED; these two values are very close to each other. The calculated CSF results are plotted in Figure 2a. The small contrast sensitivity difference between LCD and μLED is due to their average luminance difference. To visualize the difference in and readability of the patterns on these two screens, we focus on the spatial frequency ranging from 55 to 60 lines/deg. The dashed line in Figure 2b indicates the contrast sensitivity threshold of the human eye, and the patterns will not be readable if the curve is below this line. In this case, LCD and μLED share the same threshold since the contrast sensitivity threshold difference is very small. From Figure 2b, the pattern readability threshold is 56 lines/deg for LCD and 58 lines/deg for μLED. Both threshold values are close to human visual acuity (60 PPD).

The ambient light in a vehicle could vary from starlight to sunlight. The SAE 1757 standard [27] gives a good representative illumination and measurement during daytime in two typical cases: (1) Diffuse daylight with an illuminance of 5 k lux and (2) direct sunlight with 45 k lux. If we assume the surface reflectivity of the anti-reflection (AR) coating is 1.5% for both LCD and μLED panels, then the ambient light luminance will be 23.87 nits for diffuse daylight and 214.86 nits for direct sunlight. The ambient contrast ratio (ACR) of LCD and μLED will be 17.39:1 and 42.89:1 under diffuse daylight, respectively. On the other hand, the ACR of LCD and μLED will be reduced to 2.86:1 and 5.65:1 under direct sunlight. According to the ISO15008 criteria that ACR should be ≥2:1 under direct sunlight, both displays can fulfill this requirement.

Based on Equation (2), the calculated RoC results are: RoC_diffuse-LCD = 1.12, RoC_{diffuse-μLED} = 1.05, RoC_direct-LCD = 2.08, and RoC_direct-μLED = 1.43. Under the diffuse sunlight shown in Figure 3a, the recognition threshold is 55.4 and 57.8 lines/deg for LCD and μLED, respectively. Figure 3b indicates that the threshold is reduced to 52.1 lines/deg for LCD and 56.0 lines/deg for μLED. Several conclusions can be drawn, as illustrated in Figure 2 and Figure 3 and Equation (2): (1) Increasing the ambient light brightness continuously reduces the CS recognition threshold, especially for LCD due to its lower peak brightness, making the patterns more difficult to recognize. (2) Both CSFs slightly shift with increasing panel luminance. (3) The difference in the CS recognition threshold is mainly dominated by the high peak brightness of μLED instead of the light leakage of the LCD. Therefore, based on the above conclusions, we calculated the PPD as a function of display brightness under different ambient lighting conditions, as shown in Figure 3c. As the ambient light brightness increases, the patterns are more difficult to distinguish. As the reflected ambient light increases to 250 nits, the display panel with over 800 nits is required for the driver to distinguish the 55 PPD patterns.

2.3. Contrast Sensitivity Visual Test

Barton’s function was derived based on massive eye exam data obtained in lab conditions. In that case, observers had sufficient time to recognize the patterns. In realistic cases, the driving condition is more complicated than lab testing. For example, the pattern on the screen should be recognized by the driver in a very short period (~1 s) for driving safety, especially when eyesight saccades from bright surroundings to the screen. Therefore, we designed a subjective experimental comparison between two same patterns with different brightness on a single commercial 4 k 15.6” OLED laptop panel with gamma = 2.2 curves (Figure 4a). As shown in Figure 4b,c, on the left screen (representing μLED), the dark state and bright state luminance were set at 0 (gray level = 0) and 356 nits (gray level = 255), respectively. On the right screen (representing LCD), the dark state and bright state luminance were set at 0.55 nits (gray level = 11) and 142 nits (gray level = 168), respectively. The background gray level in the dark room and the bright room was fixed at 25 (1.97 nits) and 120 (67.8 nits), respectively, for the eye’s preference. The bright room test took place in an environment of ~320 lux ambient light. An additional white lamp was applied to provide ~49 nit reflectance from the screen, and its spot size exactly overlapped with the testing pattern. A total of five testers at ages ranging from 23 to 31 with normal or corrected vision were included. As indicated in

Table 1. Varying PPD as a function of distance under a fixed FoV.

PPD	Distance to Screen (cm)	Pattern Size (cm)
60	27.5	5.0
55	24.7	4.3
50	22.0	3.8
45	18.7	3.3
40	15.5	2.7

, the resolution density was controlled by adjusting the tester’s distance to the screen. The pattern size was also changed with distance to keep the FoV at 10°.

According to our simulation results shown in Figure 5a, the recognition threshold is 51.5 and 56 for LCD and μLED in the dark room, respectively. The experimental results in Figure 5b show that the patterns start to be unreadable at 45 and 40 lines/deg for the left (μLED) and right (LCD) screens, respectively. At 55 lines/deg for μLED and 50 lines/deg for LCD, the patterns are completely indistinguishable, which is very close to our simulation results.

On the other hand, the visual tests performed, as shown in Figure 5b, cannot represent realistic driving conditions because testers have sufficient time to recognize the patterns on the screen. As previously mentioned, drivers cannot stare at the screen for more than 1 s to ensure driving safety. Therefore, in quick response tests, testers are required to watch the night driving video provided by another panel, then rapidly move their sight to test patterns and recognize it within 1 s. The test results are plotted in Figure 5c. The patterns start to be unrecognizable at 27.5 and 22.5 lines/deg for μLED and LCD, respectively. At 45 lines/deg for μLED and 40 lines/deg for LCD, the patterns are completely indistinguishable. A significant degradation in performance is found for all the testers.

The bright room tests also exhibit similar results. As shown in Figure 5d, the recognition threshold is 49 and 54 lines/deg for LCD and μLED, respectively. On the other hand, according to Figure 5e,f, the recognition threshold is reduced from 50 lines/deg to 45 lines/deg for both patterns. Compared to the dark room tests, the difference is relatively small, which could be attributed to a small change in the pupil.

3. Transparent μLED for Head-Up Displays

3.1. Transmittance of a Transparent μLED

As Figure 6a indicates, the transmittance of a transparent μLED can be calculated by considering several components: (1) environmental light, which can be simplified to a D65 source, (2) surface reflectance from the μLED panel, (3) diffraction due to periodic μLED structure, and (4) the image in the human eye after considering the photopic and scotopic functions. The wavelength independent average transmittance for daylight (T_Day) and night (T_Night) can be calculated from Equations (3) and (4):

$$\overline{T_{Day}}=\frac{{{\displaystyle \int }}_{380 \mathrm{nm}}^{780 \mathrm{nm}}D\left(\lambda \right)\times P\left(\lambda \right)\times T_{Surf}{\left(\lambda \right)}^2\times T_{LED}\left(\lambda \right)d\lambda }{{{\displaystyle \int }}_{380 \mathrm{nm}}^{780 \mathrm{nm}}D\left(\lambda \right)\times P\left(\lambda \right)d\lambda },$$

(3)

$$\overline{T_{Night}}=\frac{{{\displaystyle \int }}_{380 \mathrm{nm}}^{780 \mathrm{nm}}D\left(\lambda \right)\times S\left(\lambda \right)\times T_{Surf}{\left(\lambda \right)}^2\times T_{LED}\left(\lambda \right)d\lambda }{{{\displaystyle \int }}_{380 \mathrm{nm}}^{780 \mathrm{nm}}D\left(\lambda \right)\times S\left(\lambda \right)d\lambda },$$

(4)

where D(λ), P(λ), and S(λ) represent the normalized D65, photopic, and scotopic spectrum, respectively. T_Surf = 0.5 × T_TE + 0.5 × T_TM is the surface reflectance from the AR coating and T_LED is the effective transmittance from the periodic μLED panel.

Figure 6b shows an MgF₂ AR coating (refractive index 1.377) of a 30°-tilted windshield with a sealing polymer, which has a refractive index of 1.5. The calculated reflectance is consistently lower than 2.5% from 380 nm to 780 nm (Figure 6c). For a typical vehicle, the distance from the driver to the windshield is approximately 19 inches (~48 cm). To achieve 60 PPD, the full-color pixel size is approximately 143 μm. Figure 6d shows the RGB square opaque µLEDs with a pitch size of a² in a 143 µm full-color µLED pixel, and the aperture ratio is defined as the transparent area divided by the total area. Considering a panel with 3840 × 2160 pixels, the total size is Lx = 55-cm × Ly = 31-cm.

Using the Fourier optics model for a transparent μLED from previous studies [28,29], the 2D point spread function (PSF) of the panel for an object at infinite distance can be calculated as:

$$h\left(f_x,f_y\right)\propto {\left|\left[\mathit{F}\left\{t_0\left(\xi ,\eta \right)\right\}\times comb\left(L_{x0}{f}_x,L_{y0}{f}_y\right)\right]\otimes sinc\left(L_x{f}_x,L_y{f}_y\right)\right|}^2,$$

(5)

where f_x and f_y are the spatial frequency in the x and y direction, respectively, and L_x0 and L_y0 are the size of a subpixel. For a 60 PPD full-color display panel, L_x0 = 47 μm and L_y0 = 143 μm.

If we consider the angular resolution of human normal vision acuity 20/20 to be 1 arcmin, then its corresponding length on the retina is 5 μm [28]. Therefore, on the imaging plane (retina), the effective light transmittance is the light received within 5 μm at the imaging plane; otherwise, these rays are formed into stray light, which, in turn, degrades the image quality. The effective transmittance of the panel T_LED can be calculated as:

$$T_{LED}\left(\lambda \right)=\frac{{{\displaystyle \int }}_0^{2\pi }{{\displaystyle \int }}_0^5{\left|h\left(\lambda \right)\right|}^{2}drd\theta }{{{\displaystyle \int }}_0^{2\pi }{{\displaystyle \int }}_0^5{\left|h_0\left(\lambda \right)\right|}^{2}drd\theta },$$

(6)

where h and h₀ represent the 2D PSF for a specific aperture ratio and for the unity aperture ratio, respectively.

Figure 7a indicates that T_LED remains constant for a 20 μm × 20 μm LED mesa because the diffraction order spacing is much larger than the human eye’s angular resolution, which is consistent with previous studies [28,29]. For a 143 μm pixel size, the spacing is ~65 μm for green light (λ = 550 nm), which is much larger than the distinguishable threshold (5 μm); therefore, only the wavelength-independent zeroth-order diffraction can be considered as the effective transmittance after diffraction in our case. On the other hand, transmittance increases continuously with the aperture ratio, as depicted in Figure 7b. From the simulation, we can notice that the transmittance is almost proportional to the square of the aperture ratio. This can be validated by calculating the zeroth-order diffraction intensity. For each subpixel with a mesa size of a², the aperture function can be expressed as:

$$t\left(\xi ,\eta \right)=Rect\left(\xi /L_{x0}\right)\times Rect\left(\eta /L_{y0}\right)-Rect\left(\xi /a\right)\times Rect\left(\eta /a\right)$$

(7)

The corresponding PSF is:

$$h\left(f_x,f_y\right)\propto \left|L_{x0}{L}_{y0}\times sinc\left(f_x{L}_{x0}\right)\times sinc\left(f_y{L}_{y0}\right)-a^2\times sinc\left(f_{x}a\right)\times sinc\left(f_{y}a\right)\right|$$

(8)

Since only the zeroth-order diffraction is considered, f_x and f_y both approach zero, so that the transmitted power is proportional to ${\left(L_{x0}{L}_{y0}-a^2\right)}^2$, which is the square of the aperture ratio. As a result, to achieve 70% transmittance, the corresponding aperture ratio should be ~84.5% for the 143 μm pixel size, indicating that the mesa size should be <32.5 μm.

3.2. Brightness Requirement for a Transparent μLED

For μLED based HUDs, the most challenging part is to balance the trade-off between transparency, resolution density, and brightness. Reducing μLED mesa size helps improve transparency, but the brightness is reduced due to a lower external quantum efficiency [30,31,32]. On the other hand, for a constant μLED mesa size, increasing μLED pixel size will increase the transparency of the panel due to the increased aperture ratio. In this case, the screen resolution density, and the eye recognition threshold decrease, resulting in a decreased required panel brightness. Similar to Equation (2), the ambient RoC of a transparent display can be calculated as:

$$RoC=\frac{L+2L_a\times T}{L},$$

(9)

where L_a is the ambient light luminance. Under full daylight, the environment brightness is approximately L_a = 6000 nits (or ~20,000 lux) [11].

Like Figure 3d, the human eye’s recognition threshold can be calculated as a function of mesa size and display brightness by combining Equation (9) with Barten’s model. Figure 8a indicates that it is difficult to achieve 60 PPD under such strong ambient conditions. In addition, a smaller mesa size results in a higher transparency, which increases the eye recognition threshold. For example, a ~4500 nit display is required to achieve 50 PPD for a 10 μm mesa LED, while only ~2300 nits is required for LEDs with a 45 μm mesa. Since it is difficult to perfectly match the human visual acuity of 60 PPD, we slightly reduce the resolution requirement to 55 PPD and 50 PPD, corresponding to a pixel size of 157 μm and 173 μm, respectively. The required brightness and transmittance as a function of mesa size are recalculated and shown in Figure 8b. By decreasing the human recognition threshold from 55 PPD to 50 PPD, the required brightness decreases significantly from the range of ~10,000 nits to ~4000 nits. Simultaneously, transmittance increases due to a higher aperture ratio. To keep a 70% transmittance, the maximum mesa size of the μLED increases from ~35 μm to ~40 μm. Therefore, slightly reducing the display resolution can significantly reduce the demand for the display and increase the design freedom. Overall, 50 PPD μLEDs with a 20 μm mesa size can achieve both high background transmittance (~90%) and low required display brightness (~4000 nits).

3.3. Image Quality on Emissive Transparent Displays

In the above analysis, we only focus on zeroth-order diffraction and effective transmittance within 1 arcmin on the human eye pupil. On the other hand, any higher-order diffraction introduces imaging blurs, and the wavelength-dependent diffraction order spacing causes dispersion. Figure 9 describes the simulated background image blur of the object, which is placed 100 cm away from the windshield. As shown in Figure 9b,c, a large μLED mesa (e.g., 70% aperture ratio or 45 μm mesa) can cause a stronger imaging blur (horizontal stripes on the window) and the existence of rainbow stripes, which can be suppressed as the aperture ratio increases to 90%. Figure 9d shows that the calculated multi-scale structure similarity [33] agrees with the previous analysis. It cannot perfectly achieve unity because the image content is still slightly modified during image transformation even when the aperture ratio is 100%.

4. Conclusions

In conclusion, we calculated and analyzed the performance of both infotainment displays and HUDs under different ambient lighting conditions for vehicle applications. For infotainment displays, high brightness plays a much more significant role than dark-state light leakage in determining the human recognition threshold under sunlight. In other words, if mini-LED backlit LCD can provide a comparable brightness to μLED, the eye recognition performance should be close to each other. The visual test agrees with the simulation result. In addition, due to complicated driving conditions, fast recognition of the pattern will be more difficult, especially during night driving. For HUDs at the current stage, a 50 PPD (pixel size 173 μm) μLED panel with a 20 μm mesa size can achieve reduced background image blur, lower required display brightness (~4000 nits), and higher background transmittance (~90%) simultaneously.