Variable Curvature Displays: Optical Designs and Applications for VR/AR/MR Headsets

Abstract

In the present paper, we discuss the design of a projection system with curved display and its enhancement by variably adjusting the curvature. We demonstrate that the focal surface curvature varies significantly with a change of the object position and that it can easily be computed with the Seidel aberration theory. Using this analytically derived curvature value as the starting point, we optimise a refocusable projection system with ${90}^{°}$ field of view and $F/\#=6.2$. It is demonstrated that such a system can provide stable image quality and illumination when refocusing from infinity to 1.5 m. The gain in spatial resolution is as high as 1.54 times with respect to a flat focal surface. Furthermore, we prove that a silicon die can be curved to the required shape with a safety factor of 4.3 in terms of the mechanical stress. Finally, it is shown that the developed system can be used in a virtual reality headset providing high resolution, low distortion and a flexible focusing mode.

1. Introduction

The aberration of field curvature and the means to correct it have been considered in optical design studies since the works by Joseph Petzval. During the last decade, it was practically demonstrated that the field curvature of an imaging system can be directly corrected by using a curved image sensor. Several techniques to produce devices based on curved CMOS (Complementary Metall-Oxide–Semiconductor) sensors were developed together with theoretical studies on the design and analysis of optical system using these detectors [1,2,3,4,5,6]. It was shown that such systems can provide a better image quality with fewer optical elements, and therefore reduced volume and mass. These systems also have advantages in terms of the image distortion and the illumination uniformity [3,6].

Recently, it has been demonstrated that the approach used for the production of curved sensors [7] can be applied to create curved microdisplays thanks to the similarity between CMOS detectors and OLED (Organic Light-Emitting Diode) displays [8]. This latest development opens new prospects for the creation of optical systems with curved focal surfaces. Now, the potential applications expand beyond the frames of ordinary imaging optics to projection systems. The latter area includes emerging topics such as virtual/augmented reality (VR/AR) and head-mounted displays (HMDs). The use of curved displays helps to significantly increase key parameters of the VR/AR devices such as field of view (FoV) and resolution without an increase of their mass and volume, as shown by Han [9]. Currently, the simultaneous enhancement of these parameters represents one of the challenges in the field, which urges designers to use complex solutions with extreme aspheres, FoV tiling, etc. [10,11]. In addition, the application of curved displays facilitates the exploration of new design options such as zoom projection systems. However, designing such a system requires completing some initial tasks, which include the choice of starting configuration and optimisation strategy, as well as modelling of the display curving process.

In the present paper, we consider the approach to optical design with curved display and its extension for a zoom system. The paper is organised as followed. Section 2 presents the theoretical background of the optical design with a variable curvature focal surface. In Section 3, we present the design and optimisation algorithm. Section 4 provides an optical design example with its performance analysis. In Section 5, we give estimations of the design feasibility in terms of the display curving technology. Lastly, Section 6 contains simulations demonstrating a possible application of the design for VR.

2. Theoretical Background

To obtain a good starting approximation for the optical design and to demonstrate the field curvature dependence on the system configuration, we consider Seidel aberration theory [12]. The general layout of a projection lens is shown in Figure 1. Note that hereafter we assume that the raytracing procedure is fully reversible. The figure shows tracing of two paraxial rays used for definition of the Seidel sums and the corresponding notation.

The image surface shape is determined by two Seidel sums corresponding to astigmatism and field curvature:

$$S_{III}=\sum _{i=1}^q{h}_i{P}_i{\left(\frac{{\beta }_{i+1}-{\beta }_i}{{\alpha }_{i+1}-{\alpha }_i}\right)}^2,$$

(1)

$$S_{IV}=-\sum _{i=1}^q\frac{1}{h_i}\frac{{\alpha }_{i+1}{n}_{i+1}-{\alpha }_i{n}_i}{n_i{\alpha }_i},$$

(2)

where

$$P_i={\left(\frac{{\alpha }_{i+1}-{\alpha }_i}{1/n_{i+1}-1/n_i}\right)}^2\left(\frac{{\alpha }_{i+1}}{n_{i+1}}-\frac{{\alpha }_i}{n_i}\right),$$

(3)

where $n_i$ is the index of refraction after the ith surface.

In addition, in the aberration equations, the Lagrange invariant is used in the following form:

$$I=-n_1{\alpha }_1(S-S_P){\beta }_1,$$

(4)

Then, the longitudinal aberration can be derived for the rays propagating in tangential and sagittal planes:

$$\mathsf{\Delta }{z}_t=\frac{y^{2}B}{2n_q^{\prime }(S-S_P){\alpha }_1{\alpha }_q^{\prime }{\beta }_1^2}\left(3S_{III}+I^2{S}_{IV}\right),$$

(5)

$$\mathsf{\Delta }{z}_s=\frac{y^{2}B}{2n_q^{\prime }(S-S_P){\alpha }_1{\alpha }_q^{\prime }{\beta }_1^2}\left(S_{III}+I^2{S}_{IV}\right),$$

(6)

where B is the linear magnification.

The corresponding radii of curvature are:

$$R_{t\left(s\right)}=-\frac{\mathsf{\Delta }{z}_{t\left(s\right)}^2+y^{\prime 2}}{2\mathsf{\Delta }{z}_{t\left(s\right)}}.$$

(7)

One can see that the image surface curvature changes with the change of both distance to the object S and magnification B. To demonstrate this effect and to estimate the accuracy of this theoretical approximation, we perform a series of computations for a simple singlet lens [13]. The lens is scaled to have a focal length $f^{\prime }=36$ mm and $F/\#=12.9$. The linear FoV is ±12.7 mm. Since Seidel theory works for monochromatic aberrations, we consider a single wavelength $\lambda =780$ nm, which is used in the original patent. Three positions of the objects were considered: infinity, 5 m and 1.5 m. For each computation, the image surface curvature was calculated with Equation (7). For comparison, it was also computed from the marginal ray intersection coordinate and by numerical optimisation of the root mean square (RMS) radius in the spot diagram. Finally, a calculation with a flat image surface in the nominal focus position was used to compare with. For each calculation, the lens ray aberrations were computed. The results are summarised in

Table 1. Field curvature computations for demonstrative example (FoV center/corner).

Distance, m	Criterion	R, m	Spot X, $\mathsf{\mu }$m	Spot Y, $\mathsf{\mu }$m	Spot RMS, $\mathsf{\mu }$m
1.5	Seidel Sum Tang.	128.55	12.7/46.2	12.7/27.9	9.0/27.5
	Seidel Sum Sag.	75.12	26.6/25.4	26.6/43.2	15.2/28.7
	Marginal ray	151.87	9.8/50.6	9.8/25.1	8.8/28.4
	RMS	117.44	14.6/43.4	14.6/29.9	9.4/27.2
	None	inf.	21.1/74.6	21.1/23.0	15.9/38.0
5	Seidel Sum Tang.	129.50	12.1/46.4	12.1/ 26.6	9.0/27.2
	Seidel Sum Sag.	74.23	26.3/25.1	26.3/41.9	15.1/27.9
	Marginal ray	140.67	10.6/48.7	10.6/25.	8.9/27.7
	RMS	113.59	14.8/42.5	14.8/29.2	9.5/26.7
	None	inf.	21.5/74.3	21.5/24.1	16.0/38.0
inf.	Seidel Sum Tang.	129.89	11.9/46.5	11.9/26.1	9.0/27.2
	Seidel Sum Sag.	73.87	26.2/25.0	26.2/41.4	15.0/28.0
	Marginal ray	136.53	10.9/47.8	10.9/25.1	9.0/27.4
	RMS	112.08	14.9/42.1	14.9/29.0	9.6/26.5
	None	inf.	21.6/74.1	21.6/24.4	16.2/38.0

.

This simple example clearly shows three important points. Firstly, the image quality can be significantly improved by using a curved image surface. Secondly, the curvature changes notably with a change in object position. Thirdly, the value computed with the Seidel sum for rays in the tangential plane provides a good approximation for the curvature and can be computed analytically for any optical system.

3. Design Algorithm

The analytical procedure described above is used for an initial assignment of the image surface curvature in different configurations. This idea underlies the proposed design and optimisation algorithm. The design algorithm is presented as a simplified block diagram in Figure 2. If the application implies zooming or refocusing of the optical system, after choice of the initial optical design (1), the design is split into a number of zoom configurations (2), having, e.g., different object positions. At the next step, the initial focal surface curvature for each configuration is computed through Equations (1)–(7) (3). Afterwards, the optical design is optimised for each configuration. It is practically useful to perform two optimisation loops with different quality criteria, e.g., geometrical aberrations (4a) and ensquared energy (4b). In both cases, the variables include all the radii, thicknesses and the aspheric coefficients of the optics. The boundary conditions include maintaining the image and object size as well as limitations on the center and edge thicknesses. At the first stage (4a), the merit function is based on the root mean square radii in the spot diagram used with equal weight coefficients. At the second stage (4b), we use the fraction of energy ensquared in one pixel and the weight coefficient for the edge of the FoV is doubled. The obtained optical system is subject to a detailed performance analysis (5), which includes definition of the energy concentration and some secondary indicators such as the distortion and relative illumination of the image. The main path assumes a varying image surface curvature separately for each zoom position and is marked as “CV” for “curved, variable”. For comparison, we also consider the cases of static curvature ( “CS” for “curved, static”), when the same value is assigned to all configurations, and flat surface (“FS” for “flat, static”).

In the following section, we apply this algorithm to design a refocusable lens suitable for VR applications.

4. Optical Design Example

Targeting VR devices as the primary application, and considering other solutions [14,15] in the field, we chose the following initial data for the optical design example. The pupil diameter is 6 mm and the distance between the pupil and the first lens is $\ge 18$ mm. The angular field of view is 90${}^{°}$, and the real image size is $50.8$ mm. The system is designed for the visible domain with the standard d, F and C reference wavelengths. It should be noted here that Equations (1)–(7) provide the field curvature correction only around the central wavelength. They are used to find the best starting approximation and then the correction for the entire working range is provided by numerical optimisation, as shown in Figure 2. Further, we consider the three design strategies, explained in the previous section and each time apply three configurations with the object at infinity, 5 m and 1.5 m.

4.1. Design Description

A known triplet design [16] was used as the starting point. It was scaled and optimised to match the above given parameters. An overview of the resulting “CV” design is shown in Figure 3. Note that this is an example design and some additional technological optimisation is required for its practical implementation.

The figure clearly shows the difference in the image plane position and curvature between the configuration with object at 1.5 m (“1” index) and at infinity (“2” index). The design contains three Q-type aspheres at the first, third and sixth surfaces with six aspheric coefficients each. The maximum surface slope deviation is 0.163, 0.147 and 0.107, respectively.

4.2. Performance Analysis

To demonstrate the achieved image quality, we analysed the energy concentration in the point image. The reference element is $11\times 11$ $\mathsf{\mu }$m, which corresponds to a typical display pixel size [11,17]. Additionally, the energy concentration was compared for $22\times 22$ $\mathsf{\mu }$m (i.e., $2\times 2$ pixels) area. The ensquared energy plots for all the configurations are shown in Figure 4.

The grid of subplots clearly demonstrates the advantages of introducing the image surface curvature and being able to variably adjust it. For example, the static curved (“CS”) image surface allows increasing the energy concentration for the FoV corner by factor of 1.22. The variable curvature (“CV”) solution makes the values higher by a factor of 1.54 and allows maintaining them with a large refocusing.

We may note here that some secondary performance targets were not included directly into the merit function during the optimisation, yet some gain was obtained. For instance, the “CS” demonstrates significant advantage in the relative illumination and the “CV” configuration shows a good illumination stability with refocusing. At the same time, the image distortion for these configurations appears to be slightly higher than that for the “FS”. However, the distortion values remain low for all configurations. The exact numbers are summarised in

Table 2. Auxiliary imaging performance indicators.

Distance, m	Distortion, %	Min. rel.illum.
	CV
1.5	13.2	62.0
5	13.7	60.6
inf.	13.9	63.6
	CS
1.5	12.1	79.0
5	12.1	78.3
inf.	12.1	83.4
	FS
1.5	9.2	57.8
5	9.3	59.0
inf.	9.3	69.4

.

5. Variable Curvature Display Feasibility

The image surface curvature in the designed lens varies from $-110$ to $-138.7$ mm when refocusing from infinity to $1.5$ m. The obtained ratio of the display size and curvature is close to the current limit of the curving technology [18], defined by the die breakage limit. Thus, additional modelling was performed in order to demonstrate the design feasibility.

We estimated the feasibility of variable curvature displays parameters relying on the theoretical basement described in [19] and using finite element analysis (FEA) according to the method presented in [20,21]. From this study, to reach the required radii of curvature prescribed from the optical design above, we calculated that the root mean square surface error we can reach is lower than 20 $\mathsf{\mu }$m. The difference between the desired and actual surface shape along the diagonal is shown in Figure 5. It is compared to the depth of focus, which equals $\pm 68.2\mathsf{\mu }$m for the lens with $F/\#=6.2$ and $11\times 11\mathsf{\mu }$m pixel. Thus, the calculated display surface shape can be reproduced with a satisfactory precision, although it may be improved by means of a proper mechanical design.

The corresponding amount of von Mises Stress generated in the display is lower than 130 MPa, while the breakage limit of silicon is at least 560 MPa [22]. Thus, the safety factor is about ${4.3}^{\times }$.

In addition, we performed a simple test to estimate the required precision and stability of the curving force. We computed the tolerances on the design parameters, including the display shape. Assuming that a $10\%$ increase of the aberrations is acceptable, we found the tolerance on the display radius equal to $\pm 1.2$ mm. It corresponds to the precision of the curving force within $\pm 1.1\%$. Thus, the display actuator should have a high accuracy and the display should be well isolated from all other external loads. The latter point should be accounted for in the optomechanical design, which is out of the subject of this study.

This indicates that the proposed design with variable focal plane curvature can be implemented with the known curving technique. However, we must note that this modelling is preliminary and it is intended only for estimation of the safety factor and the precision requirements. Development of an actual curving setup would require some advanced simulations and additional studies, including nonlinear effects and consideration of technological limitations.

6. Prospective Application

The primary application of the proposed solution is VR/AR helmets. A possibility to project the image at a desired position in depth without a change in the image quality would open new prospects in preparation of the content and interaction with the viewer, especially in VR movies. Figure 6 shows the optical system model overlapped on the human head model and demonstrates the dimensions and approximate packaging of the helmet. Here, we note that the display used in the design and simulations has a diagonal of 2${}^{″}$ with the sides ratio of 4:3. The sizes of currently available OLED displays are typically smaller [23,24,25], but the displays made with this technology [26] may reach larger sizes in the nearest future.

To assess the imaging performance, we performed a simulation in the direct propagation mode (i.e., from the finite distance to infinity). The simulation results are presented in Figure 7. One can see that the system provides relatively high image quality even at the FoV edge and the image distortion can be easily removed by pre-processing the projected image. The optical system has a “pincushion type” distortion, thus a distortion with the opposite sign, i.e., “barrel type”, should be introduced into the projected image to compensate.

7. Conclusions

We propose a concept, a design algorithm and an example design of projection system with a variable curvature display and show its performance advantages and physical feasibility.

The design algorithm uses Seidel aberration theory to find the initial approximation and a multi-configuration two-step numerical optimisation to finalise the design. It was demonstrated that application of this algorithm allows designing a projection system with $F/6.2$ aperture and ${90}^{°}$ field of view, which is able to re-focus the projected image from infinity to 1.5 m. The energy concentration in a point source image is higher by a factor of $1.54$ in comparison with a system with flat focal surface. A preliminary modelling shows that a silicon-based display can be curved to the calculated focal surface radius with the breakage safety factor of ${4.3}^{\times }$.

The developed optical system, as well as the corresponding design and optimisation approach, can help to improve image quality of the future VR/AR headsets and implement an operational mode with re-focusing. However, the prospective application area is much wider and can include, e.g., the design of multimedia projectors and laser scanning systems.