1. Introduction
Advances in 3D scanning technology have been widely applied to object acquisition due to the abundance of low-cost RGB-D sensors. Translucent objects are very common in realistic domestic environments, and by placing a scanner at strategically selected positions, geometric details can be captured with thoroughness and high fidelity. However, in rendering the complex scanned data of objects made from translucent materials, blurred geometric details become more pronounced as the illuminant direction approaches a horizontal orientation. According to previous works [
1,
2], highlights can contribute to the visual impression of translucency and observers can use the shape information of highlights alone to estimate curvature magnitude. In 3D color printing, a high level of detail and realism should be exhibited for the purpose of reproducing complex appearance properties and more control over the internal structure [
3]. To preserve subtle geometry details in the appearance of translucent materials, we developed a method that combines subsurface scattering approximation with generated highlight effects. We applied the directional dipole model with slant illuminant angles to the highlight-generation method and successfully maintained complicated geometry details in the rendering results for translucent materials.
Currently, many specular highlight component expressions of the BRDF (Bidirectional Reflectance Distribution Function) model employ a local orthonormal frame to calculate the highlight value. The tangential vectors of the local orthonormal frame are important for the shape of the generated highlight. Many methods used tangent vectors to simulate highlight and produce special visual effects.
Various approaches [
4,
5,
6] used the frame
to present the directions of the anisotropic highlight, and they showed that surface positions contributing to specular highlights are in planes orthogonal to the half-way vector.
The visual appearance of specified materials and the dimensions of the physical structure are related to the chosen local orthonormal frame.
Brandt and Scandolo [
7] used spectral methods to represent and process tangential vector fields on surfaces, allowing for real-time fur editing on surface meshes with global smoothly varying hair direction. Zinke and Weber [
8] used a fiber’s local coordinate system to parameterize light scattering in a filament. They reproduced view-dependent translucency and global highlights in hair fibers. Irawan and Marschner [
9] described the appearance of woven cloth that is parameterized by the local orthonormal frame
. The curvature of the yarn segment is controlled by the spine’s tangent vector, and the circular cross-section for each
is featured by
and
. Ament and Dachsbacher [
10] made an orthonormal basis at each shading point and in the tangent plane. The generated specular highlight visually emphasized the overall salient structure of the ellipsoids.
These models can describe reflectance on a surface or scattering inside a medium by analyzing specular reflection on specific structure models such as a cylinder, a curved cylinder, or voxels stored in a lattice. However, they are not well suited to surfaces that have complex geometry details.
Methods to add highlight effects on translucent materials have seldom been reported in the literature due to light scattering away inside a medium. Fleming and Bülthoff [
11] used full-scene illumination from Debevec’s light-probe database [
12] to add highlights on translucent materials. Motoyoshi [
13] also employed the lighting environment used for Debevec’s light-probe high dynamic range images [
12] to add natural highlights on translucent materials. Wang et al. [
14] added surface shading to heterogeneous translucent materials by using the Cook-Torrance model. In these methods, highlights and subsurface scattering approximation are carried out in two distinct light transport processes and thus do not affect each other.
Some computer vision methods focus on extracting specular components from captured images. The main idea is to separate the global illumination effect and the directional reflection component to obtain the meso-structure of translucent objects and detailed surface normal vectors.
Chen et al. [
15] combined polarization-difference imaging with phase shifting to separate all traces of subsurface structures and recover the geometry of translucent objects. They improved upon this by using modulated phase shifting to separate direct lighting and global light transport [
16]. They could more efficiently filter out subsurface scattering and inter-reflections that are still contained in traditional phase shifting.
The drawback of these methods is that they are essentially BRDF-based highlight-generation methods, in addition, they cannot completely separate the subsurface scattering component and the direct-illumination component, and the residual direct component may be too low and noisy compared with the translucent surfaces.
Kim and Ghosh [
17] successfully exploited polarized light-field imaging solutions to separate diffusion and specular reflection of human skin, but their method can only process highlights located in inner region of the scattering image.
The method of Sun et al. [
18] is based on color information completely without geometrical details of the surface, their method does not depend on polarization, and can remove strong specular highlight, but is tailored for overall smooth regions.
Photometric stereo can also be used to recover surface details for translucent materials [
19,
20]. However, these algorithms are operated on near flat regions, and so they are not suited to the sharp geometry features of translucent objects.
Our method simulates the highlight effects on the approximation of subsurface scattering in rendering translucent materials. It can generate highlight effects in the appropriate position for normally shaped surfaces. Moreover, the subtle geometry details are preserved while the translucency is well maintained in the rendering results, and the method can also improve the translucency effects in edge regions.
2. Background Theory
The approach of rendering translucent materials using subsurface scattering is based on the radiative transfer equation. Several BSSRDF (Bidirectional Scattering Surface Reflectance Distribution Function) models are available to estimate subsurface scattering, including the standard dipole model [
21], the quantized diffusion model [
22], the better dipole model [
23], and the directional dipole model [
24]. We chose the directional dipole model for rendering translucent materials because it considers the directions of incoming and outgoing light of the light-scattering process in the estimation of BSSRDF, and it also captures translucency effects that are present in the full path tracing and geometry details by directional illuminant light, both of which remain absent in other models.
For generating highlight effects, we employ the specular component expression of the Ward [
25], Ashikhmin [
26], and Lafortune [
27] models because they all apply a local orthonormal frame and bandwidth parameters to control the shape of highlights. When computing highlights, the NDF (normal distribution function) is the main source in simulating special highlight effects in microfacet models. By filtering NDF, Kaplanyan [
28] found small highlights across an entire pixel footprint. Guo et al. [
29] introduced the extended NDF to express visually perceived roughness due to reflections and refractions. In microfacet theory, M is a function combining Fresnel, shadowing and masking term, which represents the projected area that is visible from both the incoming and outgoing direction. Progress has been made on modeling multiple specular reflections in the v-groove cavity. Lee et al. [
30] defined the shadow masking term to show single scattering and multiple scattering inside the v-groove, their result illustrates a more specular appearance due to multiple scattering inside v-groove for high roughness values. Xie and Hanrahan [
31] derived the formula for M and NDF, with the generated highlight, their BRDF eliminated the dimmer appearance of rough surfaces and took on a more saturated color in the rendering images. The shape and intensity information of highlights are highly related to the directions of the local orthonormal frame. For objects placed in special positions, for example, the hairs located along the surface tangent vector [
4], the surface positions that can contribute to specular highlight lie in planes orthogonal to the half-way vector of the incoming and outgoing light direction; these are highlights that occur when the half-way vector is perpendicular to the surface tangent vector. When more complex shapes, such as a cylinder or a curved cylinder, are used to model an object’s shape, the local orthonormal frame describes the scattering geometry of light entering or exiting the medium, and the highlight position is distributed on the elevation and azimuthal angles of the illumination directions.
3. Highlight Effects Generation Methods
Our method is inspired by the concept of using a minimum enclosing cylinder to model a fiber for approximating outgoing radiance at a surface patch from incoming irradiance at another patch by scattering effects [
8]. We employed a directional dipole model to render the translucent appearance due to subsurface scattering of light from the incident point to the exitant point. Our goal is thus to define the highlight effects using the incident and emergent light at two different points in the approximation of subsurface scattering. By the addition of the generated highlight effects, the blurry structural details of translucent objects should become clear in the rendering result.
The geometry of light transport is shown in
Figure 1. Here, an incident light from direction
enters the material at surface point
, gets refracted and scattered inside the medium, and goes out at exitant point
.
We define the highlight effect that one would observe from at exitant point in the subsurface scattering process, in which the directional light is incident on from . The idea behind this is that BSSRDF and BRDF both describe the observed radiance contributed by the incident irradiance. In BSSRDF, the contribution is by subsurface scattering, while in BRDF it is by reflection. We use the specular highlight expression of three BRDF models to calculate our highlight effects.
Our method consists of four steps: (1) defining the local orthonormal frame, (2) choosing the illumination directions, (3) selecting the highlight component expression, and (4) generating highlight effects by a Monte Carlo method for rendering translucent materials.
3.1. Defining the Local Orthonormal Frame
We define half-way vector
as the normalized vector of the sum of incoming light
at incident point
and outgoing light
at exitant point
.
We employ the local orthonormal frame at in the highlight-calculation expression. To simplify the calculation, we define as follows: is the normal vector at , is the axis of the Cartesian coordinate system, is the cross-product of and , and is the cross-product of and .
The definition of and the choice of position to define are important in our method. When is close to , we assume the change in viewing vectors from these two points is trivial and use the viewing vector to calculate the highlight effects at .
In the calculation, and are the bandwidth parameters used to control the projection magnitude in the and directions. By using a different combination of bandwidth parameters, we can obtain varying highlight shapes for areas with complex geometry details and, moreover, enhance the visual perception of details in these areas.
3.2. Choosing the Illumination Directions
We generate highlight effects from different incident illumination directions. When we use the directional dipole model to approximate the diffusive part of the BSSRDF, the calculation process assumes that the diffusive light at the exitant point no longer depends on outgoing direction . To effectively analyze the complex shape of the generated highlight behavior, we choose elevation and azimuth angles of the incoming illumination and different bandwidth parameters and to generate highlight effects. We set fixed elevation and azimuth angles for different sampled incident points.
3.3. Selecting the Highlight Component Expression
Based on the microfacet model, the specular highlight component
of BRDF or BSDF (Bidirectional Scattering Distribution Function) can be expressed by
where
is a function combining Fresnel and shadowing terms and
is the normal distribution function.
For visualizing different appearance effects, we employ the specular highlight component of the Ward, Ashikhmin, and Lafortune models to generate the highlight effects. The normalized tangent vectors
and
specify the principal directions of the highlight. The Ashikhmin and Ward models are microfacet models, and they require parameters
,
to control the projection of
to tangent vectors
and
. Specifically, for the Ward model, we use this expression:
For the Ashikhmin model, we use the highlight expression with
where
is a Fresnel term that controls the proportion of reflected light.
For the Lafortune model, we need
to control the shape of the highlight, and the expression is
where
is incoming light
defined in the local orthonormal frame
,
is outgoing light
defined in
, and
is the exponent parameter that determines the size and shape of the highlights.
3.4. Generating Highlight Effects by A Monte Carlo Method
Since the subsurface scattering decreases exponentially with distance of the incident and exitant points, we multiply an exponential term to constrain the highlight effects within a small-scale distance.
In our rendering of translucent materials, the BSSRDF is implemented in a Monte Carlo ray tracer. We fix
and sample different incoming points
to approximate the highlight effects at
. We then estimate the highlight effects at
by averaging the highlight values of
. In summary, for methods based on the Ward and Ashikhmin models, the highlight effects’ expressions at
in a Monte Carlo estimator are defined as
for method based on the Lafortune model, the highlight effects’ expression is
where
is the sampling number of incident lights,
is the probability of sampling a mesh point on the model surface, and
is the extinction coefficient of the rendering material. The generated highlight effects by the Monte Carlo estimator should be added to the color value of the final pixel that is estimated by the directional dipole model. The proposed method is reciprocal for the three selected highlight component expressions.
4. Experiment Setup
The proper choice of the illuminant direction is important for preserving details when adding highlight effects. We set the upper hemisphere with its origin at the surface incident point and its north pole as vector
(Z axis). Therefore, in spherical coordinates, the illuminant direction can be described by elevation angle
and azimuthal angle
. To choose the incident illuminant directions of the incident points, we substitute the object surface that is visible to the viewer with the XOY plane in a simplified way. According to Adams and Elder [
32], as the illuminant direction changes from directly overhead to horizontal, the shape becomes more ambiguous and the perceived convexity due to the highlight effect becomes pronounced. To verify this phenomenon, we choose six elevation angles of 10°, 25°, 40°, 55°, 70°, and 85°. Because the general ambiguous condition of the details is not evidently affected by the azimuthal angle, we choose an azimuthal angle of 25° in our first experiment (results in
Figure 2).
To compare the results of the generated highlight effects by our model and those by traditional models, we used parameters , , which is a nearly horizontal illumination direction, for the incident light direction in the following experiments. The viewpoint is fixed, and the exitant light direction is from the exitant point to the eye. In the computation of specular highlight, we make sure the angle between the incident light and the normal vector at the incident point is less than 90°. For some small incident elevation angles, the angles are more than 90°, so the highlight result is set to zero in these regions.
We used Stanford bunny object with two materials which are from Jensen et al. [
21] and Narasimhan et al. [
33] in our experiment: marble and chocolate milk (regular). Our method can preserve the details of both the solid and liquid translucent material. In all experiments, the resolution of the resulting image is 512 × 512 pixels, and the maximum intensity of the generated highlight is set to 0.35. For marble material, in
Figure 3 and
Figure 4, the selected region (red closeups) has 1440 pixels, in
Figure 5, the selected region (purple closeups) has 20,720 pixels. For chocolate milk (regular) material, in
Figure 6,
Figure 7 and
Figure 8, the selected regions are the same as in the marble material. In every row of
Figure 3,
Figure 4,
Figure 6 and
Figure 7 from left to right, the
,
parameters are
is 2,
is 2;
is 2,
is 10;
is 10,
is 2;
is 10,
is 10. In the results of Lafortune-based methods (
Figure 5;
Figure 8), the exponent parameter
k is 10.
is 1, and, from left to right, the
,
parameters are
is 0.2,
is 0.5;
is 0.5,
is 0.2;
is −0.2,
is −0.5;
is −0.5,
is −0.2.
6. Discussion
Qi et al. [
34] investigated five image statistics of highlight areas to show how these metrics are related to the perceived gloss. We chose two metrics: (1) number of pixels, which corresponds to "percentage of highlight area" and, notably, is highly correlated with perceived gloss in the work of Qi et al. [
34]; (2) average intensity of generated highlight, which is used to analyze the perceived quality of the covered regions. We set the intensity threshold value to 0.20, and in
Table 1 and
Table 2, Number means the number of pixels that have intensity above the threshold value, while Intensity means the average intensity of highlight pixels that have an intensity value higher than the threshold value.
For methods based on the Ward and Ashikhmin models, we chose the left edge of the bunny leg’s for numerical analysis (red closeups in
Figure 3,
Figure 4,
Figure 6, and
Figure 7). From
Table 1 and
Table 2, the proposed method generated stronger highlight effects in number and average intensity at this edge, and the shape of this edge is well preserved and average intensity at this edge, and the shape of this edge is well preserved and visually enhanced. On the contrary, in the results generated by the methods based on the traditional Ward and Ashikhmin models, the values are smaller and the highlight effects at this edge are not as obvious as in the proposed method.
For chocolate milk (regular) material, the results of the generated highlight show similar effects as those for the marble material. From the “shadow-haze” view described in the work of Fleming and Bülthoff [
11], our method generates a better cue for the perception of translucency, that is, the superimposition of a sharpened edge while blurring the combined translucent colors.
For methods based on the Lafortune model, we chose the bunny’s leg region for numerical analysis (purple closeups in
Figure 5;
Figure 8). From
Table 1 and
Table 2, in the proposed method, when
is 0.2,
is 0.5, and when
is 0.5,
is 0.2, and it is evident that the values of number and average intensity are smaller than those in cases when
is −0.2,
is −0.5 and when
is −0.5,
is −0.2. This can be seen from the weaker appearance intensity of the preserved bunny leg’s details in the rendering results; In the results of methods based on the traditional Lafortune model, when
is 0.2,
is 0.5 and when
is 0.5,
is 0.2, and the values of number are distinctly larger and the values of average intensity are a little smaller than the values in cases when
is −0.2,
is −0.5 and when
is −0.5,
is −0.2.
We select a vertical line of pixels that is in the middle of the red closeup region (
Figure 3,
Figure 4,
Figure 6 and
Figure 7) to study the improvement of translucency by the generated highlight. In
Figure 9 and
Figure 10, the red line shows approximated subsurface scattering without adding highlights, and the black line shows approximated subsurface scattering with highlights. Our methods and the traditional methods all maintain the peaks of the subsurface scattering values after adding the highlight values. In our methods, the generated highlight has strong values when the value of the subsurface scattering is low, on the other hand, in the traditional methods, the generated highlight has relatively weaker values when the value of the subsurface scattering is low. For the evaluated positions overall, our method has stronger generated highlight values, and the range (difference between maximal and minimal values) of the subsurface scattering values becomes smaller after adding highlight values.
7. Conclusions
We presented a method to preserve surface details by adding highlight effects in the rendering of translucent materials. The method employs the light paths of the incident and exitant points in the light-scattering process, uses the local orthonormal frame in the incident point, combines three highlight-generation models that employ local tangent vectors, generates highlights in appropriate positions, stores surface details under different conditions while maintaining the original material’s translucency, and enhances the translucency in edge regions. All these features help to prove the validity of our method.
We compared the results of the proposed method and traditional models in three highlight expressions. Generally, the proposed method exhibits rich visual effects in regions with complex surface details and geometry features, and it increases the perception of translucency in those regions. Under some conditions, the number and average intensity of the generated highlight effects by the proposed method are smaller than the results generated by the traditional models, i.e., lower values, and the region’s details are less clear than those for higher values. However, the proposed method has the advantage of avoiding excessively bright or metal-like highly specular appearances in the rendering results.
Our method works with a single-direction light, and it can be incorporated in applications that need a fast solution. However, due to the restrictions on incident light directions, in some positions, where the angle between the incident light and the normal vector is above 90°, the highlight value is clamped, so it cannot produce highlight effects in these positions. In the future, we plan to study the relation of the generated highlight position and the local orthonormal frame. Consequently, by applying proper tangent vectors, it would be possible to produce specific highlight effects that are in accordance with a specific geometric shape.