Keywords

1 Introduction

Physically-based rendering (PBR) is basically a lighting technique that simulates a photo-realistic scene with high fidelity for a computer game, virtual reality, cinematic view, and design applications. More concretely, a scene is composed of a set of pixels with color values that can be determined by combination of quite complicated procedures such as a location of light, material property, smoothness of surface, and a location of viewer etc. To accomplish PBR, material rendering parameters have to be measured in real [1]. In usual, a task for extracting optical parameters basically require critical processes including taking of High Dynamic Range Image (HDRI) and extracting optical parameters. Each clay material has its own material-rendering properties such as scattering, absorption, Fresnel and roughness etc.

In addition, virtual reality (VR) has been gradually highlighted recently in lots of applications like an educational content, entertainment areas and various games. PBR plays a significant role in boosting immersion in VR, in which PBR helps a user to feel more realistic while running applications.

To our knowledge, no literature has been found that deals with PBR and clay art rendering. Hence, in this paper, authors propose an efficient physically-based clay art rendering scheme that reduces the execution time of acquiring optical parameters. To this end, we actually measure only a clay color using spectrophotometer and compute optical parameters without going through acquiring a raw HDRI and a sampling process that seems to be daunting tasks because it is likely a time-consuming process [2, 3].

Finally, we synthesize a few virtual clay models that are rendered by our physically-based approach. Also, we have carried out the comparisons of experiments to validate the method.

2 Method

2.1 Clay Materials

There are various kinds of clay used in clay art depending on the composition and properties. The mud clay, which is composed of basic particles, is highly viscous and flexible, and can be deformed by various models. Paper clay is made of clay material with paper fiber added. The initial color is white and color can be painted. Color clay has very high viscosity between clays. Cork clay is composed of materials using bark, not clay, and has similar texture to wood.

In addition, ball clay is composed of styrofoam particles and is painted with colors. Icing clay is a very low viscosity material and is suitable for expressing soft parts such as cream. The shape of the individual clay is shown in Fig. 1.

Fig. 1.
figure 1

Clay classification (Color figure online)

Full size image

2.2 Optical Parameters Computation

In order to create a photo-realistic synthetic image, we must actually observe and reflect the interaction of light with the physical properties of the material that constitutes the object when the light is illuminated by the material. The light and interaction are different depending on the materials such as opaque material, translucent material, transparent material, perfect reflector, and complete absorber. PBR is a technique to calculate the color value of a pixel observed in our eyes at the end of the rasterization process. At this time, there are objects constituting a scene, and each object is made of a material. The input values of PBR include light position, normal vector of surface, optical parameter (Fresnel, scattering coefficient, absorption coefficient) which represents physical characteristic of material. The input value is used to calculate the color value of the pixel finally observed in the user’s field of view. Generally, PBR performs rendering using optical parameters of objects.

In order to extract the optical parameters, several Low Dynamic Range Images (LDRI) are photographed using a spectrophotometer and optical imaging equipment, which is a customized camera, and a single HDRI is created by sampling, and optical parameters are calculated by curve-fitting. However, this method requires a complicated process and a lot of calculation time. Thus, we need to find a way to efficiently calculate optical parameters.

The following Eq. (1) is the formula to calculate the intrinsic color of an object using extracted optical parameters [4].

$$ \begin{aligned} R_{d} & = (1 - F_{dr,t} )(\int_{0}^{d} {\left( {1 - \sigma_{a} x} \right)^{2} dx\,+\,F_{dr,b} \left( {1 - \sigma_{a} d - \sigma_{s}^{ '} P\left( g \right)d} \right)^{2} )} \\ & = \left( {1 - F_{dr,t} } \right)\left( {\sigma_{s}^{ '} {\text{P}}\left( g \right)(d - \sigma_{a} d^{2} + \frac{1}{3}\sigma_{a}^{2} d^{3} )\,+\,F_{dr,b} \left( {1 - \sigma_{a} d - \sigma_{s}^{ '} P\left( g \right)d} \right)^{2} } \right) \\ \end{aligned} $$
(1)

where \( \sigma_{a} \) and \( \sigma_{s}^{{^{{\prime }} }} \) are the absorption and reduced scattering coefficients of the material respectively. P(g) is the integration of the phase function for backward scatter, and d is the depth. \( F_{dr.t} \) represents the diffuse Fresnel reflectance at the upper boundary. \( F_{dr,b} \) represents the diffuse Fresnel reflectance at the lower boundary.

We transform the Eq. (1) into the Eq. (2) in terms of \( \sigma_{a} \), the absorption coefficient and \( \sigma_{s}^{{^{{\prime }} }} \), the reduced scattering coefficient.

$$ \alpha \sigma_{s}^{{^{{\prime }} 2}} + \beta \sigma_{s}^{{^{{\prime }} }} + \gamma \,\sigma_{s}^{{^{{\prime }} }} \sigma_{a} + \delta \,\sigma_{a} =\upvarepsilon, $$
(2)

where

$$ \begin{aligned} \alpha & = d^{2} P(g)^{2} F_{dr, b} , \\ \beta & = P\left( g \right)d(1 - 2F_{dr, b} ), \\ \gamma & = P\left( g \right)d(2F_{dr, b} - d + \frac{1}{3}\sigma_{a} d^{2} ), \\ \delta & = F_{dr, b} d( - 2 + \sigma_{a} d),\,\upvarepsilon = \frac{{R_{d} }}{{(1 - F_{dr, t} )}} - F_{dr, b} \\ \end{aligned} $$

We can use the spectrophotometer to measure the value of the material’s unique color, R(d). In this case, \( F_{dr,t} \) and \( F_{dr,b} \) can be obtained by using the formula derived from R(d). The value of R(d) in Eq. (2) is known from the spectrophotometer, \( F_{dr,t} \) and \( F_{dr,b} \) can also be derived. P(g) and d are also known variables. Therefore, \( \sigma_{a} \) and \( \sigma_{s}^{{^{{\prime }} }} \) can be obtained by using a minimum mean square error (MMSE) estimator.

Table 1 below shows the optical parameters extracted by directly shooting and sampling the HDRI and the optical parameters obtained by the proposed method. As shown in Table 1, we carried out experiments on four clay materials. Optical parameters from the sampling method were obtained through a curve-fitting of samples on HDRI. We discover that the numerical values of the optical parameters between the sampling method and our method do not seem differentiable.

Table 1. Reduced scattering coefficient \( \sigma_{s}^{{^{{\prime }} }} \) and absorption coefficient \( \sigma_{a} \)
Full size table

3 Experiments

The following environment was set up to evaluate the proposed method. A desktop computer installed with Windows 10 64-bit and a GeForce GTX 1060 as graphics card was used. An Oculus Rift DK2 model was used for the VR HMD. Unity 5.4.1f. and shader code was used to implement rendering results while C# was used to compute optical parameters and to develop user interface and HMD VR facilities shown in Fig. 2.

Fig. 2.
figure 2

Clay rendering interface with HMD VR

Full size image

3.1 Comparisons of the Rendering Results

To validate our method, we have carried out the comparative rendering experiments that show BRDF (Bidirectional Reflectance Distribution Function), the sampling and our methods. Four clay materials were tested to obtain optical parameters. For BRDF rendering [6], four texture images were acquired from taking photos of four clays. And the rendering equation [4] with optical parameters as input values shown in Table 1 was employed to synthesize PBR character models. Our findings are both the sampling and ours are quite similar while BRDF rendering results are less resemble as shown in Table 2.

Table 2. The comparisons of rendering results from BRDF, sampling and ours
Full size table

3.2 Analysis of the Execution Time for Optical Parameter Calculation

An optimization strategy was investigated in Ref. [2] by showing that four LDRIs was good enough to create one HDRI to sample and extract optical parameters. Exposure Fusion [5] algorithm was used to build a HDRI on which a focusing light point was found. The radius from the center to the boundary was 5 mm with 20 pixels/mm, of which total samples was 300 samples (RGB). Given the sample points, a curve-fitting algorithm was computed to acquire optical parameters.

The proposed method as mentioned before is not necessarily going through some daunting processes such as taking LDRI and a sampling procedure to obtain optical parameters.

As shown in Fig. 3, the execution time to compute optical parameters is found about 0.040 s on average while the sampling method take 0.748 s on average that does not even include some tasks for taking LDRIs and merging them into HDRI. In short, this indication represents the proposed method enhances technical merits for PBR without losing high fidelity rendering results as well.

Fig. 3.
figure 3

Plot of the execution time between two methods

Full size image

4 Conclusion

In the previous research, the preprocessing process, which is a daunting task, is indispensable to measure optical properties of materials and extract optical parameters from acquired HDRI. This preprocessing process must take the steps of HDRI imaging, sampling, and optical parameter extraction. Therefore, in this study, we proposed an efficient method to calculate the optical parameters of an object using spectrophotometer and algorithm without going through optical parameter extraction process accompanied with physical measurement process. In order to add the interest and immersion of the user, we provided a HMD VR facility to experience clay art rendering as well.