Elsevier

Computers & Graphics

Volume 34, Issue 2, April 2010, Pages 145-157
Computers & Graphics

Technical Section
A perceptual approach for stereoscopic rendering optimization

https://doi.org/10.1016/j.cag.2009.11.004Get rights and content

Abstract

The traditional way of stereoscopic rendering requires rendering the scene for left and right eyes separately; which doubles the rendering complexity. In this study, we propose a perceptually-based approach for accelerating stereoscopic rendering. This optimization approach is based on the Binocular Suppression Theory, which claims that the overall percept of a stereo pair in a region is determined by the dominant image on the corresponding region. We investigate how binocular suppression mechanism of human visual system can be utilized for rendering optimization. Our aim is to identify the graphics rendering and modeling features that do not affect the overall quality of a stereo pair when simplified in one view. By combining the results of this investigation with the principles of visual attention, we infer that this optimization approach is feasible if the high quality view has more intensity contrast. For this reason, we performed a subjective experiment, in which various representative graphical methods were analyzed. The experimental results verified our hypothesis that a modification, applied on a single view, is not perceptible if it decreases the intensity contrast, and thus can be used for stereoscopic rendering.

Introduction

Technologies underlying 3D autostereoscopic displays have matured to the point that several commercial products are now available in the mass market. These displays are an extension of the conventional 2D displays, by their ability to emit a different image for each eye. Binocular head-mounted displays have also matured to the level that they are widely used in a number of applications. The main difficulty of these stereoscopic and autostereoscopic displays is that they require a rendering phase for each view, which multiplies the rendering time by the number of views. Consequently, there is a need to optimize solutions for stereoscopic rendering.

The traditional way of stereoscopic rendering is to handle the left and the right eye views separately, which is still the model in use in graphics APIs such as OpenGL [1]. A number of stereoscopic and multi-view rendering techniques have recently been proposed. These approaches can be categorized as pipeline-based solutions, which aim to optimize the rendering on the rasterization stage of the rendering pipeline [2], [3], [4]; and image-based solutions, where one view is rendered using the graphics rendering pipeline, and the other view is generated from this image, using the correspondences of the two views [5], [6], [7].

In this paper, we propose a new perceptually-based solution for optimization, by utilizing the suppression theory of binocular vision. According to the Binocular Suppression Theory, the less dominant view will be suppressed by the dominant one; and when the images from corresponding regions differ in an appropriate way, they fuse but the disparities are registered and used for impression of depth. It has been shown that the result of which view suppresses the other depends on the visual properties of the two images [8], [9]. Section 2.2 overviews the Binocular Suppression Theory that our solution is based on.

We investigate how binocular suppression mechanism can be utilized for optimization, by comparing the effects of different graphics rendering and modeling methods. Our aim is to identify the rendering and modeling features that do not affect the overall quality of a stereo image pair when simplified in one view. We applied our approach to a number of representative and commonly used methods used in rendering, including framebuffer upsampling, mixed-level antialiasing, specular highlight, mixed shading, mesh simplification, texture resampling, and mixed shadowing. We performed an experimental study in order to evaluate each method’s perceptual effect on the overall perceived 3D image, in terms of quality, sharpness, depth, and comfort. The experimental results show that the overall perceived stereo image quality is not affected when one of the views is modified by a technique that decreases the intensity contrast. On the other hand, when a modification that increases the intensity contrast is applied on a single view; it will be visible and the overall perceived stereo image quality will be affected towards the modified image.

The main contributions of this study are as follows:

  • a new approach for stereo rendering optimization, based on the usage of different stimuli, each with different quality, for each eye,

  • a content creation guideline, describing when it is appropriate to use each optimization,

  • a formal experimental study to verify the proposed hypothesis.

The rest of the paper is organized as follows. First, we survey previous work on stereoscopic rendering methods and Binocular Suppression Theory. Then, we explain our perceptually-based approach and the graphical methods we have used. Lastly, we provide the experiment design and our analysis of the results.

Section snippets

Stereoscopic rendering optimization

A number of techniques have been proposed to optimize stereoscopic rendering. The first group of solutions follows a graphics pipeline-based approach, by utilizing the coherence between neighboring views. Adelson and Hodges [10] simultaneously render a triangle to both images by using the x-axis coherence in device coordinates to accelerate the stereoscopic rendering process. Kalaiah and Capin [4] propose a GPU-based solution that reduces the number of vertex shader computations needed for

Approach

In this paper, we present a Binocular Suppression Theory based approach to stereoscopic graphics rendering. The proposed method exploits the fact that the overall perception of the stereo pair in a region will be determined by the dominant image on the corresponding region, instead of summation of the effect of two images. Our goal is to explore how the rendering quality can be reduced in the suppressed view, without reducing the overall perceived quality of the rendered 3D image. If such

Experiment

We have implemented the proposed methods, and performed a formal experiment to observe whether the use of each method is perceptible. We have decided to base our work on users’ subjective ratings, instead of objective evaluation in which users perform a task (such as measuring the time and error when placing an object at a depth with respect to another object). Our subjective evaluation may cause rater bias in the results due to the individual characteristics of the subjects; however, our

Experimental results and discussion

To determine the difference between the reference and test content, the Test minus Reference score was used. A score of zero means that the test sequence was rated equivalently to the reference sequence, and a negative score means that the test content was rated lower than the reference content. Error bars in the figures below show the 95% confidence interval of the mean, which corresponds to the range within which the mean is expected to fall with 95% certainty. Data points in the

Performance results

In this section, we further demonstrate the performance gain of the mixed-stereoscopic rendering approach, using the methods indicated as feasible in Table 4.

The performance gain of the mixed stereo approach over traditional stereoscopic rendering depends on the choice of the method for rendering the scene. Using advanced rendering techniques, such as BRDF, area light sources, anisotropic texture filtering, etc. will increase the advantage of our approach over the traditional approach. However,

Conclusion & future work

In this paper, we have presented a perceptually-based optimization approach for stereoscopic rendering, which makes use of the binocular suppression mechanism of the human visual system. The proposed method exploits the fact that the 3D perception of the overall stereo pair in a region is determined by the dominant image on the corresponding region, instead of summation of the effect of two images. We have also introduced an estimate of the strength of a view, called intensity contrast, and

Acknowledgments

The authors were supported by the European Commission FP7-213349 All 3D Imaging Phone project and TUBITAK. Also, we would like to thank all the participants of the subjective experiment described in this work and Onur Kucuktunc for his help on preparation of the test videos.

References (36)

  • M Wan et al.

    Interactive stereoscopic rendering of volumetric environments

    IEEE Transactions on Visualization and Computer Graphics

    (2004)
  • C. Fehn

    Depth-image based rendering (DIBR), compression and transmission for a new approach on 3DTV

    SPIE

    (2004)
  • IP Howard et al.

    Binocular fusion and rivalry

    Binocular Vision and Stereopsis

    (1995)
  • R. Blake et al.

    Visual competition

    Nature Reviews Neuroscience

    (2002)
  • S Adelson et al.

    Stereoscopic ray tracing

    The Visual Computer

    (1993)
  • L Zhang et al.

    Stereoscopic image generation based on depth images for 3D TV

    IEEE Transactions on Broadcasting

    (2005)
  • Halle M. Multiple viewpoint rendering. International Conference on Computer Graphics and Interactive Techniques, 1998;...
  • T He et al.

    Fast stereo volume rendering

    IEEE Visualization

    (1996)
  • Cited by (10)

    • Visual perception of computer-generated stereoscopic pictures: Toward the impact of image resolution

      2021, Signal Processing: Image Communication
      Citation Excerpt :

      The essence of binocular rivalry [5] is that when a stimulus is communicated to the suppressed eye, the activated neurons have an inhibitory effect on the visual center, resulting in one eye suppressing the other visual perception. However, binocular suppression [6] is a special condition of binocular rivalry when the visual stimuli no longer appear alternately between the left and right eyes; when two eyes see a similar image, the eye that sees a high-quality image will dominate, thereby suppressing the image of the other eye. The image reflected in the brain is ultimately a high-quality image, as shown in the right portion of Fig. 1.

    • Improved gloss depiction using the empirical highlight un-distortion method for 3D image-warping-based stereo rendering

      2015, Computers and Graphics (Pergamon)
      Citation Excerpt :

      Zhang et al. [8] applied a filtering technique to remove dis-occluded areas in new views resulting from DIBR. Based on the binocular suppression theory of human perception, Bulbul et al. [9] showed that stereo rendering could be accelerated by producing a high quality image for one eye and a low quality one for the other eye. Sorbier et al. [10] indicated that stereoscopic rendering can be accelerated by setting multiple rendering targets after cloning the same primitives for both views in GPUs.

    • Tile Pair-Based Adaptive Multi-Rate Stereo Shading

      2020, IEEE Transactions on Visualization and Computer Graphics
    • Efficient stereo rendering of large 3D datasets based on binocular suppression

      2015, 2015 International Conference on Big Data and Smart Computing, BIGCOMP 2015
    • Stereoscopic image quality in virtual environments

      2014, 2014 International Conference on 3D Imaging, IC3D 2014 - Proceedings
    View all citing articles on Scopus
    View full text