Correction of color information of a 3D model using a range intensity image

https://doi.org/10.1016/j.cviu.2009.03.015Get rights and content

Abstract

Most active optical range sensors record, simultaneously with the range image, the amount of light reflected at each measured surface location: this information forms what is called a range intensity image, also known as a reflectance image. This paper proposes a method that uses this type of image for the correction of the color information of a textured 3D model. This color information is usually obtained from color images acquired using a digital camera. The lighting condition for the color images are usually not controlled, thus this color information may not be accurate. On the other hand, the illumination condition for the range intensity image is known since it is obtained from a controlled lighting and observation configuration, as required for the purpose of active optical range measurement. The paper describes a method for combining the two sources of information, towards the goal of compensating for a reference range intensity image is first obtained by considering factors such as sensor properties, or distance and relative surface orientation of the measured surface. The color image of the corresponding surface portion is then corrected using this reference range intensity image. A B-spline interpolation technique is applied to reduce the noise of range intensity images. Finally, a method for the estimation of the illumination color is applied to compensate for the light source color. Experiments show the effectiveness of the correction method using range intensity images.

Introduction

Constructing textured three-dimensional (3D) models of real objects is an important issue in the field of computer vision. The texture mapping technique [1] using color images captured by a digital camera is often used for creating a realistic 3D model. However, the illumination under which color images are captured cannot usually be controlled, and thus the color information is affected by the illumination. There are several approaches to address this problem. Some methods actually control lighting conditions for the purpose of measuring reflectance properties, e.g. [2], [3], [4]. Ikeuchi and Sato [2] measure reflectance properties with a range image and a brightness image. Kay and Caelli [3] reconstruct reflectance properties from one range image and four or more color images under different point light sources. Sato et al. [4] acquire multiple range and color images by rotating an object and construct a whole 3D model with complex texture. In these methods, a Torrance–Sparrow model [5] is modified and used to separate diffuse and specular components and recover reflectance properties. Bernardini and Rushmeier [6] present a broad survey of techniques for 3D model acquisition including reflectance properties. Tominaga and Tanaka [7] and Tan et al. [8] separate the diffuse and specular components from a color image.

An active range sensor measures distance by projecting light (laser or incoherent) onto a surface and measuring a property of the reflected light (triangulation angle, time-of-flight, etc.). The amount of reflected light is a function of the reflectance properties of the measured surface point. The array of the measured power produces an intensity image with the same sampling pattern as the geometric component of the range image. This is called a range intensity image or a reflectance image. Hereafter, this paper uses the term range intensity image. The use of range intensity images for registration of a range image and a color image has been successfully demonstrated before [9], [10], [11], [12], [13].

In this paper, we focus on another useful feature of a range intensity image [14], [15]. The lighting condition yielding the range intensity image is controlled since this image is taken under known lighting conditions by virtue of the geometric sensing process. This paper investigates the application of a range intensity image to the correction of the color information of a colored 3D model. The goal is to exploit the respective advantages of a range intensity image and a color image. The controlled sensing geometry on both the illumination and observation sides, combined with the measured surface geometry, allows the inversion of an illumination model and the derivation of a corrected range intensity image representing the intrinsic reflectance properties of the surface. Using the corrected value of the range intensity image, we can adjust the intensity level of the color image. By conforming the value of the color image to that of the range intensity image, the proper intensity level is obtained without estimating the illumination for the color image. Additionally, we introduce the method to estimate the illumination color proposed by Lehmann and Palm [16], and compensate for the chromatic properties of the illuminant.

This paper is organized as follows. In Section 2, we explain the characteristics of a range intensity image and show the flow of the correction method. In Section 3, we describe the method for correcting the intensity of a color image using a range intensity image. After introducing the method to estimate the illumination color in Section 4, we show several experimental results in Section 5, and then we conclude the paper in Section 6.

Section snippets

Characteristics of a range intensity image

Fig. 1 shows an example of a range intensity image and a color image of the same object. A range intensity image is monochrome (with some exceptions such as [17]) while a color image contains multiple channels, usually red, green and blue. Thanks to sustained advances in the resolution of off-the-shelf digital cameras, a typical color image is denser than a range intensity image. Additionally, a color image usually exhibits a better S/N (signal-to-noise) ratio. Fig. 2 shows cross-sections of a

Reference range intensity image

The first step in the correction of a color image from a range intensity image requires the estimation of a reference range intensity image. A range intensity image is affected by the following factors [19]:

  • Distance to each measured point

  • Normal vector at the measured point

  • Sensor-specific characteristics

  • Reflectance properties of the illuminated surface

Attenuation due to distance has an inverse-of-square effect on the range intensity values. Normal vectors can be estimated from the range

Underlying principle

We briefly introduce the method of Lehmann and Palm [16] for the estimation of the illumination color.

Color vector of an RGB image is expressed as follows with the dichromatic reflection model.RGB=wdβdRsRβdGsGβdBsB+wsβssRsGsBThe first and second terms represent diffuse and specular components, respectively. wd and ws are the weights for diffuse and specular reflections that are defined geometrically, βd and βs are the diffuse and specular reflectance terms, respectively, and (sR,sG,sB)T

Range image sensor

The range image sensor used for the experiments is the ShapeGrabber system, with a scan head SG-100 on a PLM300 linear displacement mechanism [22]. It projects a slit of laser light and measures distances by triangulation. The number of points measured along each slit is 1280. The number of measurements in the other direction is a function of the mechanical displacement. The system provides a range intensity image along with the range data. Fig. 8 shows the ShapeGrabber sensor with a target

Conclusion

We have proposed a method that uses the range intensity image in order to improve the color information of 3D models, by taking advantage of the fact that the lighting condition for the range intensity image is controlled as part of active range sensing. We first discussed the correction of the range intensity image itself, considering the sensor-specific factors, distance, normal vectors, etc. We then proposed a method for the correction of the intensity of a color image using the corrected

References (22)

  • G. Kay et al.

    Inverting an illumination model from range and intensity maps

    CVGIP: Image Understanding

    (1994)
  • J.-A. Beraldin et al.

    Real world modelling through high resolution digital 3D imaging of objects and structures

    ISPRS Journal of Photogrammetry and Remote Sensing

    (2000)
  • J.F. Blinn et al.

    Texture and reflection in computer generated images

    Communications of the ACM

    (1976)
  • K. Ikeuchi et al.

    Determining reflectance properties of an object using range and brightness images

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1991)
  • Y. Sato, M.D. Wheeler, K. Ikeuchi, Object shape and reflectance modeling from observation, in: Proc. SIGGRAPH, vol. 97,...
  • K.E. Torrance et al.

    Theory of off-specular reflection from roughened surfaces

    Journal of the Optical Society of America

    (1967)
  • F. Bernardini et al.

    The 3D model acquisition pipeline

    Computer Graphics Forum

    (2002)
  • S. Tominaga et al.

    Estimating reflection parameters from a single color image

    IEEE Computer Graphics and Applications

    (2000)
  • R.T. Tan et al.

    Separating reflection components based on chromaticity and noise analysis

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2004)
  • F. Boughorbel, D. Page, C. Dumont, M.A. Abidi, Registration and integration of multi-sensor data for photo-realistic...
  • P. Dias, V. Sequeira, F. Vaz, J. Goncalves, Registration and fusion of intensity and range data for 3D modelling of...
  • Cited by (0)

    View full text