An improved method of photometric stereo using local shape from shading

https://doi.org/10.1016/S0262-8856(03)00096-9Get rights and content

Abstract

This paper presents an improved photometric stereo (PS) method by integrating it with a local shape from shading (SFS) algorithm. PS produces the initial estimate of image for the global accuracy and also provides the recovery of albedo, SFS supplies the more detailed information within each homogeneous area. The quality of depth obtained by integrating PS and SFS is compared with the real depth using absolute dept error function, and the improvement ranging from 2.3 to 14% over PS is obtained.

Introduction

Shape recovery is a classic problem in computer vision where aim is to get a 3D scene information from one or multiple 2D images. Techniques to recover shape are called shape-from-X techniques. Among these techniques, shape from shading (SFS) deals with the recovery of shape from a gradual variation of shading of the image while photometric stereo (PS) [16] is another method for shape recovery which differs from SFS in the number of input images. PS recovers shape from multiple intensity images of the same scene generated using a fixed viewing direction and different light source directions, while SFS provides the shape estimation from a single intensity image. SFS techniques can be divided into global and local approaches [9]. Global approaches can be further sub-divided into global minimization and global propagation approaches. Global minimization approaches obtain shape estimation by minimizing an energy function. Global propagation approaches propagate the shape information from known surface points to the whole image and thus generate an estimation from generalization. Local approaches derive shape only from the intensity information of the surface points in a small neighborhood.

Pentland [6] introduced the local SFS approach from the intensity using only its first and second derivatives. It is very sensitive to noise because of the second derivatives. Lee and Rosenfeld [2] computed the slant and tilt of the surface in the light coordinate system through the first derivative of the intensity under locally spherical surface assumption. Because of this assumption it is unusable for non-spherical surfaces. Pentland [11] introduced a new approach using the linear approximation of the reflectance function in terms of surface gradient and applied a Fourier transform to linear function to get a closed form solution for the depth at each point. This algorithm gives good results on surfaces which change linearly; however, when the surface changes have got non-linear characteristics it fails. Tsai and Shah [15] used the discrete approximation of the gradient and the depth is iteratively recovered by using linear approximation of the reflectance function. It is very fast algorithm, however, it is very sensitive to the intensity noise.

In real life applications in many areas, PS has been found to give better result and be more suitable. As mentioned earlier, in PS the shape can only be recovered from the areas that are illuminated in all input images and the quality of recovery increases with the number of image sequences.

The construction of depth from surface gradients is also a major problem. In order to overcome this problem some methods were introduced [3], [5], [8]. Specially Frankot and Chellappa [8] offered an elagant method for enforcing integrability in SFS algorithms.

SFS is also used with other shape from X modules in order to get high performance on shape recovery [10], [14], [17]. Cryer, Tsai and Shah [10] integrated stereo and SFS modules. The method recovers the depth information keeping the low frequency information from stereo and adding with high frequency information from SFS.

In order to improve the performance of shape recovery, the local SFS algorithm is integrated with PS as a new approach in this paper. PS produces the initial estimate of image to establish the global accuracy and also provides the recovery of albedo, SFS, on the other hand, supplies the more detailed information within each homogeneous area.

The algorithm has the advantage of using the most efficient parts of PS and SFS as information. Varying albedo problem of SFS methods has been overcome by using three images which are illuminated from different directions in PS. Even though some of the details of surface characteristics may be lost due to least square approach in PS, the global accuracy, in most cases, is better than SFS methods. On the other hand, in most of the cases, the performance of capturing the details of surface characteristics by local SFS algorithms is superior than PS methods.

Section snippets

An improved method of photometric stereo using local shape from shading

In order to get the better estimation of the real depth map not only PS method is applied on the input images but also local shape from shading algorithms (SFS) are applied. However, the order of combining these methods is also important. PS is more robust system than SFS because of recovering albedo and being less sensitive to noise. On the other hand some SFS algorithms can give a more detailed estimation on some local areas.

In general the global methods of SFS are very complex and slow. The

Experimental images

Experimental images are chosen so that the performance of our algorithm can be compared with those commonly known PS and SFS algorithm performances. Synthetic and real images were used to test our method.

Conclusion

We introduced a new approach for improving the performance of PS method by integrating it with ‘local’ SFS method. The results proved to be promising, in most of the cases that we have looked at. The performance of the algorithm in shape recovery, turned out to be superior compared to the performances of PS and SFS individually, when the initial estimate of an image is obtained by PS which is followed by a local improvement with SFS for each homogeneous area. The improvement ranging from 2.3 to

Acknowledgements

This work was supported by TÜBİTAK-BİLTEN (The Scientific and Technical Research Council of Turkey—Information Technologies and Electronics Research Institute).

References (17)

There are more references available in the full text version of this article.

Cited by (14)

View all citing articles on Scopus
View full text