Improved SFS 3D measurement based on BP neural network

https://doi.org/10.1016/j.imavis.2006.05.005Get rights and content

Abstract

3D surface measurement is an important requirement for modern industry. Non-contact optical method is a commonly used way to plot 2D and 3D measurement. Shape from shading (SFS) is a convenient 3D method because it can recover the 3D shape only from one image. But the existing SFS research has a lot of restriction, such as Lambertian illumination model. If the object is not under Lambertian model, the precision will decrease quickly. We propose an improved SFS based on BP neural network combining with genetic algorithm. This proposed SFS can provide the approximation of the reflectance model. So even the object is not under Lambertian light source, it can also be recovered and the precision has been greatly increased. It has been tested in the 3D reconstruction of synthetic vase, black and complex motorcar part, and really used in the online 3D measurement of work pieces.

Introduction

Research on high precision 3D measurement has become an urgent requirement for modern industry, because it can give more exact estimation about the quality of product. Among all the existing 3D measurement method, non-contact optical 3D measurement has become more and more popular. Most of existing 3D optical methods, need to obtain the geometrical information relate to the measured parts, such as structured light method, which uses laser or visual light to form structured light point, structured light strip, structured light plane, or structured grating. Now structured light method has been commonly used in steel and car industry to inspect the dimensions of some key points. But the operation of structured light is very complex, expensive, and difficult to complete 3D online shape measurement. So 3D measurement with easy operation and low cost should also be exploited. Moreover, the 3D recovery of from a historical image should also be considered.

SFS (Shape From Shading) is a process to reconstruct the 3D shape information from its 2D shading image. This is an inverse problem of image formation. The cost of SFS system is very low, because it is only composed of CCD, image-collection card, and computer. The SFS technique was firstly suggested by Horn [1] and further studied with his colleagues. The solution of the SFS was proved to be robust and stable under a controlled environment [2]. Since the reflectance model is non-linear in terms of the surface gradients, certain restrictive assumptions have to be made in order to solve the SFS problem. Firstly, surface reflection is generally assumed to be diffuse reflectance. Secondly, a far distant point light source is assumed to be known. Thirdly, images are formed by orthographic projection. Based on these assumptions, a simple Lambertian model was established such that apparent distortions on reconstructed surfaces often result in many practical applications. Torrance–Sparrow model [3] assuming that a surface is composed of small, randomly oriented, mirror-like facets. Wei and Hirzinger used a feed forward network to parameterize the object surface, but their performance would be degraded by using ineffective gradient-descent method that is of slow convergence and prone to the local minima problem, especially for a complex SFS problem [4]. Cho and Chow [5] proposed an improved neural SFS learning algorithm to tackle the shortcomings of the Wei and Hirzinger’s approaches. This SFS algorithm could enhance the solving of the SFS problem in terms of the reconstruction speed and the quality o solutions, but it is still under the restrictive condition of the Lambertian model, where the light source direction must be given. In 2000, Cho and Chow present a novel SFS neural-learning-based reflectance model [6], under which, the viewing direction and the light source direction are no longer required. But the optimization of weight suffers from the well-known local minima problem.

It is the purpose of this paper to propose an efficient solution to SFS, which can realize robust and high precision 3D measurement only from one image. The image can be a historical picture, or captured by CCD or camera. Compared with existing SFS, the proposed method does not care much about the direction of the light source, because it can estimate the light source direction even not under Lambertian model. And the proposed method can reach high precision. The structure of this paper is as follows. In Section 2, a BP neura network-based reflectance model for SFS, combined with genetic algorithms, is studied. In Section 3, the results and precision of 3D measurement of synthetic vase is analyzed. In Section 4, the application in the 3D measurement of online work piece is introduced. In Section 5, the results on the 3D measurement of black and complex work piece is provided. The last is for the conclusion.

Section snippets

Improved SFS method based on BP neural network and genetic algorithm

SFS is well worked for Lambertian reflectance model. However, in most practical cases, the intensity and direction of illumination is uncertainty. In order to generate a much more practical reflectance model, incorporating more physical reflectance parameters and effects, is inevitable. It is very difficult to solve the non-linear equation of a very complicated SFS model. Because the reflectance model can be viewed as an arbitrary continuous function, we proposed a neural network framework to

Result on the 3D measurement of synthetic vase

One synthetic vase was selected to test the precision of the proposed SFS method. The true depth maps of the synthetic were generated mathematically by the following function:z(x,y)=[0.15-0.1y(6y+1)2(y-1)2(3y-2)]2-x2,where −0.6 ⩽ x ⩽ 0.6 and 0.0 ⩽ y ⩽ 1.0.

The synthetic object was generated using true depth maps by 3D numerical control machine. The contour plot of this function is shown in Fig. 2(a).

Different illuminate conditions (i.e., known illuminate directions and unknown directions) were employed

Results on the online 3D measurement of work piece

Online 3D surface measurement of work piece has become more and more important in modern industry, because it can give vivid information about the status of the work piece and the quality of product can be easily judged. Although a lot of researches using different technique has been focus on 3D research, but now there is no excellent instrument really used in the industry, for the purpose of automatic 3D surface defect detection. Some 3D methods will bring hazard to people. For example, X-ray

Results on the 3D measurement of black and complex work piece

In order to test the applicability of the proposed SFS algorithm, some complex part in motorcar are selected in our experiment. A black work piece in shown in Fig. 4(a). The 3D recovery of black object is a difficult problem in vision measurement, because the light is absorbed by black object. Using the SFS method, the black object is also be recovered. The 3D recovery result is shown in Fig. 4(b). Although the precision of the image edge is not very high now, the center object is well

Conclusions

SFS technique can reconstruct the 3D shape of object based on single image. But the existing SFS suffer from low precision when the measured object is not under the Lambertian model. We proposed an improved SFS based on BP neural network combining with genetic algorithm, which can work even the light source direction is unkown, because it can estimate the light source direction. Through the BP neural network model, the SFS algorithm has become more robust and effective for most application. The

Acknowledgements

The financial support of YeShenghua, a famous academician in China, and my colleges FenLan Li, Jin Liu, HuiZhi Wen, YanFeng Li, LiMin Zhang, are all greatly acknowledgement.

References (6)

There are more references available in the full text version of this article.

Cited by (5)

View full text
Copyright © 2006 Elsevier B.V. All rights reserved.