Neural Network Based 3D Model Reconstruction  with Highly Distorted Stereoscopic Sensors

Abstract

In stereoscopic vision, there are two artificial eyes implemented so that it can obtain two separate views of the scene and simulate the binocular depth perception of human beings. Traditionally, camera calibration and 3D reconstruction of such a vision sensor are performed by geometrical solutions. However, the traditional camera model is very complicated since nonlinear factors in it and needs to approximate the light projection scheme by a number of parameters. It is even very difficult to model some highly distorted vision sensors, such as fish-eye lens. In order to simplify both the camera calibration and 3D reconstruction procedures, this work presents a method based on neural networks which is brought forward according to the characteristics of neural network and stereoscopic vision. The relation between spatial points and image points is established by training the network without the parameters of the cameras, such as focus, distortions besides the geometry of the system. The training set for our neural network consists of a variety of stereo-pair images and corresponding 3D world coordinates. Then the 3D reconstruction of a new s cene is simply using the trained network. Such a method is more similar to how human’s eyes work. Simulations and real data are used to demonstrate and evaluate the procedure. We observe that the errors obtained our experimentation are accurate enough for most machine-vision applications.

This work is supported by the National Natural Science Foundation of China (NSFC No. 60405009 and 60374056) and Zhejiang Natural Science Foundation of China (ZJNSF No.Y104185).