An uncalibrated lightfield acquisition system
Introduction
Traditionally, 3D graphics systems use geometric modeling where the scene is represented as a set of geometric primitives and lights. Such scenes are rendered using the standard geometric rendering pipeline, which involves modeling and viewing transforms, projection, clipping, perspective division and scan conversion. A relatively newer approach to rendering is Image Based Rendering (IBR) [8], [1]. IBR offers many advantages:
- •
The results of IBR are far more photo-realistic and modeling of scenes is easier since images are used to model the scene.
- •
IBR techniques are less computationally intensive and hence suitable for real-time rendering.
- •
The rendering speed is independent of the scene complexity.
Lightfield Rendering as proposed by Levoy and Hanrahan [7] is an IBR technique that relies on a convenient representation of the radiance information of a scene as a function of position and direction [9]. The lightfield is similar to the Plenoptic Function [2] except that in a space free of occluders it becomes a 4D function unlike the Plenoptic Function which is a 5D function. Levoy and Hanrahan [7] have suggested a convenient representation of the lightfield using two parallel, parameterized planes or lightslabs as shown in Fig. 1. The intensity of any ray intersecting the two planes is a function of four parameters, u, v, s and t, where (u,v) is the intersection of the ray with the outer plane and (s,t) is the intersection with the inner plane. As has been shown in their paper, an image taken from a viewpoint on one of the planes becomes a 2D slice of the 4D lightfield.
The process of lightfield rendering involves two steps—acquisition of the lightfield, and rendering using the acquired data. To acquire the lightfield, a series of images is taken from different viewpoints associated to one of the two planes used in the lightfield representation. An image taken in this manner is just a set of rays between the viewpoint of the camera and pixels on the image. Continuing in this way, the complete lightfield can be generated. Rendering from the acquired lightfield is a simple task—to generate an image from a new viewpoint, rays are cast from the centre of projection of desired viewpoint to each pixel of the desired image. Intersections of these rays are computed with the planes of the lightfield and nearest ray(s) from the lightfield are looked up for intensity values.
Convenient acquisition of lightfields for real world scenes has been one of the longstanding problems1 associated with lightfield rendering. Expensive, complex and bulky setups have been used for the purpose and this has been prohibitive to experimenting with Lightfield rendering. In this paper, we present a method of lightfield acquisition, which is convenient, fast and does not require large calibration objects to be visible. Our method of acquisition does not require explicit calibration, i.e. recovery of full projection matrix. We compute homographies induced between the camera plane and the lightfield plane(s) and apply for ray intersections with lightfield planes. It may be noted that the lightfield generated is, however, calibrated by virtue of its representation. Thus, rendering through a new view point is done by simple ray casting onto the lightfield.
Section snippets
Prior work
The lightfield acquisition system designed by Levoy and Hanrahan [7] consists of a single camera on a robot arm that translates in a two-dimensional plane as it captures images of an object. The camera has a narrow field of view lens, so the system allows the camera to be rotated towards the object. The images captured thus cannot be used as direct entries to the lightfield database but have to be transformed so as to align them along a plane parallel to the translation plane.
The camera is
Motivation
Let us first examine what exactly has to be done in order to incorporate the information captured by a particular image frame into the lightfield. Essentially, the lightfield is nothing but a set of rays—all rays that pass through the bounded parallel planes of a lightslab are elements of the lightfield.
In order to update the lightfield with information gathered from a particular frame, we need to determine the set of rays which the frame has captured and which intersect both of the lightfield
Implementation and results
The schematic of the system that we have implemented is shown in Fig. 7.
We have used a Sony digital video camera with a bttv848 frame grabber card to grab frames. All processing is done on a 1.4 GHz Pentium 4 system with 1 GB memory. The frames are fed to the point detector module, which uses color identification and region growing to identify the eight points (four on each plane). We are not tracking these points, however, we expect this module to become faster if tracking is done. Once the
Conclusion
Lightfield is a convenient method of representing the radiance in a space free of occluders. The method requires no information about the surface properties or geometry of the scene and hence is suitable for real world scenes. However, a few problems are associated with lightfield rendering, lightfield acquisition is one of them.
We have presented a lightfield capturing system which is easily reproducible, cheap and convenient to use. Our method facilitates lightfield capture using a handheld
References (10)
- C. Bregler, M. F. Cohen, R. Szeliski, P. Debevec, L. McMillan and F. X. Sillion, Image-Based Modeling, Rendering and...
- et al.
The Plenoptic function and the elements of early vision
(1991) - et al.
The Lumigraph
(1996) - et al.
Multiple View Geometry in Computer Vision
(September 2000) - et al.
Dynamically reparameterized lightfields
(2000)
Cited by (1)
Seabed modeling and distractor extraction for mobile AUVs using light field filtering
2011, Proceedings - IEEE International Conference on Robotics and Automation