Multi perspective panoramic imaging
Introduction
Image sensors have recently become almost as inexpensive and available as scalar sensors which are used for temperature or light measurements. The Stanford Multi-Camera Array project is an early example for the simultaneous usage of more than 100 inexpensive CCD cameras [1], [2], [3], [4]. Other projects are currently emerging in the field of sensor networks. The ESB sensor node platform by the FU-Berlin is one instance of a small, wireless and video-enabled device [5].
Other than scalar values which can be displayed on a virtual map or which can simply be aggregated, it is not obvious how to display a massive amount of (possibly uncalibrated) images, particularly in a way that makes sense for a human observer.
Consolidating all images into a single one could be a possible solution. Similar attempts have been made in the field of panoramic images, in which a series of pictures are stitched to one another to produce a continuous view. For a long time, panoramic images have been considered feasible only if all images have the same focal point, respectively, if the camera does not alter its location. In this paper, we devise a novel method for creating panoramic views from images with varying focal points. The specific problems which arise here are described in the following section, along with descriptions of prior attempts to create panoramic images from movies. Section 3 suggests a basic warping scheme as a solution. In Section 4, we identify some shortcomings of the basic scheme which are solved by an extension. The evaluation in Section 5 shows some examples and analyzes the new degree of freedom, but also fundamental limitations, of multi-perspective still imaging and video in general. The outlook in Section 6 sketches future improvements to reduce the amount of human interaction.
Section snippets
Related work
In the context of our paper, we will distinguish between monoperspective panoramic images and multi-perspective imaging.
Warping for panoramic stitching
In this section, we will show that panoramic images are possible, even if the focal point of the camera changes significantly. Of course, the resulting image will imply several changes in perspective and, unlike existing approaches, these changes will by no means be continuous. But, as we will see, this does not necessarily result in an unnatural output.
Fig. 4 shows a building from two different perspectives with a certain overlap. In conventional panoramic image generation, semi-transparent
Extended warping
A typical result of the above-described simple warping scheme can be seen in Fig. 8. We identified four different kinds of artifacts, namely expansions, contractions, undefined areas, and reflections (see Fig. 9).
The emergence of expansions and contractions are most obvious. If two neighboring abscissas exhibit a large opening angle in the source image, but a smaller angle in the target image, this means that a large image patch in the source area will be squeezed into a small patch within the
Evaluation and limitations
Fig. 12, Fig. 13 show examples of panoramic images with changing perspectives. Intuitively speaking, warping and stitching images work best if the object being shown can be unfolded, theoretically. This is often true for buildings. In particular, stitching works without problems if the camera follows a straight path. The reason is that succeeding images can be concatenated to existing ones if the above-mentioned trajectory exists and if it is not occluded from two neighboring viewpoints. We can
Conclusion and outlook
A new approach for producing panoramic images from photos with varying centers of projection is proposed. A trajectory has to be found in neighboring images which serves as a cut. A vertical warping scheme distorts neighboring images such that they fit together. Another horizontal warping process makes the distorted parts of the image near the cut converge against the natural image near the left and right borders of the panorama. Finally, artifacts originating from the warping and from the lens
Acknowledgements
We want to thanks Mr. Allen Wrenn and Mrs. Weyerer for their suggestions and improvements which contributed significantly to the quality of this work.
References (27)
- B. Wilburn, N. Joshi, V. Vaish, E. Talvala, E. Antunez, A. Barth, A. Adams, M. Levoy, M. Horowitz, High performance...
- B. Wilburn, N. Joshi, V. Vaish, M. Levoy, M. Horowitz, High speed video using a dense camera array, in: Proc. of the...
- V. Vaish, B. Wilburn, N. Joshi, M. Levoy, Using plane and parallax for calibrating dense camera arrays, in: Proc. of...
- B. Wilburn, M. Smulski, H.K. Lee, M. Horowitz, The light field video camera, in: Proc. of Media Processors 2002, SPIE...
- E. Kappe, A. Liers, H. Ritter, J. Schiller, Low-power image transmission in wireless sensor networks using scatterweb...
Barnard’s Photographic Views of the Sherman Campaign
(1866)Automatic video segmentation employing object/camera modeling techniques
(2005)Video mosaics for virtual environments
IEEE Computer Graphics and Applications
(1996)Methods of Plane Projective Geometry Based on the Use of General Homogeneous Coordinates
(1946)General Homogeneous Coordinates in Space of Three Dimensions
(1951)
Cited by (17)
Image mosaicing: A deeper insight
2019, Image and Vision ComputingCitation Excerpt :Firstly, the images are aligned using a common global coordinate instead of the local coordinate of each individual image of the scene. In the next stage, the aligned images are projected over a common compositing manifold for warping [21]. The compositing surface/manifold provides a global frame to the aligned images for the generation of a single image mosaic.
A positioning method for the optimal seam in binocular visual image stitching
2020, Proceedings of SPIE - The International Society for Optical EngineeringColour Vignetting Correction for Microscopy Image Mosaics Used for Quantitative Analyses
2018, BioMed Research InternationalExperimental assessment of rectification algorithm in vision-based deflection measurement system
2017, Nondestructive Testing and EvaluationReal-time multi-camera video stitching based on improved optimal stitch line and multi-resolution fusion
2017, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)Distortion-Free Image Mosaicing for Tunnel Inspection Based on Robust Cylindrical Surface Estimation through Structure from Motion
2016, Journal of Computing in Civil Engineering