research-article

Image registration for foveated panoramic sensing

Authors:

James H. ElderAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 8, Issue 2

Article No.: 17, Pages 1 - 20

https://doi.org/10.1145/2168996.2168997

Published: 22 May 2012 Publication History

Abstract

This article addresses the problem of registering high-resolution, small field-of-view images with low-resolution panoramic images provided by a panoramic catadioptric video sensor. Such systems may find application in surveillance and telepresence systems that require a large field of view and high resolution at selected locations. Although image registration has been studied in more conventional applications, the problem of registering panoramic and conventional video has not previously been addressed, and this problem presents unique challenges due to (i) the extreme differences in resolution between the sensors (more than a 16:1 linear resolution ratio in our application), and (ii) the resolution inhomogeneity of panoramic images. The main contributions of this article are as follows. First, we introduce our foveated panoramic sensor design. Second, we show how a coarse registration can be computed from the raw images using parametric template matching techniques. Third, we propose two refinement methods allowing automatic and near real-time registration between the two image streams. The first registration method is based on matching extracted interest points using a closed form method. The second registration method is featureless and based on minimizing the intensity discrepancy allowing the direct recovery of both the geometric and the photometric transforms. Fourth, a comparison between the two registration methods is carried out, which shows that the featureless method is superior in accuracy. Registration examples using the developed methods are presented.

References

[1]

Amintabar, A. and Boufama, B. 2008. Homography-based plane identification and matching. In Proceedings of the IEEE International Conference on Image Processing.

[2]

Bay, H., Ess, A., Tuytelaars, T., and Gool, L. V. 2008. SURF: Speeded Up Robust Features. Comput Vision Image Understand. 110, 3, 346--359.

Digital Library

[3]

Boult, T. E., Gao, X., Micheals, R., and Eckmann, M. 2004. Omni-directional visual surveillance. Image Vision Comput. 22, 7, 515--534.

[4]

Brown, M. and Lowe, D. G. 2007. Automatic panoramic image stitching using invariant features. Int. J. Comput. Vision 74, 1, 59--73.

Digital Library

[5]

Chen, J., Chen, C., and Chen, Y. 2003. Fast algorithm for robust template matching with M-estimators. IEEE Trans. Signal Process 51, 1, 230--243.

Digital Library

[6]

Conroy, T. L. and Moore, J. B. September 1999. Resolution invariant surfaces for panoramic vision systems. InProceedings of the IEEE Conference on Computer Vision.

[7]

Danilidis, K. and Geyer, C. 2000. Omnidirectional vision: Theory and algorithms. In Proceedings of the IEEE International Conference on Patter Recognition.

Digital Library

[8]

Dornaika, F. and Elder, J. 2002. Image registration for foveated omnidirectional sensing. In Proceedings of the European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 2353.

Digital Library

[9]

Dufourneau, Y., Schmid, C., and Horaud, R. 2000. Matching images with different resolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

[10]

Fischler, M. A. and Bolles, R. C. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24, 6, 381--395.

Digital Library

[11]

Fletcher, R. 1990. Practical Methods of Optimization. Wiley, New York.

Digital Library

[12]

Harris, C. and Stephens, M. 1988. A combined corner and edge detector. In Proceedings of the Alvey Vision Conference.

[13]

He Q. and Chu, C. H. 2006. Planar surface detection in image pairs using homographic constraints. In Proceedings of the 2nd International Symposium on Advances in Visual Computing. Lecture Notes in Computer Science, Vol. 4291.

Digital Library

[14]

Kanatani, K. and Ohta, N. 1999. Accuracy bounds and optimal computation of homography for image mosaicing applications. In Proceedings of the IEEE Conference on Computer Vision.

[15]

Kim, D. H., Yoon, Y. I., and Choi, J. S. 2003. An efficient method to build panoramic image mosaics. Patt. Recog. Lett 24, 14, 2421--2429.

Digital Library

[16]

Lin, S. S. and Bajcsy, R. 2006. Single-view-point omnidirectional catadioptric cone mirror imager. IEEE Trans. Patt. Anal. Machine Intell. 28, 5, 840--845.

Digital Library

[17]

Lowe, D. 2004. Distinctive image features from scale invariant keypoints. Int. J. Comput. Vision 60, 2, 91--100.

Digital Library

[18]

Ohta, J. 2007. Smart CMOS Image Sensors and Applications. CRC Press, Taylor & Francis Group.

[19]

Peng, G., Xie, S., and Cheng, L. 2005. An HVSM for improving the homing ability of visual robots. Int. J. Intell. Syst. Techn. Appl. 1, 2, 18--31.

Digital Library

[20]

Pilu, M. 1997. A direct method for stereo correspondence based on singular value decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

Digital Library

[21]

Prime, S. L., Tsotsos, L., Keith, G. P., and Crawford, J. D. 2007. Visual memory capacity in transsaccadic integration. Exp. Brain Resear. 180, 4, 609--628.

[22]

Scaramuzza, D. and Siegwart, R. 2007. A practical toolbox for calibrating omnidirectional cameras. In Vision Systems: Applications, Intech.

[23]

Schwartz, W. B., Kembhavi, A., Harwood, D., and Davis, L. S. 2009. Human detection using partial least squares analysis. In Proceedings of the IEEE International Conference on Computer Vision.

[24]

Spacek, L. and Burbridge, C. 2007. Instantaneous robot self-localisation and motion estimation with omnidirectional vision. Rob. Auton. Syst. 55, 667--674.

Digital Library

[25]

Swaminathan, R., Grossberg, M. D., and Nayar, S. K. 2006. Non-single viewpoint catadioptric cameras: Geometry and analysis. Int. J. Comput. Vision 66, 3, 211--229.

Digital Library

[26]

Tanaka, K., Sano, M., Ohara, S., and Okudaira, M. 2000. A parametric template method and its application to robust matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

[27]

Torre, F., Vallespi, C., Rybski, P. E., Veloso, M., and Kanade, T. 2005. Learning to track multiple people in omnidirectional video. In Proceedings of the IEEE International Conference on Robotics and Automation.

[28]

Traver, V. J. and Bernardino, A. 2010. A review of log-polar imaging for visual perception in robotics. Rob. Auton. Syst. 58, 4, 378--398.

Digital Library

[29]

Wu, Y., Kanade, T., Li, C., and Cohn, J. 2000. Image registration using wavelet-based motion model. Int. J. Comput. Vision 38, 2, 129--152.

Digital Library

[30]

Zitová, B. and Flusser, J. 2003. Image registration methods: a survey. Image Vision Comput. 21, 977--1000.

Cited By

Syawaludin MKim SHwang J(2021)Planar-Equirectangular Image StitchingElectronics10.3390/electronics1009112610:9(1126)Online publication date: 10-May-2021
https://doi.org/10.3390/electronics10091126
Syawaludin MLee MHwang J(2020)Foveation Pipeline for 360° Video-Based TelemedicineSensors10.3390/s2008226420:8(2264)Online publication date: 16-Apr-2020
https://doi.org/10.3390/s20082264
Yeh JChang CChia TChiang SHuang P(2016)Cooperative dual camera surveillance system for real-time object searching and close-up viewing2016 2nd International Conference on Intelligent Green Building and Smart Grid (IGBSG)10.1109/IGBSG.2016.7539432(1-5)Online publication date: Jun-2016
https://doi.org/10.1109/IGBSG.2016.7539432
Show More Cited By

Index Terms

Image registration for foveated panoramic sensing

Recommendations

Image Registration for Foveated Omnidirectional Sensing
ECCV '02: Proceedings of the 7th European Conference on Computer Vision-Part IV

This paper addresses the problem of registering highresolution, small FOV images with low-resolution panoramic images provided by an omnidirectional catadioptric video sensor. Such systems may find application in surveillance and telepresence systems ...
Image Registration of Sectioned Brains

The physical (microtomy), optical (microscopy), and radiologic (tomography) sectioning of biological objects and their digitization lead to stacks of images. Due to the sectioning process and disturbances, movement of objects during imaging for example, ...
Joint segmentation and registration for 4D lung CT images based on Markov random field
ICIMCS '18: Proceedings of the 10th International Conference on Internet Multimedia Computing and Service

The study of segmentation and registration of lung volume in medical images has been an active area with the emergence and development of 4D CT (Computed Tomography) medical imaging techniques. Precise image segmentation and registration methods are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 8, Issue 2

May 2012

144 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2168996

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 May 2012

Accepted: 01 January 2011

Revised: 01 April 2010

Received: 01 January 2010

Published in TOMM Volume 8, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
438
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Syawaludin MKim SHwang J(2021)Planar-Equirectangular Image StitchingElectronics10.3390/electronics1009112610:9(1126)Online publication date: 10-May-2021
https://doi.org/10.3390/electronics10091126
Syawaludin MLee MHwang J(2020)Foveation Pipeline for 360° Video-Based TelemedicineSensors10.3390/s2008226420:8(2264)Online publication date: 16-Apr-2020
https://doi.org/10.3390/s20082264
Yeh JChang CChia TChiang SHuang P(2016)Cooperative dual camera surveillance system for real-time object searching and close-up viewing2016 2nd International Conference on Intelligent Green Building and Smart Grid (IGBSG)10.1109/IGBSG.2016.7539432(1-5)Online publication date: Jun-2016
https://doi.org/10.1109/IGBSG.2016.7539432
Lin CChen KChen SChen CHung Y(2015)Large-Area, Multilayered, and High-Resolution Visual Monitoring Using a Dual-Camera SystemACM Transactions on Multimedia Computing, Communications, and Applications10.1145/264586211:2(1-23)Online publication date: 7-Jan-2015
https://dl.acm.org/doi/10.1145/2645862

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents