Abstract
We present a sequential mosaicing algorithm for a calibrated rotating camera which can for the first time build drift-free, consistent spherical mosaics in real-time, automatically and seamlessly even when previously viewed parts of the scene are re-visited. Our mosaic is composed of elastic triangular tiles attached to a backbone map of feature directions over the unit sphere built using a sequential EKF SLAM (Extend Kalman Filter Simultaneous Localization And Mapping) approach.
This method represents a significant advance on previous mosaicing techniques which either require off-line optimization or which work in real-time but use local alignment of nearby images and ultimately drift. We demonstrate the system’s real-time performance with real-time mosaicing results from sequences with 360 degrees pan. The system shows good global mosaicing ability despite the challenging conditions: hand-held simple low-resolution webcam, varying natural outdoor illumination, and people moving in the scene.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bar-Shalom, Y., & Fortmann, T. E. (1988). Mathematics in science and engineering: Vol. 179. Tracking and data association. San Diego: Academic Press.
Brown, M., & Lowe, D. (2003). Recognising panoramas. In ICCV (pp. 1218–1225). Nice.
Capel, D., & Zisserman, A. (1998). Automated mosaicing with super-resolution zoom. In CVPR (pp. 885–891). June 1998.
Castellanos, J. A., & Tardós, J. D. (1999). Mobile robot localization and map building: a multisensor fusion approach. Boston: Kluwer Academic.
Castellanos, J. A., Montiel, J. M. M., Neira, J., & Tardós, J. D. (1994). Sensor influence in the performance of simultaneous mobile robot localization and map building. In Lecture notes in computer science (Vol. 250, pp. 287–296). Berlin: Springer.
Civera, J., Davison, A. J., & Montiel, J. M. M. (2008). Inverse depth parametrization for monocular SLAM. IEEE Transactions on Robotics and Automation, Accepted.
Davison, A. J. (2003). Real-time simultaneous localization and mapping with a single camera. In ICCV.
Davison, A. J., & Murray, D. W. (1998). Mobile robot localization using active vision. In ICCV (pp. 809–825). Freiburg, Germany.
de Agapito, L., Hayman, E., & Reid, I. A. (2001). Self-calibration of rotating and zooming cameras. IJCV, 45(2), 107–127.
Eustice, R. M., Singh, H., Leonard, J. J., Walter, M., & Ballard, R. (2005). Visually navigating the RMS titanic with SLAM information filters. In RSS.
Feder, H., Leonard, J., & Smith, C. (1999). Adaptive mobile robot navigation and mapping. International Journal of Robotics Research, 18(7), 650–668.
Harris, C., & Stephens, M. (1988). A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference (pp. 147–151).
Jung, I., & Lacroix, S. (2003). High resolution terrain mapping using low altitude aerial stereo imagery. In ICCV. Nice.
Kim, D., & Hong, K. (2006). Real-time mosaic using sequential graph. Journal of Electronic Imaging, 15(2), 47–63.
Kim, J. H., & Sukkarieh, S. (2003). Airborne simultaneous localisation and map building. In ICRA (pp. 406–411).
Marks, R., Rock, S., & Lee, M. (1995). Real-time video mosaicking of the ocean floor. IEEE Journal of Oceanic Engineering, 20(3), 229–241.
Mikhail, E. M., Bethel, J. S., & McGlone, J. C. (2001). Introduction to modern photogrammetry. New York: Wiley.
Montiel, J. M. M., & Davison, A. J. (2006). A visual compass based on SLAM. In ICRA (pp. 1917–1922).
Morimoto, C., & Chellappa, R. (1997). Fast 3D stabilization and mosaic construction. In Proc. CVPR (pp. 660–665).
Ortín, D., Montiel, J. M. M., & Zisserman, A. (2003). Automated multisensor polyhedral model acquisition. In ICRA.
Renka, R. J. (1997). Algorithm 772: Stripack: Delaunay triangulation and Voronoi diagram on the surface of a sphere. ACM Transactions on Mathematical Software, 23(3), 416–434.
Sawhney, H., Hsu, S., & Kumar, R. (1998). Robust video mosaicing through topology inference and local to global alignment. In ECCV.
Smith, R. C., & Cheeseman, P. (1986). On the representation and estimation of spatial uncertainty. International Journal Robotics Research, 5(4), 56–68.
Steedly, D., Pal, C., & Szeliski, R. (2005). Efficiently stitching large panoramas from video. In ICCV.
Szeliski, R., & Shum, H. (1997). Creating full view panoramic image mosaics and environmental maps. In Proc. SIGGRAPH (pp. 251–258).
Triggs, B., McLauchlan, P., Hartley, R., & Fitzgibbon, A. (2000). Bundle adjustment—a modern synthesis. In W. Triggs, A. Zisserman, & R. Szeliski (Eds.), Vision algorithms: theory and practice. Lecture notes in computer science (pp. 298–375). Berlin: Springer.
Zhu, Z., Xu, G., Riseman, E. M., & Hanson, A. R. (2006). Fast construction of dynamic and multi-resolution 360° panoramas from video sequences. Image and Vision Computing, 24(1), 13–26.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Civera, J., Davison, A.J., Magallón, J.A. et al. Drift-Free Real-Time Sequential Mosaicing. Int J Comput Vis 81, 128–137 (2009). https://doi.org/10.1007/s11263-008-0129-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-008-0129-5