Skip to main content
SpringerLink
Log in

Panorama from Representative Frames of Unconstrained Videos Using DiffeoMeshes

  • Conference paper
  • First Online:
Book cover Computer Vision – ACCV 2018 (ACCV 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11363))

Included in the following conference series:

  • 3154 Accesses

Abstract

Panorama creation from unconstrained hand-held videos is a challenging task due to the presence of large parallax, moving objects and motion blur. Alignment of the frames taken from a hand-held video is often very difficult to perform. The method proposed here aims to generate a panorama view of the video shot given as input. The proposed framework for panorama creation consists of four stages: The first stage performs a sparse frame selection based on alignment and blur score. A global order for aligning the selected frames is generated by computing a Minimum Spanning Tree with the most connected frame as the root of the MST. The third stage performs frame alignment using a novel warping model termed as DiffeoMeshes, a demon-based diffeomorphic registration process for mesh deformation, whereas the fourth stage renders the panorama. For evaluating the alignment performance, experiments were first performed on a standard dataset consisting of pairs of images. We have also created and experimented on a dataset of 20 video shots for generating panorama. Our proposed method performs better than the existing state-of-the-art methods in terms of alignment error and panorama rendering quality.

Partially supported by TCS Foundation, India.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Brown, M., Lowe, D.G., et al.: Recognising panoramas. In: ICCV (2003)

    Google Scholar 

  2. Liu, F., Hu, Y.H., Gleicher, M.L.: Discovering panoramas in web videos. In: ACMMM, pp. 329–338 (2008)

    Google Scholar 

  3. Li, Y., Kang, S.B., Joshi, N., Seitz, S.M., Huttenlocher, D.P.: Generating sharp panoramas from motion-blurred videos. In: CVPR (2010)

    Google Scholar 

  4. Hu, J., Zhang, D.Q., Yu, H., Chen, C.W.: Long scene panorama generation for indoor environment. In: ICIP (2014)

    Google Scholar 

  5. Freeman, S.C., Freeman, L.C.: The networkers network: a study of the impact of a new communications medium on sociometric structure (1979)

    Google Scholar 

  6. Liu, S., Yuan, L., Tan, P., Sun, J.: Bundled camera paths for video stabilization. ACM Trans. Graph. (TOG) 32, 78 (2013)

    Google Scholar 

  7. Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends® Comput. Graph. Vis. 2, 1–104 (2006)

    Article  MathSciNet  Google Scholar 

  8. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74, 59–73 (2007)

    Article  Google Scholar 

  9. Szeliski, R., Shum, H.Y.: Creating full view panoramic image mosaics and environment maps. In: SIGGRAPH (1997)

    Google Scholar 

  10. Gao, J., Kim, S.J., Brown, M.S.: Constructing image panoramas using dual-homography warping. In: CVPR (2011)

    Google Scholar 

  11. Lin, W.Y., Liu, S., Matsushita, Y., Ng, T.T., Cheong, L.F.: Smoothly varying affine stitching. In: CVPR (2011)

    Google Scholar 

  12. Liu, F., Gleicher, M., Jin, H., Agarwala, A.: Content-preserving warps for 3D video stabilization. ACM Trans. Graph. (TOG) 28, 44 (2009)

    Google Scholar 

  13. Lin, C.C., Pankanti, S.U., Natesan Ramamurthy, K., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: CVPR (2015)

    Google Scholar 

  14. Zaragoza, J., Chin, T.J., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving DLT. In: CVPR (2013)

    Google Scholar 

  15. Chang, C.H., Sato, Y., Chuang, Y.Y.: Shape-preserving half-projective warps for image stitching. In: CVPR (2014)

    Google Scholar 

  16. Chen, Y.-S., Chuang, Y.-Y.: Natural image stitching with the global similarity prior. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 186–201. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_12

    Chapter  Google Scholar 

  17. Lin, K., Jiang, N., Liu, S., Cheong, L.F., Lu, J., Do, M.: Direct photometric alignment by mesh deformation. In: CVPR (2017)

    Google Scholar 

  18. Bartoli, A.: Groupwise geometric and photometric direct image registration. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2098–2108 (2008)

    Article  Google Scholar 

  19. Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: CVPR (2014)

    Google Scholar 

  20. Jacob, G.M., Das, S.: Large parallax image stitching using an edge-preserving Diffeomorphic warping process. In: Blanc-Talon, J., Helbert, D., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2018. LNCS, vol. 11182, pp. 521–533. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01449-0_44

    Chapter  Google Scholar 

  21. Lin, K., Jiang, N., Cheong, L.-F., Do, M., Lu, J.: SEAGULL: seam-guided local alignment for parallax-tolerant image stitching. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 370–385. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_23

    Chapter  Google Scholar 

  22. Davis, J.: Mosaics of scenes with moving objects. In: CVPR (1998)

    Google Scholar 

  23. Kourogi, M., Kurata, T., Hoshino, J., Muraoka, Y.: Real-time image mosaicing from a video sequence. In: ICIP (1999)

    Google Scholar 

  24. Shum, H.Y., Szeliski, R.: Construction and refinement of panoramic mosaics with global and local alignment. In: ICCV (1998)

    Google Scholar 

  25. Hsieh, J.W.: Fast stitching algorithm for moving object detection and mosaic construction. Image Vis. Comput. 22, 291–306 (2004)

    Article  Google Scholar 

  26. Zhi, Q., Cooperstock, J.R.: Toward dynamic image mosaic generation with robustness to parallax. IEEE Trans. Image Process. 21, 366–378 (2012)

    Article  MathSciNet  Google Scholar 

  27. Zeng, L., Zhang, S., Zhang, J., Zhang, Y.: Dynamic image mosaic via sift and dynamic programming. Mach. Vis. Appl. 25, 1271–1282 (2014)

    Article  Google Scholar 

  28. Thirion, J.P.: Image matching as a diffusion process: an analogy with Maxwell’s demons. Med. Image Anal. 2, 243–260 (1998)

    Article  Google Scholar 

  29. Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Diffeomorphic demons: efficient non-parametric image registration. NeuroImage 45, S61–S72 (2009)

    Article  Google Scholar 

  30. Fadaeieslam, M.J., Soryani, M., Fathy, M.: Efficient key frames selection for panorama generation from video. J. Electron. Imaging 20, 023015 (2011)

    Article  Google Scholar 

  31. Steedly, D., Pal, C., Szeliski, R.: Efficiently registering video into panoramic mosaics. In: ICCV (2005)

    Google Scholar 

  32. Kim, S.H., et al.: Key frame selection algorithms for automatic generation of panoramic images from crowdsourced geo-tagged videos. In: Pfoser, D., Li, K.-J. (eds.) W2GIS 2013. LNCS, vol. 8470, pp. 67–84. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-55334-9_5

    Chapter  Google Scholar 

  33. Elhamifar, E., Sapiro, G., Sastry, S.S.: Dissimilarity-based sparse subset selection. IEEE Trans. Pattern Anal. Mach. Intell. 38, 2182–2197 (2016)

    Article  Google Scholar 

  34. Crete, F., Dolmiere, T., Ladret, P., Nicolas, M.: The blur effect: perception and estimation with a new no-reference perceptual blur metric. In: Human Vision and Electronic Imaging XII, vol. 6492, p. 64920I (2007)

    Google Scholar 

  35. Supplementary Document. http://www.cse.iitm.ac.in/~vplab/ACCV_18/Supplementary.pdf

  36. Opsahl, T., Agneessens, F., Skvoretz, J.: Node centrality in weighted networks: generalizing degree and shortest paths. Soc. Netw. 32, 245–251 (2010)

    Article  Google Scholar 

  37. Erkan, G., Radev, D.R.: LexPageRank: Prestige in multi-document text summarization. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (2004)

    Google Scholar 

  38. Santos-Ribeiro, A., Nutt, D.J., McGonigle, J.: Inertial demons: a momentum-based Diffeomorphic registration framework. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 37–45. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46726-9_5

    Chapter  Google Scholar 

  39. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004)

    Article  MathSciNet  Google Scholar 

  40. Goldstein, T., Studer, C., Baraniuk, R.: A field guide to forward-backward splitting with a FASTA implementation. arXiv eprint abs/1411.3406 (2014)

    Google Scholar 

  41. Igarashi, T., Moscovich, T., Hughes, J.F.: As-rigid-as-possible shape manipulation. ACM Trans. Graph. (TOG) 24, 1134–1141 (2005)

    Article  Google Scholar 

  42. Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. (TOG) 2, 217–236 (1983)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Visualization and Perception Lab, Department of Computer Science and Engineering, Indian Institute of Technology, Madras, Chennai, India

    Geethu Miriam Jacob & Sukhendu Das

Authors
  1. Geethu Miriam Jacob
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sukhendu Das
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Geethu Miriam Jacob .

Editor information

Editors and Affiliations

  1. IIIT Hyderabad, Hyderabad, India

    C. V. Jawahar

  2. ANU, Canberra, ACT, Australia

    Hongdong Li

  3. Simon Fraser University, Burnaby, BC, Canada

    Greg Mori

  4. ETH Zurich, Zurich, Zürich, Switzerland

    Konrad Schindler

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 3324 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jacob, G.M., Das, S. (2019). Panorama from Representative Frames of Unconstrained Videos Using DiffeoMeshes. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds) Computer Vision – ACCV 2018. ACCV 2018. Lecture Notes in Computer Science(), vol 11363. Springer, Cham. https://doi.org/10.1007/978-3-030-20893-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-20893-6_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20892-9

  • Online ISBN: 978-3-030-20893-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation