Skip to main content
SpringerLink
Log in

Disparity-preserving image rectangularization for stereoscopic panorama

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This study aims at generating a long strip stereoscopic panorama with a rectangular boundary from a stereoscopic video. The issues arising from this goal are how to automatically select appropriate frames to reduce geometric distortion in image stitching, how to preserve disparity under image warping, and how to generate a rectangular panoramic stereoscopic image without the loss of boundary information. To deal with these issues and to generate visually smooth stereoscopic panorama, a disparity-aware image warping is proposed. Moreover, the image warping method is performed on the irregular left and right panoramic images simultaneously with a hybrid control mesh to generate a rectangular panorama while preserving the spatial shape and disparity as much as possible. Experimental results on various stereo video contents show that the proposed method can effectively preserve both the spatial shapes and pixel disparity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Agarwala A, Zheng KC, Pal C, Agrawala M, Cohen M, Curless B, Salesin D, Szeliski R (2005) Panoramic video textures. ACM Trans Graph 24(3):821–827

    Article  Google Scholar 

  2. Barnes C, Shechtman E, Finkelstein A, Goldman DB (2009) Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans Graph 28(3):24:1–24:11

    Article  Google Scholar 

  3. Barron JT, Poole B (2016) The fast bilateral solver. In: European conference on computer vision. Springer, pp 617–632

  4. Brown M, Lowe DG (2007) Automatic panoramic image stitching using invariant features. Int J Comput Vision 74(1):59–73

    Article  Google Scholar 

  5. Capeto U (2014) Depth map automatic generator 5 (dmag5). http://3dstereophoto.blogspot.com/2014/05/depth-map-automatic-generator-5-dmag5.html. Accessed 2019 May 10

  6. Capeto U (2015) Depth map automatic generator 9 (dmag9). http://3dstereophoto.blogspot.com/2015/12/depth-map-automatic-generator-9-dmag9.html. Accessed 2019 May 10

  7. Capeto U (2019) 3d stereoscopic photography, 3d software. http://3dstereophoto.blogspot.com/p/software.html. Accessed 2019 May 10

  8. Chang CH, Liang CK, Chuang YY (2011) Content-aware display adaptation and interactive editing for stereoscopic images. IEEE Trans Multi 13(4):589–601

    Article  Google Scholar 

  9. Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput Vision 59(2):167–181

    Article  Google Scholar 

  10. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun of the ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  11. Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926

    Article  Google Scholar 

  12. Guo Y, Liu F, Shi J, Zhou ZH, Gleicher M (2009) Image retargeting using mesh parametrization. IEEE Trans Multi 11(5):856–867

    Article  Google Scholar 

  13. He K, Chang H, Sun J (2013) Rectangling panoramic images via warping. ACM Trans Graph 32(4):79:1–79:10

    MATH  Google Scholar 

  14. Jin Y, Liu L, Wu Q (2010) Nonhomogeneous scaling optimization for realtime image resizing. Vis Comput 26(6-8):769–778

    Article  Google Scholar 

  15. Kopf J, Kienzle W, Drucker S, Kang SB (2012) Quality prediction for image completion. ACM Trans Graph 31(6):131:1–131:8

    Google Scholar 

  16. Lin SS, Yeh IC, Lin CH, Lee TY (2013) Patch-based image warping for content-aware retargeting. IEEE Trans Multi 15(2):359–368

    Article  Google Scholar 

  17. Lin SS, Lin CH, Chang SH, Lee TY (2014) Object-coherence warping for stereoscopic image retargeting. IEEE Trans Circuits Syst Video Techn 24 (5):759–768

    Article  Google Scholar 

  18. Lin SS, Lin CH, Kuo YH, Lee TY (2016) Consistent volumetric warping using floating boundaries for stereoscopic video retargeting. IEEE Trans Circuits Syst Video Techn 26(5):801–813

    Article  Google Scholar 

  19. Liu F, Hu YH, Gleicher ML (2008) Discovering panoramas in web videos. In: Proceedings of the 16th ACM international conference on multimedia, pp 329–338

  20. Niu Y, Feng WC, Liu F (2012) Enabling warping on stereoscopic images. ACM Trans Graph 31(6):183:1–183:7

    Article  Google Scholar 

  21. Oettermann S (1997) The panorama: history of a mass medium, 1st edn. Zone Books

  22. Rhemann C, Rother C (2012) Fast cost-volume filtering for visual correspondence and beyond. In: CVPR

  23. Shewchuk JR (1996) Triangle: engineering a 2d quality mesh generator and delaunay triangulator. In: Selected papers from the workshop on applied computational geormetry, towards geometric engineering, pp 203–222

  24. Shum HY, Szeliski R (2000) Systems and experiment paper: construction of panoramic image mosaics with global and local alignment. Int J Comput Vision 36(2):101–130

    Article  Google Scholar 

  25. Sorkine O, Cohen-Or D, Lipman Y, Alexa M, Rössl C, Seidel HP (2004) Laplacian surface editing. In: Proceedings of the 2004 eurographics/ACM SIGGRAPH symposium on geometry processing, pp 175–184

  26. Suzuki S, Abe K (1985) Topological structural analysis of digitized binary images by border following. Comput Vis Graph Image Process 30(1):32–46

    Article  MATH  Google Scholar 

  27. Yan T, Huang Z, Lau RWH, Xu Y (2013) Seamless stitching of stereo images for generating infinite panoramas. In: Proceedings of the 19th ACM symposium on virtual reality software and technology, pp 251–258

  28. Yan W, Hou C, Lei J, Fang Y, Gu Z, Ling N (2017) Stereoscopic image stitching based on a hybrid warping model. IEEE Transactions on Circuits and Systems for Video Technology 27(9):1934–1946

    Article  Google Scholar 

  29. Yoon KJ, Kweon IS (2006) Adaptive support-weight approach for correspondence search. IEEE transactions on pattern analysis and machine intelligence (4): 650–656

  30. Zaragoza J, Chin TJ, Brown MS, Suter D (2013) As-projective-as-possible image stitching with moving dlt. In: Proceedings of the 2013 IEEE conference on computer vision and pattern recognition, pp 2339–2346

  31. Zhang F, Liu F (2014) Parallax-tolerant image stitching. In: Proceedings of the 2014 IEEE conference on computer vision and pattern recognition, pp 3262–3269

  32. Zhang F, Liu F (2015) Casual stereoscopic panorama stitching. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 2002–2010

Download references

Acknowledgements

We thank the anonymous reviewers for valuable comments and Cheng-Yue Qiu and Yun-Chen Lin for taking some exprimental videos and preproessing results. This work was supported by Ministry of Science and Technology under no. 108-2221-E-155-033-MY2, 108-2221-E-019-038-MY2, 108-2221-E-006-038-MY3 and 107-2221-E-006-196-MY3.

Author information

Authors and Affiliations

  1. Yuan Ze University, Taoyuan City, Taiwan

    I-Cheng Yeh

  2. National Taiwan Ocean University, Keelung City, Taiwan

    Shih-Syun Lin

  3. MStar Semiconductor, Inc., Hsinchu, Taiwan

    Shuo-Tse Hung

  4. National Cheng-Kung University, Tainan City, Taiwan

    Tong-Yee Lee

Authors
  1. I-Cheng Yeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shih-Syun Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuo-Tse Hung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tong-Yee Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tong-Yee Lee.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yeh, IC., Lin, SS., Hung, ST. et al. Disparity-preserving image rectangularization for stereoscopic panorama. Multimed Tools Appl 79, 26123–26138 (2020). https://doi.org/10.1007/s11042-020-09159-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09159-z

Keywords

Search

Navigation