Skip to main content
SpringerLink
Log in

Content-and-disparity-aware stereoscopic video stabilization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Filming stereoscopic videos has become easier with the development of science and technology, and such videos now proliferate on the Internet. Meanwhile, video stabilization is an important research topic. Thus, this study presents a method of stabilizing stereoscopic videos with preserving the disparities between objects in the frames. First, the feature points must be tracked and separated into many groups. We posit that the shaky motion is caused not only by translations but also by rotations. Thus, directly smoothing the path will not produce a similar trajectory so that we solve the shakiness of the turning before smoothing the path. To address such shakiness, we initially estimate the rotation angles between two adjacent frames. By determining the angle changes of all the frames, we can find out the preference of rotation in a video. Furthermore, the inconsistent angular velocity can be alleviated and the shakiness of the turning is solved by rotating the frame appropriately. Then, the Bézier curve is utilized to smooth the trajectories. We split a trajectory into a set of subtrajectories and subsequently smooth the latter independently. Unlike previous researches, we split the trajectory according to the feature tracking rate to obtain similar trajectories in the original video path. After making subtrajectories smooth, we merge them to attain a smoothed trajectory. The joint of the two subtrajectories is replaced by their interpolation. Finally, we optimize the smoothness and context preservation to stabilize videos without requiring extensive clipping.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. (Accessed date 30 June 2017) Voodoo camera tracker: A tool for the integration of virtual and real scenes. ftp://ftp.tnt.uni-hannover.de/pub/digilab/

  2. Fan D, Wang W, Cheng M, Shen J (2019) Shifting more attention to video salient object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8546–8556

  3. Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(10):1915–1926

    Article  Google Scholar 

  4. Goldstein A, Fattal R (2012) Video stabilization using epipolar geometry. ACM Trans Graph 31(5):126:1–126:10

    Article  Google Scholar 

  5. Grundmann M, Kwatra V, Essa I (2011) Auto-directed video stabilization with robust l1 optimal camera paths. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 225–232

  6. Grundmann M, Kwatra V, Han M, Essa I (2010) Efficient hierarchical graph-based video segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2141–2148

  7. Grundmann M, Kwatra V, Castro D, Essa I (2012) Effective calibration free rolling shutter removal, pp 1–8

  8. Guo H, Liu S, Zhu S, Zeng B (2016) Joint bundled camera paths for stereoscopic video stabilization. In: Proceedings of the IEEE International Conference on Image Processing, pp 1071–1075

  9. Guo H, Liu S, He T, Zhu S, Zeng B, Gabbouj M (2016) Joint video stitching and stabilization from moving cameras. IEEE Trans Image Process 25(11):5491–5503

    Article  MathSciNet  Google Scholar 

  10. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  11. Jia C, Sinno Z, Evans BL (2014) Real-time 3d rotation smoothing for video stabilization. In: Proceedings of the Asilomar Conference on Signals, Systems and Computers, pp 673–677

  12. Lee K-Y, Chuang Y-Y, Chen B-Y, Ouhyoung M (2009) Video stabilization using robust feature trajectories. In: Proceedings of the International Conference on Computer Vision, pp 1397–1404

  13. Lin S, Lin C, Yeh I, Chang S, Yeh C, Lee T (2013) Content-aware video retargeting using object-preserving warping. IEEE Trans Vis Comput Graph 19(10):1677–1686

    Article  Google Scholar 

  14. Liu F, Gleicher M, Jin H, Agarwala A (2009) Content-preserving warps for 3d video stabilization. ACM Trans Graph 28(3):44:1–44:9

    Google Scholar 

  15. Liu F, Gleicher M, Wang J, Jin H, Agarwala A (2011) Subspace video stabilization. ACM Trans Graph 30(1):4:1–4:10

    Article  Google Scholar 

  16. Liu S, Wang Y, Yuan L, Bu J, Tan P, Sun J (2012) Video stabilization with a depth camera. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 89–95

  17. Liu S, Yuan L, Tan P, Sun J (2013) Bundled camera paths for video stabilization. ACM Trans Graph 32(4):78:1–78:10

    Google Scholar 

  18. Matsushita Y, Ofek E, Ge W, Tang X, Shum H-Y (2006) Full-frame video stabilization with motion inpainting. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(7):1150–1163

    Article  Google Scholar 

  19. Morimoto C, Chellappa R (1998) Evaluation of image stabilization algorithms. In: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, 5, pp 2789–2792

  20. Nie G, Cheng M, Liu Y, Liang Z, Fan D, Liu Y, Wang Y (2019) Multi-level context ultra-aggregation for stereo matching. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3278–3286

  21. Smith BM, Zhang L, Jin H, Agarwala A (2009) Light field video stabilization. In: Proceedings of the IEEE International Conference on Computer Vision, pp 341–348

  22. Wang W, Shen J, Xie J, Cheng M, Ling H, Borji A (2019) Revisiting video saliency prediction in the deep learning era. In: IEEE Transactions on Pattern Analysis and Machine Intelligence https://doi.org/10.1109/TPAMI.2019.2924417

  23. Wang Y, Liu F, Hsu P, Lee T (2013) Spatially and temporally optimized video stabilization. IEEE Trans Vis Comput Graph 19(8):1354–1361

    Article  Google Scholar 

  24. Zhang L, Xu Q, Huang H (2017) A global approach to fast video stabilization. IEEE Transactions on Circuits and Systems for Video Technology 27 (2):225–235

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported in part by the Ministry of Science and Technology (contracts MOST-108-2221-E-019-038-MY2, MOST-108-2221-E-006-038-MY3 and MOST-107-2221-E-006-196-MY3) of Taiwan.

Author information

Authors and Affiliations

  1. National Taiwan Ocean University, Keelung, Taiwan

    Shih-Syun Lin

  2. National Cheng-Kung University, Tainan, Taiwan

    Thi Ngoc Hanh Le, Pang-Yu Wu & Tong-Yee Lee

Authors
  1. Shih-Syun Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thi Ngoc Hanh Le
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pang-Yu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tong-Yee Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tong-Yee Lee.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, SS., Le, T.N.H., Wu, PY. et al. Content-and-disparity-aware stereoscopic video stabilization. Multimed Tools Appl 80, 1545–1564 (2021). https://doi.org/10.1007/s11042-020-09767-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09767-9

Keywords

Search

Navigation