Skip to main content
SpringerLink
Log in

A survey on digital video stabilization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Shakes and jitters are an eventual result of involuntary camera movements during video recording. Digital video stabilization is the elimination of these errors with smart algorithms. This process is usually performed in three steps which are camera motion estimation, motion correction, and stable video synthesis. In the literature, methods differ by the way they perform these steps. The recent success of deep learning has pioneered learning-based video stabilization approaches. This paper provides a detailed explanation of video stabilization methods by analyzing and comparing the applied approaches from past to present.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Ali MK, Yu S, Kim TH (2020) Learning deep video stabilization without optical flow. arXiv:2011.09697

  2. Auberger S, Miro C (2005) Digital video stabilization architecture for low cost devices. In: ISPA 2005. Proceedings of the 4th international symposium on image and signal processing and analysis, 2005, IEEE, pp 474–479

  3. Battiato S, Gallo G, Puglisi G, Scellato S (2007) Sift features tracking for video stabilization. In: 14Th international conference on image analysis and processing (ICIAP 2007), IEEE, pp 825–830

  4. Battiato S, Puglisi G, Bruna A (2008) A robust video stabilization system by adaptive motion vectors filtering. In: 2008 IEEE International conference on multimedia and expo, IEEE, pp 373–376

  5. Bay H, Tuytelaars T, Van Gool L (2006) Surf: Speeded up robust features. In: European conference on computer vision, Springer, pp 404–417

  6. Bradley A, Klivington J, Triscari J, van der Merwe R (2021) Cinematic-l1 video stabilization with a log-homography model. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 1041–1049

  7. Broggi A, Grisleri P, Graf T, Meinecke M (2005) A software video stabilization system for automotive oriented applications. In: 2005 IEEE 61St vehicular technology conference, vol 5. IEEE, pp 2760–2764

  8. Buehler C, Bosse M, McMillan L (2001) Non-metric image-based rendering for video stabilization. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, vol 2. IEEE, pp II–II

  9. Calonder M, Lepetit V, Strecha C, Fua P (2010) Brief: Binary robust independent elementary features. In: European conference on computer vision, Springer, pp 778–792

  10. Cao M, Zheng L, Jia W, Liu X (2020) Real-time video stabilization via camera path correction and its applications to augmented reality on edge devices. Comput Commun 158:104–115

    Article  Google Scholar 

  11. Chang HC, Lai SH, Lu KR (2006) A robust real-time video stabilization algorithm. J Vis Commun Image Represent 17(3):659–673

    Article  Google Scholar 

  12. Chen BY, Lee KY, Huang WT, Lin JS (2008) Capturing intention-based full-frame video stabilization. In: Computer graphics forum, vol 27. Wiley online library, pp 1805–1814

  13. Chen K, Tu J, Yao J (2018) Generalized content-preserving warps for image stitching. arXiv:1809.06783

  14. Chen YT, Tseng KW, Lee YC, Chen CY, Hung YP (2021) Pixstabnet: Fast multi-scale deep online video stabilization with pixel-based warping. In: 2021 IEEE International conference on image processing (ICIP), IEEE, pp 1929–1933

  15. Choi J, Kweon IS (2020) Deep iterative frame interpolation for full-frame video stabilization. ACM Trans Graph (TOG) 39(1):1–9

    Article  Google Scholar 

  16. Choi J, Park J, Kweon IS (2021) Self-supervised real-time video stabilization. arXiv:2111.05980

  17. Djuric PM, Kotecha JH, Zhang J, Huang Y, Ghirmai T, Bugallo MF, Miguez J (2003) Particle filtering. IEEE Signal Proc Mag 20(5):19–38

    Article  Google Scholar 

  18. Farid H, Woodward JB (1997) Video stabilization and enhancement. TR2007-605. Dartmouth College, Computer Science

  19. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  20. Garbin SJ, Kowalski M, Johnson M, Shotton J, Valentin J (2021) Fastnerf: High-fidelity neural rendering at 200fps. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 14346–14355

  21. Grundmann M, Kwatra V, Essa I (2011) Auto-directed video stabilization with robust l1 optimal camera paths. In: CVPR 2011, IEEE, pp 225–232

  22. Guilluy W, Oudre L, Beghdadi A (2021) Video stabilization: overview, challenges and perspectives. Signal Process Image Commun 116015:90

    Google Scholar 

  23. Hu R, Shi R, Shen If, Chen W (2007) Video stabilization using scale-invariant features. In: 2007 11Th international conference information visualization (IV’07), IEEE, pp 871–877

  24. Huang CH, Yin H, Tai YW, Tang CK (2019) Stablenet: Semi-online, multi-scale deep video stabilization. arXiv:1907.10283

  25. Huang H, Lin L, Tong R, Hu H, Zhang Q, Iwamoto Y, Han X, Chen YW, Wu J (2020) Unet 3+: a full-scale connected unet for medical image segmentation. In: ICASSP 2020-2020 IEEE International conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 1055–1059

  26. Ito MS, Izquierdo E (2020) Distortion-free video stabilization. In: 2020 IEEE International conference on multimedia & expo workshops (ICMEW), IEEE, pp 1–4

  27. Ito MS, Izquierdo E (2021) Deep homography-based video stabilization. In: 2020 25Th international conference on pattern recognition (ICPR), IEEE, pp 8819–8825

  28. Jaderberg M, Simonyan K, Zisserman A, et al. (2015) Spatial transformer networks. In: Advances in neural information processing systems, pp 2017–2025

  29. Kaba K, Sarıgül M, Avcı M, Kandırmaz HM (2018) Estimation of daily global solar radiation using deep learning model. Energy 162:126–135

    Article  Google Scholar 

  30. Karageorgos K, Dimou A, Axenopoulos A, Daras P, Alvarez F (2017) Semantic filtering for video stabilization. In: 2017 14Th IEEE international conference on advanced video and signal based surveillance (AVSS), IEEE, pp 1–6

  31. ki Paik J, Park YC, Kim DW (1992) An adaptive motion decision system for digital image stabilizer based on edge pattern matching. IEEE Trans Consum Electron 38(3):607–616

    Article  Google Scholar 

  32. Kim SK, Kang SJ, Wang TS, Ko SJ (2013) Feature point classification based global motion estimation for video stabilization. IEEE Trans Consum Electron 59(1):267–272

    Article  Google Scholar 

  33. Kosiorek AR, Strathmann H, Zoran D, Moreno P, Schneider R, Mokrá S, Rezende DJ (2021) Nerf-vae: a geometry aware 3d scene generative model. In: International conference on machine learning, PMLR, pp 5742–5752

  34. Kwon O, Shin J, Paik J (2005) Video stabilization using kalman filter and phase correlation matching. In: International conference image analysis and recognition, Springer, pp 141–148

  35. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  36. Lee KY, Chuang YY, Chen BY, Ouhyoung M (2009) Video stabilization using robust feature trajectories. In: 2009 IEEE 12Th international conference on computer vision, IEEE, pp 1397–1404

  37. Lee YC, Tseng KW, Chen YT, Chen CC, Chen CS, Hung YP (2021) 3d video stabilization with depth estimation by cnn-based optimization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10621–10630

  38. Liang YM, Tyan HR, Chang SL, Liao HY, Chen SW (2004) Video stabilization for a camcorder mounted on a moving vehicle. IEEE Trans Veh Technol 53(6):1636–1648

    Article  Google Scholar 

  39. Litvin A, Konrad J, Karl WC (2003) Probabilistic video stabilization using kalman filtering and mosaicking. In: Image and video communications and processing 2003, vol 5022. International Society for Optics and Photonics, pp 663–674

  40. Liu F, Gleicher M, Jin H, Agarwala A (2009) Content-preserving warps for 3d video stabilization. ACM Trans Graph (TOG) 28(3):1–9

    Google Scholar 

  41. Liu F, Gleicher M, Wang J, Jin H, Agarwala A (2011) Subspace video stabilization. ACM Trans Graph (TOG) 30(1):1–10

    Article  Google Scholar 

  42. Liu S, Tan P, Yuan L, Sun J, Zeng B (2016) Meshflow: Minimum latency online video stabilization. In: European conference on computer vision, Springer, pp 800–815

  43. Liu S, Yuan L, Tan P, Sun J (2013) Bundled camera paths for video stabilization. ACM Trans Graph (TOG) 32(4):1–10

    Google Scholar 

  44. Liu S, Yuan L, Tan P, Sun J (2014) Steadyflow: Spatially smooth optical flow for video stabilization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4209–4216

  45. Liu YL, Lai WS, Yang MH, Chuang YY, Huang JB (2021) Hybrid neural fusion for full-frame video stabilization. arXiv:2102.06205

  46. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  47. Lucas BD, Kanade T et al (1981) An iterative image registration technique with an application to stereo vision

  48. Matsushita Y, Ofek E, Ge W, Tang X, Shum HY (2006) Full-frame video stabilization with motion inpainting. IEEE Trans Pattern Anal Mach Intell 28(7):1150–1163

    Article  Google Scholar 

  49. Niskanen M, Silvén O, Tico M (2006) Video stabilization performance assessment. In: 2006 IEEE International conference on multimedia and expo, IEEE, pp 405–408

  50. Pinto B, Anurenjan P (2011) Video stabilization using speeded up robust features. In: 2011 International conference on communications and signal processing, IEEE, pp 527–531

  51. Raj R, Rajiv P, Kumar P, Khari M, Verdú E, Crespo RG, Manogaran G (2020) Feature based video stabilization based on boosted haar cascade and representative point matching algorithm. Image Vis Comput 103957:101

    Google Scholar 

  52. Ratakonda K (1998) Real-time digital video stabilization for multi-media applications. In: ISCAS’98. Proceedings of the 1998 IEEE international symposium on circuits and systems (Cat. No. 98CH36187), vol 4. IEEE, pp 69–72

  53. Roberto Me, Maia HdA, Pedrini H (2022) Survey on digital video stabilization: concepts, methods, and challenges. ACM Comput Surv (CSUR) 55(3):1–37

    Article  Google Scholar 

  54. Rosten E, Porter R, Drummond T (2008) Faster and better: a machine learning approach to corner detection. IEEE Trans Pattern Anal Mach Intell 32 (1):105–119

    Article  Google Scholar 

  55. Rublee E, Rabaud V, Konolige K, Bradski G (2011) Orb: an efficient alternative to sift or surf. In: 2011 International conference on computer vision, IEEE, pp 2564–2571

  56. Sarıgül M, Ozyildirim BM, Avci M (2019) Differential convolutional neural network. Neural Netw 116:279–287

    Article  Google Scholar 

  57. Shen X, Wang C, Li X, Yu Z, Li J, Wen C, Cheng M, He Z (2019) Rf-net: an end-to-end image matching network based on receptive field. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8132–8140

  58. Shen Y, Guturu P, Damarla T, Buckles BP, Namuduri KR (2009) Video stabilization using principal component analysis and scale invariant feature transform in particle filter framework. IEEE Trans Consum Electron 55(3):1714–1721

    Article  Google Scholar 

  59. Shi J, et al. (1994) Good features to track. In: 1994 Proceedings of IEEE conference on computer vision and pattern recognition, IEEE, pp 593–600

  60. Shi Z, Shi F, Lai WS, Liang CK, Liang Y (2022) Deep online fused video stabilization. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 1250–1258

  61. Shih ML, Su SY, Kopf J, Huang JB (2020) 3d photography using context-aware layered depth inpainting. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8028–8038

  62. Sobel I, Feldman G (1968) A 3x3 isotropic gradient operator for image processing, a talk stanford artif

  63. Song C, Zhao H, Jing W, Zhu H (2012) Robust video stabilization based on particle filtering with weighted feature points. IEEE Trans Consum Electron 58(2):570–577

    Article  Google Scholar 

  64. Targ S, Almeida D, Lyman K (2016) Resnet in resnet: Generalizing residual architectures. arXiv:1603.08029

  65. Tico M, Vehvilainen M (2005) Constraint motion filtering for video stabilization. In: IEEE International conference on image processing 2005, vol 3. IEEE, pp III–569

  66. Ullman S (1979) The interpretation of structure from motion. Proc R Soc London Ser B Biol Sci 203(1153):405–426

    Google Scholar 

  67. Vlahović N, Ilić N, Stanković M (2018) Deep learning in video stabilization homography estimation. In: 2018 14Th symposium on neural networks and applications (NEUREL), IEEE, pp 1–5

  68. Walha A, Wali A, Alimi AM (2015) Video stabilization with moving object detecting and tracking for aerial video surveillance. Multimed Tools Appl 74(17):6745–6767

    Article  Google Scholar 

  69. Wang JM, Chou HP, Chen SW, Fuh CS (2009) Video stabilization for a hand-held camera based on 3d motion model. In: 2009 16Th IEEE international conference on image processing (ICIP), IEEE, pp 3477–3480

  70. Wang M, Yang GY, Lin JK, Shamir A, Zhang SH, Lu SP, Hu SM (2018) Deep online video stabilization. arXiv:1802.08091

  71. Wang M, Yang GY, Lin JK, Zhang SH, Shamir A, Lu SP, Hu SM (2018) Deep online video stabilization with multi-grid warping transformation learning. IEEE Trans Image Process 28(5):2283–2292

    Article  MathSciNet  Google Scholar 

  72. Wang YS, Liu F, Hsu PS, Lee TY (2013) Spatially and temporally optimized video stabilization. IEEE Trans Vis Comput Graph 19(8):1354–1361

    Article  Google Scholar 

  73. Welch G, Bishop G et al (1995) An introduction to the kalman filter

  74. Wu H, Xiao L, Wei Z (2021) Simultaneous video stabilization and rolling shutter removal. IEEE Trans Image Process 30:4637–4652

    Article  Google Scholar 

  75. Xu J, Chang Hw, Yang S, Wang M (2012) Fast feature-based video stabilization without accumulative global motion estimation. IEEE Trans Consum Electron 58(3):993–999

    Article  Google Scholar 

  76. Xu SZ, Hu J, Wang M, Mu TJ, Hu SM (2018) Deep video stabilization using adversarial networks. In: Computer graphics forum, vol 37. Wiley Online Library, pp 267–276

  77. Xu Y, Zhang J, Maybank SJ, Tao D (2020) Dut: Learning video stabilization by simply watching unstable videos. arXiv:2011.14574

  78. Xu Y, Zhang J, Tao D (2021) Out-of-boundary view synthesis towards full-frame video stabilization. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 4842–4851

  79. Yang J, Schonfeld D, Chen C, Mohamed M (2006) Online video stabilization based on particle filters. In: 2006 International conference on image processing, IEEE, pp 1545–1548

  80. Yang J, Schonfeld D, Mohamed M (2009) Robust video stabilization based on particle filter tracking of projected camera motion. IEEE Trans Circ Syst Video Technol 19(7):945–954

    Article  Google Scholar 

  81. Yu J, Ramamoorthi R (2018) Selfie video stabilization. In: Proceedings of the European conference on computer vision (ECCV), pp 551–566

  82. Yu J, Ramamoorthi R (2019) Robust video stabilization by optimization in cnn weight space. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3800–3808

  83. Yu J, Ramamoorthi R (2020) Learning video stabilization using optical flow. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8159–8167

  84. Yu J, Ramamoorthi R, Cheng K, Sarkis M, Bi N (2021) Real-time selfie video stabilization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12036–12044

  85. Zeng Y, Fu J, Chao H (2020) Learning joint spatial-temporal transformations for video inpainting. In: European conference on computer vision, Springer, pp 528–543

  86. Zhang G, Hua W, Qin X, Shao Y, Bao H (2009) Video stabilization based on a 3d perspective camera model. Vis Comput 25(11):997

    Article  Google Scholar 

  87. Zhang G, Qin X, Hua W, Wong TT, Heng PA, Bao H (2007) Robust metric reconstruction from challenging video sequences. In: 2007 IEEE Conference on computer vision and pattern recognition, IEEE, pp 1–8

  88. Zhang Y, Zhang Y, Cai W (2018) Separating style and content for generalized style transfer. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8447–8455

  89. Zhao M, Ling Q (2020) Adaptively meshed video stabilization. IEEE Trans Circ Syst Video Technol

  90. Zhao M, Ling Q (2020) Pwstablenet: Learning pixel-wise warping maps for video stabilization. IEEE Trans Image Process 29:3582–3595

    Article  MATH  Google Scholar 

  91. Zhou Z, Jin H, Ma Y (2013) Plane-based content preserving warps for video stabilization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2299–2306

  92. Zhu X, Guo K, Fang H, Chen L, Ren S, Hu B (2021) Cross view capture for stereo image super-resolution. IEEE Trans Multimed

  93. Zhu X, Guo K, Ren S, Hu B, Hu M, Fang H (2021) Lightweight image super-resolution with expectation-maximization attention mechanism. IEEE Trans Circ Syst Video Technol

  94. Zhu X, Liu X, Lei Z, Li SZ (2017) Face alignment in full pose range: a 3d total solution. IEEE Trans Pattern Anal Mach Intell 41(1):78–92

    Article  Google Scholar 

Download references

Acknowledgements

This work has been supported by project number 120E447 from the TUBITAK (Scientific and Technological Research Council of Turkey).

Author information

Authors and Affiliations

  1. Cukurova University, Computer Engineering Department, Adana, Turkey

    Mehmet Sarıgül

  2. Iskenderun Technical University, Computer Engineering Department, Iskenderun/Hatay, Turkey

    Mehmet Sarıgül

Authors
  1. Mehmet Sarıgül
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mehmet Sarıgül.

Ethics declarations

Conflict of Interests

The author declares that there is no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sarıgül, M. A survey on digital video stabilization. Multimed Tools Appl 82, 40181–40207 (2023). https://doi.org/10.1007/s11042-023-14726-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14726-1

Keywords

Search

Navigation