Skip to main content
SpringerLink
Log in

Video stabilization algorithm for field robots in uneven terrain

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

Field robots equipped with visual sensors have been used to automate several services. In many scenarios, these robots are tele-operated by a remote operator who controls the robot motion based on a live video feed from the robot’s cameras. In other cases, like surveillance and monitoring applications, the video recorded by the robot is later analyzed or inspected manually. A shaky video is produced on an uneven terrain. It could also be caused due to loose and vibrating mechanical frame on which the camera has been mounted. Jitters or shakes in these videos are undesired for tele-operation, and to maintain desired quality of service. In this paper, we present an algorithm to stabilize the undesired jitters in a shaky video using only the camera information for different areas of vineyard based on terrain profile. The algorithm works by tracking robust feature points in the successive frames of the camera, smoothing the trajectory, and generating desired transformations to output a stabilized video. We have tested the algorithm in actual field robots in uneven terrains used for agriculture, and found the algorithm to produce good results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Auysakul J, Xu H, Pooneeth V (2018) A hybrid motion estimation for video stabilization based on an imu sensor. Sensors 18(8), https://doi.org/10.3390/s18082708, https://www.mdpi.com/1424-8220/18/8/2708

  2. Ballester C, Garrido L, Lazcano V, Caselles V (2012) A tv-l1 optical flow method with occlusion detection. In: Pinz A, Pock T, Bischof H, Leberl F (eds) Pattern Recognition. Springer, Heidelberg, pp 31–40

    Chapter  Google Scholar 

  3. Chen J, DuanStudent Member H, SongStudent Member Y, Cai Z, Yang G (2022) Optical flow computation for video under the dynamic illumination. IEEE Transactions on Multimedia pp 1–16, https://doi.org/10.1109/TMM.2022.3207583

  4. Dobrčki T, Oh Y, Ko H, Kim T, Kim D, Hong BW (2022a) Incremental learning for optical flow. In: 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), pp 588–590, https://doi.org/10.1109/ICTC55196.2022.9952534

  5. Dobrički T, Zhuang X, Won KJ, Hong BW (2022b) Survey on unsupervised learning methods for optical flow estimation. In: 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), pp 591–594, https://doi.org/10.1109/ICTC55196.2022.9952910

  6. Gray JL, Naman AT, Taubman DS (2022) Gradient consistency based multi-scale optical flow. In: 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP), pp 1–6, https://doi.org/10.1109/MMSP55362.2022.9949072

  7. Jeon S, Yoon I, Jang J, Yang S, Kim J, Paik J (2017) Robust video stabilization using particle keypoint update and l1-optimized camera path. Sensors 17(2), https://doi.org/10.3390/s17020337, https://www.mdpi.com/1424-8220/17/2/337

  8. Koh YJ, Sim JY, Kim CS (2014) Robust video stabilization based on mesh grid warping of rolling-free features. In: 2014 IEEE International Conference on Image Processing (ICIP), pp 5781–5785, https://doi.org/10.1109/ICIP.2014.7026169

  9. Kouskoulis G, Antoniou C, Spyropoulou I (2019) A method for the treatment of pedestrian trajectory data noise. Transportation Research Procedia 41:782–798, https://doi.org/10.1016/j.trpro.2019.09.126, urban Mobility - Shaping the Future Together mobil.TUM 2018 - International Scientific Conference on Mobility and Transport Conference Proceedings

  10. Li Z, Pundlik S, Luo G (2013) Stabilization of magnified videos on a mobile device for visually impaired. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp 54–55, https://doi.org/10.1109/CVPRW.2013.15

  11. Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence - Volume 2, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, IJCAI’81, p 674-679

  12. Luo Y, Ying X, Li R, Wan Y, Hu B, Ling Q (2022) Multi-scale optical flow estimation for video infrared small target detection. In: 2022 2nd International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI), pp 129–132, https://doi.org/10.1109/CEI57409.2022.9950186

  13. Matsushita Y, Ofek E, Ge W, Tang X, Shum HY (2006) Full-frame video stabilization with motion inpainting. IEEE Trans Pattern Anal Mach Intell 28(7):1150–1163. https://doi.org/10.1109/TPAMI.2006.141

    Article  Google Scholar 

  14. Milanović PD, Popadić IV, Kovačević BD (2021) Gyroscope-based video stabilization for electro-optical long-range surveillance systems. Sensors 21(18), https://doi.org/10.3390/s21186219, https://www.mdpi.com/1424-8220/21/18/6219

  15. Mohammadi M, Fathi M, Soryani M (2011) A new decoder side video stabilization using particle filter. In: 2011 18th International Conference on Systems, Signals and Image Processing, pp 1–4

  16. Molnár J, Chetverikov D, Fazekas S (2010) Illumination-robust variational optical flow using cross-correlation. Comput Vis Image Underst 114(10):1104–1114. https://doi.org/10.1016/j.cviu.2010.07.006

    Article  Google Scholar 

  17. Park S, Kwak N (2015) Illumination robust optical flow estimation by illumination-chromaticity decoupling. In: 2015 IEEE International Conference on Image Processing (ICIP), pp 1910–1914, https://doi.org/10.1109/ICIP.2015.7351133

  18. Ravankar A, Ravankar AA, Hoshino Y, Emaru T, Kobayashi Y (2016) On a hopping-points svd and hough transform-based line detection algorithm for robot localization and mapping. Int J Adv Rob Syst 13(3):98. https://doi.org/10.5772/63540

    Article  Google Scholar 

  19. Ravankar A, Ravankar AA, Hoshino Y, Emaru T, Kobayashi Y (2016) On a hopping-points svd and hough transform based line detection algorithm for robot localization and mapping. Int J Adv Rob Syst 13(3):98. https://doi.org/10.5772/63540

    Article  Google Scholar 

  20. Ravankar A, A Ravankar A, Watanabe M, Hoshino Y, Rawankar A (2020a) Development of a low-cost semantic monitoring system for vineyards using autonomous robots. Agriculture 10(5), https://doi.org/10.3390/agriculture10050182, https://www.mdpi.com/2077-0472/10/5/182

  21. Ravankar A, Ravankar AA, Rawankar A, Hoshino Y (2021) Autonomous and safe navigation of mobile robots in vineyard with smooth collision avoidance. Agriculture 11(10), https://doi.org/10.3390/agriculture11100954, https://www.mdpi.com/2077-0472/11/10/954

  22. Ravankar AA, Ravankar A, Emaru T, Kobayashi Y (2020b) Hpprm: Hybrid potential based probabilistic roadmap algorithm for improved dynamic path planning of mobile robots. IEEE Access 8:221,743–221,766, https://doi.org/10.1109/ACCESS.2020.3043333

  23. Ravankar AA, Ravankar AA, Watanabe M, Hoshino Y, Rawankar A (2020) Multi-robot path planning for smart access of distributed charging points in map. Artif Life Robot 26:52–60. https://doi.org/10.1007/s10015-020-00612-8

    Article  Google Scholar 

  24. Tsoligkas NA, Xu D, French I, Luo Y (2006) A motion model based video stabilisation algorithm. In: 2006 World Automation Congress, pp 1–6, https://doi.org/10.1109/WAC.2006.375744

  25. Wang X, Liu S (2021) Vehicle trajectory optimization based on limiting average algorithm. IEEE Access 9:9595–9599. https://doi.org/10.1109/ACCESS.2020.3047386

    Article  Google Scholar 

  26. Wang Y, Huang Q, Zhang D, Chen Y (2017) Digital video stabilization based on block motion estimation. In: 2017 International Conference on Computer Technology, Electronics and Communication (ICCTEC), pp 894–897, https://doi.org/10.1109/ICCTEC.2017.00198

  27. Yoon I, Jeon S, Jeong S, Paik J (2015) Video stabilization using classification-based homography estimation for consumer video camera. In: 2015 IEEE 5th International Conference on Consumer Electronics - Berlin (ICCE-Berlin), pp 116–117, https://doi.org/10.1109/ICCE-Berlin.2015.7391209

  28. Zhang G, Yu L, Wang W (2010) Video stabilization algorithm based on video object segmentation. In: 2010 2nd International Conference on Future Computer and Communication, vol 2, pp V2–509–V2–512, https://doi.org/10.1109/ICFCC.2010.5497504

Download references

Acknowledgements

This work was supported by JSPS KAKENHI Grant Number 21K14117.

Author information

Authors and Affiliations

  1. Faculty of Engineering, Kitami Institute of Technology, Kitami, Hokkaido, Japan

    Abhijeet Ravankar

  2. Department of Electronics and Telecommunication Engineering, Thakur College of Engineering and Technology, Mumbai, India

    Arpit Rawankar

  3. Department of Robotics, Smart Robots Design Laboratory, Tohoku University, Sendai, Japan

    Ankit A. Ravankar

Authors
  1. Abhijeet Ravankar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arpit Rawankar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ankit A. Ravankar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abhijeet Ravankar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was presented in part at the joint symposium of the 27th International Symposium on Artificial Life and Robotics, the 7th International Symposium on BioComplexity, and the 5th International Symposium on Swarm Behavior and Bio-Inspired Robotics (Online, January 25–27, 2022).

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ravankar, A., Rawankar, A. & Ravankar, A.A. Video stabilization algorithm for field robots in uneven terrain. Artif Life Robotics 28, 502–508 (2023). https://doi.org/10.1007/s10015-023-00883-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-023-00883-x

Keywords

Search

Navigation