Skip to main content
Springer Nature Link
Log in

Real-Time Object Detection and Localization for Vision-Based Robot Manipulator

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Traditionally, the robot manipulators were assigned a monotonous task. Therefore, in an aim of simplifying the interactions between unstructured surrounding and robots, as well as to carry out the more complex operations, cognitive abilities such as visual perception are appended to systems for the sake of intelligent operations. Hence, this article directed towards the multistage process involved in the design and development of vision-based robot manipulator suitable for pick and place operation in real-time entailing external disturbances. The intact system development occurs in three phases: first, the robot manipulator with 3-DOF is designed, developed and also enhanced by integrating the vision source. Second, an algorithm for object detection and localization in real time is framed in which segmented object matrices are returned by the former and the position of the stabilized object is returned by the latter. Third, entire hardware–software integration is achieved to perform the desired operation. The algorithm developed in this paper proves to be good for the developed vision-based manipulator, as we achieve quite a good accuracy in object detection algorithm whereas 98.41–99.12% accuracy is achieved in the object localization algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Ahuja N et al. Computer controlled robotic arm. In: Proceedings of the 16th IEEE symposium computer-based medical systems; 2003, pp. 361–6.

  2. Liyanage MH, Krouglicof N, Gosine R. Design and control of a high performance Scara type robotic arm with rotary hydraulic actuators Department of Mechanical Engineering, Faculty of Engineering & Applied Science, Memorial University, Newfoundland, Canada. Electrical Computer Engineering; 2009, pp. 827–32.

  3. Sen Gupta G, Mukhopadhyay SC, Finnie M. WiFi-based control of a robotic arm with remote vision. In: 2009 IEEE instrumentation and measurement technology conference. I2MTC 2009, no. May; 2009, pp. 557–62.

  4. Hao WG, Leck YY, Hun LC. 6-DOF PC-Based Robotic Arm (PC-ROBOARM) with efficient trajectory planning and speed control. no. May; 2011, pp. 1–7.

  5. Szabó R, Gontean A. Creating a programming language for the AL5 type robotic arms. In: 2013 36th international conference on telecommunications and signal processing. TSP 2013; 2013, p. 62–5.

  6. Szabo R, Gontean A. Remotely commanding the Lynxmotion AL5 type robotic arms. In: 2013 21st telecommunications forum Telfor, TELFOR, TELFOR 2013—proceedings paper, no. 2; 2013, p. 889–892.

  7. Spong MW, Vidyasagar M. Robot dynamics and control. New York: Wiley; 1989.

    Google Scholar 

  8. . Kanellakis C, Kyritsis G, Tsilomitrou O, Manesis S. A low-cost stereoscopic µP-based vision system for industrial light objects grasping. In: 2015 23rd Mediterranean conference on control and automation (MED); 2015, p. 759–65.

  9. Batra V, Jadon C, Kumar V. A cognitive framework on object recognition and localization for robotic vision. In: 2020 Indo—Taiwan 2ndinternational conference on computing, analytics and networks (Indo—Taiwan ICAN), Rajpura, India; 2020, p. 125–31.

  10. Batra V, Kumar V. Learn in lab series: image processing (part-I). Seattle: Independently Published at Amazon; 2020.

    Google Scholar 

  11. Kumar R, Lal S, Kumar S, Chand P. Object detection and recognition for a pick and place Robot. In: Asia-Pacific world congress on computer science and engineering; 2014, p. 1–7.

  12. Bui HM, Lech M, Cheng E, Neville K, Burnett IS. Object recognition using deep convolutional features transformed by a recursive network structure. IEEE Access. 2016;4:10059–66.

    Article  Google Scholar 

  13. Loncomilla P, Ruiz-del-Solar J, Martínez L. Object recognition using local invariant features for robotic applications: a survey. Pattern Recognit. 2016;60:499–514.

    Article  Google Scholar 

  14. Bazzani L, Bergamo A, Anguelov D, Torresani L. Self-taught object localization with deep networks. In: 2016 IEEE winter conference on applications of computer vision, WACV 2016; 2016

  15. Oyama E et al. Inverse kinematics learning for robotic arms with fewer degrees of freedom by modular neural network systems. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS; 2005, p. 833–40.

  16. Manigpan S, Kiattisin S, Leelasantitham A. A simulation of 6R industrial articulated robot arm using backpropagation neural network. In: ICCAS 2010—international conference on control, automation and systems; 2010, p. 823–826.

  17. Puheim M, Bundzel M, Madarasz L. Forward control of robotic arm using the information from stereo-vision tracking system. In: CINTI 2013 proceedings of the 14th IEEE international symposium on computational intelligence and informatics; 2013, p. 57–62

  18. Astanin S, Antonelli D, Chiabert P, Alletto C. Reflective workpiece detection and localization for flexible robotic cells. Robot Comput Integr Manuf. 2017;44:190–8.

    Article  Google Scholar 

  19. Zhu H, Yi H, Chellali R, Feng L. Object recognition and localization algorithm base on NAO robot. In: RO-MAN 2018—27th IEEE international conference on robot and human interactive communication; 2018, p. 483–86.

  20. Kazemi M, Gupta KK, Mehrandezh M. Randomized kinodynamic planning for robust visual servoing. IEEE Trans Robot. 2013;29(5):1197–211.

    Article  Google Scholar 

  21. Jinxiang Z, Chuanwei S, Xiaowei C. Study on the identification and localization of corn stalk under complex background. In: ICCSS 2017—2017 international conference on informative and cybernetics for computational social systems; 2017, p. 290–94.

  22. Hong YF, Chang YM, Li CHG. Real-time visual-based localization for mobile robot using structured-view deep learning. IEEE Int Conf Autom Sci Eng. 2019;2019:1353–8.

    Google Scholar 

  23. Kot T, Bobovský Z, Brandstötter M, Krys V, Virgala I, Novák P. Finding optimal manipulator arm shapes to avoid collisions in a static environment. Appl Sci. 2021;11(1):64. https://doi.org/10.3390/app11010064.

    Article  Google Scholar 

  24. Dai L, Yu Y, Zhai D-H, Huang T, Xia Y. Robust model predictive tracking control for robot manipulators with disturbances. IEEE Trans Industr Electron. 2021;68(5):4288–97. https://doi.org/10.1109/TIE.2020.2984986.

    Article  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the Department of Mechanical Engineering, Chitkara University Punjab, India, for providing the required platform and facilities for conducting experimental work in the Robotics and Mechatronics Research Laboratory at their premises.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Chitkara University Institute of Engineering and Technology, Chitkara University, Punjab, India

    Varun Batra & Vijay Kumar

Authors
  1. Varun Batra
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Vijay Kumar
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Varun Batra.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Applications of Cloud Computing, Data Analytics and Building Secure Networks” guest-edited by Rajnish Sharma, Pao-Ann Hsiung and Sagar Juneja”.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Batra, V., Kumar, V. Real-Time Object Detection and Localization for Vision-Based Robot Manipulator. SN COMPUT. SCI. 2, 175 (2021). https://doi.org/10.1007/s42979-021-00561-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-021-00561-4

Keywords