Skip to main content
Log in

Autonomous robotic capture of non-cooperative target using visual servoing and motion predictive control

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

This paper presents a framework for autonomous capture operation of a non-cooperative mobile target in a 3-dimensional workspace using a robotic manipulator with visual servoing. The visual servoing with an eye-in-hand configuration is based on motion predictive control using Kalman filter for the on-line state and parameter estimation of the target. A transitional decision making process is developed to guide the robotic manipulator between the different phases of the capture operation by employing a custom metric that translates visual misalignments between the end-effector and the target into a guidance measurement. These phases include the target acquisition and approach stage and the alignment and capture phase. Experiments have been carried out on a custom designed and built robotic manipulator with 6 degrees of freedom. The objective is to evaluate the performance of the proposed motion predictive control scheme for the autonomous capturing task and to demonstrate the robustness of the proposed control scheme in the presence of noise and unexpected disturbances in vision system, sensory-motor coordination and constraints for the execution in real environments. Experimental results of the visual servoing control scheme integrated with the motion predictive Kalman filter indicate the feasibility and applicability of the proposed control scheme. It shows that when the target motion is properly predicted, the tracking and capture performance has been improved significantly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Bradski, G., & Kaehler, A. (2008). Learning OpenCV: Computer Vision With the OpenCV Library. Sebastopol: O’Reilly Media Inc.

    Google Scholar 

  • Chung, T., Hollinger, G., & Isler, V. (2011). Search and pursuit-evasion in mobile robotics a survey. Autonomous Robots, 31, 299–316.

    Article  Google Scholar 

  • Cretual, A., & Chaumetter, F. (2001). Application of motion-based visual servoing to target tracking. The International Journal of Robotics Research, 20, 878–890.

    Article  Google Scholar 

  • Edan, Y. (1995). Design of an autonomous agricultural robot. Applied Intelligence, 5, 45–50.

    Article  Google Scholar 

  • Fang, Y., Liu, X., & Zhang, X. (2011). Motion-estimation-based visual servoing of nonholonomic mobile robots. IEEE Transactions on Robotics, 6, 1167–1175.

    Google Scholar 

  • Fang, Y., Liu, X., & Zhang, X. (2012). Adaptive active visual servoing of nonholonomic mobile robots. IEEE Transactions on Industrial Electronics, 1, 489–497.

    Google Scholar 

  • Fang Y, Behal A, Dixon W, Dawson D (2002) “Adaptive 2.5d visual servoing of kinematically redundant robot manipulators”. In: Proceedings of the 41st IEEE Conference on Decision and Control, vol. 3, (pp. 2860–2865), December 2002.

  • Fumagall, M., Ivaldi, S., Randazzo, M., et al. (2012). Force feedback exploiting tactile and proximal force/torque sensing. Autonomous Robots, 33, 381–398.

    Article  Google Scholar 

  • Ghadyok, V., Goldin, J., & Ren, W. (2012). On the design and development of attitude stabilization, vision-based navigation, and aerial gripping for a low-cost quadrotor. Autonomous Robots, 33, 41–68.

    Article  Google Scholar 

  • Hsiao, K., Kaelbling, L., & Lozano-Perez, T. (2011). Robust grasping under object pose uncertainty. Autonomous Robots, 31, 253–268.

    Article  Google Scholar 

  • Ignakov, D., Okouneva, G., & Liu, G. (2012). Localization of a door handle of unknown geometry using a single camera for door-opening with a mobile manipulator. Autonomous Robots, 33, 415–426.

    Article  Google Scholar 

  • Klein, C., & Huang, C. (1983). Review of pseudoinverse control for use with kinematically redundant manipulators. IEEE Transactions on Systems, 10, 245–250.

    Google Scholar 

  • Kurniawati, H., Bandyopadhyay, T., & Patrikalakis, N. (2012). Global motion planning under uncertain motion, sensing, and environment map. Autonomous Robots, 33, 255–272.

    Article  Google Scholar 

  • Liang, J., & Ma, O. (2011). Angular velocity tracking for satellite rendezvous and docking. Acta Astronautica, 69, 1019–1028.

    Article  Google Scholar 

  • Lippiello, V., Siciliano, B., & Villani, L. (2008). Interaction control of robot manipulators using force and vision. International Journal of Optomechatronics, 2, 257–274.

    Article  Google Scholar 

  • Y.-C. Liu and N. Chopra, Semi-autonomous teleoperation in task space with redundant slave robot under communication delays. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 679–684), September 2011.

  • McKerrow, P. J. (1991). Introduction to Robotics. Sydney: Addison-Wesley.

    Google Scholar 

  • A. Mittal and N. Paragios, Motion-based background subtraction using adaptive kernel density estimation. In: Proceedings of the 2004 IEEE Computer on Computer Vision and Pattern Recognition, 2004, vol. 2, (pp. II-302–II-309), June 2004.

  • A. Ogilvie, J. Allport, M. Hannah, et al., Autonomous robotic operations for on-orbit satellite servicing. In: Proc. SPIE 6958, Sensors and Systems for Space Applications II, no. 695809, April 2008.

  • Rekleitis, I., Martin, E., Rouleau, G., L’Archeveque, R., Parsa, K., & Dupuis, E. (2007). Autonomous capture of a tumbling satellite. Journal of Field Robotics, 24(4), 275–296.

    Article  Google Scholar 

  • F. Roe, R. Howard, and L. Murphy, Automated rendezvous and capture system development and simulation for nasa. In: P. Motaghedi (Ed.), Modeling, Simulation and Calibration of Space-Based Systems, Proceedings of the SPIE.

  • De Schutter, J. (1987). A study of active compliant motion control methods for rigid manipulators based on a generic scheme. IEEE International Conference on Robotics and Automation, 4, 1060–1065.

    Google Scholar 

  • Slaughter, D., Giles, D., & Downey, D. (2008). Autonomous robotic weed control systems: A review. Computers and Electronics in Agriculture, 61, 63–98.

    Article  Google Scholar 

  • Wang, H., & Xie, Y. (2009). Adaptive jacobian position/force tracking control of free-flying manipulators. Robotics and Autonomous Systems, 57, 173–181.

    Article  MathSciNet  Google Scholar 

  • Wong, A., Mayorga, R., & Rong, L. (1996). Liang X (1996) A vision based on-line motion planning of robot manipulators. In: Proceedings of the. IEEE/RSJ international conference on intelligent robots and systems - robotic intelligence interacting with dynamic worlds, vol. 1–3, (pp. 940–948).

  • Zhang, S., Ding, Y., Hao, K., & Zhang, D. (2012). An efficient two-step solution for vision-based pose determination of a parallel manipulator. Robotics and Computer-Integrated Manufacturing, 2, 182–189.

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Ontario Graduate Scholarships in Science and Technology (OGSST).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zheng H. Zhu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Larouche, B.P., Zhu, Z.H. Autonomous robotic capture of non-cooperative target using visual servoing and motion predictive control. Auton Robot 37, 157–167 (2014). https://doi.org/10.1007/s10514-014-9383-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-014-9383-2

Keywords

Navigation