Skip to main content

Advertisement

Springer Nature Link
Log in

Optimized vision-based robot motion planning from multiple demonstrations

  • Published:
Autonomous Robots Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

This paper combines workspace models with optimization techniques to simultaneously address whole-arm collision avoidance, joint limits and camera field of view (FOV) limits for vision-based motion planning of a robot manipulator. A small number of user demonstrations are used to generate a feasible domain over which the whole robot arm can servo without violating joint limits or colliding with obstacles. Our algorithm utilizes these demonstrations to generate new feasible trajectories that keep the target in the camera’s FOV and achieve the desired view of the target (e.g., a pre-grasping location) in new, undemonstrated locations. To fulfill these requirements, a set of control points are selected within the feasible domain. Camera trajectories that traverse these control points are modeled and optimized using either quintic splines (for fast computation) or general polynomials (for better constraint satisfaction). Experiments with a seven degree of freedom articulated arm validate the proposed scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  • Calinon, S., D’halluin, F., Sauser, E. L., Caldwell, D. G., & Billard, A. (2010). Learning and reproduction of gestures by imitation. IEEE Robotics & Automation Magazine, 17(2), 44–54.

    Article  Google Scholar 

  • Chan, A., Croft, E. A., & Little, James J. (2011). Constrained manipulator visual servoing (cmvs): Rapid robot programming in cluttered workspaces. In IEEE/RSJ int. conf. on intelligent robots and systems, IROS’11 (pp. 2825–2830). San Francisco, California.

  • Chan, A., Croft, E. A., & Little, J. J. (2013). Modeling nonconvex workspace constraints from diverse demonstrations sets for constrained manipulator visual servoing. In IEEE int. conf. on robotics and automation, ICRA’13 (pp. 3047–3053) Kongresszentrum Karlsruhe, Karlsruhe, Germany.

  • Chaumette, F. (1998). Potential problems of stability and convergence in image-based and position-based visual servoing. In D. Kriegman, G. Hager, & S. Morse (Eds.), The confluence of vision and control (Vol. 237 , pp. 66–78). Lecture notes in control and information sciences, Springer.

  • Chaumette, F., & Hutchinson, S. (2006). Visual servo control, part I: Basic approaches. IEEE Robotics and Automation Magazine, 13(4), 82–90.

    Article  Google Scholar 

  • Chaumette, F., & Hutchinson, S. (2007). Visual servo control, part II: Advanced approaches. IEEE Robotics and Automation Magazine, 14(1), 109–118.

    Article  Google Scholar 

  • Chesi, G. (2009). Visual servoing path-planning via homogeneous forms and LMI optimizations. IEEE Transactions on Robotics, 25(2), 281–291.

    Article  Google Scholar 

  • Chesi, G., & Hashimoto, K. (Eds.). (2010). Visual servoing via advanced numerical methods. Berlin: Springer.

    MATH  Google Scholar 

  • Chesi, G., & Hung, Y. S. (2007). Global path-planning for constrained and optimal visual servoing. IEEE Transactions on Robotics, 23(5), 1050–1060.

    Article  Google Scholar 

  • Chesi, G., & Shen, T. (2012). Conferring robustness to path-planning for image-based control. IEEE Transactions on Control Systems Technology, 20(4), 950–959.

    Article  Google Scholar 

  • Chesi, G., & Vicino, A. (2004). Visual servoing for large camera displacements. IEEE Transactions on Robotics, 20(4), 724–735.

    Article  Google Scholar 

  • Croft, E. A., Benhabib, B., & Fenton, R. G. (1995). Near-time optimal robot motion planning for online applications. Journal of Robotic Systems, 12(8), 553–567.

    Article  MATH  Google Scholar 

  • Dasgupta, B., Gupta, A., & Singla, E. (2009). A variational approach to path planning for hyper-redundant manipulators. Robotics and Autonomous Systems, 57(2), 194–201.

    Article  Google Scholar 

  • Hosoda, K., Sakamoto, K. & Asada, M. (1995). Trajectory generation for obstacle avoidance of uncalibrated stereo visual servoing without 3d reconstruction. In IEEE/RSJ Int. conf. on intelligent robots and systems, IROS’95 (pp. 29–34). Pittsburgh, PA: IEEE Computer Society.

  • Kavraki, L. E., Svestka, P., Latombe, J.-C., & Overmars, M. H. (1996). Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Transactions on Robotics and Automation, 12(4), 566–580.

    Article  Google Scholar 

  • Khansari-Zadeh, S. M., & Billard, A. (2011). Learning stable non-linear dynamical systems with gaussian mixture models. IEEE Transaction on Robotics, 27(5), 943–957.

    Article  Google Scholar 

  • Kingston, Z., Dantam, N., & Kavraki, L. (2015). Kinematically constrained workspace control via linear optimization. 2015 IEEE-RAS 15th international conference on humanoid robots (humanoids) (pp. 758–764). Seoul, Korea.

  • Klanke, S., Lebedev, D., Haschke, R., Steil, J., & Ritter, H. (2006). Dynamic path planning for a 7-dof robot arm. 2006 IEEE/RSJ international conference on intelligent robots and systems (pp. 3879–3884). Beijing, China.

  • Ladd, A. M., & Kavraki, Lydia E. (2002). Generalizing the analysis of prm. In Proceedings of the 2002 IEEE international conference on robotics and automation (ICRA 2002) (pp. 2120–2125). Washington, DC.

  • Latombe, J. C. (1991). Robot motion planning. Dordrecht: Kluwer Academic Publishers.

    Book  MATH  Google Scholar 

  • Lingelbach, F. (2004). Path planning using probabilistic cell decomposition. In IEEE int. conf. on robotics and automation, ICRA’2004 (pp. 467–472). New Orleans, LA, USA.

  • Mainprice, J., Hayne, R., & Berenson, D. (2015). Using inverse optimal control to predict human reaching motion in collaborative tasks. In workshop on human movement understanding and neuromechanics, Seattle, USA.

  • Mezouar, Y., & Chaumette, F. (2002). Path planning for robust image-based control. IEEE Transactions on Robotics and Automation, 18(4), 534–549.

    Article  Google Scholar 

  • Moon, A., Parker, C. A. C., Croft, E. A. & Van der Loos, H. F. M. (September 2011). Did you see it hesitate? Empirically grounded design of hesitation trajectories for collaborative robots. In IEEE/RSJ int. conf. on intelligent robots and systems, IROS’11. (pp. 1994–1999). San Francisco, California.

  • Pastor, P., Kalakrishnan, M., Meier, F., Stulp, F., Buchli, J., Theodorou, E., et al. (2013a). From dynamic movement primitives to associative skill memories. Robotics and Autonomous Systems, 61(4), 351–361.

    Article  Google Scholar 

  • Pastor, P., Kalakrishnan, M., Meier, F., Stulp, F., Buchli, J., Theodorou, E., et al. (2013b). From dynamic movement primitives to associative skill memories. Robotics and Autonomous Systems, 61(4), 351–361.

    Article  Google Scholar 

  • Ramer, C., Reitelshofer, S., & Franke, J. (2013). A robot motion planner for 6-dof industrial robots based on the cell decomposition of the workspace. 2013 44th international symposium on robotics (ISR) (pp. 1–4). Seoul, Korea.

  • Shen, T., & Chesi, G. (2012). Visual servoing path-planning for cameras obeying the unified model. Advanced Robotics, 26(8–9), 843–860.

    Google Scholar 

  • Shen, T., Radmard, S., Chan, A., Croft, E. A., & Chesi, G. (2013). Motion planning from demonstrations and polynomial optimization for visual servoing applications. In IEEE/RSJ int. conf. on intelligent robots and systems, IROS’13 (pp. 578–583). Tokyo Big Sight, Japan.

  • Shukla, A., Singla, E., Wahi, P., & Dasgupta, B. (2013). A direct variational method for planning monotonically optimal paths for redundant manipulators in constrained workspaces. Robotics and Autonomous Systems, 61(2), 209–220. ISSN 0921-8890.

  • Spong, M. W., Hutchinson, S., & Vidyasagar, M. (2006). Robot modeling and control. Hoboken: Wiley.

    Google Scholar 

  • Steinmetz, F. (2014). Programming by demonstration for in-contact tasks using dynamic movement primitives. Master’s thesis, LuleåUniversity of Technology.

  • Tahri, O., Araujo, H., Chaumette, F., & Mezouar, Y. (2013). Robust image-based visual servoing using invariant visual information. Robotics and Autonomous Systems, 61(12), 1588–1600.

    Article  Google Scholar 

  • Wang, S., Guo, H., Cao, X., Chai, X., Wen, F., & Yuan, K. (2015). Motion planning for vision-based stevedoring tasks on industrial robots. 2015 IEEE international conference on mechatronics and automation (ICMA) (pp. 1264–1269). Beijing, China.

  • Yang, S. X., & Meng, M. Q.-H. (2000). Real-time collision-free path planning of robot manipulators using neural network approaches. Autonomous Robots, 9(1), 27–39.

    Article  Google Scholar 

  • Yershov, D. S., & LaValle, S. M. (2011). Simplicial dijkstra and a* algorithms for optimal feedback planning. In IEEE/RSJ int. conf. on intelligent robots and systems, IROS’11. (pp. 3862–3867), San Francisco, CA.

  • Zhou, F., & De la Torre, F. (2016). Generalized canonical time warping. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 279–294.

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by Hunan Provincial Natural Science Foundation of China under Grant 2017JJ3211 and Scientific Research Fund of Hunan Provincial Education Department under Grant 17B161.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Central South University, Changsha, P. R. China

    Tiantian Shen

  2. Department of Electronic Information Engineering, College of Polytechnic, Hunan Normal University, Changsha, P. R. China

    Tiantian Shen

  3. CARIS Lab, Department of Mechanical Engineering, University of British Columbia, Vancouver, Canada

    Sina Radmard, Ambrose Chan & Elizabeth A. Croft

  4. Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong Kong, Hong Kong S.A.R.

    Graziano Chesi

Authors
  1. Tiantian Shen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Sina Radmard
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ambrose Chan
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Elizabeth A. Croft
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Graziano Chesi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Tiantian Shen.

Additional information

Portions of this paper were submitted to the 2013 IEEE-IROS Shen et al. 2013.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, T., Radmard, S., Chan, A. et al. Optimized vision-based robot motion planning from multiple demonstrations. Auton Robot 42, 1117–1132 (2018). https://doi.org/10.1007/s10514-017-9667-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-017-9667-4

Keywords