Skip to main content
SpringerLink
Log in

Robotic assembly solution by human-in-the-loop teaching method based on real-time stiffness modulation

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

We propose a novel human-in-the-loop approach for teaching robots how to solve assembly tasks in unpredictable and unstructured environments. In the proposed method the human sensorimotor system is integrated into the robot control loop though a teleoperation setup. The approach combines a 3-DoF end-effector force feedback with an interface for modulation of the robot end-effector stiffness. When operating in unpredictable and unstructured environments, modulation of limb impedance is essential in terms of successful task execution, stability and safety. We developed a novel hand-held stiffness control interface that is controlled by the motion of the human finger. A teaching approach was then used to achieve autonomous robot operation. In the experiments, we analysed and solved two part-assembly tasks: sliding a bolt fitting inside a groove and driving a self-tapping screw into a material of unknown properties. We experimentally compared the proposed method to complementary robot learning methods and analysed the potential benefits of direct stiffness modulation in the force-feedback teleoperation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. For the sake of generality of introduction we use term “modulation of the impedance” when either stiffness, damping or mass is modulated in real time.

  2. Tutor is a teleoperator that teaches the robot a new behaviour.

  3. Note that the strategies are devised for autonomous robot operation and not for the comfort of the tutor during the teaching stage.

References

  • Ajoudani, A., Godfrey, S., Bianchi, M., Catalano, M., Grioli, G., Tsagarakis, N., et al. (2014). Exploring teleimpedance and tactile feedback for intuitive control of the pisa/iit softhand. IEEE Transactions on Haptics, 7(2), 203–215.

    Article  Google Scholar 

  • Ajoudani, A., Tsagarakis, N. G., & Bicchi, A. (2012). Tele-impedance: Teleoperation with impedance regulation using a body–machine interface. The International Journal of Robotics Research, 31(13), 1642–1656.

    Article  Google Scholar 

  • Albu-Schäffer, A., Ott, C., Frese, U., & Hirzinger, G. (2003). Cartesian impedance control of redundant robots: Recent results with the DLR-light-weight-arms. In 2003 IEEE international conference on robotics and automation (ICRA), vol. 3, pp. 3704–3709.

  • Albu-Schäffer, A., Ott, C., & Hirzinger, G. (2007). A unified passivity-based control framework for position, torque and impedance control of flexible joint robots. The International Journal of Robotics Research, 26(1), 23–39.

    Article  MATH  Google Scholar 

  • Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5), 469–483.

    Article  Google Scholar 

  • Babič, J., Hale, J. G., & Oztop, E. (2011). Human sensorimotor learning for humanoid robot skill synthesis. Adaptive Behavior—Animals, Animats, Software Agents, Robots, Adaptive Systems, 19, 250–263.

    Google Scholar 

  • Ben Amor, H., Berger, E., Vogt, D., & Jung, B. (2009). Kinesthetic bootstrapping: Teaching motor skills to humanoid robots through physical interaction. In Proceedings of the 32nd Annual German conference on advances in artificial intelligence, pp. 492–499.

  • Ben Amor, H., Neumann, G., Kamthe, S., Kroemer, O., & Peters, J. (2014). Interaction primitives for human–robot cooperation tasks. In 2014 IEEE international conference on robotics and automation (ICRA), pp. 2831–2837.

  • Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008). Robot programming by demonstration. In B. Siciliano & O. Khatib (Eds.), Springer handbook of robotics (pp. 1371–1394). Berlin: Springer.

    Chapter  Google Scholar 

  • Buchli, J., Stulp, F., Theodorou, E., & Schaal, S. (2011). Learning variable impedance control. The International Journal of Robotics Research, 30(7), 820–833.

    Article  Google Scholar 

  • Burdet, E., Osu, R., Franklin, D. W., Milner, T. E., & Kawato, M. (2001). The central nervous system stabilizes unstable dynamics by learning optimal impedance. Nature, 414(6862), 446–449.

    Article  Google Scholar 

  • Calinon, S., Sardellitti, I., & Caldwell, D. (2010). Learning-based control strategy for safe human–robot interaction exploiting task and robot redundancies. In 2010 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 249–254.

  • Daniel, R., & McAree, P. (1998). Fundamental limits of performance for force reflecting teleoperation. The International Journal of Robotics Research, 17(8), 811–830.

    Article  Google Scholar 

  • De Luca, A., Albu-Schäffer, A., Haddadin, S., & Hirzinger, G. (2006). Collision detection and safe reaction with the DLR-III lightweight manipulator arm. In 2006 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 1623–1630.

  • Emge, N., Prebeg, G., Uygur, M., & Jaric, S. (2013). Effects of muscle fatigue on grip and load force coordination and performance of manipulation tasks. Neuroscience Letters, 550, 46–50.

    Article  Google Scholar 

  • Evrard, P., Gribovskaya, E., Calinon, S., Billard, A., & Kheddar, A. (2009). Teaching physical collaborative tasks: Object-lifting case study with a humanoid. In IEEE-RAS international conference on humanoid robots, pp. 399–404.

  • Fite, K. B., Speich, J. E., & Goldfarb, M. (2001). Transparency and stability robustness in two-channel bilateral telemanipulation. Journal of Dynamic Systems, Measurement, and Control, 123(8), 400–407.

    Article  Google Scholar 

  • Garabini, M., Passaglia, A., Belo, F., Salaris, P., & Bicchi, A. (2011). Optimality principles in variable stiffness control: The vsa hammer. In 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 3770–3775.

  • Hannaford, B., & Anderson, R. (1988). Experimental and simulation studies of hard contact in force reflecting teleoperation. In 1988 IEEE international conference on robotics and automation (ICRA), (pp. 584–589). Philadelphia, USA.

  • Hersch, M., Guenter, F., Calinon, S., & Billard, A. (2008). Dynamical system modulation for robot learning via kinesthetic demonstrations. IEEE Transactions on Robotics, 24(6), 1463–1467.

    Article  Google Scholar 

  • Hogan, N. (1984). Adaptive control of mechanical impedance by coactivation of antagonist muscles. IEEE Transactions on Automatic Control, 29(8), 681–690.

    Article  MATH  Google Scholar 

  • Hogan, N. (1985). Impedance control: An approach to manipulation. I—Theory. II—Implementation. III—Applications. ASME Transactions Journal of Dynamic Systems and Measurement Control B, 107, 1–24.

    Article  MATH  Google Scholar 

  • Ijspeert, A. J., Nakanishi, J., Hoffmann, H., Pastor, P., & Schaal, S. (2013). Dynamical movement primitives: Learning attractor models for motor behaviors. Neural Computation, 25(2), 328–373.

    Article  MathSciNet  MATH  Google Scholar 

  • Ijspeert, A. J., Nakanishi, J., & Schaal, S. (2002a). Learning rhythmic movements by demonstration using nonlinear oscillators. In 2002 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 958–963.

  • Ijspeert, A . J., Nakanishi, J., & Schaal, S. (2002b). Movement imitation with nonlinear dynamical systems in humanoid robots. In 2002 IEEE international conference on robotics and automation (ICRA), vol. 2, pp. 1398–1403.

  • Imamizu, H., Miyauchi, S., Tamada, T., Sasaki, Y., Takino, R., Pütz, B., et al. (2000). Human cerebellar activity reflecting an acquired internal model of a new tool. Nature, 403(6766), 192–195.

    Article  Google Scholar 

  • Kawato, M. (1999). Internal models for motor control and trajectory planning. Current Opinion in Neurobiology, 9(6), 718–727.

    Article  Google Scholar 

  • Kim, W., Hannaford, B., & Fejczy, A. (1992). Force-reflection and shared compliant control in operating telemanipulators with time delay. IEEE Transactions on Robotics and Automation, 8(2), 176–185.

    Article  Google Scholar 

  • Kober, J., Wilhelm, A., Oztop, E., & Peters, J. (2012). Reinforcement learning to adjust parametrized motor primitives to new situations. Autonomous Robots, 33(4), 361–379.

    Article  Google Scholar 

  • Kormushev, P., Calinon, S., & Caldwell, D. (2010). Robot motor skill coordination with em-based reinforcement learning. In 2010 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 3232–3237.

  • Kormushev, P., Calinon, S., & Caldwell, D. G. (2011). Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input. Advanced Robotics, 25(5), 581–603.

    Article  Google Scholar 

  • Kronander, K., & Billard, A. (2012). Online learning of varying stiffness through physical human–robot interaction. In 2012 IEEE international conference on robotics and automation (ICRA), pp. 1842–1849.

  • Kronander, K., & Billard, A. (2014). Learning compliant manipulation through kinesthetic and tactile human–robot interaction. IEEE Transactions on Haptics, 7(3), 367–380.

    Article  Google Scholar 

  • Kronander, K., Burdet, E., & Billard, A. (2014). Task transfer via collaborative manipulation for insertion assembly. Workshop on human–robot interaction for industrial manufacturing, robotics, science and systems.

  • Kuchenbecker, K. J., & Niemeyer, G. (2006). Induced master motion in force-reflecting teleoperation. Journal of Dynamic Systems, Measurement, and Controlh, 128(4), 800–810.

    Article  Google Scholar 

  • Kushida, D., Nakamura, M., Goto, S., & Kyura, N. (2001). Human direct teaching of industrial articulated robot arms based on force-free control. Artificial Life and Robotics, 5(1), 26–32.

    Article  Google Scholar 

  • Lee, D., & Ott, C. (2011). ncremental kinesthetic teaching of motion primitives using the motion refinement tube. Autonomous Robots, 31(2), 115–131.

    Article  Google Scholar 

  • Liang, P., Yang, C., Li, Z., & Li, R. (2015). Writing skills transfer from human to robot using stiffness extracted from sEMG. In 2015 IEEE international conference on cyber technology in automation, control, and intelligent systems (CYBER), pp. 19–24.

  • Oztop, E., Lin, L.-H., Kawato, M., & Cheng, G. (2006). Dexterous skills transfer by extending human body schema to a robotic hand. In 2006 6th IEEE-RAS international conference on humanoid robots, pp. 82–87.

  • Paraschos, A., Daniel, C., Peters, J., & Neumann, G. (2013). Probabilistic movement primitives. In C. Burges, L. Bottou, M. Welling, Z. Ghahramani, & K. Weinberger (Eds.), Advances in neural information processing systems 26 (pp. 2616–2624). Red Hook: Curran Associates Inc.

    Google Scholar 

  • Peternel, L., Noda, T., Petrič, T., Ude, A., Morimoto, J., & Babič, J. (2016a). Adaptive control of exoskeleton robots for periodic assistive behaviours based on EMG feedback minimisation. PLoS ONE, 11(2), e0148942.

    Article  Google Scholar 

  • Peternel, L., Petrič, T., & Babič, J. (2015). Human-in-the-loop approach for teaching robot assembly tasks using impedance control interface. In 2015 IEEE international conference on robotics and automation (ICRA), pp. 1497–1502.

  • Peternel, L., Petrič, T., Oztop, E., & Babič, J. (2014). Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach. Autonomous robots, 36(1–2), 123–136.

    Article  Google Scholar 

  • Peternel, L., Tsagarakis, N., & Ajoudani, A. (2016b). Towards multi-modal intention interfaces for human–robot co-manipulation. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 2663–2669.

  • Petrič, T., Gams, A., Ijspeert, A. J., & Žlajpah, L. (2011). On-line frequency adaptation and movement imitation for rhythmic robotic tasks. The International Journal of Robotics Research, 30(14), 1775–1788.

    Article  Google Scholar 

  • Rozo, L., Bruno, D., Calinon, S., & Caldwell, D. G. (2015). Learning optimal controllers in human-robot cooperative transportation tasks with position and force constraints. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS).

  • Schaal, S., & Atkeson, C. G. (1998). Constructive incremental learning from only local information. Neural Computation, 10(8), 2047–2084.

    Article  Google Scholar 

  • Schindlbeck, C., & Haddadin, S. (2015). Unified passivity-based cartesian force/impedance control for rigid and flexible joint robots via task-energy tanks. In 2015 IEEE international conference on robotics and automation (ICRA), pp. 440–447.

  • Theodorou, E., Buchli, J., & Schaal, S. (2010). A generalized path integral control approach to reinforcement learning. Journal of Machine Learning Research, 11, 3137–3181.

    MathSciNet  MATH  Google Scholar 

  • Todd, G., Gandevia, S. C., & Taylor, J. L. (2010). Change in manipulation with muscle fatigue. European Journal of Neuroscience, 32(10), 1686–1694.

    Article  Google Scholar 

  • Tonietti, G., Schiavi, R., & Bicchi, A. (2005). Design and control of a variable stiffness actuator for safe and fast physical human/robot interaction. In 2005 IEEE international conference on robotics and automation (ICRA), pp. 526–531.

  • Tsumugiwa, T., Yokogawa, R., & Hara, K. (2002). Variable impedance control with virtual stiffness for human-robot cooperative peg-in-hole task. In 2002 IEEE/RSJ international conference on intelligent robots and systems (IROS), vol. 2, pp. 1075–1081.

  • Van Der Linde, R. Q., Lammertse, P., Frederiksen, E., & Ruiter, B. (2002). The hapticmaster, a new high-performance haptic interface. In Proceedings of Eurohaptics, 2002, 1–5.

  • Walker, D., Salisbury, J., & Niemeyer, G. (2011). Demonstrating the benefits of variable impedance to telerobotic task execution. In IEEE international conference on robotics and automation (ICRA), 2011 , (pp. 1348–1353).

  • Walker, D., Wilson, R., & Niemeyer, G. (2010). User-controlled variable impedance teleoperation. In IEEE International Conference on Robotics and Automation (ICRA), 2010 , (pp. 5352–5357).

  • Wolf, S., & Hirzinger, G. (2008). A new variable stiffness design: Matching requirements of the next robot generation. In IEEE International Conference on Robotics and Automation (ICRA), 2008, (pp. 1741–1746).

  • Wolpert, D. M., Miall, R. C., & Kawato, M. (1998). Internal models in the cerebellum. Trends in Cognitive Sciences, 2(9), 338–347.

    Article  Google Scholar 

  • Yang, C., Ganesh, G., Haddadin, S., Parusel, S., Albu-Schäffer, A., & Burdet, E. (2011). Human-like adaptation of force and impedance in stable and unstable interactions. IEEE Transactions on Robotics, 27(5), 918–930.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Barry Ridge for narrating the supplementary video.

Author information

Authors and Affiliations

  1. Department of Automation Biocybernetics and Robotics, Jožef Stefan Institute, Jamova cesta 39, 1000, Ljubljana, Slovenia

    Luka Peternel, Tadej Petrič & Jan Babič

  2. HRII Lab, Department of Advanced Robotics, Istituto Italiano di Technologia, Via Morego 30, 16163, Genoa, Italy

    Luka Peternel

Authors
  1. Luka Peternel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tadej Petrič
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jan Babič
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luka Peternel.

Additional information

This work was supported by European Community Framework Programme 7 through the CoDyCo Project (Contract No. 600716).

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 20490 KB)

Supplementary material 2 (mp4 2391 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peternel, L., Petrič, T. & Babič, J. Robotic assembly solution by human-in-the-loop teaching method based on real-time stiffness modulation. Auton Robot 42, 1–17 (2018). https://doi.org/10.1007/s10514-017-9635-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-017-9635-z

Keywords

Search

Navigation