Skip to main content
Springer Nature Link
Log in

Autonomous reconfiguration of robot shape by using Q-learning

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

A modular robot can be built with a shape and function that matches the working environment. We developed a four-arm modular robot system which can be configured in a planar structure. A learning mechanism is incorporated in each module constituting the robot. We aim to control the overall shape of the robot by an accumulation of the autonomous actions resulting from the individual learning functions. Considering that the overall shape of a modular robot depends on the learning conditions in each module, this control method can be treated as a dispersion control learning method. The learning object is cooperative motion between adjacent modules. The learning process proceeds based on Q-learning by trial and error. We confirmed the effectiveness of the proposed technique by computer simulation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Fukuda T, Nakagawa S (1988) Dynamically reconfigurable robotics system. Proceedings of the IEEE International Conference on Robotics and Automation, IEEE Computer Society Press, pp 1581–1586

  2. Chirikjian GS (1993) Metamorphic hyper-redundant manipulators. Proceedings of the JSME International Conference on Advanced Mecatronics, The Japan Society of Mechanical Engineers, pp 467–472

  3. Yim M (1994) New locomotion gaits. Proceedings of the IEEE ICRA, IEEE Computer Society Press, pp 2508–2514

  4. Paredis C, Brown HB, Khosla P (1996) A rapidly deployable manipulator system. Proceedings of the IEEE ICRA, IEEE Computer Society Press, pp 1434–1439

  5. Pamecha A, Ebert-Uphoff I, Chirikjian G (1997) Useful metric for modular robot motion planning. IEEE Trans Robotics Autom 13:531–545

    Article  Google Scholar 

  6. Rus D, Vona M (2001) Crystalline robots: self-reconfiguration with compressible unit modules. Auton Robots 10:107–124

    Article  MATH  Google Scholar 

  7. Chiang C-H, Chirikjian G (2001) Modular robot motion planning using similarity metrics. Auton Robots 10:91–106

    Article  MATH  Google Scholar 

  8. Shen W-M, Salemi B, Will P (2002) Hormone-inspired adaptive communication and distributed control for CONRO. IEEE Trans Robotics Autom 18(5):700–712

    Article  Google Scholar 

  9. Walter J, Welch J, Amato N (2000) Distributed reconfiguration of metamorphic robot chains. Proceedings of the 19th Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing (PODC 2000), Portland, pp 171–180

  10. Bulter Z, Kotay K, Rus D, et al (2001) Cellular automata for decentralized control of self-reconfigurable robots. Proceedings of the 2001 IEEE International Conference on Robotics and Automation, Workshop on Robotics: Modular Robots, IEEE Computer Society Press

  11. Watkins CJCH, Dayan P (1992) Q-learning. Machine Learning 8:279–292

    MATH  Google Scholar 

  12. Sutton RS, Barti AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    Google Scholar 

  13. Kotay K, Rus D (1999) Locomotion versatility through self-reconfiguration. Robotics Auton Syst 26:217–232

    Article  Google Scholar 

  14. Rus D, Vona M (2001) Crystalline robots: self-reconfiguration with unit-compressible modules. Auton Robots 10:107–124

    Article  MATH  Google Scholar 

  15. Ito K (2000) Intelligence and emergence. NTT Publishing, Japan, pp 60–70

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Electro-Communications, 1-5-1 Chofu-ga-oka, Chofu, Tokyo, 182-8585, Japan

    Satoshi Shiba & Masafumi Uchida

  2. Meisei University, Tokyo, Japan

    Akio Nozawa

  3. Aoyama Gakuin University, Kanagawa, Japan

    Hirotoshi Asano & Hideto Ide

  4. Kogakuin University, Tokyo, Japan

    Hitoshi Onogaki & Syuichi Yokoyama

  5. National Institute of Advanced Industrial Science and Technology (AIST), Ibaraki, Japan

    Tota Mizuno

Authors
  1. Satoshi Shiba
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Masafumi Uchida
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Akio Nozawa
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Hirotoshi Asano
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Hitoshi Onogaki
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Tota Mizuno
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Hideto Ide
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Syuichi Yokoyama
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Masafumi Uchida.

Additional information

This work was presented in part at the 14th International Symposium on Artificial Life and Robotics, Oita, Japan, February 5–7, 2009

About this article

Cite this article

Shiba, S., Uchida, M., Nozawa, A. et al. Autonomous reconfiguration of robot shape by using Q-learning. Artif Life Robotics 14, 213–218 (2009). https://doi.org/10.1007/s10015-009-0656-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-009-0656-3

Key words