Abstract
The hierarchical organisation of behaviour is a fundamental means through which robots and organisms can acquire and produce sophisticated and flexible behaviours that allow them to solve multiple tasks in multiple conditions. Recently, the research on this topic has been receiving increasing attention. On the one hand, machine learning and robotics are recognising the fundamental importance of the hierarchical organisation of behaviour for building robots that scale up to solve complex tasks, possibly in a cumulative fashion. On the other hand, research in psychology and neuroscience is finding increasing evidence that modularity and hierarchy are pivotal organisation principles of behaviour and of the brain. This book reviews the state of the art in computational and robotic models of the hierarchical organisation of behaviour. Each contribution reviews the main works of the authors on this subject, the open challenges, and promising research directions. Together, the contributions give a good coverage of the most important models, findings, and challenges of the field. This introductory chapter presents the general aims and scope of the book and briefly summarises the contents of each chapter.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bakker, B., & Schmidhuber, J. (2004). Hierarchical reinforcement learning based on subgoal discovery and subpolicy specialization. In F. Groen, N. Amato, A. Bonarini, E. Yoshida, B. Kruse (Eds.), Proceedings of the 8-th conference on intelligent autonomous systems (IAS-8) (pp. 438–445).
Baldassarre, G., & Mirolli, M. (2010). What are the key open challenges for understanding the autonomous cumulative learning of skills? The Newsletters of the Autonomous Mental Development Technical committee (IEEE CIS AMD Newsletters), 7(1), 11.
Balleine, B. W., & Dickinson, A. (1998). Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology, 37(4–5), 407–419.
Botvinick, M., & Plaut, D.C. (2004). Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. Psychological Review, 111(2), 395–429.
Cooper, R., & Shallice, T. (2000). Contention scheduling and the control of routine activities. Cognitive Neuropsychology, 17(4), 297–338.
Demiris, Y., & Khadhouri, B. (2006). Hierarchical attentive multiple models for execution and recognition of actions. Robotics and Autonomous Systems, 54(5), 361–369.
Fischer, K. W. (1980). A theory of cognitive development: the control and construction of hierarchies of skills. Psychological Review, 87(6), 477–531.
French, R. M. (1999). Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4), 128–135.
Fuster, J. M. (2001). The prefrontal cortex–an update: time is of the essence. Neuron, 30, 319–333.
Graybiel, A. M. (1998). The basal ganglia and chunking of action repertoires. Neurobiology of Learning and Memory, 70(1–2), 119–136.
Graziano, M. (2006). The organization of behavioral repertoire in motor cortex. The Annual Review of Neuroscience, 29, 105–134.
Hart, S., & Grupen, R. (2011). Learning generalizable control programs. IEEE Transactions on Autonomous Mental Development, 3(1), 216–231.
McCloskey, M., & Cohen, N. (1989). Catastrophic interference in connectionist networks: the sequential learning problem. In G. H. Bower (Ed.), The psychology of learning and motivation (vol. 24, pp. 109–165). San Diego: Academic.
Meunier, D., Lambiotte, R., Bullmore, E. T. (2010). Modular and hierarchically modular organization of brain networks. Front Neuroscience, 4, 200.
Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. The Annual Review of Neuroscience, 24, 167–202.
Miller, G. A., Galanter, E., Pribram, K. H. (1960). Plans and the structure of behavior. New York: Adams-Bannister-Cox.
Redgrave, P., & Gurney, K. (2006). The short-latency dopamine signal: a role in discovering novel actions? Nature Reviews Neuroscience, 7(12), 967–975.
Schneider, D. W., & Logan, G. D. (2006). Hierarchical control of cognitive processes: switching tasks in sequences. Journal of Experimental Psychology. General, 135(4), 623–640.
Singh, S. (1992). Transfer of learning by composing solutions of elemental sequential tasks. Machine Learning, 8(3), 323–339.
Yamashita, Y., & Tani, J. (2008). Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Computational Biology, 4(11), e1000220.
Yin, H. H., & Knowlton, B. J. (2006). The role of the basal ganglia in habit formation. Nature Reviews Neuroscience, 7(6), 464–476.
Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., Reynolds, J. R. (2007). Event perception: a mind-brain perspective. Psychological Bulletin, 133(2), 273–293.
Acknowledgements
This chapter and a large part of the effort that led to this book have been supported by the Project “IM-CLeVeR—Intrinsically Motivated Cumulative Learning Versatile Robots” funded by the European Commission under the 7th Framework Programme (FP7/2007–2013), “Challenge 2—Cognitive Systems, Interaction, Robotics”, Grant Agreement No. ICT-IP-231722. Support or co-support from other institutions, where present, is described in the “Acknowledgments” section of each chapter. The editors of the book thank the EU reviewers (Benjamin Kuipers, Luc Berthouze, and Yasuo Kuniyoshi) and the EU Project Officer (Cécile Huet) for their valuable advice and encouragement. For more information on the IM-CLeVeR project, and for additional multimedia material, see the project web site: http://www.im-clever.eu/. We also thank Simona Bosco for her editorial help with some contributions.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Baldassarre, G., Mirolli, M. (2013). Computational and Robotic Models of the Hierarchical Organization of Behavior: An Overview. In: Baldassarre, G., Mirolli, M. (eds) Computational and Robotic Models of the Hierarchical Organization of Behavior. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39875-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-39875-9_1
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39874-2
Online ISBN: 978-3-642-39875-9
eBook Packages: Computer ScienceComputer Science (R0)