Skip to main content
SpringerLink
Log in

Brain-Inspired Active Learning Architecture for Procedural Knowledge Understanding Based on Human-Robot Interaction

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Improving robots with self-learning ability is one of the critical challenges for the researchers in the area of cognitive robotics and artificial general intelligence. This robot will decide when, where, and what to learn in a continuous visual environment by itself. Here we focus on the procedural knowledge learning, which is sequential and considered harder to understand compared with declarative knowledge in the cognitive system. Inspired by the architecture of the human brain which has integrated well different kinds of cognitive functions, a Brain-inspired Active Learning Architecture (BALA) is proposed for procedural knowledge understanding based on Baxter robot and human interaction. The BALA model contains four main parts: inspired by Primary Visual Pathway, a Convolutional Neural Network (CNN) is constructed for spatial information abstraction; inspired by the Hippocampus Pathway (especially the recurrent loops in CA3 sub-region), a Recurrent Neural Network (RNN) is built for sequential information processing related with procedural knowledge; inspired by the Prefrontal Cortex, a Knowledge Graph based on Bag Of Words (BOW) is constructed for declarative knowledge generation and association; inspired by the Basal Ganglia Pathway, we select Q matrix for Reinforcement Learning (RL). The CNN and RNN parts will be firstly pre-trained on ImageNet dataset and standard Youtube Video-Scene dataset respectively. Then, the RNN, Knowledge Graph, and Q matrix will be dynamically updated in the Baxter robot’s interactive learning procedure with human cooperators. The BALA could actively and incrementally recognize different kinds of procedural knowledge. In 22-type daily-life videos with procedure knowledge (e.g., opening the door, wiping the table, or taking the phone), the BALA model gets the best performance compared with standard CNN, RNN, RL, and other integrative methods. The BALA model is a small step on integrative intelligence interaction between the Baxter robot and human cooperator.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://github.com/thomasaimondy/BALA

References

  1. Shen Y-Y, Liu C-L. Incremental adaptive learning vector quantization for character recognition with continuous style adaptation. Cognitive Computation 2018;10(2):334–346.

    Article  Google Scholar 

  2. Reyes O, Altalhi AH, Ventura S. Statistical comparisons of active learning strategies over multiple datasets. Knowl-Based Syst 2018;145:274–288.

    Article  Google Scholar 

  3. Zhu Z, Hu H. Robot learning from demonstration in robotic assembly: a survey. Robotics 2018;7(2):17.

    Article  Google Scholar 

  4. Bhat AA, Mohan V. Goal-directed reasoning and cooperation in robots in shared workspaces: an internal simulation based neural framework. Cognitive Computation 2018;10(4):558–576.

    Article  Google Scholar 

  5. Parisi GI, Kemker R, Part JL, Kanan C, Wermter S. 2019. Continual lifelong learning with neuralnetworks: a review. Neural Networks.

  6. Hao W, Fan J, Zhang Z, Zhu G. End-to-end lifelong learning: a framework to achieve plasticities of both the feature and classifier constructions. Cognitive Computation 2018;10(2):321–333.

    Article  Google Scholar 

  7. Kaliukhovich DA, Beeck HO. Hierarchical stimulus processing in rodent primary and lateral visual cortex as assessed through neuronal selectivity and repetition suppression. Journal of Neurophysiology 2018;120(3):926–941.

    Article  Google Scholar 

  8. He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. Proceedings of the IEEE international conference on computer vision; 2015. p. 1026–1034.

  9. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. Imagenet: a large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition. IEEE; 2009. p. 248–255.

  10. Zhang T, Zeng Y, Xu B. Hcnn: a neural network model for combining local and global features towards human-like classification. International Journal of Pattern Recognition and Artificial Intelligence 2016; 30(01):1655004.

    Article  MathSciNet  Google Scholar 

  11. Venugopalan S, Rohrbach M, Donahue J, Mooney R, Darrell T, Saenko K. Sequence to sequence-video to text. Proceedings of the IEEE international conference on computer vision; 2015 . p. 4534–4542.

  12. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al. Human-level control through deep reinforcement learning. Nature 2015;518(7540):529.

    Article  Google Scholar 

  13. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, et al. Mastering the game of go with deep neural networks and tree search. Nature 2016;529(7587):484.

    Article  Google Scholar 

  14. Oh J, Guo X, Lee H, Lewis RL, Singh S. Action-conditional video prediction using deep networks in atari games. Advances in neural information processing systems; 2015. p. 2863– 2871.

  15. Yang Y, Loog M. A benchmark and comparison of active learning for logistic regression. Pattern Recogn 2018;83:401–415.

    Article  Google Scholar 

  16. Tebbe J, Gao Y, Sastre-Rienietz M, Zell A . A table tennis robot system using an industrial kuka robot arm. German conference on pattern recognition. Springer; 2018. p. 33–45.

  17. Yang Y, Li Y, Fermuller C, Aloimonos Y . Robot learning manipulation action plans by watching unconstrained videos from the world wide web. Twenty-ninth AAAI conference on artificial intelligence; 2015.

  18. Zlatintsi A, Rodomagoulakis I, Koutras P, Dometios AC, Pitsikalis V, Tzafestas CS, Maragos P. Multimodal signal processing and learning aspects of human-robot interaction for an assistive bathing robot. IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2018; 2018. p. 3171–3175.

  19. Toprak S, Navarro-Guerrero N, Wermter S. Evaluating integration strategies for visuo-haptic object recognition. Cognitive computation 2018;10(3):408–425.

    Article  Google Scholar 

  20. Camacho-Collados J, Pilehvar MT. From word to sense embeddings: a survey on vector representations of meaning. J Artif Intell Res 2018;63:743–788.

    Article  MathSciNet  Google Scholar 

  21. Osa T, Pajarinen J, Neumann G, Bagnell JA, Abbeel P, Peters J, et al. An algorithmic perspective on imitation learning. Foundations and Trends®; in Robotics 2018;7(1-2):1–179.

    Article  Google Scholar 

  22. Amato C. Decision-making under uncertainty in multi-agent and multi-robot systems: planning and learning. IJCAI; 2018 . p. 5662–5666.

  23. Pérez-Sánchez B, Fontenla-Romero O, Guijarro-Berdiñas B. A review of adaptive online learning for artificial neural networks. Artif Intell Rev 2018;49(2):281–299.

    Article  Google Scholar 

  24. Hester T, Vecerik M, Pietquin O, Lanctot M, Schaul T, Piot B, Horgan D, Quan J, Sendonaris A, Osband I, et al. Deep q-learning from demonstrations. Thirty-second AAAI conference on artificial intelligence; 2018.

  25. Zhao F, Yi Z, Wang G, Bai J, Xu B. A brain-inspired decision making model based on top-down biasing of prefrontal cortex to basal ganglia and its application in autonomous uav explorations. Cognitive Computation 2018;10(2):296–306.

    Article  Google Scholar 

  26. Zhu J-J, Bento J. 2017. Generative adversarial active learning. arXiv:1702.07956.

  27. Zhou Z, Shin JY, Zhang L, Gurudu SR, Gotway MB, Liang J . Fine-tuning convolutional neural networks for biomedical image analysis: actively and incrementally. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 . p. 4761–4772.

  28. Konyushkova K, Sznitman R, Fua P. 2017. Learning active learning from data. Neural Information Processing Systems, pp 4225–4235.

  29. Cohn DA, Ghahramani Z, Jordan MI. Active learning with statistical models. J Artif Intell Res 1996;4(1):129–145.

    Article  Google Scholar 

  30. Chernova S, Veloso MM. Interactive policy learning through confidence-based autonomy. J Artif Intell Res 2009;34(1):1–25.

    MathSciNet  MATH  Google Scholar 

  31. Ugur E, Dogar MR, Cakmak M, Sahin E. Curiosity-driven learning of traversability affordance on a mobile robot. 2007 IEEE 6th international conference on development and learning; 2007. p. 13–18.

  32. Oudeyer P-Y, Kaplan F, Hafner VV. Intrinsic motivation systems for autonomous mental development. IEEE Trans Evol Comput 2007;11(2):265–286.

    Article  Google Scholar 

  33. Schembri M, Mirolli M, Baldassarre G. 2007. Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot, pp 282–287.

  34. Zhang T, Yi Z, Xu B. A computational approach towards the microscale mouse brain connectome from the mesoscale. Journal of Integrative Neuroscience 2017;16(3):291–306.

    Article  Google Scholar 

  35. Zhang T, Zeng Y, Zhao D, Wang L, Zhao Y, Xu B. Hmsnn: hippocampus inspired memory spiking neural network. 2016 IEEE international conference on systems, man, and cybernetics (SMC). IEEE; 2016. p. 002301–002306.

  36. Zhang T, Zeng Y, Zhao D, Shi M . A plasticity-centric approach to train the non-differential spiking neural networks. Thirty-second AAAI conference on artificial intelligence; 2018.

  37. Zhang T, Yi Z, Zhao D, Xu B. Brain-inspired balanced tuning for spiking neural networks. IJCAI; 2018. p. 1653–1659.

  38. Hochreiter S, Schmidhuber J. Long short-term memory. Neural computation 1997;9(8):1735–1780.

    Article  Google Scholar 

  39. Chen DL, Dolan WB. Collecting highly parallel data for paraphrase evaluation. Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies. Association for Computational Linguistics; 2011 . p. 190–200.

  40. Torabi A, Pal C, Larochelle H, Courville A. 2015. Using descriptive video services to create a large data source for video annotation research. arXiv:1503.01070.

  41. Rohrbach A, Rohrbach M, Tandon N, Schiele B. A dataset for movie description. Proceedings of the IEEE conference on computer vision and pattern recognition; 2015. p. 3202–3212.

Download references

Funding

This study is supported by National Natural Science Foundation of China (No. 61806195), the Strategic Priority Research Program of Chinese Academy of Sciences (Grant No. XDB32070100), the Beijing Municipality of Science and Technology (Grant No. Z181100001518006), the Major Research Program of Shandong Province (Grant No. 2018CXGC1503) and the CETC Joint Fund (Grant No. 6141B08010103).

Author information

Authors and Affiliations

  1. Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Tielin Zhang, Yi Zeng, Mengting Shi & Enmeng Lu

  2. Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China

    Yi Zeng

  3. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Yi Zeng

  4. University of Chinese Academy of Sciences, Beijing, China

    Yi Zeng & Mengting Shi

  5. Research Center for Intelligent Security Technology, Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences, Beijing, China

    Ruihan Pan

Authors
  1. Tielin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ruihan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mengting Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Enmeng Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Tielin Zhang or Yi Zeng.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, T., Zeng, Y., Pan, R. et al. Brain-Inspired Active Learning Architecture for Procedural Knowledge Understanding Based on Human-Robot Interaction. Cogn Comput 13, 381–393 (2021). https://doi.org/10.1007/s12559-020-09753-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-020-09753-1

Keywords

Search

Navigation