Skip to main content
SpringerLink
Log in

SOAR Improved Artificial Neural Network for Multistep Decision-making Tasks

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Recently, artificial neural networks (ANNs) have been applied to various robot-related research areas due to their powerful spatial feature abstraction and temporal information prediction abilities. Decision-making has also played a fundamental role in the research area of robotics. How to improve ANNs with the characteristics of decision-making is a challenging research issue. ANNs are connectionist models, which means they are naturally weak in long-term planning, logical reasoning, and multistep decision-making. Considering that a small refinement of the inner network structures of ANNs will usually lead to exponentially growing data costs, an additional planning module seems necessary for the further improvement of ANNs, especially for small data learning. In this paper, we propose a state operator and result (SOAR) improved ANN (SANN) model, which takes advantage of both the long-term cognitive planning ability of SOAR and the powerful feature detection ability of ANNs. It mimics the cognitive mechanism of the human brain to improve the traditional ANN with an additional logical planning module. In addition, a data fusion module is constructed to combine the probability vector obtained by SOAR planning and the original data feature array. A data fusion module is constructed to convert the information from the logical sequences in SOAR to the probabilistic vector in ANNs. The proposed architecture is validated in two types of robot multistep decision-making experiments for a grasping task: a multiblock simulated experiment and a multicup experiment in a real scenario. The experimental results show the efficiency and high accuracy of our proposed architecture. The integration of SOAR and ANN is a good compromise between logical planning with small data and probabilistic classification with big data. It also has strong potential for more complicated tasks that require robust classification, long-term planning, and fast learning. Some potential applications include recognition of grasping order in multiobject environment and cooperative grasping of multiagents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. https://github.com/thomasaimondy/SoarImprovedANN

  2. https://github.com/thomasaimondy/patchlinemod

References

  1. Kotseruba I, Tsotsos JK. 40 years of cognitive architectures: core cognitive abilities and practical applications. Artif Intell Rev;40:1–78.

  2. Anderson JR, Bothell D, Byrne MD, Douglass S, Lebiere C, Qin Y. An integrated theory of the mind. Psychol Rev 2004;111(4):1036.

    Article  Google Scholar 

  3. Anderson JR. Human symbol manipulation within an integrated cognitive architecture. Cogn Sci 2005;29(3): 313–341.

    Article  Google Scholar 

  4. Laird JE, Newell A, Rosenbloom PS. Soar: an architecture for general intelligence. Artif Intell 1987;33 (1):1–64.

    Article  Google Scholar 

  5. Laird JE. 2012. The Soar cognitive architecture. MIT Press, Cambridge.

  6. French RM. Catastrophic forgetting in connectionist networks. Trends Cogn Sci 1999;3(4):128–135.

    Article  Google Scholar 

  7. Eliasmith C, Trujillo O. The use and abuse of large-scale brain models. Curr Opinion Neurobiol 2014;25: 1–6.

    Article  Google Scholar 

  8. Hawkins J, Ahmad S. Why neurons have thousands of synapses, a theory of sequence memory in neocortex. Front Neural Circ 2016;10:23.

    Google Scholar 

  9. Sun R, Peterson T, Merrill E. A hybrid architecture for situated learning of reactive sequential decision making. Appl Intell 1999;11(1):109–127.

    Article  Google Scholar 

  10. O’Reilly RC, Wyatte D, Herd S, Mingus B, Jilk DJ. Recurrent processing during object recognition. Front Psychol 2013;4:124.

    Article  Google Scholar 

  11. Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw 2015;61:85–117.

    Article  Google Scholar 

  12. He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. Proceedings of the IEEE international conference on computer vision; 2015. p. 1026–1034.

  13. Wang Z, Wang X, Wang G. Learning fine-grained features via a cnn tree for large-scale classification. Neurocomputing 2018;275:1231–1240.

    Article  Google Scholar 

  14. Dahl GE, Yu D, Li D, Acero A. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans Audio Speech Lang Process 2012;20(1):30–42.

    Article  Google Scholar 

  15. Redmon J, Divvala S, Girshick R, Farhadi A. You only once: look Unified, real-time object detection. Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 779–788.

  16. Maturana D, Scherer S. Voxnet: a 3d convolutional neural network for real-time object recognition. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE; 2015. p. 922–928.

  17. Oh J, Guo X, Lee H, Lewis RL, Singh S. Action-conditional video prediction using deep networks in atari games. In: Advances in neural information processing systems; 2015. p. 2863–2871.

  18. Weisz G, Budzianowski P, Su P-H, Gasic M. Sample efficient deep reinforcement learning for dialogue systems with large action spaces. IEEE/ACM Trans Audio Speech Lang Process (TASLP) 2018;26(11):2083–2097.

    Article  Google Scholar 

  19. Zen H, Sak H. 2015. Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2015. P. 4470–4474.

  20. Finn C, Levine S. 2017. Deep visual foresight for planning robot motion. In: IEEE International Conference on Robotics and Automation (ICRA). IEEE; 2017. p. 2786–2793.

  21. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves Ax, Riedmiller M, Fidjeland AK, Ostrovski G, et al. Human-level control through deep reinforcement learning. Nature 2015;518(7540):529.

    Article  Google Scholar 

  22. Ge L, Ren Z, Li Y, Xue Z, Wang Y, Cai J, Yuan J. 3d hand shape and pose estimation from a single rgb image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2019. p. 10833–10842.

  23. Dong J, Jiang W, Huang Q, Bao H, Zhou X. Fast and robust multi-person 3d pose estimation from multiple views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2019. p. 7792–7801.

  24. Huajun Z, Jin Z, Rui W, Tan M. Multi-objective reinforcement learning algorithm and its application in drive system. In 2008 34th Annual Conference of IEEE Industrial Electronics. IEEE; 2008. p. 274–279.

  25. Hester T, Vecerik M, Pietquin O, Lanctot M, Piot B, Horgan D, Quan J, Sendonaris A, Osband I, et al. Deep q-learning from demonstrations. In: Thirty-Second AAAI Conference on Artificial Intelligence; 2018.

  26. Ellefsen KO, Torresen J. Self-adapting goals allow transfer of predictive models to new tasks; 2019.

  27. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M. Mastering the game of go with deep neural networks and tree search. Nature 2016;529(7587):484–489.

    Article  Google Scholar 

  28. Yang Y, Yi L, Fermuller C, Aloimonos Y. Robot learning manipulation action plans by “watching” unconstrained videos from the world wide web. In: Twenty-ninth Aaai Conference on Artificial Intelligence; 2015.

  29. Volodymyr M, Koray K, David S, Rusu AA, Joel Vx, Bellemare MG, Alex G, Martin R, Fidjeland AK, Georg O. Human-level control through deep reinforcement learning. Nature 2015;518(7540): 529.

    Article  Google Scholar 

  30. Zhang H, Lan X, Zhou X, Tian Z, Zhang Y, Zheng N. 2018. Visual manipulation relationship network for autonomous robotics. In: IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids). IEEE; 2018. p. 118–125.

  31. Zeng A, Song S, Lee J, Rodriguez A, Funkhouser T. 2019. Tossingbot: learning to throw arbitrary objects with residual physics.

  32. Chen H-Z, Tian G-H, Liu G-L. A selective attention guided initiative semantic cognition algorithm for service robot. Int J Autom Comput 2018;15(5):559–569.

    Article  Google Scholar 

  33. Van Dang C, Pham TX, Gil K-J, Shin Y-B, Kim J-W, et al. Implementation of a refusable human-robot interaction task with humanoid robot by connecting soar and ros. J Korea Robot Soc 2017;12(1):55–64.

    Article  Google Scholar 

  34. Puigbo J-Y, Pumarola A, Angulo C, Tellez R. Using a cognitive architecture for general purpose service robot control. Connect Sci 2015;27(2):105–117.

    Article  Google Scholar 

  35. Zheng J, Cai F, Chen W, Feng C, Chen H. Hierarchical neural representation for document classification. Cogn Comput 2019;11(2):317–327.

    Article  Google Scholar 

  36. Zhou K, Wei R, Xu Z, Zhang Q, Lu H, Zhang G. 2019. An air combat decision learning system based on a brain-like cognitive mechanism. Cognitive Computation.

  37. Liu P, Qin X. A new decision-making method based on interval-valued linguistic intuitionistic fuzzy information. Cogn Comput 2019;11(1):125–144.

    Article  Google Scholar 

  38. Doumanoglou A, Kouskouridas R, Malassiotis S, Kim T-K. Recovering 6d object pose and predicting next-best-view in the crowd. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2016. p. 3583–3592.

  39. Hodan T, Michel F, Brachmann E, Kehl W, GlentBuch A, Kraft D, Drost B, Vidal J, Ihrke S, Zabulis X, et al. Bop: benchmark for 6d object pose estimation. In: Proceedings of the European Conference on Computer Vision (ECCV); 2018. p. 19–34.

  40. Hinterstoisser S, Lepetit V, Ilic S, Holzer S, Bradski G, Konolige K, Navab N. Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In: Asian conference on computer vision. Springer; 2012. p. 548–562.

  41. Hinterstoisser S, Holzer S, Cagniart C, Ilic S, Konolige K, Navab N, Lepetit V. Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In: IEEE International Conference on Computer Vision; 2012.

  42. Van Der Maaten L. Accelerating t-sne using tree-based algorithms. J Mach Learn Res 2014;15(1):3221–3245.

    MathSciNet  MATH  Google Scholar 

  43. van der Maaten L, Hinton G. Visualizing data using t-sne. J Mach Learn Res 2008;9:2579–2605.

    MATH  Google Scholar 

Download references

Funding

This study is funded by the Beijing Natural Science Foundation (No. 4182008), the National Natural Science Foundation of China (No. 61873008), the National Natural Science Foundation of China (No. 61806195), and the Beijing Academy of Artificial Intelligence (BAAI).

Author information

Author notes

    Authors and Affiliations

    1. Faculty of Information Technology, Beijing University of Technology, Beijing, China

      Guoyu Zuo & Tingting Pan

    2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

      Tielin Zhang

    3. School of Software and Microelectronics, Peking University, Beijing, China

      Yang Yang

    Authors
    1. Guoyu Zuo
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Tingting Pan
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Tielin Zhang
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Yang Yang
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Guoyu Zuo.

    Ethics declarations

    Conflict of Interest

    The authors declare that they have no conflict of interest.

    Ethical Approval

    This article does not contain any studies with human participants or animals performed by any of the authors.

    Additional information

    Publisher’s Note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Guoyu Zuo, Tingting Pan have equal contribution to this work and should be regarded as co-first authors. Tielin Zhang and Yang Yang contributed to this paper equally.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Zuo, G., Pan, T., Zhang, T. et al. SOAR Improved Artificial Neural Network for Multistep Decision-making Tasks. Cogn Comput 13, 612–625 (2021). https://doi.org/10.1007/s12559-020-09716-6

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s12559-020-09716-6

    Keywords

    Search

    Navigation