Skip to main content
SpringerLink
Log in

Behavioral Decision-Making of Mobile Robot in Unknown Environment with the Cognitive Transfer

  • Regular Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

How to improve the behavioral decision-making ability and adaptability to unknown environments is of great importance for an agent. The traditional decision-making methods usually suffer from long training time, due to the large amount of training samples, or low adaptability to the unknown environments, or lack of the continuous learning capacities, etc. In response to these problems, this work proposes a novel motivated developmental network (MDN) to improve the decision-making ability of the agent. During the environment exploration, if the agent encounters an unknown environment, new layers and neurons are dynamically inserted to the MDN, according to the task requirements. Through the interaction between internal neurons and the inserted new neurons, the agent can autonomously develop and learn in the unknown environments without training data, but the behavioral decision-making at this stage is random. To further improve the agent’s decision-making ability, in the off-task process, through the gated self-organization mechanism, the agent can selectively recall the specific knowledge in its “brain”, and the MDN will transfer and generate a large amount of new data, according to the recalled knowledge, then the new layers and neurons will be inserted to memorize the new knowledge. Hence the knowledge base of the MDN becomes more and more complete, thereby improving its decision-making ability and adaptability to the new environment. To demonstrate the performance of the MDN model, a mobile robot navigation in different environments are executed. The experimental results illustrate that the agent can not only autonomously learn in static environments, but also has better decision-making ability in unknown dynamic environment, i.e., better adaptability to the new environment. Comparison with other algorithms further demonstrate the potential of the proposed MDN model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Li, H., Savkin, A.V.: An algorithm for safe navigation of mobile robots by a sensor network in dynamic cluttered industrial environments. Robot. Comput. Integr. Manuf. 54, 65–82 (2018)

    Article  Google Scholar 

  2. Qiu, Q., Fan, Z., Meng, Z., Zhang, Q., Cong, Y., Li, B., Wang, N., Zhao, C.: Extended ackerman steering principle for the coordinated movement control of a four wheel drive agricultural mobile robot. Comput. Electron. Agric. 152, 40–50 (2018)

    Article  Google Scholar 

  3. Buyurgan, N., Lehlou, N.: A terrain risk assessment method for military surveillance applications for mobile assets. Comput. Ind. Eng. 88, 88–99 (2015)

    Article  Google Scholar 

  4. Sword, C.M.: Viable alternative mine operating system: A novel underwater robotic excavation system for flooded open-cut mines. Energy Procedia 125, 50–55 (2017)

    Article  Google Scholar 

  5. Salzmann-Erikson, M., Erikssonm, H: Absorbability: applicability and availability in nursing and care robots. A thematic analysis of twitter postings. Telematics Inform. 35(5), 1553–1560 (2018)

    Article  Google Scholar 

  6. Sanguino, T.J.M.: 50 years of rovers for planetary exploration. A retrospective review for future directions. Robot. Auton. Syst. 94, 172–185 (2017)

    Article  Google Scholar 

  7. Bai, L., Guan, J., Chen, X., Hou, J., Duan, W.: An optional passive/active transformable wheel-legged mobility concept for search and rescue robots. Robot. Auton. Syst. 107, 145–155 (2018)

    Article  Google Scholar 

  8. Calzado, J., Lindsay, A., Chen, C., Samuels, G., Olszewska, J.I: Sami: Interactive, multi-sense robot architecture. In: Proceedings of 22nd IEEE International Conference on Intelligent Engineering Systems, June 21–23, pp 317–322, Las Palmas de Gran Canaria, Spain (2018)

  9. Sekiguchi, S., Yorozu, A., Kuno, K., Okada, M., Takahashi, M.: Human-friendly control system design for two-wheeled service robot with optimal control approach. Robot. Auton. Syst. 131, 1–16 (2020)

    Article  Google Scholar 

  10. Mohanta, J.C., Keshari, A.: A knowledge based fuzzy-probabilistic roadmap method for mobile robot navigation. Appl. Soft Comput. 79, 391–409 (2019)

    Article  Google Scholar 

  11. Hacohen, S., Shoval, S., Shvalb, N.: Applying probability navigation function in dynamic uncertain environments. Robot. Auton. Syst. 87, 237–246 (2017)

    Article  Google Scholar 

  12. Goto, Y., Fujita, M., Nide, N.: Impletation of 3-valued paraconsistent logic programming towards decision making system of agents. J. Syst. Sci. Syst. Eng. 27(3), 323–339 (2018)

    Article  Google Scholar 

  13. Rath, A.K., Das, D. R., Parhi, H. C., Muni, M.K., Kumar, P.B.: Analysis and use of fuzzy intelligent technique for navigation of humanoid robot in obstacle prone zone. Def. Technol. 14(6), 677–682 (2018)

    Article  Google Scholar 

  14. Turnwald, A., Wollherr, D.: Human-like motion planning based on game theoretic decision making. Int. J. Soc. Robot. 11, 151–170 (2019)

    Article  Google Scholar 

  15. Liu, P., Yu, H., Cang, S.: Optimized adaptive tracking control for an underactuated vibro-driven capsule system. Nonlinear Dyn. 94, 1803–1817 (2018)

    Article  Google Scholar 

  16. Liu, P., Huda, M.N., Tang, Z., Sun, L.: A self-propelled robotic system with a visco-elastic joint: dynamics and motion analysis. Eng. Comput. 36, 655–669 (2020)

    Article  Google Scholar 

  17. Ahmed, S.A., Topalov, A.V., Shakev, N.G., Popov, V.L.: Model-free detection and following of moving objects by an omnidirectional mobile robot using 2d range data. IFAC PapersOnLine 51(22), 226–231 (2018)

    Article  Google Scholar 

  18. Magro, A.V., Manso, L.J., Macharet, D.G., Bustos, P.: Socially aware robot navigation system in human-populated and interactive environments based on an adaptive spatial density function and space affordances. Pattern Recogn. Lett. 118, 72–84 (2019)

    Article  Google Scholar 

  19. Abeyrathna, K.D., Granmo, O.C., Yakovlev, R., Shafikand, A., Goodwin, M.: A novel multi-step finite-state automaton for arbitrarily deterministic tsetlin machine learning. In: Proc Artificial Intelligence XXXVII: 40th SGAI International Conference on Artificial Intelligence, December 15–17, pp 108–114, Cambridge, UK (2020)

  20. Aleluya, E.R.M., Zamayla, A.D., Tamula, S.L.M.: Decision-making system of soccer-playing robots using finite state machine based on skill hierarchy and path planning through bezier polynomials. Procedia Comput. Sci. 135, 230–237 (2018)

    Article  Google Scholar 

  21. Chen, X., Tian, G., Miao, Y.: Driving rule acquisition and decision algorithm to unmanned vehicle in urban traffic. Trans. Bjing Inst. Technol. 37(5), 491–496 (2017)

    Google Scholar 

  22. Klose, P., Mester, R.: Simulated autonomous driving in a realistic driving environment using deep reinforcement learning and a deterministic finite state machine. In: Proceedings of the 2nd International Conference on Applications of Intelligent Systems, pp 1–6 (2019)

  23. Li, J., Tan, Y.: A probabilistic finite state machine based strategy for multi-target search using swarm robotics. Appl. Soft Comput. 77, 467–487 (2019)

    Article  Google Scholar 

  24. Olszewska, J.I., Toman, J.: Open: New path-planning algorithm for real-world complex environment. In: Proceedings of the International Conference on Innovative Techniques and Applications of Artificial Intelligence, pp 237–244 (2016)

  25. Boloor, A., Garimella, K., He, X., Gill, C.: Attacking vision-based perception in end-to-end autonomous driving models. J. Syst. Archit. 110, 1–13 (2020)

    Article  Google Scholar 

  26. Liang, Y., Yan, Z., Zhang, Q., Liang, H., Ji, X., Liu, Y., Liu, R.: A decision-making model based on basal ganglia account of action prediction. In: Proceedings of the 2019 IEEE International Conference on Robotics and Biomimetics, December 6–8, pp 1705–1710, Dali, Yunnan, China (2019)

  27. Wang, C., Zhang, X., Cong, L., Li, J., Zhang, J.: Research on intelligent collision avoidance decision-making of unmanned ship in unknown environments. Evol. Syst. 10, 649–658 (2019)

    Article  Google Scholar 

  28. Zheng, Z., Wu, X., Weng, J: Emergent neural turing machine and its visual navigation. Neural Netw. 10, 116–130 (2019)

    Article  Google Scholar 

  29. Wang, D., Hu, Y., Ma, T.: Mobile robot navigation with the combination of supervised learning in cerebellum and reward-based learning in basal ganglia. Cogn. Syst. Res. 59, 1–14 (2020)

    Article  Google Scholar 

  30. Wang, D., Wang, H., Liu, L.: Unknown environment exploration of multi-robot system with the fordpso. Swarm Evol. Comput. 26, 157–174 (2016)

    Article  Google Scholar 

  31. Gao, W., Tang, Q., Ye, B., Yang, Y., Yao, J.: An enhanced heuristic ant colony optimization for mobile robot path planning. Soft Comput. 24, 6139–6150 (2020)

    Article  Google Scholar 

  32. Faridi, A.Q., Sharma, S., Shukla, A., Tiwari, R., Dhar, J.: Multi-robot multi-target dynamic path planning using artificial bee colony and evolutionary programming in unknown environment. Intell. Serv. Robot. 11, 171–186 (2018)

    Article  Google Scholar 

  33. Ding, H.: Motion path planning of soccer training auxiliary robot based on genetic algorithm in fixed-point rotation environment. J. Ambient Intell. Humanized Comput. https://doi.org/10.1007/s12652-020-01877-4 (2020)

  34. Rao, D.C., Kabat, M.R., Das, P.K., Jena, P.K.: Cooperative navigation planning of multiple mobile robots using improved krill herd. Arab. J. Sci. Eng. 43, 7869–7891 (2018)

    Article  Google Scholar 

  35. Gonzalez-Billandon, J., Sciutti, A., Sandini, G., Rea, F.: Towards a cognitive architecture for self-supervised transfer learning for objects detection with a humanoid robot. In: Proceedings of 2020 Joint IEEE 10th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), Oct 26–30, pp 1–8, Valparaiso, Chile (2020)

  36. Hou, S., Dong, B., Wang, H., Wu, G.: Inspection of surface defects on stay cables using a robot and transfer learning. Autom. Constr. 119, 1–14 (2020)

    Article  Google Scholar 

  37. Carlucho, I., Paula, M.D., Acosta, G.G.: An adaptive deep reinforcement learning approach for mimo pid control of mobile robots. ISA Trans. 102, 280–294 (2020)

    Article  Google Scholar 

  38. Bing, Z., Lemke, C., Cheng, L., Huang, K., Knol, A.: Energy-efficient and damage-recovery slithering gait design for a snake-like robot based on reinforcement learning and inverse reinforcement learning. Neural Netw. 129, 323–333 (2020)

    Article  Google Scholar 

  39. Cuayahuitl, H.: A data-efficient deep learning approach for deployable multimodal social robots. Neurocomputing 396, 587–598 (2020)

    Article  Google Scholar 

  40. Rincon, L., Coronado, E., Law, C., Venture, G.: Adaptive cognitive robot using dynamic perception with fast deep-learning and adaptive on-line predictive control. In: Proceedings of IFToMM World Congress on Mechanism and Machine Science, pp 2429–2438 (2019)

  41. Liu, P., Yu, H., Cang, S.: Adaptive neural network tracking control for underactuated systems with matched and mismatched disturbances. Nonlinear Dyn. 98, 1447–1464 (2019)

    Article  Google Scholar 

  42. Bryndin, E.: Development of sensitivity and active behavior of cognitive robot by means artificial intelligence. Int. J. Robot. Res. Dev. 10(1), 1–11 (2020)

    Google Scholar 

  43. Goel, A.K., Fitezerald, T., Parashar, P.: Analogy and Meta Reasoning: Cognitive Strategies for Robot Learning. Academic Press, Salt Lake City, UT USA (2020)

    Google Scholar 

  44. Olszewska, J.I., Houghtaling, M., Goncalves, P.J.S., Fabiano, N., Haidegger, T., Carbonera, J.L., Patterson, W.R., Ragavan, S.V., Fiorini, S.R., Prestes, E.: Robotic standard development life cycle in action. J. Intell. Robot. Syst. 98, 119–131 (2020)

    Article  Google Scholar 

  45. Weng, J.: Why have we passed neural networks no not abstract well. Nat. Intell. INNS Mag. 1 (1), 13–22 (2011)

    Google Scholar 

  46. Wang, D., Wang, J., Liu, L.: Developmental network: An internal emergent object feature learning. Neural Process. Lett. 48, 1135–1159 (2018)

    Article  Google Scholar 

  47. Wu, X., Bo, Y., Weng, J.: Information-dense actions as contexts. Neurocomputing 311, 164–175 (2018)

    Article  Google Scholar 

  48. Weng, J.: Natural and Artificial Intelligence: Introduction to Computational Brain-Mind. BMI Press, Okemos, Michigan USA (2012)

    Google Scholar 

  49. Avery, M.C., Krichmar, J.L.: Neuromodulatory systems and their interactions: A review of models, theories, and experiments. Front. Neural Circ. 11, 1–18 (2017)

    Google Scholar 

  50. Dasgupta, S., Worgotter, F., Manoonpong, P.: Neuromodulatory adaptive combination of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control. Front. Neural Circ. 8, 1–21 (2014)

    Google Scholar 

  51. Krichmar, J.L.: The neuromodulatory system: a framework for survival and adaptive behavior in a challenging world. Adapt. Behav. 16(6), 385–399 (2008)

    Article  Google Scholar 

  52. Wang, D., Duan, Y., Weng, J.: Motivated optimal developmental learning for sequential tasks without using rigid time-discounts. IEEE Tran. Neural Netw. Learn. Syst. 29(10), 4917–4931 (2018)

    Article  Google Scholar 

  53. Barr, R.: Transfer of learning between 2d and 3d sources during infancy: Informing theory and practice. Dev. Rev. 30(2), 128–154 (2010)

    Article  Google Scholar 

  54. Solgi, M., Liu, T., Weng, J.: A computational developmental model for specificity and transfer in perceptual learning. J. Vis. 13(1), 1–23 (2013)

    Article  Google Scholar 

  55. Liu, J.: Optimization of stochastic computing based deep learning systems with parallel finite state machine implementation. In: Proceedings of the 2020 4th International Conference on Algorithms, Computing and Systems, , September 26–28, 20120, pp 22–26, Berlin, German (2020)

  56. Waterman, M.W., Frezzo, D.C., Wang, M.X.: Adaptive learning using finite state machine logic. In: Proceedings of the Seventh ACM Conference on Learning@Scale, pp 237–240, Virtual Event USA (2020)

  57. Wang, D., Xin, J.: Emergent spatio-temporal multimodal learning using a developmental network. Appl. Intell. 49, 1306–1323 (2019)

    Article  Google Scholar 

  58. Sanchez, J.A., Romero, V.: Computation of moments for probabilistic finite-state automata. Inform. Sci. 516, 388–400 (2020)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This research is supported by the National Natural Science Funds of China under Grants 61873245 and 61876169, Natural Science Funds of Henan Province under Grant 202300410483, and Scientific Problem Tackling of Henan Province under Grant 192102210256.

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Zhengzhou University, 450001, Zhengzhou, People’s Republic of China

    Dongshu Wang & Heshan Wang

  2. Henan Branch of China Mobile Communications Group Co. Ltd, 450018, Zhengzhou, Henan, China

    Kai Yang

  3. The People’s Bank of China, Zhengzhou Central Sub-Branch, 450018, Zhengzhou, People’s Republic of China

    Lei Liu

Authors
  1. Dongshu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heshan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heshan Wang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Yang, K., Wang, H. et al. Behavioral Decision-Making of Mobile Robot in Unknown Environment with the Cognitive Transfer. J Intell Robot Syst 103, 7 (2021). https://doi.org/10.1007/s10846-021-01451-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-021-01451-w

Keywords

Search

Navigation