Abstract
Four knowledge sharing techniques based on fuzzy-Q learning are investigated in this paper. These knowledge sharing techniques are ‘Shared Memory’, ‘Adaptive Weighted Strategy Sharing’, ‘Exploration Guided Method’, and ‘Greatest Mass Method’. Different robot expertness measures are applied to these knowledge sharing techniques in order to improve learning performance. We proposed a new robot expertness measure based on regret evaluation. The regret takes uncertainty bounds of two best actions, i.e. greedy action and the second best action, into account. Simulations were performed to compare the effectiveness of the three expertness measures i.e. expertness based on accumulated rewards, on average move and on regret measure, when applied to different sharing techniques. Our proposed measure resulted in better performance than the other expertness measures. Analysis and comparison of different knowledge sharing techniques are also provided herein.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Tan, M.: Multi-agent reinforcement learning: Independent vs cooperative agents. In: Proc. 10th Int. Conf. Machine Learning (1993)
Ahmadabadi, M.N., Asadpour, M.: Expertness Based Cooperative Q-Learning. IEEE Trans. SMC–Part B 32(1), 66–76 (2002)
Ahmadabadi, M.N., Asadpour, M., Khodanbakhsh, S.H., Nakano, E.: Expertness Measuring in Cooperative Learning. In: IEEE Int. Conf. on IROS, pp. 2261–2267 (2000)
Ahmadabadi, M.N., Asadpour, M., Nakano, E.: Cooperative Q-learning: The Knowledge Sharing Issue. J. Adv. Robotics. 15(8), 815–832 (2002)
Bitaghsir, A.A., Moghimi, A., Lesani, M., Keramati, M.M., Ahmadabadi, M.N., Arabi, B.N.: Successful Cooperation between Heterogeneous Fuzzy Q-learning Agents. In: IEEE Int. Conf. SMC (2004)
Dixon, K.R., Malak, R.J., Khosla, P.K.: Incorporating Prior Knowledge and Previously Learned Information into Reinforcement Learning Agents. Tech. report. Institute for Complex Engineered Systems, Carnegie Mellon University (2000)
Malak, R.J., Khosla, P.K.: A framework for the Adaptive Transfer of Robot Skill Knowledge Using Reinforcement Learning Agents. In: Proc. IEEE Int. Conf. ICRA (2001)
Moreno, D.L., Regueiro, C.B., Iglesias, R., Barro, S.: Using Prior Knowledge to Improve Reinforcement Learning in Mobile Robotics. In: Proc. Towards Autonomous Robotics Systems. Univ. of Essex, UK (2004)
Ritthipravat, P., Maneewarn, T., Laowattana, D., Wyatt, J.: A Modified Approach to Fuzzy Q-Learning for Mobile Robots. In: Proc. IEEE Int. Conf. SMC (2004)
Ritthipravat, P., Maneewarn, T., Wyatt, J.C., Laowattana, D.: Comparison and analysis of expertness measure in knowledge sharing among robots. In: Ali, M., Dapoigny, R. (eds.) IEA/AIE 2006. LNCS, vol. 4031, pp. 60–69. Springer, Heidelberg (2006)
Peterson, T.S., Owens, N.E., Carroll, J.L.: Towards Automatic Shaping in Robot Navigation. In: Proc. IEEE Int. Conf. ICRA (2001)
Carroll, J.L., Peterson, T.S., Owens, N.E.: Memory-guided Exploration in Reinforcement Learning. In: Proc. IEEE Int. Conf. IJCNN (2001)
Whitehead, S., Karlsson, J., Karlsson, J.: Learning Multiple Goal Behavior via Task Decomposition and Dynamic Policy Merging. In: Connell, J.H., Mahadevan, S. (eds.) Robot Learning, Kluwer Academic Publishers, Norwell, MA (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ritthipravat, P., Maneewarn, T., Wyatt, J., Laowattana, D. (2006). Fuzzy-Q Knowledge Sharing Techniques with Expertness Measures: Comparison and Analysis. In: Grigoriev, D., Harrison, J., Hirsch, E.A. (eds) Computer Science – Theory and Applications. CSR 2006. Lecture Notes in Computer Science, vol 3967. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11753728_55
Download citation
DOI: https://doi.org/10.1007/11753728_55
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34166-6
Online ISBN: 978-3-540-34168-0
eBook Packages: Computer ScienceComputer Science (R0)