Abstract
Learning agents have to deal with the exploration-exploitation dilemma. The choice between exploration and exploitation is very difficult in dynamic systems; in particular in large scale ones such as economic systems. Recent research shows that there is neither an optimal nor a unique solution for this problem. In this paper, we propose an adaptive approach based on meta-rules to adapt the choice between exploration and exploitation. This new adaptive approach relies on the variations of the performance of the agents. To validate the approach, we apply it to economic systems and compare it to two adaptive methods originally proposed by Wilson: one local and one global. Moreover, we compare different exploration strategies and focus on their influence on the performance of the agents.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Azoulay-Schwartz, R., Kraus, S., Wilkenfeld, J.: Exploration vs. exploitation: choosing a supplier in an environment of incomplete information. Elsevier Science (2003)
Baum, J.A.C., Rao, H.: Handbook of Organizational Change and Development: Evolutionary Dynamics of Organizational Populations and Communities. Oxford University Press, Oxford (1999)
Butz, M.V., Wilson, S.W.: An algorithmic description of XCS. Journal of Soft Computing 6, 144–153 (2002)
Carmel, D., Markovitch, S.: Exploration Strategies forModel-Based Learning inMulti-agent Systems. In: Jennings, N., Sycara, K., Georgeff, M. (eds.) Autonomous Agents and Multi-agent systems, vol. 2(2), pp. 141–172 (1999)
Gittings, J.C.: Multi-armed bandit allocation indices. John Wiley and sons, New York (1989)
Roux-Dufort, C.: L’apprentissage organisationnel et le d´eveloppement de l’organisation. D´eveloppement de l’organisation, Nouveaux regards. Durand, R., Economica, pp. 111–134 (2002)
Kaelbling, L.P., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
March, J.G., Simon, H.A.: Les organisations, Dunod edn. (1991)
Meuleau, N., Bourgine, P.: Exploration of multi-state environments: Local measure and backpropagation of uncertainty. Machine Learning 35(2), 117–154 (1999)
Miramontes Hercog, L., Fogarty, T.C.: Social Simulation Using a Multi-agent Model Based on Classifier Systems: The emergence of Vacillating Behavior in the “ El Farol” Bar Problem. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 2001. LNCS (LNAI), vol. 2321, pp. 88–111. Springer, Heidelberg (2002)
Rejeb, L., Guessoum, Z.: Adaptive Firms. In: Proc. AISTA 2004 International Conference on Advances in Intelligent Systems - Theory and Applications. In co-operation with the IEEE Computer Society. Luxembourg November (2004)
Penrose, E.T.: The theory of the growth of the firm. Basil Blackwell, Malden (1959)
Peres-Uribe, A., Hirsbrunner, B.: The risk of Exploration in multi-agent learning systems: a case study. In: Proc. Agents 2000 Joint workshop on learning agents, Barcelona, June 3–7, pp. 33–37 (2000)
Sutton, R.S., Barto, A.G.: Reinforcement learning, an introduction. MIT Press, Cambridge (1998)
Thrun, S.B.: The role of exploration in learning control. In: Sofge, D.A. (ed.) Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches. Florence, Kentucky: Van Nostrand Reinhold (1992)
Watkins, C., Dayan, P.: Q-Learning. Machine Learning 8, 279–292 (1999)
Wiering, M.: Explorations in Efficient Reinforcement Learning. Ph.D. thesis (February 1999)
Wilson, S.W.: Classifiers Fitness Based on Accuracy. Evolutionary computation 3(2), 149–175 (1995)
Wilson, S.W.: Explore/Exploit Strategies in Autonomy. In: Maes, P., Mataric, M., Pollac, J., Meyer, J.-A., Wilson, S. (eds.) From Animals to Animats 4, Proc. of the 4th International Conference of Adaptive Behavior, Cambridge (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rejeb, L., Guessoum, Z., M’Hallah, R. (2006). An Adaptive Approach for the Exploration-Exploitation Dilemma and Its Application to Economic Systems. In: Tuyls, K., Hoen, P.J., Verbeeck, K., Sen, S. (eds) Learning and Adaption in Multi-Agent Systems. LAMAS 2005. Lecture Notes in Computer Science(), vol 3898. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11691839_10
Download citation
DOI: https://doi.org/10.1007/11691839_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33053-0
Online ISBN: 978-3-540-33059-2
eBook Packages: Computer ScienceComputer Science (R0)