Abstract
Positive transfer learning (TL) occurs when, after gaining experience from learning how to solve a (source) task, the same learner can exploit this experience to improve performance and/or learning on a different (target) task. TL methods are typically complex, and case-based reasoning can support them in multiple ways. We introduce a method for recognizing intent in a source task, and then applying that knowledge to improve the performance of a case-based reinforcement learner in a target task. We report on its ability to significantly outperform baseline approaches for a control task in a simulated game of American football. We also compare our approach to an alternative approach where source and target task learning occur concurrently, and discuss the tradeoffs between them.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Blaylock, N., Allen, J.: Fast hierarchical goal schema recognition. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, pp. 796–801. AAAI Press, Boston (2006)
Bui, H.: A general model for online probabilistic plan recognition. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pp. 1309–1318. Morgan Kaufmann, Acapulco (2002)
Bransford, J.D., Brown, A.L., Cocking, R.R. (eds.): How people learn: Brain, mind, experience, and school. National Academy Press, Washington (2000)
Chang, C.-C., Lin, C.-J.: LIBSVM: A library for support vector machines (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvm
Fagan, M., Cunningham, P.: Case-based plan recognition in computer games. In: Ashley, K.D., Bridge, D.G. (eds.) ICCBR 2003. LNCS, vol. 2689, pp. 161–170. Springer, Heidelberg (2003)
Falkenhainer, B., Forbus, K.D., Gentner, D.: The structure-mapping engine: Algorithm and examples. Artificial Intelligence 41(1), 1–63 (1989)
Gentner, D.: The mechanisms of analogical learning. In: Buchanan, B.G., Wilkins, D.C. (eds.) Readings in knowledge acquisition and learning: Automating the construction and improvement of expert systems. Morgan Kaufmann, San Francisco (1993)
Goel, A., Bhatta, S.: Design patterns: A unit of analogical transfer in creative design. Advanced Engineering Informatics 18(2), 85–94 (2004)
von Hessling, A., Goel, A.: Abstracting reusable cases from reinforcement learning. In: Aha, D.W., Wilson, D.C. (eds.) Computer gaming and simulation environments: Proceedings of the ICCBR Workshop (2005); S. Bruninghaus (ed.) Workshop Proceedings of the Sixth ICCBR. DePaul University, Chicago
Hinrichs, T., Forbus, K.: Analogical learning in a turn-based strategy game. In: Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, pp. 853–858. Professional Book Center, Hyderabad (2007)
Kautz, H.: A formal theory of plan recognition. Doctoral dissertation, University of Rochester, Rochester, NY (1987)
Kerkez, B., Cox, M.T.: Incremental case-based plan recognition with local predictions. International Journal on Artificial Intelligence Tools 12(4), 413–463 (2003)
Klenk, M., Forbus, K.D.: Measuring the level of transfer learning by an AP physics problem-solver. In: Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, pp. 446–451. AAAI Press, Vancouver (2007)
Kuhlmann, G., Stone, P.: Graph-based domain mapping for transfer learning in general games. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS, vol. 4701, pp. 188–200. Springer, Heidelberg (2007)
Liao, L., Fox, D., Kautz, H.A.: Location-based activity recognition using relational Markov networks. In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, pp. 773–778. Professional Book Center, Edinburgh (2005)
Liu, Y., Stone, P.: Value-function-based transfer for reinforcement learning using structure mapping. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, pp. 415–420. AAAI Press, Boston (2006)
López de Mantaras, R., McSherry, D., Bridge, D.G., Leake, D.B., Smyth, B., Craw, S., Faltings, B., Maher, M.L., Cox, M.T., Forbus, K.D., Keane, M., Aamodt, A., Watson, I.D.: Retrieval, reuse, revision and retention in case-based reasoning. Knowledge Engineering Review 20(3), 215–240 (2005)
Marx, Z., Rosenstein, M.T., Kaelbling, L.P., Dietterich, T.G.: Transfer learning with an ensemble of background tasks. In: Silver, D., Bakir, G., Bennett, K., Caruana, R., Pontil, M., Russell, S., Tadepalli, P. (eds.) Inductive Transfer: 10 Years Later: Papers from the NIPS Workshop, Whistler, BC, Canada (2005), http://iitrl.acadiau.ca/itws05/
Molineaux, M., Aha, D.W.: TIELT: A testbed for gaming environments. In: Proceedings of the Twentieth National Conference on Artificial Intelligence, pp. 1690–1691. AAAI Press, Pittsburgh (2005)
Molineaux, M., Aha, D.W., Moore, P.: Learning continuous action models in a real-time strategy environment. In: Proceedings of the Twenty-First International FLAIRS Conference, pp. 257–262. AAAI Press, Coconut Grove (2008)
Molineaux, M., Aha, D.W., Sukthankar, G.: Beating the defense: Using plan recognition to inform learning agents. To appear in Proceedings of the Twenty-Second International FLAIRS Conference. AAAI Press, Sanibel Island (2009)
Natarajan, S., Bui, H.H., Tadepalli, P., Kersting, K., Wong, W.-K.: Logical hierarchical hidden Markov models for modeling user activities. In: Železný, F., Lavrač, N. (eds.) ILP 2008. LNCS, vol. 5194, pp. 192–209. Springer, Heidelberg (2008)
Natarajan, S., Tadepalli, P., Fern, A.: A relational hierarchical model for decision-theoretic assistance. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds.) ILP 2007. LNCS, vol. 4894, pp. 175–190. Springer, Heidelberg (2008)
Perkins, D.N., Salomon, G.: Transfer of learning. In: Husen, T., Postelwhite, T.N. (eds.) International Handbook of Educational Research, pp. 6452–6457. Pergamon Press, Oxford (1994)
Raina, R., Ng, A.Y., Koller, D.: Constructing informative priors using transfer learning. In: Proceedings of the Twenty-Third International Conference on Machine Learning, pp. 713–720. ACM, Pittsburgh (2006)
Rosenstein, M.T., Marx, Z., Kaelbling, L.P., Dietterich, T.G.: To transfer or not to transfer. In: Silver, D., Bakir, G., Bennett, K., Caruana, R., Pontil, M., Russell, S., Tadepalli, P. (eds.) Inductive Transfer: 10 Years Later: Papers from the NIPS Workshop, Whistler, BC, Canada (2005), http://iitrl.acadiau.ca/itws05/Papers/ITWS10-RosensteinM05_ITWS.pdf
Shapiro, D., Könik, T., O’Rorke, P.: Achieving far transfer in an integrated cognitive architecture. In: Proceedings of the Twenty-Third Conference on Artificial Intelligence, pp. 1325–1330. AAAI Press, Chicago (2008)
Sharma, M., Holmes, M., Santamaria, J.C., Irani, A., Isbell Jr., C.L., Ram, A.: Transfer learning in real-time strategy games using hybrid CBR/RL. In: Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, Hyderabad, India, pp. 1041–1046 (2007), http://www.aaai.org/Papers/IJCAI/2007/IJCAI07-168.pdf
Simon, H.A.: Search and reasoning in problem solving. Artificial Intelligence 21, 7–29 (1983)
Stracuzzi, D.: Memory organization and knowledge transfer. In: Banerjee, B., Liu, Y., Youngblood, G.M. (eds.) Structural Knowledge Transfer for Machine Learning: Papers from the ICML Workshop, Pittsburgh, PA (2006), http://orca.st.usm.edu/~banerjee/icmlws06/
Sukthankar, G., Molineaux, M., Aha, D.W.: Recognizing and exploiting opponent intent in Rush Football (Technical Note AIC-09-062). Naval Research Laboratory, Navy Center for Applied Research in Artificial Intelligence, Washington (2008)
Sutton, R., Barto, A.: Reinforcement learning: An introduction. MIT Press, Cambridge (1998)
Thorndike, E.L., Woodworth, R.S.: The influence of improvement in one mental function upon the efficiency of other functions (I). Psychological Review 8, 247–261 (1901)
Vapnik, V.: Statistical learning theory. Wiley & Sons, New York (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Aha, D.W., Molineaux, M., Sukthankar, G. (2009). Case-Based Reasoning in Transfer Learning. In: McGinty, L., Wilson, D.C. (eds) Case-Based Reasoning Research and Development. ICCBR 2009. Lecture Notes in Computer Science(), vol 5650. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02998-1_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-02998-1_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02997-4
Online ISBN: 978-3-642-02998-1
eBook Packages: Computer ScienceComputer Science (R0)