Skip to main content

Advertisement

Springer Nature Link
Log in

A learning system based on lazy metareasoning

  • Regular Paper
  • Published:
Progress in Artificial Intelligence Aims and scope Submit manuscript

Abstract

Metareasoning has been widely studied in the literature, with a wide variety of algorithms and partially overlapping methodological approaches. However, these methods are typically either not targeted toward practical machine learning systems or alternatively are focused on achieving the best possible performance for a particular domain, with extensive human tuning and research, and vast computing resources. In this paper, our goal is to create systems that perform sustained autonomous learning, with automatically determined domain-specific optimizations for any given domain, and without requiring human assistance. We present Alma, a metareasoning architecture that creates and selects reasoning methods based on empirically observed performance. This is achieved by using lazy learning at the metalevel, and automatically training and combining reasoning methods at run-time. In experiments across diverse data sets, we demonstrate the ability of Alma to successfully reason about learner performance in different domains and achieve a better overall result than any of the individual reasoning methods, even with limited computing time available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Notes

  1. Algorithms with stochastic behavior are modeled as different random number generator states being different functions.

  2. Standard UCB1 can provably achieve asymptotic zero-regret for the basic multi-armed bandit problem, while this has not been shown for UCB1TUNED. However, the zero-regret proof doesn’t apply to the harder metareasoning problem Alma is addressing, and for our components, we are primarily interested in practical performance.

References

  1. Aha, D.W. (ed.): Lazy Learning. Kluwer, Norwell (1997)

    MATH  Google Scholar 

  2. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002). https://doi.org/10.1023/A:1013689704352

    Article  MATH  Google Scholar 

  3. Bates, J.M., Granger, C.W.: The combination of forecasts. J. Oper. Res. Soc. 20(4), 451–468 (1969)

    Article  Google Scholar 

  4. Bennett, J., Lanning, S., et al.: The Netflix prize. In: Proceedings of the KDD Cup and Workshop, vol. 2007, p. 35. New York, NY, USA (2007)

  5. Bonissone, P.P.: Lazy meta-learning: creating customized model ensembles on demand. In: IEEE World Congress on Computational Intelligence, pp. 1–23. Springer, Berlin (2012)

  6. Brügmann, B.: Monte Carlo Go. In: AAAI Fall Symposium on Games: Playing, Planning, and Learning (1993)

  7. Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A.: Ensemble selection from libraries of models. In: Proceedings of the Twenty-First International Conference on Machine Learning, p. 18. ACM (2004)

  8. Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)

    Book  MATH  Google Scholar 

  9. Chakrabarti, D., Kumar, R., Radlinski, F., Upfal, E.: Mortal multi-armed bandits. In: Koller, D., Schuurmans, D., Bengio, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 21, pp. 273–280. Curran Associates, Inc. (2009)

  10. Chan, P.K., Stolfo, S.J.: Experiments on multistrategy learning by meta-learning. In: Proceedings of the Second International Conference on Information and Knowledge Management, pp. 314–323. ACM (1993)

  11. Clemen, R.T.: Combining forecasts: a review and annotated bibliography. Int. J. Forecast. 5(4), 559–583 (1989)

    Article  Google Scholar 

  12. Cox, M.: Introspective Multistrategy Learning: Constructing a Learning Strategy Under Reasoning Failure. Ph.D. Thesis, College of Computing, Georgia Institute of Technology (1996)

  13. Cox, M.T., Eiselt, K., Kolodner, J., Nersessian, N., Recker, M., Simon, T.: Introspective multistrategy learning: on the construction of learning strategies. Artif. Intell. 112, 1–55 (1999)

    Article  Google Scholar 

  14. Dietterich, T.G.: Ensemble methods in machine learning. In: Proceedings of the First International Workshop on Multiple Classifier Systems, MCS’00, pp. 1–15. Springer, London, UK (2000). http://dl.acm.org/citation.cfm?id=648054.743935

  15. Fox, S., Leake, D.B.: Introspective reasoning for index refinement in case-based reasoning. J. Exp. Theor. Artif. Intell. 13(1), 63–88 (2001). https://doi.org/10.1080/09528130010029794

    Article  MATH  Google Scholar 

  16. Francis, A.G., Ram, A.: The utility problem in case-based reasoning. In: AAAICBR-93, Proceedings of the 1993 Case-Based Reasoning Workshop (1993)

  17. Gelly, S., Wang, Y.: Exploration Exploitation in Go: UCT for Monte-Carlo Go. In: Twentieth Annual Conference on Neural Information Processing Systems (NIPS 2006) (2006)

  18. Guyon, I., Chaabane, I., Escalante, H.J., Escalera, S., Jajetic, D., Lloyd, J.R., Macià, N., Ray, B., Romaszko, L., Sebag, M., Statnikov, A., Treguer, S., Viegas, E.: A brief review of the ChaLearn AutoML challenge: any-time any-dataset learning without human intervention. In: Hutter, F., Kotthoff, L., Vanschoren, J. (eds.) Proceedings of the Workshop on Automatic Machine Learning, Proceedings of the Machine Learning Research, vol. 64, pp. 21–30. PMLR, New York, NY, USA (2016). http://proceedings.mlr.press/v64/guyon_review_2016.html

  19. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11(1), 10–18 (2009)

    Article  Google Scholar 

  20. Holte, R.: Very simple classification rules perform well on most commonly used datasets. Mach. Learn. 11, 63–91 (1993)

    Article  MATH  Google Scholar 

  21. Houeland, T.G., Aamodt, A.: Towards an introspective architecture for meta-level reasoning in clinical decision support systems. In: ICCBR 2009, 7th Workshop on CBR in the Health Sciences (2009)

  22. Houeland, T.G., Aamodt, A.: The Utility Problem for Lazy Learners—Towards a Non-eager Approach, pp. 141–155. Springer, Berlin (2010). https://doi.org/10.1007/978-3-642-14274-1_12

  23. Houeland, T.G., Bruland, T., Aamodt, A., Langseth, H.: Extended abstract: combining CBR and BN using metareasoning. In: Kofod-Petersen, A., Heintz, F., Langseth, H. (eds.) SCAI, Frontiers in Artificial Intelligence and Applications, vol. 227, pp. 189–190. IOS Press (2011). https://doi.org/10.3233/978-1-60750-754-3-189

  24. Kocsis, L., Szepesvári, C.: Bandit based monte-carlo planning. ECML-06. Number 4212 in LNCS, pp. 282–293. Springer, Berlin (2006)

  25. Kotthoff, L., Thornton, C., Hoos, H.H., Hutter, F., Leyton-Brown, K.: Auto-weka 2.0: automatic model selection and hyperparameter optimization in weka. J. Mach. Learn. Res. 17, 1–5 (2016)

    MATH  Google Scholar 

  26. Krawczyk, B., Minku, L.L., Gama, J., Stefanowski, J., Woźniak, M.: Ensemble learning for data stream analysis: a survey. Inf. Fusion 37, 132–156 (2017)

    Article  Google Scholar 

  27. Laird, J.: The Soar Cognitive Architecture. MIT Press, Cambridge (2012)

    Google Scholar 

  28. Lemke, C., Budka, M., Gabrys, B.: Metalearning: a survey of trends and technologies. Artif. Intell. Rev. 44(1), 117–130 (2015)

    Article  Google Scholar 

  29. Lenz, M.: CABATA: a hybrid CBR system. In: Richter, M.M., Wess, S., Althoff, K.D., Maurer, F. (eds.) First European Workshop on Case-Based Reasoning (EWCBR-93): Posters and Presentations (volume I), pp. 204–209 (1993)

  30. Li, B., Hoi, S.C.H.: Online portfolio selection: a survey. ACM Comput. Surv. 46(3), 35:1–35:36 (2014). https://doi.org/10.1145/2512962

    MATH  Google Scholar 

  31. Lichman, M.: UCI Machine Learning Repository (2013). http://archive.ics.uci.edu/ml. Accessed 26 May 2017

  32. Masoudnia, S., Ebrahimpour, R.: Mixture of experts: a literature survey. Artif. Intell. Rev. 42, 275–293 (2014). https://doi.org/10.1007/s10462-012-9338-y

  33. Mendes-Moreira, J., Soares, C., Jorge, A.M., Sousa, J.F.D.: Ensemble approaches for regression: a survey. ACM Comput. Surv. (CSUR) 45(1), 10 (2012)

    Article  MATH  Google Scholar 

  34. Mitchell, T.M.: The Need for Biases in Learning Generalizations. Tech. rep (1980)

  35. Nascimento, D.S., Canuto, A.M., Coelho, A.L.: An empirical analysis of meta-learning for the automatic choice of architecture and components in ensemble systems. In: 2014 Brazilian Conference on Intelligent Systems (BRACIS), pp. 1–6. IEEE (2014)

  36. Radhakrishnan, J., Ontañ, S., Ram, A.: Goal-driven learning in the gila integrated intelligence architecture. In: Boutilier, C. (ed.) IJCAI, pp. 1205–1210 (2009). http://dblp.uni-trier.de/db/conf/ijcai/ijcai2009.html

  37. Rice, J.R.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976). https://doi.org/10.1016/S0065-2458(08)60520-3

    Article  Google Scholar 

  38. Rubin, J., Watson, I.: On combining decisions from multiple expert imitators for performance. In: Walsh, T. (ed.) IJCAI, pp. 344–349. IJCAI/AAAI (2011)

  39. Salzberg, S.L.: On comparing classifiers: pitfalls to avoid and a recommended approach. Data Min. Knowl. Discov. 1(3), 317–328 (1997). https://doi.org/10.1023/A:1009752403260

    Article  Google Scholar 

  40. Schaffer, C.: Selecting a classification method by cross-validation. Mach. Learn. 13(1), 135–143 (1993). https://doi.org/10.1007/BF00993106

    Google Scholar 

  41. Schapire, R.E.: The Boosting Approach to Machine Learning: An Overview. Springer, New York, pp. 149–171 (2003). https://doi.org/10.1007/978-0-387-21579-2_9

  42. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., Hassabis, D.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016). https://doi.org/10.1038/nature16961

    Article  Google Scholar 

  43. Simm, J.: Survey of Hyperparameter Optimization in NIPS2014. http://github.com/jaak-s/nips2014-survey (2015). Accessed 26 May 2017

  44. Thompson, W.R.: On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3–4), 285–294 (1933)

    Article  MATH  Google Scholar 

  45. Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of the KDD-2013, pp. 847–855 (2013)

  46. Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. Artif. Intell. Rev. 18(2), 77–95 (2002). https://doi.org/10.1023/A:1019956318069

    Article  Google Scholar 

  47. Wallis, K.F.: Combining forecasts–forty years later. Appl. Financ. Econ. 21(1–2), 33–41 (2011). https://doi.org/10.1080/09603107.2011.523179

    Article  Google Scholar 

  48. Watson, I.: A case study of maintenance of a commercially fielded case-based reasoning system. Comput. Intell. 17, 387–398 (2001)

    Article  Google Scholar 

  49. Wolpert, D.H.: Stacked generalization. Neural Netw. 5(2), 241–259 (1992). https://doi.org/10.1016/S0893-6080(05)80023-1

    Article  Google Scholar 

  50. Woźniak, M., Graña, M., Corchado, E.: A survey of multiple classifier systems as hybrid systems. Inf. Fusion 16, 3–17 (2014)

    Article  Google Scholar 

  51. Zhang, X.S., Shrestha, B., Yoon, S., Kambhampati, S., et al.: An ensemble architecture for learning complex problem-solving techniques from demonstration. ACM Trans. Intell. Syst. Technol. 3(4), 75:1–75:38 (2012)

    Article  Google Scholar 

  52. Zhang, X.S., Yoon, S., DiBona, P., Appling, D.S., Ding, L., et al.: An ensemble learning and problem solving architecture for airspace management. In: Haigh, K.Z., Rychtyckyj, N. (eds.) Proceedings of Twenty-First Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-09). AAAI (2009)

  53. Zoph, B., Le, Q.V.: Neural Architecture Search with Reinforcement Learning (2017). http://arxiv.org/abs/1611.01578

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, Norwegian University of Science and Technology, Trondheim, Sør-Trøndelag, Norway

    Tor Gunnar Houeland & Agnar Aamodt

Authors
  1. Tor Gunnar Houeland
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Agnar Aamodt
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Tor Gunnar Houeland.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Houeland, T.G., Aamodt, A. A learning system based on lazy metareasoning. Prog Artif Intell 7, 129–146 (2018). https://doi.org/10.1007/s13748-017-0138-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13748-017-0138-0

Keywords