Abstract
The concept of “task” is at the core of artificial intelligence (AI): Tasks are used for training and evaluating AI systems, which are built in order to perform and automatize tasks we deem useful. In other fields of engineering theoretical foundations allow thorough evaluation of designs by methodical manipulation of well understood parameters with a known role and importance; this allows an aeronautics engineer, for instance, to systematically assess the effects of wind speed on an airplane’s performance and stability. No framework exists in AI that allows this kind of methodical manipulation: Performance results on the few tasks in current use (cf. board games, question-answering) cannot be easily compared, however similar or different. The issue is even more acute with respect to artificial general intelligence systems, which must handle unanticipated tasks whose specifics cannot be known beforehand. A task theory would enable addressing tasks at the class level, bypassing their specifics, providing the appropriate formalization and classification of tasks, environments, and their parameters, resulting in more rigorous ways of measuring, comparing, and evaluating intelligent behavior. Even modest improvements in this direction would surpass the current ad-hoc nature of machine learning and AI evaluation. Here we discuss the main elements of the argument for a task theory and present an outline of what it might look like for physical tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Classical planning has hierarchical task networks [4], but subtask decomposition is almost always done manually and there is no real analysis of tasks on a general level. Some people working on AI evaluation — one of task theory’s primary applications — attempt to analyze some properties of task-environments, but they don’t go beyond complexity and difficulty-related measures [6].
- 2.
More discussion of various evaluation methods can be found in our previous publication [11].
- 3.
Recall that a task theory would include the limitations that the body of an agent imposes — its interface to the task-environment.
- 4.
We use approximate rather than precise equivalence between X and its goal value \(G_X\) because we intend for our theory to describe real-world task-environments, which always must come with error bounds.
References
Bieger, J., Thórisson, K.R., Garrett, D.: Raising AI: tutoring matters. In: Goertzel, B., Orseau, L., Snaider, J. (eds.) AGI 2014. LNCS, vol. 8598, pp. 1–10. Springer, Heidelberg (2014)
Garrett, D., Bieger, J., Thórisson, K.R.: Tunable and generic problem instance generation for multi-objective reinforcement learning. In: Proceedings of the IEEE Symposium Series on Computational Intelligence 2014. IEEE, Orlando, Florida (2014)
Genesereth, M., Thielscher, M.: General game playing. Synth. Lect. Artif. Intell. Mach. Learn. 8(2), 1–229 (2014)
Georgievski, I., Aiello, M.: An Overview of Hierarchical Task Network Planning. CoRR abs/1403.7426 (2014). http://arxiv.org/abs/1403.7426
Hernández-Orallo, J.: AI Evaluation: past, present and future. CoRR abs/1408.6908 (2014). http://arxiv.org/abs/1408.6908
Hernández-Orallo, J.: Stochastic tasks: difficulty and levin search. In: Bieger, J., Goertzel, B., Potapov, A. (eds.) AGI 2015. LNCS, vol. 9205, pp. 90–100. Springer, Heidelberg (2015)
Legg, S., Veness, J.: An approximation of the universal intelligence measure. In: Dowe, D.L. (ed.) Solomonoff Festschrift. LNCS, vol. 7070, pp. 236–249. Springer, Heidelberg (2013)
Levine, J., Congdon, C.B., Ebner, M., Kendall, G., Lucas, S.M., Miikkulainen, R., Schaul, T., Thompson, T., Lucas, S.M., Mateas, M.: General video game playing. Artif. Comput. Intell. Games 6, 77–83 (2013)
Marcus, G., Rossi, F., Veloso, M. (eds.): Beyond the Turing Test, AI Magazine, vol. 37, 1 edn. AAAI (2016)
Robinson, P.: Task complexity, task difficulty, and task production: exploring interactions in a componential framework. Appl. Linguist. 22(1), 27–57 (2001)
Thórisson, K.R., Bieger, J., Schiffel, S., Garrett, D.: Towards flexible task environments for comprehensive evaluation of artificial intelligent systems and automatic learners. In: Bieger, J., Goertzel, B., Potapov, A. (eds.) AGI 2015. LNCS, vol. 9205, pp. 187–196. Springer, Heidelberg (2015)
Wang, P.: The assumptions on knowledge and resources in models of rationality. Int. J. Mach. Conscious. 3(01), 193–218 (2011)
Wooldridge, M.: An Introduction to MultiAgent Systems. Wiley, New York (2009)
Acknowledgments
The authors would like to thank Eric Nivel for insightful comments. This work was sponsored by the School of Computer Science at Reykjavik University, by a Centers of Excellence Grant (IIIM) from the Science & Technology Policy Council of Iceland, and by a grant from the Future of Life Institute.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Thórisson, K.R., Bieger, J., Thorarensen, T., Sigurðardóttir, J.S., Steunebrink, B.R. (2016). Why Artificial Intelligence Needs a Task Theory. In: Steunebrink, B., Wang, P., Goertzel, B. (eds) Artificial General Intelligence. AGI 2016. Lecture Notes in Computer Science(), vol 9782. Springer, Cham. https://doi.org/10.1007/978-3-319-41649-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-41649-6_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-41648-9
Online ISBN: 978-3-319-41649-6
eBook Packages: Computer ScienceComputer Science (R0)