Abstract
Without a concrete measure of the “complicatedness” of tasks that artificial agents can reliably perform, assessing progress in AI is difficult. Only by providing evidence of progress towards more complicated tasks can developers aiming for general machine intelligence (GMI) ascertain their progress towards that goal. No such measure for this exists at present. In this work we propose a new measure of the intricacy of tasks, especially designed to describe their physical composition and makeup. Our intricacy is a multi-dimensional measurement that depends purely on objective physical properties of tasks and the environment in which they are to be performed. From this task intricacy measure, a relation to the knowledge of learners can allow calculation of the difficulty of a particular task for a particular learner. The method is intended for both narrow-AI and GMI-aspiring systems. Here we discuss some of the implications of our intricacy measure and suggest ways in which it may be used in AI research and system evaluation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
With complexity we mean the intuitive concept as used in every-day language, not the concept as used in computer science.
- 2.
For a more detailed description of our understanding of causal knowledge and its implications see [3].
- 3.
While we take the non-axiomatic approach we still assume that the underlying environment follows certain rules, i.e. causal structures.
- 4.
We assume that the “designer’s perspective” includes a complete access and overview to a task’s full set of variables.
- 5.
For further information on the level of detail see [3]. How knowledge representation of the agent affects the intricacy by changing the level of detail is a problem that still needs to be addressed.
- 6.
References
Adams, S., et al.: Mapping the landscape of human-level artificial general intelligence. AI Mag. 33(1), 25–42 (2012)
Bartlett, P.L., Mendelson, S.: Rademacher and gaussian complexities: risk bounds and structural results. J. Mach. Learn. Res. 3, 463–482 (2002)
Belenchia, M., Thórisson, K.R., Eberding, L.M., Sheikhlar, A.: Elements of task theory. In: Proceedings of the International Conference on Artificial General Intelligence. Springer (2021, in submission)
Bellemare, M.G., Naddaf, Y., Veness, J., Bowling, M.: The arcade learning environment: an evaluation platform for general agents (extended abstract). In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp. 4148–4152 (2015)
Bieger, J., Thórisson, K.R., Steunebrink, B.R., Thorarensen, T., Sigurdardóttir, J.S.: Evaluation of general-purpose artificial intelligence: Why, what & how. In: EGPAI 2016 - Evaluating General-Purpose A.I., Workshop held in conjuction with the European Conference on Artificial Intelligence (2016)
Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Learnability and the vapnik-chervonenkis dimension. J. ACM (JACM) 36(4), 929–965 (1989)
Eberding, L.M., Thórisson, K.R., Sheikhlar, A., Andrason, S.P.: SAGE: task-environment platform for evaluating a broad range of AI learners. In: Goertzel, B., Panov, A.I., Potapov, A., Yampolskiy, R. (eds.) AGI 2020. LNCS (LNAI), vol. 12177, pp. 72–82. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52152-3_8
Hernández-Orallo, J.: The Measure of all Minds: Evaluating natural and artificial intelligence. Cambridge University Press, Cambridge (2017)
Hernández-Orallo, J., et al.: A new ai evaluation cosmos: Ready to play the game? AI Mag. 38(3), 66–69 (2017)
Martınez-Plumed, F., Hernández-Orallo, J.: Ai results for the atari 2600 games: difficulty and discrimination using irt. EGPAI, Evaluating General-Purpose Artificial Intelligence 33 (2016)
Nivel, E., et al.: Bounded recursive self-improvement (2013)
Riedl, M.O.: The lovelace 2.0 test of artificial creativity and intelligence. arXiv preprint arXiv:1410.6142 (2014)
Świechowski, M., Park, H., Mańdziuk, J., Kim, K.J.: Recent advances in general game playing. The Scientific World Journal 2015 (2015)
Thórisson, K.R., Bieger, J., Schiffel, S., Garrett, D.: Towards flexible task environments for comprehensive evaluation of artificial intelligent systems & automatic learners. In: Proceedings of the International Conference on Artificial General Intelligence, pp. 187–196 (2015)
Thórisson, K.R., Bieger, J., Thorarensen, T., Sigurðardóttir, J.S., Steunebrink, B.R.: Why artificial intelligence needs a task theory. In: Steunebrink, B., Wang, P., Goertzel, B. (eds.) AGI -2016. LNCS (LNAI), vol. 9782, pp. 118–128. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41649-6_12
Thórisson, K.R., Talbot, A.: Cumulative learning with causal-relational models. In: Iklé, M., Franz, A., Rzepka, R., Goertzel, B. (eds.) AGI 2018. LNCS (LNAI), vol. 10999, pp. 227–237. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-97676-1_22
Turing, A.M.: I.-Computing machinery and intelligence. Mind LIX(236), 433–460 (10 1950). https://doi.org/10.1093/mind/LIX.236.433
Wang, P.: Rigid flexibility: The logic of intelligence. Springer Science, vol. 34 (2006)
Acknowledgments
This work was supported in part by Cisco Systems, the Icelandic Institute for Intelligent Machines and Reykjavik University.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Eberding, L.M., Belenchia, M., Sheikhlar, A., Thórisson, K.R. (2022). About the Intricacy of Tasks. In: Goertzel, B., Iklé, M., Potapov, A. (eds) Artificial General Intelligence. AGI 2021. Lecture Notes in Computer Science(), vol 13154. Springer, Cham. https://doi.org/10.1007/978-3-030-93758-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-93758-4_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93757-7
Online ISBN: 978-3-030-93758-4
eBook Packages: Computer ScienceComputer Science (R0)