Teleporting Universal Intelligent Agents

Abstract

When advanced AIs begin to choose their own destiny, one decision they will need to make is whether or not to transfer or copy themselves (software and memory) to new hardware devices. For humans this possibility is not (yet) available and so it is not obvious how such a question should be approached. Furthermore, the traditional single-agent reinforcement-learning framework is not adequate for exploring such questions, and so we base our analysis on the “multi-slot” framework introduced in a companion paper. In the present paper we attempt to understand what an AI with unlimited computational capacity might choose if presented with the option to transfer or copy itself to another machine. We consider two rigorously executed formal thought experiments deeply related to issues of personal identity: one where the agent must choose whether to be copied into a second location (called a“slot”), and another where the agent must make this choice when, after both copies exist, one of them will be deleted. These decisions depend on what the agents believe their futures will be, which in turn depends on the definition of their value function, and we provide formal results.