Designing embedded agents to optimize end-user objectives

Abstract

We formulate the design of discrete-state stochastic control systems as optimizing a performance objective specified in user-oriented terms, i.e. terms that need not be perceivable by the controller or agent being designed. This addresses a user acceptance issue: while agent designs (control algorithms) are limited to distinctions about state supported by artificial perception systems, end users want to evaluate performance using terms such as safety, opportunity, and throughput. We elucidate a feedback from evaluation to agent design via a sensitivity analysis, obtaining the gradient of a time-averaging objective function w.r.t. state transitions influenced by the agent. This gradient leads to a methodology for iteratively improving a system's performance, as perceived by others.

This work was supported in part by NASA contract 9-19040.