Finding partitions for learning control of dynamic systems

Abstract

When a dynamic systems is controlled by a learning controller, the state space is required to be coarsely partitioned to make the learning task computationally feasible. This partition forms the representation of the dynamic system to the learning algorithm. However, such representations normally make the system non-Markovian and thus hard to control, do not naturally allow for asymptotic approach of the setpoint, and often necessitate large control actions. By analysing the partitioning function as an information channel providing partial observation of the underlying Markovian process, it can be shown that the problems of hidden state, selective perception and partially observable systems are not brought about by having the wrong number of cells, which could be improved by pruning or splitting nodes, nor can it be improved by augmenting observations with a history of observations, but is rather due to a poor choice of base representation. Using an information loss metric and sliding mode design heuristics, it is possible to find more appropriate partitions of the state space which remove or reduce the learning and controlling problems associated with partitioned representations of dynamic systems. The practical benefits of these partitions are demonstrated controlling a benchmark process.