Requirements for the Learning of Multiple Dynamics

Abstract

The aim of this work is to discover the principles of learning a group of dynamical systems. The learning mechanism, which is referred to as the Learning Algorithm of Multiple Dynamics (LAMD), is expected to satisfy the following four requirements. (i) Given a set of time-series sequences for training, estimate the dynamics and their latent variables. (ii) Order the dynamical systems according to the similarities between them. (iii) Interpolate intermediate dynamics from the given dynamics. (iv) After training, the LAMD should be able to identify or classify new sequences. For this purpose several algorithms have been proposed, such as the Recurrent Neural Network with Parametric Bias and the modular network SOM with recurrent network modules. In this paper, it is shown that these types of algorithms do not satisfy the above requirements, but can be improved by normalization of estimated latent variables. This confirms that the estimation process of latent variables plays an important role in the LAMD. Finally, we show that a fully latent space model is required to satisfy the requirements, for which purpose a SOM with a higher-rank, such as a SOM², is best suited.