Multi-agent Based Manifold Denoising

Abstract

Manifold learning plays a central role in many Machine Learning (ML) methods where it assumes information lies on a low-dimensional manifold, but the presence of high dimensional noise may defect their performance. In this contribution, we propose a novel (swarm) algorithm to suppress the noise of manifolds of potentially varying dimensionalities. Inspired by colonial insects this method employs multiple agents with different strategies moving through the data space in parallel. During this process, they use local information to reconstruct the manifolds and then move data objects close to them. Moreover, principles of evolutionary game theory are used to encourage agents to select better strategies and hence optimize the hyper-parameters automatically. While other denoising techniques can be seen as single-agent approaches, the new algorithm is a multi-agent approach which makes it more flexible and suitable for scenarios including multiple manifolds. In the experiments, we simulate several situations from a simple manifold with a specific noise level, to more complex manifolds where there are variations on the density, noise level or dimensionalities. Furthermore, we demonstrate the improvement of the proposed algorithm for the performance of the Parzen Window (PW) density estimator.