Online Semi-supervised Learning from Evolving Data Streams with Meta-features and Deep Reinforcement Learning

Abstract

Online semi-supervised learning (SSL) from data streams is an emerging area of research with many applications due to the fact that it is often expensive, time-consuming, and sometimes even unfeasible to collect labelled data from streaming domains. State-of-the-art online SSL algorithms use clustering techniques to maintain micro-clusters, or, alternatively, employ wrapper methods that utilize pseudo-labeling based on confidence scores. Current approaches may introduce false behaviour or make limited use of labelled instances, thus potentially leading to important information being overlooked. In this paper, we introduce the novel Online Reinforce SSL algorithm that uses various K Nearest Neighbour (KNN) classifiers to learn meta-features across diverse domains. Our Online Reinforce SSL algorithm features a meta-reinforcement learning agent trained on multiple-source streams obtained by extracting meta-features and subsequently transferring this meta-knowledge to our target domain. That is, the predictions of the KNN learners are used to select pseudo-labels for the target domain as instances arrive via an incremental learning paradigm. Extensive experiments on benchmark datasets demonstrate the value of our approach and confirm that Online Reinforce SSL outperforms both the state-of-the-art and a self-training baseline.