Why Semi-supervised Learning Makes Sense: A Pedagogical Note

Abstract

The main idea behind semi-supervised learning is that when we do not have enough human-generated labels, we train a machine learning system based on what we have, and we add the resulting labels (called pseudo-labels) to the training sample. Interesting, this idea works well, but why is somewhat a mystery: we did not add any new information so why is this working? There exist explanations for this empirical phenomenon, but most of these explanations are based on complicated math. In this paper, we provide a simple intuitive explanation.