Extended MDL principle for feature-based inductive transfer learning

Abstract

Transfer learning provides a solution in real applications of how to learn a target task where a large amount of auxiliary data from source domains are given. Despite numerous research studies on this topic, few of them have a solid theoretical framework and are parameter-free. In this paper, we propose an Extended Minimum Description Length Principle (EMDLP) for feature-based inductive transfer learning, in which both the source and the target data sets contain class labels and relevant features are transferred from the source domain to the target one. Unlike conventional methods, our encoding measure is based on a theoretical background and has no parameter. To obtain useful features to be used in the target task, we design an enhanced encoding length by adopting a code book that stores useful information obtained from the source task. With the code book that builds connections between the source and the target tasks, our EMDLP is able to evaluate the inferiority of the results of transfer learning with the add sum of the code lengths of five components: those of the corresponding two hypotheses, the two data sets with the help of the hypotheses, and the set of the transferred features. The proposed method inherits the nice property of the MDLP that elaborately evaluates the hypotheses and balances the simplicity of the hypotheses and the goodness-of-the-fit to the data. Extensive experiments using both synthetic and real data sets show that the proposed method provides a better performance in terms of the classification accuracy and is robust against noise.