The recognition of manga (Japanese comics) characters is an essential step in industrial applications, such as manga character retrieval, content analysis and copyright protection. However, conventional methods for manga character recognition are mainly based on handcrafted features which are not robust enough for manga of various style. The emergence of deep learning based methods provides representational features, which has a huge demand for labeled data. In this paper, we propose a framework to exploit unlabeled manga data to facilitate the discriminative capability of deep feature representations for manga character recognition (i.e., unsupervised learning on manga images), which does not rely on any manual annotation. Specifically, we first train an initial feature model using an anime character dataset. Then, we adopt a Progressive Main Characters Mining (PMCM) strategy which iterates between two steps: 1) produce selected data with estimated labels from unlabeled data, 2) update the feature model by the selected data. These two steps are mutually promoted in essence. Experimental results on Manga109 dataset, to which we introduce new head annotations, demonstrate the effectiveness of the proposed framework and the usefulness in manga character verification and retrieval.