"Learning with side-information" is attracting more and more attention in machine learning problems. In this paper, we propose a general iterative framework for relevant linear feature extraction. It efficiently utilizes both the side-information and unlabeled data to enhance gradually algorithms' performance and robustness. Both good relevant feature extraction and reasonable similarity matrix estimation can be realized. Specifically, we adopt relevant component analysis (RCA) under this framework and get the derived iterative self-enhanced relevant component analysis (ISERCA) algorithm. The experimental results on several data sets show that ISERCA outperforms RCA.