One of problems with incremental learning approaches is that the quality of learning samples cannot be controlled since they are provided on-line. Current incremental learning algorithms ignore the issue by accepting each sample unconditionally and assimilating it into the existing recognition system. However, improper sample, caused by a partial occlusion, badly illumination condition and unusual pose, deteriorates the overall performance of the recognition system significantly. In order to overcome this issue, in this paper, based on hypothesis and verification paradigm, we propose a criterion for sample selection, and present a novel system for automatically evaluating the qualities of new added samples during the incremental learning procedure. Following this, a subspace is learned by using only the samples that are considered to be good after verification. The high computation cost, incurred by inclusion of sample verification into incremental learning framework, is greatly reduced using relative distance filtering. The experimental results demonstrated the superiority of our approach, in both recognition accuracy and robustness.