This paper introduces the minimal local reconstruction error (MLRE) as a similarity measure and presents a MLRE-based classier. From the geometric meaning of the minimal local reconstruction error, we derive that the MLRE-based classifier is a generalization of the conventional nearest neighbor classier and the nearest neighbor line and plane classifiers. We further apply the MLRE measure to characterize the within-class and between-class local scatters and then develop a MLRE measure based discriminant feature extraction method. The proposed MLRE-based feature extraction method is in line with the MLRE-based classification method in spirit, thus the two methods can be seamlessly combined in applications. The experimental results on the CENPARMI handwritten numeral database and the FERET face image database show effectiveness of the proposed MLRE-based feature extraction and classification method.