The facial pore feature is one of the crucial indicators for face recognition and skin evaluation. However, pores are tiny, which are difficult to detect and analysis based on the digital image. We proposed a new facial pore detection algorithm that combines the characteristics of skin pigment distribution and optimal scale, which can effectively eliminate the effect of complicated skin interferences from facial pore detection processing. First, considering the dissimilarity of skin pigment distribution on different pigment layers, we used SURF and SIFT algorithms to detect the skin features on different pigment layers and calculated the threshold by DBSCAN. Then, the Euclid distance was calculated to describe the positions similarity of the same detected points on different layers. Last, the optimal scales were set as thresholds to screen off the interferences of skin features except for pores. The experiment results confirm the improvement of the pore detection accuracy.