A feature extraction method, named as centralized Gabor gradient histogram (CGGH), was proposed for facial gender recognition. By combining centralized binary pattern (CBP) and Gabor gradient magnitude, CGGH captures discriminative information at different scales and orientations. Moreover, the center-based nearest neighbor (CNN) classifier was selected to do the final classification, which was superior to traditional pattern classifier. The experimental results clearly show that the superiority of the proposed method over other compared methods and demonstrate that CNN classifier can enhance the performance of CGGH in facial gender recognition.