Facial expression recognition by artificial intelligence has become a hotspot and produced promising results in recent years. However, most recent work focuses on posed expressions whose involved muscles and dynamics are different from those of spontaneous ones. So this paper proposed a novel two-layer approach, combining dynamic head motion features with static appearance features, to recognize spontaneous expressions. Different from other methods, our method improved an existing eyes location algorithm and explored new head motion features calculated from locations of pupils to recognize fear firstly, and then used appearance features extracted by the AAM algorithm to recognize the remaining images. Comparative experiments on the natural visible and infrared facial expression database (NVIE) show the effectiveness of our approach.