Local binary pattern (LBP) has limitation in extracting the edge and direction information, which is vital to infrared face recognition. A new infrared face recognition algorithm fusion of LBP and histogram of oriented gradients (HOG) is proposed. First, LBP operator is adopted to extract the texture feature of an infrared face, and then the edge features of the original infrared face are extracted by using HOG operator. Finally, multiple kernel learning (MKL) is applied to fuse the texture features and edge features. Experiments are conducted on infrared face database of variable ambient temperature. The results show that the fusion of LBP and HOG perform better than traditional LBP or HOG features for infrared face recognition, the proposed method is more robust to ambient temperatures.