In this paper, an integrated algorithm to detect humans in thermal imagery was introduced. In recent years, histogram of oriented gradient (HOG) is a quite popular algorithm for person detection in visible imagery. We implement the pedestrian detection in infrared imagery with this algorithm by adjusting the parameters. Simultaneously, we have increased some other geometric characteristics, such as mean contrast, which is used as features for the detection. After analyzing the property of the infrared imagery, which is designed to meet the shortfall of the HOG in infrared imagery, the combined vectors are fed to a linear SVM for object/non-object classification and we get the detector at the same time. After that, the detection window is scanned across the image at multiple positions and scales, which is followed by the combination of the overlapping detections. At last, a pedestrian is described by a final detection, and we have detected the pedestrians in the thermal imagery. Experimental results with OSU Thermal Pedestrian Database are reported to demonstrate the excellent performance of our algorithms.