Due to the low signal to noise ratio and limited spatial resolution, small target detection in an infrared image is a challenging task. Existing methods often have high false alarm rates and low probabilities of detection when infrared small targets submerge in the background clutter. In this paper, the Convolutional Neural Network (CNN) is adapted to extract the hidden features of small targets from infrared imagery with a proposed technique for a large amount of training data generation. The Point Spread Function (PSF) is employed to model the small target data and generate positive samples. The random background image patches are selected as the negative samples. In this way, the detection problem is skillfully converted into a problem of pattern classification using CNN. Extensive synthetic and real small targets were tested to evaluate the performance of this novel small target detection framework. The experimental results indicate that the proposed algorithm is simple and effective with satisfactory detection accuracy.