Our Pt-Net is a novel object detection network based on a pre-trained and multi-feature VGG-16 network. Firstly, Pt-Net is initialized by a pre-trained VGG-16 model and its own CNN output via a linear combination. Secondly, Pt-Net generates proposals via particle filter method on Conv5 feature map and crops the multi-feature maps which are combined by fusing hierarchical CNN features in corresponding positions. After that, we apply multi-feature concatenation for the cropped parts for more image feature information and adopt a novel two-dimensional overlap area loss function for localization. Finally, we apply our Pt-Net on both object detection task and face detection task which are trained on the PASCAL VOC dataset and WIDER FACE dataset. Pt-Net can achieve a mAP of 76.8% on the detection of PASCAL VOC 2007 dataset and state-of-the-art results on the FDDB benchmark at 43 fps on an NVIDIA GTX 1070p GPU.