3D object detection in the real crowded scene is still a challenging task due to occlusion and density change. We propose a part-aware 3D single-stage detector with local and non-local attention (PLNL-3DSSD) to fully use part information and inter-object relation. A primary part feature fusion is proposed for encoding the entire box feature vector by introducing semantic parts dividing. We develop a parallel part branch for robust and accurate object detection. We also develop 10-cal and non-local attention in set abstraction for enhancing data flow transfer between objects. Our method ranks second in single-stage 3D object detector on the KITTI 3D car detection benchmark while ensuring satisfactory efficiency.