Using deep-learing based object detection technology to detect contraband is to locate and classify it in X-RAY luggage images. It can circumvent the problem of missed and false detection caused by traditional manual security inspection. This paper proposes a novel anchor-free detector named Scale Adaptive CenterNet (SA-CenterNet), which models the contraband as its center point at multiple scales. Our method uses keypoint estimation to find center points and regresses to object’s sizes to obtain bounding boxes. SA-CenterNet introduces a scale adaptive module which uses multi-level feature maps to predict occluded contraband with the different sizes and the same center point. In additIon, we propose a feature enhancement module (FEM) to better extract the feature of small and obscure contraband. We use lightweight backbone and group convolution in detection head to reduce the amount of calculation and speed up the inference speed. Experiments demonstrate that our method achieves 95.2%AP50 at 105 FPS on our dataset named UNICOMP, which surpasses the state-of-the-art methods. Our method can robustly detect contraband passing the security inspection machine in real time, saving manpower and material resources.