Safety monitoring system based on passive millimeter wave (PMMW) is getting much more popular among security check field due to its advantages as no radiation, no contact, which has been wildly applied in public area as airports, railway stations, etc, to detect dangerous substances hidden under clothing. In this paper, an algorithm of Classifying Target After Segmenting (CTAS) is proposed to solve real-time detecting problem. In segmentation section, the accuracy of target segmentation is improved by applying the modified traditional maximum entropy segmentation algorithm. In classification section, a new neural network which's structure is similar to LeNet-5 is built by using Inception Module. Depthwise Separable convolution is applied to enhance the model's calculation performance. The test result shows that the overall classification accuracy of the test set image is 98.9%, and the calculation speed fits the real-time requirement properly.