Skip to main content
SpringerLink
Log in

Material-aware Cross-channel Interaction Attention (MCIA) for occluded prohibited item detection

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

A Correction to this article was published on 30 May 2022

This article has been updated

Abstract

For security inspection, detecting prohibited items in X-ray images is challenging since they are usually occluded by non-prohibited items. In X-ray images, different materials present different colors and textures. On this basis, we exploit the material characteristics to detect occluded prohibited items. Moreover, the occlusion mainly exists between prohibited items and non-prohibited ones, belonging to inter-class occlusion. We propose a Material-aware Cross-channel Interaction Attention (MCIA) module which can use the material information of X-ray images to deal with the inter-class occlusion. Specifically, MCIA is composed of Material Perception (MP) and Cross-channel Interaction (CI). MP captures distinctive material information of X-ray images and CI gets the local cross-channel interaction to convert material information into channel-wise weights. By combining MP and CI, MCIA effectively helps the network to highlight the core features of prohibited items while suppressing non-prohibited items. Meanwhile, we design the MCIA-Net and MCIA-FPN by placing our MCIA module behind each stage in ResNet. Our MCIA-Net and MCIA-FPN can be used as backbones to detect occluded prohibited items. Note that MCIA-FPN also takes into account the prohibited items of various sizes. Our MCIA-Net and MCIA-FPN have been comprehensively validated on the SIXray dataset and OPIXray dataset. The experimental results prove the superiority of our method. Furthermore, our proposed MCIA module outperforms several widely used attention mechanisms and effectively improves the performance of Faster R-CNN and Cascade R-CNN in detecting occluded prohibited items.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Change history

References

  1. Akcay, S., Breckon, T.: Towards automatic threat detection: A survey of advances of deep learning within x-ray security imaging. arXiv preprint arXiv:2001.01293 (2020)

  2. Akcay, S., Breckon, T.P.: An evaluation of region based object detection strategies within x-ray baggage security imagery. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 1337–1341. IEEE (2017)

  3. Akçay, S., Kundegorski, M.E., Devereux, M., Breckon, T.P.: Transfer learning using convolutional neural networks for object classification within x-ray baggage security imagery. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 1057–1061. IEEE (2016)

  4. Baştan, M., Yousefi, M.R., Breuel, T.M.: Visual words on baggage x-ray images. In: International Conference on Computer Analysis of Images and Patterns, pp. 360–368. Springer (2011)

  5. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)

  6. Bodla, N., Singh, B., Chellappa, R., Davis, L.S.: Soft-nms–improving object detection with one line of code. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5561–5569 (2017)

  7. Cai, Z., Vasconcelos, N.: Cascade r-cnn: Delving into high quality object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6154–6162 (2018)

  8. Cao, Y., Xu, J., Lin, S., Wei, F., Hu, H.: Gcnet: Non-local networks meet squeeze-excitation networks and beyond. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 0 (2019)

  9. Cui, Y., Oztan, B.: Automated firearms detection in cargo x-ray images using retinanet. In: Anomaly Detection and Imaging with X-Rays (ADIX) IV, vol. 10999, p. 109990P. International Society for Optics and Photonics (2019)

  10. Dai, J., Li, Y., He, K., Sun, J.: R-fcn: Object detection via region-based fully convolutional networks. arXiv preprint arXiv:1605.06409 (2016)

  11. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)

  12. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

  13. Hassan, T., Bettayeb, M., Akçay, S., Khan, S., Bennamoun, M., Werghi, N.: Detecting prohibited items in x-ray images: A contour proposal learning approach. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 2016–2020. IEEE (2020)

  14. Hassan, T., Khan, S.H., Akcay, S., Bennamoun, M., Werghi, N.: Deep cmst framework for the autonomous recognition of heavily occluded and cluttered baggage items from multivendor security radiographs. arXiv preprint arXiv:1912.04251 (2019)

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer vision and Pattern Recognition, pp. 770–778 (2016)

  16. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  17. Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510 (2017)

  18. Jaccard, N., Rogers, T.W., Griffin, L.D.: Automated detection of cars in transmission x-ray images of freight containers. In: 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 387–392. IEEE (2014)

  19. Kundegorski, M.E., Akçay, S., Devereux, M., Mouton, A., Breckon, T.P.: On using feature descriptors as visual words for object detection within x-ray baggage security screening (2016)

  20. Lee, H., Kim, H.E., Nam, H.: Srm: A style-based recalibration module for convolutional neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1854–1862 (2019)

  21. Li, Z., Zhou, F.: Fssd: feature fusion single shot multibox detector. arXiv preprint arXiv:1712.00960 (2017)

  22. Liang, K.J., Heilmann, G., Gregory, C., Diallo, S.O., Carlson, D., Spell, G.P., Sigman, J.B., Roe, K., Carin, L.: Automatic threat recognition of prohibited items at aviation checkpoint with x-ray imaging: a deep learning approach. In: Anomaly Detection and Imaging with X-Rays (ADIX) III, vol. 10632, p. 1063203. International Society for Optics and Photonics (2018)

  23. Liang, K.J., Sigman, J.B., Spell, G.P., Strellis, D., Chang, W., Liu, F., Mehta, T., Carin, L.: Toward automatic threat recognition for airport x-ray baggage screening with deep convolutional object detection. arXiv preprint arXiv:1912.06329 (2019)

  24. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

  25. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

  26. Liu, J., Leng, X., Liu, Y.: Deep convolutional neural network based object detector for x-ray baggage security imagery. In: 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), pp. 1757–1761. IEEE (2019)

  27. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21–37. Springer (2016)

  28. Liu, Z., Duan, Q., Shi, S., Zhao, P.: Multi-level progressive parallel attention guided salient object detection for rgb-d images. The Visual Computer pp. 1–12 (2020)

  29. Liu, Z., Li, J., Shu, Y., Zhang, D.: Detection and recognition of security detection object based on yolo9000. In: 2018 5th International Conference on Systems and Informatics (ICSAI), pp. 278–282. IEEE (2018)

  30. Mery, D., Svec, E., Arias, M.: Object recognition in baggage inspection using adaptive sparse representations of x-ray images. In: Image and Video Technology, pp. 709–720. Springer (2015)

  31. Miao, C., Xie, L., Wan, F., Su, C., Liu, H., Jiao, J., Ye, Q.: Sixray: A large-scale security inspection x-ray benchmark for prohibited item discovery in overlapping images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2119–2128 (2019)

  32. Nam, H., Kim, H.E.: Batch-instance normalization for adaptively style-invariant neural networks. Adv. Neural. Inf. Process. Syst. 31, 2558–2567 (2018)

    Google Scholar 

  33. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  34. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

  35. Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

  36. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2016)

    Article  Google Scholar 

  37. Riffo, V., Mery, D.: Automated detection of threat objects using adapted implicit shape model. IEEE Trans. Syst. Man, Cyber: Syst. 46(4), 472–482 (2015)

    Article  Google Scholar 

  38. Shajini, M., Ramanan, A.: An improved landmark-driven and spatial–channel attentive convolutional neural network for fashion clothes classification. The Visual Computer pp. 1–10 (2020)

  39. Shi, W., Du, H., Mei, W., Ma, Z.: (sarn) spatial-wise attention residual network for image super-resolution. The Visual Computer pp. 1–12 (2020)

  40. Steitz, J.M.O., Saeedan, F., Roth, S.: Multi-view x-ray r-cnn. In: German Conference on Pattern Recognition, pp. 153–168. Springer (2018)

  41. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., Hu, Q.: Eca-net: Efficient channel attention for deep convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11534–11542 (2020)

  42. Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794–7803 (2018)

  43. Wei, Y., Tao, R., Wu, Z., Ma, Y., Zhang, L., Liu, X.: Occluded prohibited items detection: An x-ray security inspection benchmark and de-occlusion attention module. arXiv preprint arXiv:2004.08656 (2020)

  44. Woo, S., Park, J., Lee, J.Y., So Kweon, I.: Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)

  45. Zhang, T., Gao, F., Dong, J., Du, Q.: Remote sensing image translation via style-based recalibration module and improved style discriminator. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2021)

    Google Scholar 

  46. Zhang, Yt., Zhang, Hg., Zhao, If., Yang, Jf.: Automatic detection of prohibited items with small size in x-ray images. Optoelectron. Lett. 16(4), 313–317 (2020)

    Article  Google Scholar 

Download references

Funding

No funding was received to assist with the preparation of this manuscript.

Author information

Authors and Affiliations

  1. School of Information and Electronics, Beijing Institute of Technology, Beijing, 100081, China

    Man Wang, Wenbo Mei & Shaui Wang

  2. School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing, 100081, China

    Huiqian Du

  3. The Inner Mongolia Autonomous Region Public Security Bureau, Hohhot, 010051, China

    Dasen Yuan

Authors
  1. Man Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huiqian Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenbo Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shaui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dasen Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huiqian Du.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: there were errors in the affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, M., Du, H., Mei, W. et al. Material-aware Cross-channel Interaction Attention (MCIA) for occluded prohibited item detection. Vis Comput 39, 2865–2877 (2023). https://doi.org/10.1007/s00371-022-02498-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02498-y

Keywords

Search

Navigation