Skip to main content
SpringerLink
Log in

Discovering HOI Semantics from Massive Image Data

  • Conference paper
  • First Online:
Database and Expert Systems Applications (DEXA 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12924))

Included in the following conference series:

  • 780 Accesses

Abstract

Human-Object Interaction (HOI) plays an important role in human-centric scene understanding. However, the commonly used two-stage methods have large computational costs and a slow inferring speed. The existing one-stage methods detect HOIs by detecting the central points or the union boxes of human and objects, which need to process a large scale of regions and many unnecessary features. In this paper, we propose a novel one-stage method for discovering HOI semantics from massive image data. In particular, we present two new designs in our method, namely action classification and displacement prediction. Further, we design a special HOI score calculation strategy, which can decay the HOI score of the results that have bad matching result. We evaluate our method on the popular HICO-DET benchmark and compare our proposal with a number of existing approaches. The results show that our method outperforms existing methods in discovering HOI semantics. abstract environment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bansal, A., Rambhatla, S.S., Shrivastava, A., Chellappa, R.: Detecting human-object interactions via functional generalization. In: Proceedings of AAAI, vol. 34, pp. 10460–10469 (2020)

    Google Scholar 

  2. Chao, Y.W., Wang, Z., He, Y., Wang, J., Deng, J.: Hico: a benchmark for recognizing human-object interactions in images. In: Proceedings of ICCV, pp. 1017–1025 (2015)

    Google Scholar 

  3. Fang, H.S., Xie, Y., Shao, D., Lu, C.: DIRV: dense interaction region voting for end-to-end human-object interaction detection. arXiv preprint arXiv:2010.01005 (2020)

  4. Gao, C., Xu, J., Zou, Y., Huang, J.-B.: DRG: dual relation graph for human-object interaction detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 696–712. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58610-2_41

    Chapter  Google Scholar 

  5. Gao, C., Zou, Y., Huang, J.B.: ican: instance-centric attention network for human-object interaction detection. arXiv preprint arXiv:1808.10437 (2018)

  6. Kim, B., Choi, T., Kang, J., Kim, H.J.: UnionDet: union-level detector towards real-time human-object interaction detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12360, pp. 498–514. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58555-6_30

    Chapter  Google Scholar 

  7. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  8. Liao, Y., Liu, S., Wang, F., Chen, Y., Qian, C., Feng, J.: Ppdm: parallel point detection and matching for real-time human-object interaction detection. In: Proceedings of CVPR, pp. 482–490 (2020)

    Google Scholar 

  9. Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of ICCV, pp. 2980–2988 (2017)

    Google Scholar 

  10. Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29

    Chapter  Google Scholar 

  11. Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of NIPS (2015)

    Google Scholar 

  12. Tian, Q., Wan, S., Jin, P., Xu, J., Zou, C., Li, X.: A novel feature fusion with self-adaptive weight method based on deep learning for image classification. In: Hong, R., Cheng, W.-H., Yamasaki, T., Wang, M., Ngo, C.-W. (eds.) PCM 2018. LNCS, vol. 11164, pp. 426–436. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00776-8_39

    Chapter  Google Scholar 

  13. Wan, B., Zhou, D., Liu, Y., Li, R., He, X.: Pose-aware multi-level feature network for human object interaction detection. In: Proceedings of ICCV, pp. 9469–9478 (2019)

    Google Scholar 

  14. Wan, S., Jin, P., Yue, L.: An approach for image retrieval based on visual saliency. In: Proceedings of IASP, pp. 172–175 (2009)

    Google Scholar 

  15. Wang, H., Zheng, W., Yingbiao, L.: Contextual heterogeneous graph network for human-object interaction detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12362, pp. 248–264. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58520-4_15

    Chapter  Google Scholar 

  16. Wang, T., Yang, T., Danelljan, M., Khan, F.S., Zhang, X., Sun, J.: Learning human-object interaction detection using interaction points. In: Proceedings of CVPR, pp. 4116–4125 (2020)

    Google Scholar 

  17. Yang, X., Wan, S., Jin, P.: Domain-invariant region proposal network for cross-domain detection. In: Proceedings of ICME, pp. 1–6 (2020)

    Google Scholar 

  18. Yang, X., Wan, S., Jin, P., Zou, C., Li, X.: MHEF-TripNet: mixed triplet loss with hard example feedback network for image retrieval. In: Zhao, Y., Barnes, N., Chen, B., Westermann, R., Kong, X., Lin, C. (eds.) ICIG 2019. LNCS, vol. 11903, pp. 35–46. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-34113-8_4

    Chapter  Google Scholar 

  19. Yu, F., Wang, D., Shelhamer, E., Darrell, T.: Deep layer aggregation. In: Proceedings of CVPR, pp. 2403–2412 (2018)

    Google Scholar 

  20. Zhong, X., Ding, C., Qu, X., Tao, D.: Polysemy deciphering network for human-object interaction detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12365, pp. 69–85. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58565-5_5

    Chapter  Google Scholar 

  21. Zhou, P., Chi, M.: Relation parsing neural network for human-object interaction detection. In: Proceedings of ICCV, pp. 843–851 (2019)

    Google Scholar 

  22. Zhou, X., Wang, D., Krähenbühl, P.: Objects as points. arXiv preprint arXiv:1904.07850 (2019)

Download references

Acknowledgments

This paper is supported by the National Science Foundation of China (grant no. 62072419).

Author information

Authors and Affiliations

  1. University of Science and Technology of China, Hefei, China

    Mingguang Zheng, Shouhong Wan & Peiquan Jin

  2. Key Laboratory of Electromagnetic Space Information, CAS, Hefei, China

    Mingguang Zheng, Shouhong Wan & Peiquan Jin

Authors
  1. Mingguang Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shouhong Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Peiquan Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peiquan Jin .

Editor information

Editors and Affiliations

  1. University of Vienna, Vienna, Austria

    Christine Strauss

  2. Johannes Kepler University Linz, Linz, Oberösterreich, Austria

    Gabriele Kotsis

  3. Vienna University of Technology, Vienna, Austria

    A Min Tjoa

  4. Johannes Kepler University of Linz, Linz, Austria

    Ismail Khalil

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zheng, M., Wan, S., Jin, P. (2021). Discovering HOI Semantics from Massive Image Data. In: Strauss, C., Kotsis, G., Tjoa, A.M., Khalil, I. (eds) Database and Expert Systems Applications. DEXA 2021. Lecture Notes in Computer Science(), vol 12924. Springer, Cham. https://doi.org/10.1007/978-3-030-86475-0_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86475-0_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86474-3

  • Online ISBN: 978-3-030-86475-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation