Skip to main content
SpringerLink
Log in

Human-object interaction detection via interactive visual-semantic graph learning

  • Letter
  • Special Focus on Deep Learning for Computer Vision
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Conclusion

In this study, we propose modeling the context visually and semantically by combining a visual graph and a semantic graph and learning a vital context in the HOI problem using a group of graph update-modules, including graph inner update modules and graph cross update modules. We fuse the contextual features from the visual graph and semantic graph with the visual characteristics of the human-object pairs in a network to detect HOIs. We evaluate our proposed model on two challenging datasets, HICO-DET and V-COCO, and demonstrate excellent performance. Our work can provide a reference for modeling contextual information in the HOI problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Gao C, Zou Y L, Huang J-B. iCAN: instance-centric attention network for human-object interaction detection. 2018. ArXiv:1808.10437

  2. Wang T, Anwer R M, Khan M H, et al. Deep contextual attention for human-object interaction detection. In: Proceedings of IEEE International Conference on Computer Vision, 2019

  3. Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality. In: Proceedings of Advances in Neural Information Processing Systems, 2013. 3111–3119

  4. Veličković P, Cucurull G, Casanova A, et al. Graph attention networks. 2017. ArXiv:1710.10903

Download references

Acknowledgements

This work was supported by National Key Research and Development Program of China (Grant No. 2018AAA0100802).

Author information

Authors and Affiliations

  1. College of Artificial Intelligence, Beijing Normal University, Beijing, 100875, China

    Tongtong Wu, Fuqing Duan & Liang Chang

  2. School of Computer and Control Engineering, University of Chinese Academy of Sciences, Beijing, 100049, China

    Ke Lu

Authors
  1. Tongtong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fuqing Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liang Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ke Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fuqing Duan.

Additional information

Supporting information

Appendixes A–C. The supporting information is available online at info.scichina.com and link.springer.com. The supporting materials are published as submitted, without typesetting or editing. The responsibility for scientific accuracy and content remains entirely with the authors.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, T., Duan, F., Chang, L. et al. Human-object interaction detection via interactive visual-semantic graph learning. Sci. China Inf. Sci. 65, 160108 (2022). https://doi.org/10.1007/s11432-021-3427-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-021-3427-2

Search

Navigation