Conclusion
In this study, we propose modeling the context visually and semantically by combining a visual graph and a semantic graph and learning a vital context in the HOI problem using a group of graph update-modules, including graph inner update modules and graph cross update modules. We fuse the contextual features from the visual graph and semantic graph with the visual characteristics of the human-object pairs in a network to detect HOIs. We evaluate our proposed model on two challenging datasets, HICO-DET and V-COCO, and demonstrate excellent performance. Our work can provide a reference for modeling contextual information in the HOI problem.
References
Gao C, Zou Y L, Huang J-B. iCAN: instance-centric attention network for human-object interaction detection. 2018. ArXiv:1808.10437
Wang T, Anwer R M, Khan M H, et al. Deep contextual attention for human-object interaction detection. In: Proceedings of IEEE International Conference on Computer Vision, 2019
Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality. In: Proceedings of Advances in Neural Information Processing Systems, 2013. 3111–3119
Veličković P, Cucurull G, Casanova A, et al. Graph attention networks. 2017. ArXiv:1710.10903
Acknowledgements
This work was supported by National Key Research and Development Program of China (Grant No. 2018AAA0100802).
Author information
Authors and Affiliations
Corresponding author
Additional information
Supporting information
Appendixes A–C. The supporting information is available online at info.scichina.com and link.springer.com. The supporting materials are published as submitted, without typesetting or editing. The responsibility for scientific accuracy and content remains entirely with the authors.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Wu, T., Duan, F., Chang, L. et al. Human-object interaction detection via interactive visual-semantic graph learning. Sci. China Inf. Sci. 65, 160108 (2022). https://doi.org/10.1007/s11432-021-3427-2
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11432-021-3427-2