Abstract
Human-object interaction detection (HOI) is a fundamental task in computer vision, which requires locating instances and predicting their interactions. To tackle HOI, we attempt to capture the global context information in HOI scenes by explicitly encoding the global features using our novel channel aware attention mechanism. Our observation is that the context of an image, including people, objects and background plays important roles in HOI prediction. To leverage such information, we propose a channel aware attention, which applies global average pooling on the features to learn their channel-wise inter-dependency. Based on the channel aware attention, we develop a channel aware module and a channel aware encoder. Handling features in channel dimensions makes it convenient to encode the global features as well as to learn semantic features. Empirically, our model outstrips the strong baseline by 3.2 points on V-COCO and 0.79 points on HICO-DET respectively. The visual analysis demonstrates that our method is able to capture abundant interaction-related features by attending to relevant regions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gupta, S., Malik, J.: Visual semantic role labeling. arXiv preprint arXiv:1505.04474 (2015)
Chao, Y.-W., Liu, Y., Liu, M., Zeng, H., Deng, J.: Learning to detect human-object interactions. In: WACV (2018)
Tamura, M., Ohashi, H., Yoshinaga, T.: QPIC: query-based pairwise human-object interaction detection with image-wide contextual information. In: CVPR, pp. 10405–10414 (2021)
Zou, C., et al.: End-to-end human object interaction detection with hoi transformer. In: CVPR, pp. 11825–11834 (2021)
Vaswani, A.,et al.: Attention is all you need. In: NeurIPS (2017)
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: ECCV, pp. 213–229 (2020)
Gkioxari, G., Girshick, R., Dollár, P., He, K.: Detecting and recognizing human-object interactions. In: CVPR, pp. 8359–8367 (2018)
Gao, C., Zou, Y., Huang, J.-B.: iCAN: instance-centric attention network for human-object interaction detection. arXiv preprint arXiv:1808.10437 (2018)
Wang, T., et al.:, Deep contextual attention for human-object interaction detection. In: ICCV, pp. 5694–5702 (2019)
Li, Y.-L., Liu, X., Wu, X., Li, Y., Lu, C.: Hoi analysis: integrating and decomposing human-object interaction. In: NeurIPS, pp. 5011–5022 (2020)
Hou, Z., Peng, X., Qiao, Y., Tao, D.: Visual compositional learning for human-object interaction detection. In: ECCV, pp. 584–600 (2020)
Gao, C., Xu, J., Zou, Y., Huang, J.-B.: DRG: dual relation graph for human-object interaction detection. In: ECCV, pp. 696–712 (2020)
Zhang, F.Z., Campbell, D., Gould, S.: Spatially conditioned graphs for detecting human-object interactions. In: ICCV, pp. 13 319–13 327 (2021)
Liao, Y., Liu, S., Wang, F., Chen, Y., Qian, C., Feng, J.: PPDM: parallel point detection and matching for real-time human-object interaction detection. In: CVPR, pp. 482–490 (2020)
Wang, T., Yang, T., Danelljan, M., Khan, F.S., Zhang, X., Sun, J.: Learning human-object interaction detection using interaction points. In: CVPR, pp. 4116–4125 (2020)
Kim, B., Choi, T., Kang, J., Kim, H.J.: UnionDet: union-level detector towards real-time human-object interaction detection. In: ECCV, pp. 498–514 (2020)
Chen, M., Liao, Y., Liu, S., Chen, Z., Wang, F., Qian, C.: Reformulating HOI detection as adaptive set prediction. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: CVPR, pp. 7132–7141 (2018)
Kim, B., Lee, J., Kang, J., Kim, E.-S., Kim, H.J.: HOTR: end-to-end human-object interaction detection with transformers. In: CVPR, pp. 74–83 (2021)
Chan, S., Tao, J., Zhou, X., Bai, C., Zhang, X.: Siamese implicit region proposal network with compound attention for visual tracking. IEEE Trans. Image Process. 31, 1882–1894 (2022)
Bai, C., Li, H., Zhang, J., Huang, L., Zhang, L.: Unsupervised adversarial instance-level image retrieval. IEEE Trans. Multimedia 23, 2199–2207 (2021)
Li, Y.-L., et al.: Transferable interactiveness knowledge for human-object interaction detection. In: CVPR (2019)
Hou, Z., Yu, B., Qiao, Y., Peng, X., Tao, D.: Detecting human-object interaction via fabricated compositional learning. In: CVPR, pp. 14646–14655 (2021)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
Lin, T.-Y. et al.: Microsoft COCO: common objects in context. In: ECCV, pp. 740–755 (2014)
Acknowledgments
This work was supported by Zhejiang Provincial Natural Science Foundation of China (No. LY21F020024), National Natural Science Foundation of China (No. 62272395), and Qin Chuangyuan Innovation and Entrepreneurship Talent Project (No. QCYRCXM-2022-359).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, Z., Meng, J., Yue, Y., Zhang, Z. (2024). Human-Object Interaction Detection with Channel Aware Attention. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1961. Springer, Singapore. https://doi.org/10.1007/978-981-99-8126-7_25
Download citation
DOI: https://doi.org/10.1007/978-981-99-8126-7_25
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8125-0
Online ISBN: 978-981-99-8126-7
eBook Packages: Computer ScienceComputer Science (R0)