Skip to main content
SpringerLink
Log in

Enhanced Gaze Following via Object Detection and Human Pose Estimation

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11962))

Included in the following conference series:

Abstract

The aim of gaze following is to estimate the gaze direction, which is useful for the understanding of human behaviour in various applications. However, it is still an open problem that has not been fully studied. In this paper, we present a novel framework for gaze following problem, where both the front/side face case and the back face case are taken into account. For the front/side face case, head pose estimation is applied to estimate the gaze, and then object detection is used to further refine the gaze direction by selecting the object that intersects with the gaze in a certain range. For the back face case, a deep neural network with the human pose information is proposed for gaze estimation. Experiments are carried out to demonstrate the superiority of the proposed method, as compared with the state-of-the-art method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Fathi, A., Hodgins, J.K., Rehg, J.M.: Social interactions: a first-person perspective. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1226–1233 (2012)

    Google Scholar 

  2. Gkioxari, G., Girshick, R.B., Dollár, P., He, K.: Detecting and recognizing human-object interactions. CoRR abs/1704.07333 (2017). http://arxiv.org/abs/1704.07333

  3. Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 478–500 (2010)

    Article  Google Scholar 

  4. Krafka, K., et al.: Eye tracking for everyone. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2176–2184 (2016)

    Google Scholar 

  5. Marin-Jimenez, M.J., Zisserman, A., Eichner, M., Ferrari, V.: Detecting people looking at each other in videos. Int. J. Comput. Vis. 106(3), 282–296 (2014)

    Article  Google Scholar 

  6. Park, H.S., Jain, E., Sheikh, Y.: Predicting primary gaze behavior using social saliency fields. In: Proceedings of IEEE International Conference on Computer Vision, pp. 3503–3510 (2013)

    Google Scholar 

  7. Park, H.S., Shi, J.: Social saliency prediction. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4777–4785 (2015)

    Google Scholar 

  8. Parks, D., Borji, A., Itti, L.: Augmented saliency model using automatic 3D head pose detection and learned gaze following in natural scenes. Vis. Res. 116, 113–126 (2015)

    Article  Google Scholar 

  9. Recasens, A., Vondrick, C., Khosla, A., Torralba, A.: Following gaze in video. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 1444–1452 (2017)

    Google Scholar 

  10. Recasens, A., Khosla, A., Vondrick, C., Torralba, A.: Where are they looking? In: Proceedings of Advances in Neural Information Processing Systems, pp. 199–207 (2015)

    Google Scholar 

  11. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  12. Ruiz, N., Chong, E., Rehg, J.M.: Fine-grained head pose estimation without keypoints. CoRR abs/1710.00925 (2017). http://arxiv.org/abs/1710.00925

  13. Santini, T., Fuhl, W., Kasneci, E.: CalibMe: fast and unsupervised eye tracker calibration for gaze-based pervasive human-computer interaction. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 2594–2605. ACM (2017)

    Google Scholar 

  14. Shepherd, S.V.: Following gaze: gaze-following behavior as a window into social cognition. Front. Integr. Neurosci. 4, 1–13 (2010)

    Google Scholar 

  15. Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., Li, S.Z.: \(\text{S}^3\)FD: single shot scale-invariant face detector. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 192–201 (2017)

    Google Scholar 

  16. Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4511–4520 (2015)

    Google Scholar 

Download references

Acknowledgement

This work is partly supported by the Fundamental Research Funds for the Central Universities (No. 3072019CFJ0602).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Harbin Engineering University, Harbin, 150001, China

    Jian Guan & Jianguo Sun

  2. Department of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, 518055, China

    Liming Yin, Shuhan Qi, Xuan Wang & Qing Liao

  3. Peng Cheng Laboratory, Shenzhen, 518000, China

    Qing Liao

Authors
  1. Jian Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liming Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianguo Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuhan Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qing Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuan Wang .

Editor information

Editors and Affiliations

  1. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Yong Man Ro

  2. National Chiao Tung University, Hsinchu, Taiwan

    Wen-Huang Cheng

  3. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Junmo Kim

  4. National Cheng Kung University, Tainan City, Taiwan

    Wei-Ta Chu

  5. Tsinghua University, Beijing, China

    Peng Cui

  6. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Jung-Woo Choi

  7. National Tsing Hua University, Hsinchu, Taiwan

    Min-Chun Hu

  8. Ghent University, Ghent, Belgium

    Wesley De Neve

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Guan, J., Yin, L., Sun, J., Qi, S., Wang, X., Liao, Q. (2020). Enhanced Gaze Following via Object Detection and Human Pose Estimation. In: Ro, Y., et al. MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science(), vol 11962. Springer, Cham. https://doi.org/10.1007/978-3-030-37734-2_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37734-2_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37733-5

  • Online ISBN: 978-3-030-37734-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation