Skip to main content
SpringerLink
Log in

Development of head detection and tracking systems for visual surveillance

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

This paper proposes a technique for the detection of head nod and shake gestures based on eye tracking and head motion decision. The eye tracking step is divided into face detection and eye location. Here, we apply a motion segmentation algorithm that examines differences in moving people’s faces. This system utilizes a Hidden Markov Model-based head detection module that carries out complete detection in the input images, followed by the eye tracking module that refines the search based on a candidate list provided by the preprocessing module. The novelty of this paper is derived from differences in real-time input images, preprocessing to remove noises (morphological operators and so on), detecting edge lines and restoration, finding the face area, and cutting the head candidate. Moreover, we adopt a K-means algorithm for finding the head region. Real-time eye tracking extracts the location of eyes from the detected face region and is performed at close to a pair of eyes. After eye tracking, the coordinates of the detected eyes are transformed into a normalized vector of x-coordinate and y-coordinate. Head nod and shake detector uses three hidden Markov models (HMMs). HMM representation of the head detection can estimate the underlying HMM states from a sequence of face images. Head nod and shake can be detected by three HMMs that are adapted by a directional vector. The directional vector represents the direction of the head movement. The vector is HMMs for determining neutral as well as head nod and shake. These techniques are implemented on images, and notable success is notified.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Ekman P, Huang T, Sejnowski T, Hager J (1993) “Final Report to NSF of the planning Workshop on Facial Expression Understanding”, Technical report, National Science Foundation, Human Interaction Lab., UCSF, CA 94143, 1993

  2. Othman H, Aboulnasr T (2000) Low complexity 2-D Hidden Markov Model for face recognition. In: Proceedings of ISCAS 2000

  3. Lee KD, Nam MY, Chung KY, Lee YH, Kang UG (2013) Context and profile based cascade classifier for efficient people detection and safety care system. Multimed Tools Appl 63(1):27–44

    Article  Google Scholar 

  4. Kapoor A, Picard RW (2001) A real-time head nod and shake detector. In: Proceedings of the workshop on perceptive user interfaces, pp 1–5

  5. Tan W, Rong G (2003) A real-time head nod and shake detector using HMMs. Expert Syst Appl 25:461–466

    Article  Google Scholar 

  6. Kim H, Reitmayr G, Woo W (2013) IMAF: in situ indoor modeling and annotation framework on mobile phones. Pers Ubiquit Comput 17(3):571–582

    Article  Google Scholar 

  7. Davis JW, Vakes S (2001) “A Perceptual User Interface for Recognizing Head Gesture Acknowledgements”, ACM Workshop on Perceptual User Interfaces, pp 1–7

  8. Toyama K (1998) Look, Ma—No Hands! Hands free cursor control with real-time 3D face tracking. In: Proceedings of the workshop on perceptual user interfaces, pp 49–54

  9. Chen Q, Wu H, Fukumoto T, Yachida M (1998) 3D head pose estimation without feature tracking. In: Proceedings of the IEEE international conference on automatic face and gesture recognition

  10. Heinzman J, Zelinsky A (1998) 3-D facial pose and gaze point estimation using a robust real-time tracking paradigm. In: Proceedings of the IEEE international conference on automatic face and gesture recognition

  11. Kawato S, Ohya J (2000) Real-time detection of nodding and head-shaking by directly detecting and tracking the between-eyes. In: Proceedings of the IEEE international conference on automatic face and gesture recognition, pp 40–45

  12. Qing L, Shan S, Gao W, Du B (2005) Object recognition under generic illumination based on harmonic relighting. Int J Pattern Recognit Artif Intell 19(4):513–531

    Article  Google Scholar 

  13. Lumina RL, Shapiro G, Zuniga O (1983) A new connected components algorithm for virtual memory computers. Comput Vis Graph Image Process 22:287–300

    Article  Google Scholar 

  14. Abdel-Mottaleb M, Elgammal A (1999) Face detection in complex environments from color images. In: Proceedings of the IEEE international conference on image processing, vol 3, pp 622–626

  15. Runkle P, Bharadwaj P, Couchman L, Carin L (1999) Hidden Markov Models for multi-aspect target classification. IEEE Trans Signal Process 47(7):2035–2040

    Article  Google Scholar 

  16. Yang T, Pan Q, Li J, Cheng Y, Zhao C (2004) Real-time head tracking system with an active camera. In: Proceedings of the world congress on intelligent control and automation, Hangzhou, PR China

  17. Pitas I (1993) Digital image processing algorithms. Prentice Hall, Englewood Cliffs

    Google Scholar 

  18. Poniszewska-Maranda A (2010) Conception approach of access control in heterogeneous information systems using UML. Telecommun Syst 45:177–190

    Article  Google Scholar 

  19. Song CW, Chung KY, Jung JJ, Rim KW, Lee JH (2011) Localized approximation method using inertial compensation in WSNs. Info Int Interdiscip J 14(11):3591–3600

    Google Scholar 

  20. Portet F, Vacher M, Golanski C, Roux C, Meillon B (2013) Design and evaluation of a smart home voice interface for the elderly: acceptability and objection aspects. Pers Ubiquit Comput 17(1):127–144

    Article  Google Scholar 

  21. Lopez P, Orfila A, Palomar E, Castro H (2012) A secure distance-based RFID identification protocol with an off-line back-end database. Pers Ubiquit Comput 16(3):351–365

    Article  Google Scholar 

  22. Rabiner L (1989) A tutorial on Hidden Markov Models and selected applications in speech recognition. Proc IEEE 77(2):257–285

    Article  Google Scholar 

  23. Jung YG, Han MS, Chung KY, Lee SJ (2011) A study of a valid frequency range using correlation analysis of throat signal. Info Int Interdiscip J 14(11):3791–3799

    Google Scholar 

  24. Nefian AV, Hayes MH (1998) Face detection and recognition using Hidden Markov Models. In: Proceedings of the IEEE international conference on image processing, vol 1, pp 141–145

  25. Weiming H, Tieniu T, Fellow, IEEE, Wang L, Maybank S (2004) A survey on visual surveillance of object motion and behaviors. IEEE Trans Syst Man Cybern C 34(3):334–352

    Google Scholar 

  26. Hynes M, Wang H, McCarrick E, Kilmartin L (2011) Accurate monitoring of human physical activity levels for medical diagnosis and monitoring using off-the-shelf cellular handsets. Pers Ubiquit Comput 15(7):667–678

    Article  Google Scholar 

  27. Baum Le, Petrie T, Soules G, Weiss N (1970) A maximization techniques occurring in the statistical analysis of probabilistic functions of Markov chains. Ann Math Stat 41(1):164–171

    Article  MATH  MathSciNet  Google Scholar 

  28. Djuric PM, Chun JH (2002) An MCMC sampling approach to estimation of nonstationary Hidden Markov Models. IEEE Trans Signal Process 50(5):1113–1123

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Education, Science and Technology (No. 2012-0004478).

Author information

Authors and Affiliations

  1. HCI Lab., Department of Computer and Information Engineering, Inha University, Yonghyeon 1,4-dong, Nam-gu, Incheon, 402-751, Korea

    Sung-Kwan Kang

  2. School of Computer Information Engineering, Sangji University, Usan-dong, Wonju-si, 220-702, Gangwon-do, Korea

    Kyung-Yong Chung

  3. Department of Computer and Information Engineering, Inha University, Yonghyeon 1,4-dong, Nam-gu, Incheon, 402-751, Korea

    Jung-Hyun Lee

Authors
  1. Sung-Kwan Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kyung-Yong Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jung-Hyun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kyung-Yong Chung.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kang, SK., Chung, KY. & Lee, JH. Development of head detection and tracking systems for visual surveillance. Pers Ubiquit Comput 18, 515–522 (2014). https://doi.org/10.1007/s00779-013-0668-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-013-0668-9

Keywords

Search

Navigation