Skip to main content

Advertisement

SpringerLink
Log in

Online learning design of an image-based facial expression recognition system

  • Original Research Paper
  • Published:
Intelligent Service Robotics Aims and scope Submit manuscript

Abstract

In order to serve people and support them in daily life, a domestic or service robot needs to accommodate itself to various individuals. Emotional and intelligent human–robot interaction plays an important role for a robot to gain attention of its users. Facial expression recognition is a key factor in interactive robotic applications. In this paper, an image-based facial expression recognition system that adapts online to a new face is proposed. The main idea of the proposed learning algorithm is to adjust parameters of the support vector machine (SVM) hyperplane for learning facial expressions of a new face. After mapping the input space to Gaussian-kernel space, support vector pursuit learning (SVPL) is employed to retrain the hyperplane in the new feature space. To expedite the retraining step, we propose to retrain a new SVM classifier by using only samples classified incorrectly in previous iteration in combination with critical historical sets. After adjusting the hyperplane parameters, the new classifier will recognize more effectively previous unrecognizable facial datasets. Experiments of using an embedded imaging system show that the proposed system recognizes new facial datasets with a recognition rate of 92.7%. Furthermore, it also maintains a satisfactory recognition rate of 82.6% of old facial samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. An JL, Wang ZO, Ma ZP (2003) An incremental learning algorithm for support vector machine. In: Proceedings of international conference on machine learning and cybernetics, pp 1153–1156

  2. Andrian H, Song KT (2005) Embedded CMOS imaging system for real-time robotic vision. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems, pp 3694–3699

  3. Bourel F, Chibelushi CC, Low AA (2002) Robust facial expression recognition using a state-based model of spatially-localised facial dynamics. In: Proceedings of IEEE fifth international conference on automatic face and gesture recognition, pp 106–111

  4. Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. Adv Neural Inf Process Syst 13: 409–415

    Google Scholar 

  5. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University, New York

    Google Scholar 

  6. Fujita M (2004) On activating human communications with pet-type robot AIBO. In: Proceedings of IEEE vol 92(11), pp 1804–1813

  7. Guo G, Dyer CR (2005) Learning from examples in the small sample case: face expression recognition. IEEE Trans Syst Man Cybern B Cybern 35(3): 477–488

    Article  Google Scholar 

  8. Hao PY (2008) A fuzzy model of support vector classification algorithm. In: Proceedings of IEEE international conference on fuzzy systems, pp 299–304

  9. Han MJ, Hsu JH, Song KT, Chang FY (2008) A new information fusion method for bimodal robotic emotion recognition. J Comput 3(7): 39–47

    Google Scholar 

  10. Hong JW, Han MJ, Song KT, Chang FY (2007) A fast learning algorithm for robotic emotion recognition. In: Proceedings of IEEE international symposium on computational intelligence in robotics and automation, pp 25–30

  11. Intelligent system control integration laboratory. http://isci.cn.nctu.edu.tw/VideoClip/ISR/

  12. Kotsia I, Pitas I (2007) Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans Image Process 16(1): 172–187

    Article  MathSciNet  Google Scholar 

  13. Lai JH, Yuen PC, Chen WS, Lao S, Kawade M (2001) Robust facial feature point detection under nonlinear illuminations. In: Proceedings of IEEE workshop on recognition, analysis and tracking of faces and gestures in real-time systems, pp 168–174

  14. Lee D, Lee J (2007) Equilibrium-based support vector machine for semisupervised classification. IEEE Trans Neural Netw 18(2): 578–583

    Article  Google Scholar 

  15. Liu Y, He Q, Chen Q (2004) Incremental batch learning with support vector machines. In: Proceedings of fifth world congress on intelligent control and automation, pp 1857–1861

  16. Liu Y, He Q, Tang Y (2004) Support vector pursuit learning. In: Proceedings of IEEE international conference on system, man and cybernetics, pp 5841–5846

  17. Lund HH (2004) Modern artificial intelligence for human–robot interaction. Proc IEEE 92(11): 1821–1838

    Article  Google Scholar 

  18. Lyons M, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with Gabor wavelets. In: Proceedings of the third IEEE conference on face and gesture recognition, pp 200–205

  19. Mitra P, Murthy CA, Pal SK (2000) Data condensation in large databases by incremental learning with support vector machines. In: Proceedings of IEEE international conference on pattern recognition, pp 708–711

  20. Pantic M, Rothkrantz LJM (2000) Automatic analysis of facial expressions: the state of the art. IEEE Trans Pattern Anal Mach Intell 22(12): 1424–1445

    Article  Google Scholar 

  21. Pololu Corporation: Pololu serial 8-servo controller. http://www.pololu.com/products/pololu/0727/

  22. Roh SG, Yang KW, Park JH, Moon H, Kim HS, Lee H, Choi HR (2009) A modularized personal robot DRPI: design and implementation. IEEE Trans Robotics 25(2): 414–425

    Article  Google Scholar 

  23. Saatci Y, Town C (2006) Cascaded classification of gender and facial expression using active appearance models. In: Proceedings of IEEE seventh international conference on automatic face and gesture recognition, pp 393–398

  24. Takacs G, Pataki B (2007) Lower bounds on the Vapnik–Chervonenkis dimension of convex polytope classifiers. In: Proceedings of IEEE international conference on intelligent engineering systems, pp 145–148

  25. Valstar M, Pantic M (2006) Fully automatic facial action unit detection and temporal analysis. In: Proceedings of IEEE international conference on computer vision and pattern recognition workshop, pp 149–156

  26. Vapnik V (1995) The nature of statistic learning theory. Springer, New York

    Google Scholar 

  27. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of IEEE sixth international conference on computer vision and pattern recognition, pp 511–518

  28. Wu C, Wang X, Bai D, Zhang H (2008) Fast SVM incremental learning based on the convex hulls algorithm. In: Proceedings of IEEE international conference on computational intelligence and security, pp 249–252

  29. Wu YW, Liu W, Wang JB (2008) Application of emotional recognition in intelligent tutoring system. In: Proceedings of IEEE international conference on knowledge discovery and data mining workshop, pp 449–452

  30. Xiao R, Wang J, Zhang F (2000) An approach to incremental SVM learning algorithm. In: Proceedings of IEEE twelfth international conference on tools with artificial intelligence, pp 268–273

  31. Xuchun L, Yan Z, Sung E (2005) Sequential bootstrapped support vector machines—an SVM accelerator. In: Proceedings of international conference on neural networks, pp 1437–1442

  32. Yang Y, Ge SS, Lee TH, Wang C (2008) Facial expression recognition and tracking for intelligent human–robot interaction. Intelligent Service Robotics 1(2): 143–157

    Article  Google Scholar 

  33. Yeasin M, Bullot B, Harma R (2006) Recognition of facial expressions and measurement of levels of interest from video. IEEE Trans Multimedia 8(3): 500–508

    Article  Google Scholar 

  34. Zeng WH, Ma J (2004) A novel incremental SVM learning algorithm. In: Proceedings of international conference on CSCW in design, pp 658–662

  35. Zhou G, Zhan Y, Zhang J (2006) Facial expression recognition based on selective feature extraction. In: Proceedings of IEEE sixth international conference on intelligent systems design and applications, pp 412–417

Download references

Author information

Authors and Affiliations

  1. Institute of Electrical and Control Engineering, National Chiao Tung University, Room 709, Engineering Building 5, 1001 Ta Hsueh Road, Hsinchu, 300, Taiwan, ROC

    Kai-Tai Song, Meng-Ju Han & Jung-Wei Hong

Authors
  1. Kai-Tai Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meng-Ju Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jung-Wei Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai-Tai Song.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Song, KT., Han, MJ. & Hong, JW. Online learning design of an image-based facial expression recognition system. Intel Serv Robotics 3, 151–162 (2010). https://doi.org/10.1007/s11370-010-0066-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11370-010-0066-3

Keywords

Search

Navigation