Skip to main content

Advertisement

Springer Nature Link
Log in

Facial expression recognition of a speaker using thermal image processing and reject criteria in feature vector space

  • Original Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

In our previously developed method for the facial expression recognition of a speaker, the positions of feature vectors in the feature vector space in image processing were generated with imperfections. The imperfections, which caused misrecognition of the facial expression, tended to be far from the center of gravity of the class to which the feature vectors belonged. In the present study, to omit the feature vectors generated with imperfections, a method using reject criteria in the feature vector space was applied to facial expression recognition. Using the proposed method, the facial expressions of two subjects were discriminable with 86.8 % accuracy for the three facial expressions of “happy”, “neutral”, and “others” when they exhibited one of the five intentional facial expressions of “angry”, “happy”, “neutral”, “sad”, and “surprised”, whereas these expressions were discriminable with 78.0 % accuracy by the conventional method. Moreover, the proposed method effectively judged whether the training data were acceptable for facial expression recognition at the moment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Yoshitomi Y, Kimura S, Hira E, et al (1996) Facial expression recognition using infrared rays image processing. In: Proceedings of the Annual Convention IPS Japan, Osaka, Japan, Sept 4–6, 1996, 2:339–340

  2. Yoshitomi Y, Kimura S, Hira E, et al (1997) Facial expression recognition using thermal image processing. In: IPSJ SIG notes, CVIM103-3, Kyoto, Japan, Jan 23–24, 1997, pp 17–24

  3. Yoshitomi Y, Miyawaki N, Tomita S, et al (1997) Facial expression recognition using thermal image processing and neural network. In: Proceedings of 6th IEEE International Workshop on Robot and Human Communication, Sendai, Japan, Sep 29–Oct 1, 1997, pp 380–385

  4. Sugimoto Y, Yoshitomi Y, Tomita S (2000) A method for detecting transitions of emotional states using a thermal face image based on a synthesis of facial expressions. J Robot Auton Syst 31:147–160

    Article  Google Scholar 

  5. Yoshitomi Y, Kim SIll, Kawano T, et al (2000) Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face. In: Proceedings of 6th IEEE International Workshop on Robot and Human Interactive Communication, Osaka, Japan, Sep 27–29, 2000, pp 178–183

  6. Ikezoe F, Ko R, Tanijiri T et al (2004) Facial expression recognition for speaker using thermal image processing (in Japanese). Trans Human Interface Soc 6(1):19–27

    Google Scholar 

  7. Nakano M, Ikezoe F, Tabuse M et al (2009) A study on the efficient facial expression using thermal face image in speaking and the influence of individual variations on its performance (in Japanese). J IEEE 38(2):156–163

    Google Scholar 

  8. Koda Y, Yoshitomi Y, Nakano M, et al (2009) Facial expression recognition for a speaker of a phoneme of vowel using thermal image processing and a speech recognition system. In: Proceedings of 18th IEEE International Symposium on Robot and Human Interactive Communication, Toyama, Japan, Sep 29–Oct 1, 2009, pp 955–960

  9. Yoshitomi Y (2010) Facial expression recognition for speaker using thermal image processing and speech recognition system. In: Fujita H, Sasaki J (eds) Proceedings of 10th WSEAS International Conference on Applied Computer Science, Appi Kogen, Iwate, Japan, Oct 4–6, 2010, pp 182–186

  10. Fujimura T, Yoshitomi Y, Asada T et al (2011) Facial expression recognition of a speaker using front-view face judgment, vowel judgment, and thermal image processing. J Artif Life ics 16(3):411–417

    Article  Google Scholar 

  11. Yoshitomi Y, Asada T, Shimada K et al (2011) Facial expression recognition of a speaker using vowel judgment and thermal image processing. J Artif Life Robot 16(3):318–323

    Article  Google Scholar 

  12. Yoshitomi Y (2012) Facial expression recognition of speaker using vowel judgment and features of thermal face image. In: Anderson D, Yang HJ, Varacha P (eds) Proceedings of 1st WSEAS International Conference on Information Technology and Computer Networks, Vienna, Austria, Nov 10–12, 2012, pp 139–145

  13. Nakanishi Y, Yoshitomi Y, Asada T et al (2013) Robust facial expression recognition of a speaker using thermal image processing and updating of fundamental training-data. J Artif Life Robot 17(3):342–349

    Google Scholar 

  14. Yoshitomi Y, Tabuse M, Asada T (2012) Facial expression recognition using thermal image processing. In: Carvalho VH (ed) Image processing: methods, applications and challenges. Nova Science Publisher, New York, pp 57–85

    Google Scholar 

  15. Asada T, Yoshitomi Y, Tabuse M (2012) A system for facial expression recognition of a speaker using front-view face judgment, vowel judgment and thermal image processing. J Artif Life Robot 17(2):263–269

    Google Scholar 

  16. Yoshitomi Y, Tabuse M, Asada T (2011) Vowel judgment for facial expression recognition of a speaker. In: Ipšić I (ed) Speech technologies. InTech, Rijeka, pp 405–424

    Google Scholar 

  17. Socolinsky DA, Selinger A (2002) A comparative analysis of face recognition performance with visible and thermal infrared imagery. In: Kasturi R, Laurendeau D, Suen C et al (eds) Proceedings of 16th International Conference on Pattern Recognition, Quebec City, Canada, Aug 11–15, 2002, 4, pp 217–222

  18. Khan MM, Ward RD, Ingleby M (2004) Automated classification and recognition of facial expressions using infrared thermal imaging. In: Proceedings of 2004 IEEE Conference on Cybernetics and Intelligent Systems, Singapore, Dec 1–3, 2004, 1, pp 202–206

  19. Khan MM, Ingleby M, Ward RD (2006) Automated facial expression classification and affect interpretation using infrared measurement of facial skin temperature variations. ACM Trans Auton Adapt Syst 1(1):91–113

    Google Scholar 

  20. Jiang G, Song X, Zheng F et al (2006) Facial expression recognition using thermal image. In: Proceedings of 27th Annual International Conference of the Engineering in Medicine and Biology Society, Shanghai, China, Jan 17–18, 2006, pp 631–633

  21. Hernández B, Olague G, Hammoud R et al (2007) Visual learning of texture descriptors for facial expression recognition in thermal imagery. Computer Vis and Image Underst 16(2–3):258–269

    Google Scholar 

  22. Kuno H (1994) Infrared rays engineering (in Japanese). IEICE, Tokyo, p 22

    Google Scholar 

  23. Kuno H (1994) Infrared rays engineering (in Japanese). IEICE, Tokyo, p 45

    Google Scholar 

  24. Kawahara T et al (2010) Open-source large vocabulary CSR engine Julius. Julius rev.4.1.5.1. http://julius.sourceforge.jp/. Accessed 12 Mar 2013

Download references

Acknowledgments

This work was supported by KAKENHI (22300077).

Author information

Authors and Affiliations

  1. Graduate School of Life and Environmental Sciences, Kyoto Prefectural University, 1-5 Nakaragi-cho, Shimogamo, Sakyo-ku, Kyoto, 606-8522, Japan

    Yuu Nakanishi, Yasunari Yoshitomi, Taro Asada & Masayoshi Tabuse

Authors
  1. Yuu Nakanishi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yasunari Yoshitomi
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Taro Asada
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Masayoshi Tabuse
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yasunari Yoshitomi.

About this article

Cite this article

Nakanishi, Y., Yoshitomi, Y., Asada, T. et al. Facial expression recognition of a speaker using thermal image processing and reject criteria in feature vector space. Artif Life Robotics 19, 76–88 (2014). https://doi.org/10.1007/s10015-013-0136-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-013-0136-7

Keywords