Skip to main content
SpringerLink
Log in

Towards Synchronous Model of Non-emotional Conversational Gesture Generation in Humanoids

  • Conference paper
  • First Online:
Intelligent Computing

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 283))

Abstract

We need social robotics to assist humans with elderly care, health care, child education, entertainment, space exploration and hospitality. This task requires sharing a common space with human beings along with humanlike humanoid-human interactions that involve combining facial-expression analysis, speech understanding, gesture analysis and generation. Facial expression analysis, speech understanding, speech generation and hand-gesture recognition using camera and depth-sensors are increasingly being explored. Limited gesture generation is being explored in avatars (virtual) and more recently in humanoids. However, gesture generation is limited to head-gestures, and random and meaningless hand-gestures, which lack detailed classification of gestures and their translation to humanoid gesture generation. In this paper, we extend the hand-gesture classification in the psychology-domain to include haptic gestures and extend the conceptual dependency model for computational hand-gesture generation. We also present a model to generate different subclasses of hand-gestures based upon the notion of scene, object attributes, shallow anaphora analysis and speech-action temporal alignment at the sentence level.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Fong, T., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robot. Auton. Syst. 42(3–4), 143–166 (2003)

    Article  Google Scholar 

  2. Radford, N.A., et al.: Valkyrie: NASA’s first bipedal humanoid robot. J. Field Robot. 32(3), 297–419 (2015)

    Article  Google Scholar 

  3. Ishii Y., Watanabe, T.: An embodied avatar mediated communication system with virtual actor for human interaction analysis. In: Proceedings of the 16th IEEE International Symposium on Robot and Human Interactive Communication, Jeju, South Korea, pp. 37–42 (2007)

    Google Scholar 

  4. Thalmann N. M., Zhang, Z.: Assistive social robots for people with special needs. In: Proceedings of the International Conference on Contemporary Computing and Informatics, Mysore, India, pp. 1374–1380 (2014)

    Google Scholar 

  5. Loghmani, M.R., Patten, T., Vincze, M.: Towards socially assistive robots for elderly: an end-to-end object search framework. In: Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Athens, Greece, pp. 190–194 (2018)

    Google Scholar 

  6. García, D.H., Esteban, P.G., Lee, H.R., Romeo, M., Senft, E., Billing, E.: Social robots in therapy and care. In: Proceedings of the 14th ACM/IEEE International Conference on Human-Robot Interaction, Daegu, South Korea, pp. 669–670 (2019)

    Google Scholar 

  7. Kirstein, F., Risager, R.V.: Social robots in educational institutions they came to stay: introducing, evaluating, and securing social robots in daily education. In: Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction, Christchurch, New Zealand, pp. 453–454 (2016)

    Google Scholar 

  8. Duffy, B.R.: Anthropomorphism and the social robot. Robot. Auton. Syst. 42(3–4), 177–190 (2003)

    Article  Google Scholar 

  9. Wainer, J., Feil-seifer, D.J., Shell, D.A., Mataric, M.J.: The role of physical embodiment in human-robot interaction. In: Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication, Hatfield, UK, pp. 117–122 (2006)

    Google Scholar 

  10. Liu, H., Wang, L.: Gesture recognition for human-robot collaboration: a review. Proc. Int. J. Ind. Ergon. 68, 355–367 (2018)

    Article  Google Scholar 

  11. Moni, M. A.: HMM-based hand gesture recognition: a review on techniques and approaches. In: Proceedings of the 2nd IEEE International Conference on Computer Science and Information Technology, Beijing, China, pp. 433–437 (2009)

    Google Scholar 

  12. Noroozi, F., Corneanu, C., Kamińska, D., Sapiński, T., Escalera, S., Anbarjafari, G.: Survey on emotional body gesture recognition. IEEE Trans. Affect. Comput. (2018). https://doi.org/10.1109/TAFFC.2018.2874986

    Article  Google Scholar 

  13. Wagner, P., Malisz, Z., Kopp, S.: Gesture and Speech in Interaction - an overview. Speech Commun. 57, 209–232 (2014)

    Article  Google Scholar 

  14. Iverson, J.M., Goldin-Meadow, S.: Why do people gesture as they speak. Nature 396, 228 (1998)

    Article  Google Scholar 

  15. Andrist, S., Tan, X.Z., Gleicher, M, Mutlu, B.: Conversational gaze aversion for humanlike robots. In: Proceedings of the 9th ACM/IEEE International Conference on Human-Robot Interaction, Bielefeld, Germany, pp. 25–32 (2014)

    Google Scholar 

  16. Efron, D.: Gesture and Environment. King’s Crown Press, Morningside Heights (1941)

    Google Scholar 

  17. McNeill, D.: Hand and Mind: What Gestures Reveal about Thought. The University of Chicago Press, Chicago (1992)

    Google Scholar 

  18. Ekman, P., Frisen, W.V.: The repertoire of nonverbal behavior: categories, origins. usage and coding. Semotica 1, 49–98 (1969)

    Google Scholar 

  19. Krauss, R.M., Chen, Y., Gottesman, R.F.: Lexical gestures and lexical access: a process model. In: Language and gesture, pp. 261–283. Cambridge University Press, New York (2001)

    Google Scholar 

  20. Singh, A., Bansal, A.K.: Declarative modeling and implementation of robotic head-based gestures for human-robot interaction. Int. J. Comput. Appl. 26(2), 49–66 (2019)

    Google Scholar 

  21. Schank, R.C.: Conceptual dependency: a theory of natural language understanding. Cogn. Psychol. 3, 552–631 (1972)

    Article  Google Scholar 

  22. Lytinen, S.L.: Conceptual dependency and its descendants. Comput. Math. Appl. 23(2–5), 51–73 (1992)

    Article  MathSciNet  Google Scholar 

  23. Chein, M., Mugnier, M.L.: Conceptual graphs: fundamental notions. Revue d’Inteligence Artificielle 6(4), 365–406 (1992)

    Google Scholar 

  24. Ghayoumi, M., Thafar, M., Bansal, A.K.: A formal approach for multimodal integration to derive emotions. J. Vis. Lang. Sentient Syst. 2, 48–54 (2016)

    Google Scholar 

  25. Allen, J.F.: Maintaining knowledge about temporal intervals. Commun. ACM 26(11), 832–843 (1983)

    Article  Google Scholar 

  26. Wang, L., Tan, T., Ning, H., Hu, W.: Silhouette analysis-based gait recognition for human identification. IEEE Trans. Pattern Anal. Mach. Intell. 25(12), 1505–1518 (2003)

    Article  Google Scholar 

  27. Xu, D., Chen, Y.L., Lin, C., Kong X., Wu, X.: Real-time dynamic gesture recognition system based on depth perception for robot navigation. In: Proceedings of the IEEE International Conference on Robotics and Biomimetics, Guangzhou, China, pp. 689–694 (2012)

    Google Scholar 

  28. Park, H.S., Kim, E.Y., Jang, S.S., Park, S.H., Park, M.H., Kim, H.J.: HMM-based gesture recognition for robot control. In: Marques, J.S., Pérez de la Blanca, N., Pina, P. (eds.) IbPRIA. LNCS, vol. 3522, pp. 607–614. Springer, Heidelberg (2005). https://doi.org/10.1007/11492429_73

    Chapter  Google Scholar 

  29. Yang, Z., Li, Y., Chen, W., Zheng, Y.: Dynamic hand gesture recognition using hidden Markov models. In: Proceedings of the 7th International Conference on Computer Science & Education, Melbourne, VIC, Australia, pp. 360–365 (2012)

    Google Scholar 

  30. Shin, S., Sung, W.: Dynamic hand gesture recognition for wearable devices with low complexity recurrent neural networks. In: Proceedings of the IEEE International Symposium on Circuits and Systems, Montreal, QC, Canada (2016). https://arxiv.org/pdf/1608.04080.pdf

  31. Chen, X., Guo, H., Wang G., Zhang, L.: Motion feature augmented recurrent neural network for skeleton-based dynamic hand gesture recognition. In: Proceedings of the IEEE International Conference on Image Processing, Beijing, China, pp. 2881–2885 (2017)

    Google Scholar 

  32. Lai, K., Yanushkevich, S.N.: CNN+RNN depth and skeleton based dynamic hand gesture recognition. In: Proceedings of the 24th International Conference on Pattern Recognition, Beijing, China (2018). https://doi.org/10.1109/ICPR.2018.8545718

  33. Hakim, N.L., Shih, T.K., Arachchi, S.P.K., Aditya, W., Chen, Y., Lin, C.: Dynamic hand gesture recognition using 3DCNN and LSTM with FSM. Sensors 19(24), 5429 (2019). https://doi.org/10.3390/s19245429

    Article  Google Scholar 

  34. Nam, Y., Wohn, N., Lee-Kwang, H.: Modeling and recognition of hand gesture using colored Petri nets. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 29(5), 514–521 (1999)

    Article  Google Scholar 

  35. Ng-Thow-Hing, V., Luo, P., Okita, S., Synchronized gesture and speech production for humanoid robots. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, pp. 4617–4624 (2010)

    Google Scholar 

  36. Fadli, H., Machbub C., Hidayat, E.: Human gesture imitation on NAO humanoid robot using kinect based on inverse kinematics method. In: Proceedings of the International Conference on Advanced Mechatronics, Intelligent Manufacture, and Industrial Automation, Surabaya, Indonesia, pp. 116–120 (2017)

    Google Scholar 

  37. Huisman, G.: Social touch technology: a survey of haptic technology for social touch. IEEE Trans. Haptics 10(3), 391–408 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Kent State University, Kent, OH, 44242, USA

    Aditi Singh & Arvind K. Bansal

Authors
  1. Aditi Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arvind K. Bansal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aditi Singh .

Editor information

Editors and Affiliations

  1. Faculty of Science and Engineering, Saga University, Saga, Japan

    Kohei Arai

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Singh, A., Bansal, A.K. (2022). Towards Synchronous Model of Non-emotional Conversational Gesture Generation in Humanoids. In: Arai, K. (eds) Intelligent Computing. Lecture Notes in Networks and Systems, vol 283. Springer, Cham. https://doi.org/10.1007/978-3-030-80119-9_47

Download citation

Publish with us

Policies and ethics

Search

Navigation