Skip to main content
SpringerLink
Log in

Enhancing the Communication Spectrum in Collaborative Virtual Environments

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10072))

Included in the following conference series:

Abstract

The importance of interpersonal and group communication has been studied and recognized for thousands of years. With recent technological advances, humans have enabled remote interaction through shared virtual spaces; however, research is still needed to develop methods for expressing many important non-verbal communication cues. Our work explores the methods for enhancing the communication spectrum in collaborative virtual environments. Our primary contribution is a machine learning framework that maps human facial data to avatars in the virtual world. We developed a synthetic training process to create labeled data to alleviate the burden of manual annotation. Additionally, we describe a collaborative virtual environment that can utilize both verbal and non-verbal cues for improved user communication and interaction. Finally, we present results demonstrating the success of our method in a sample collaborative scenario.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.unity3d.com.

  2. 2.

    http://www.mixamo.com.

References

  1. Mehrabian, A.: Nonverbal Communication. Transaction Publishers, New Brunswick (1977)

    Google Scholar 

  2. Fabri, M., Moore, D.: The use of emotionally expressive avatars in collaborative virtual environments. In: Virtual Social Agents, vol. 88 (2005)

    Google Scholar 

  3. Ekman, P., Rosenberg, E.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS). Oxford University Press, New York (1997)

    Google Scholar 

  4. Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., Sasse, M.A.: The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 529–536. ACM (2003)

    Google Scholar 

  5. Tanenbaum, J., Seif El-Nasr, M., Nixon, M.: Nonverbal Communication in Virtual Worlds: Understanding and Designing Expressive Characters. ETC Press, Pittsburgh (2014)

    Google Scholar 

  6. Guye-Vuillème, A., Capin, T.K., Pandzic, S., Thalmann, N.M., Thalmann, D.: Nonverbal communication interface for collaborative virtual environments. Virtual Reality 4, 49–59 (1999)

    Article  Google Scholar 

  7. Cootes, T.F., Edwards, G.J., Taylor, C.J., et al.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23, 681–685 (2001)

    Article  Google Scholar 

  8. Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2879–2886. IEEE (2012)

    Google Scholar 

  9. Baltrusaitis, T., Robinson, P., Morency, L.P.: Constrained local neural fields for robust facial landmark detection in the wild. In: International Conference on Computer Vision Workshops, pp. 354–361 (2013)

    Google Scholar 

  10. Sintel: Sintel face rig (2016). https://durian.blender.org/about/

  11. Mixamo: Adobe mixamo (2016). https://www.mixamo.com/faceplus

  12. Faceware: Faceware technology (2016). http://facewaretech.com/products/software/

  13. Realsense: Intel realsense technology (2016). https://software.intel.com/en-us/articles/realsense-overview

  14. Hsieh, P.L., Ma, C., Yu, J., Li, H.: Unconstrained realtime facial performance capture. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1675–1683 (2015)

    Google Scholar 

  15. Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28, 807–813 (2010)

    Article  Google Scholar 

  16. Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: IEEE Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)

    Google Scholar 

  17. Lucey, P., Cohn, J., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: Computer Vision and Pattern Recognition Workshops, pp. 94–101 (2010)

    Google Scholar 

  18. Li, H., Trutoiu, L., Olszewski, K., Wei, L., Trutna, T., Hsieh, P.L., Nicholls, A., Ma, C.: Facial performance sensing head-mounted display. ACM Trans. Graph. (TOG) 34, 47 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computing Sciences, Villanova University, Villanova, Pennsylvania, USA

    Edward Kim & Christopher Moritz

Authors
  1. Edward Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher Moritz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edward Kim .

Editor information

Editors and Affiliations

  1. University of Nevada , Reno, Nevada, USA

    George Bebis

  2. NASA Ames Research Center , Moffett Field, California, USA

    Richard Boyle

  3. Lawrence Berkeley National Laboratory , Berkeley, California, USA

    Bahram Parvin

  4. Desert Research Institute , Reno, Nevada, USA

    Darko Koracin

  5. The Australian National University , O'Malley, Aust Capital Terr, Australia

    Fatih Porikli

  6. Pilot AI Labs , Redwood City, California, USA

    Sandra Skaff

  7. University of Florida , Gainesville, Florida, USA

    Alireza Entezari

  8. Google Inc. , Mountain View, California, USA

    Jianyuan Min

  9. Osaka University , Osaka, Japan

    Daisuke Iwai

  10. The MOVES Institute , Monterey, California, USA

    Amela Sadagic

  11. University of Arizona , Tucson, Arizona, USA

    Carlos Scheidegger

  12. Université Paris-Sud , Orsay, France

    Tobias Isenberg

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Kim, E., Moritz, C. (2016). Enhancing the Communication Spectrum in Collaborative Virtual Environments. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2016. Lecture Notes in Computer Science(), vol 10072. Springer, Cham. https://doi.org/10.1007/978-3-319-50835-1_61

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50835-1_61

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50834-4

  • Online ISBN: 978-3-319-50835-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation