Skip to main content
Springer Nature Link
Log in

HandTalker: A Multimodal Dialog System Using Sign Language and 3-D Virtual Human

  • Conference paper
  • First Online:
Advances in Multimodal Interfaces — ICMI 2000 (ICMI 2000)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1948))

Included in the following conference series:

  • 1024 Accesses

Abstract

In this paper, we describe HandTalker: a system we designed for making friendly communication reality between deaf people and normal hearing society. The system consists of GTS (Gesture/Sign language To Spoken language) part and STG (Spoken language To Gesture/Sign language) part. GTS is based on the technology of sign language recognition, and STG is based on 3D virtual human synthesis. Integration of the sign language recognition and 3D virtual human techniques greatly improves the system performance. The computer interface for deaf people is data-glove, camera and computer display, and the interface for hearing-abled is microphone, keyboard, and display. HandTalker now can support no domain limited and continuously communication between deaf and hearing-abled Chinese people.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

REFERENCES

  1. Jiyong Ma, Wen Gao, Jiangqin Wu and Chunli Wang, A Continuous Chinese Sign Language recognition system, pp428–433, 28-31 March, FG.2000, Grenoble, France.

    Google Scholar 

  2. Ho-Sub Yoon. Jung Soh. Byung-Woo Min. Hyun Seung Yang. Recognition of alphabetical hand gestures using hidden Markov model. IEICE Transactions on Fundamentals of Electronics Communications & Computer Sciences, vol.E82-A, no.7, July 1999, pp.1358–66.

    Google Scholar 

  3. Vogler C., Metaxas D. Parallel hidden Markov models for American Sign Language recognition. Proceedings of the Seventh IEEE International Conference on Computer Vision. IEEE Comput. Soc. Part vol.1, 1999, pp.116–22vol.1. Los Alamitos, CA, USA

    Google Scholar 

  4. Wen Gao, Yibo Song, Baocai Yin, Jie Yan and Ying Liu. Synthesis of sign language,sound and corresponding facial expression driven by text in multimodal interface. In Proceedings of the First International Conference on Multimodal Interface, 1996, 244–248.

    Google Scholar 

  5. Wen Gao, Mingbao Liu. A hierarchical approach to human face detection in a complex background. In Proceedings of the First International Conference on Multimodal Interface,1996,289–292.

    Google Scholar 

  6. Yuille, A.L., Hallinan, P.W. & Cohen, D.S.Feature extraction from faces using deformable templates. International Journal of Computer Vision, 1992,8, 99–111.

    Article  Google Scholar 

  7. Wen Gao, Jie Yan, Baocai Yin, Yibo Song, An individual facial image synthesis system for virtual human, In Proceedings of the Second International Conference on Multimodal Interface,1999, 20–25

    Google Scholar 

  8. P. Ekman and W. V. Friesen. Facial action coding system. Consulting Psychologists Press Inc., California, 1978

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Microsoft Research China, China

    Jie Yan

  2. Harbin Institute of Technology, Harbin, China

    Wen Gao, Jiyong Ma, Xilin Chen, Wei Zeng, Hongming Zhang & Jiangqin Wang

  3. Institute of Computing Technology, Chinese Academy of Sciences, 100080, China

    Wen Gao, Jiyong Ma & Shiguan Shan

Authors
  1. Wen Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiyong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiguan Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xilin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hongming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jie Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jiangqin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Automation, Chinese Academy of Sciences, P.O.Box 2728, 100080, Beijing, China

    Tieniu Tan

  2. Computer Department, Media Laboratory, Tsinghua University, 100084, Beijing, China

    Yuanchun Shi

  3. Institute of Computing Technology, Chinese Academy of Sciences, P.O.Box 2704, 100080, Beijing, China

    Wen Gao

Rights and permissions

Reprints and permissions

Copyright information

© 2000 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gao, W. et al. (2000). HandTalker: A Multimodal Dialog System Using Sign Language and 3-D Virtual Human. In: Tan, T., Shi, Y., Gao, W. (eds) Advances in Multimodal Interfaces — ICMI 2000. ICMI 2000. Lecture Notes in Computer Science, vol 1948. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-40063-X_74

Download citation

  • DOI: https://doi.org/10.1007/3-540-40063-X_74

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-41180-2

  • Online ISBN: 978-3-540-40063-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation