Jie Yang
ABSTRACT
A sign is something that suggests the presence of a fact, condition, or quality. Signs are everywhere in our lives. They make our lives easier when we are familiar with them. But sometimes they pose problems. For example, a tourist might not be able to understand signs in a foreign country. This paper discusses problems of automatic sign recognition and translation. We present a system capable of capturing images, detecting and recognizing signs, and translating them into a target language. We describe methods for automatic sign extraction and translation. We use a user-centered approach in system development. The approach takes advantage of human intelligence if needed and leverage human capabilities. We are currently working on Chinese sign translation. We have developed a prototype system that can recognize Chinese sign input from a video camera that is a common gadget for a tourist, and translate the signs into English or voice stream. The sign translation, in conjunction with spoken language translation, can help international tourists to overcome language barriers. The technology can also help a visually handicapped person to increase environmental awareness.
- Black, A. W., Taylor, P., and Caley, R., Festival, www.cstr.ed.ac.uk/projects/festival.html, The Centre for Speech Technology Research (CSTR) at the University of Edinburgh, 1998.]]Google Scholar
- Brown, R. D., Adding Linguistic Knowledge to a Lexical Example-Based Translation System. Proceedings of the Eighth International Conference on Theoretical and Methodological Issues in Machine Translation (TMI-99), pp. 22--32, Chester, England, August, 1999.]]Google Scholar
- Brown, R. D., Automated Dictionary Extraction for "Knowledge-Free" Example-Based Translation. Proceedings of the Seventh International Conference on Theoretical and Methodological Issues in Machine Translation (TMI-97), pp. 111--118, Santa Fe, New Mexico, July, 1997.]]Google Scholar
- Brown, R. D., Automated Generalization of Translation Examples. Proceedings of the Eighteenth International Conference on Computational Linguistics (COLING-2000), pp. 125--131, 2000.]] Google ScholarDigital Library
- Brown, R. D., Example-based machine translation in the pangloss system. Proceedings of the 16th International Conference on Computational Linguistics, pp. 169--174, 1996.]] Google ScholarDigital Library
- Cui, Y. and Huang, Q., Character Extraction of License Plates from Video. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 502--507, 1997.]] Google ScholarDigital Library
- Gao, J., Yang, J., Zhang, Y., Waibel, A., Text Detection and Translation from Natural Scenes, Technical Report CMU-CS-01-139, Computer Science Department, Carnegie Mellon University, June, 2001.]]Google Scholar
- Gao, J. and Yang, J., "An Adaptive Algorithm for Text Detection from Natural Scenes," Proceedings of Computer Vision and Pattern Recognition (CVPR 2001).]]Google Scholar
- Hogan, C. and Frederking, R. E., An Evaluation of the Multi-engine MT Architecture. Machine Translation and the Information Soup: Proceedings of the Third Conference of the Association for Machine Translation in the Americas (AMTA '98), vol. 1529 of Lecture Notes in Artificial Intelligence, pp. 113--123. Springer-Verlag, Berlin, October.]] Google ScholarDigital Library
- Hutchins, John W., Machine Translation: Past, Present, Future, Ellis Horwood Limited, England, 1986.]] Google ScholarDigital Library
- Jain, A. K. and Yu, B., Automatic text location in images and video frames. Pattern Recognition, vol. 31, no. 12, pp. 2055--2076, 1998.]]Google ScholarCross Ref
- Kubler, Cornelius C., "Read Chinese Signs". Published by Chheng & Tsui Company, 1993.]]Google Scholar
- Li, H. and Doermann, D., Automatic Identification of Text in Digital Video Key Frames, Proceedings of IEEE International Conference of Pattern Recognition, pp. 129--132, 1998.]] Google ScholarDigital Library
- Li, H. and Doermann, D., Superresolution-Based Enhancement of Text in Digital Video. ICPR, pp. 847--850, 2000.]]Google Scholar
- Lienhart, R., Automatic Text Recognition for Video Indexing, Proceedings of ACM Multimedia 96, pp. 11--20, 1996.]] Google ScholarDigital Library
- Ohya, J., Shio, A., and Akamatsu, A., Recognition of characters in scene images. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 2, pp. 214--220, 1994.]] Google ScholarDigital Library
- Sato, T., Kanade, T., Hughes, E. K., and Smith, M. A., Video OCR for digital news archives. IEEE Int. Workshop on Content-Based Access of Image and Video Database, 1998.]] Google ScholarDigital Library
- Taylor, M. J., Zappala, A., Newman, W. M., and Dance, C. R., Documents through cameras, Image and Vision Computing, vol. 17, no. 11, pp. 831--844, 1999.]]Google ScholarCross Ref
- Waibel, A., Interactive Translation of Conversational Speech, Computer, vol. 29, no. 7, 1996.]] Google ScholarDigital Library
- Watanabe, Y., Okada, Y., Kim, Y. B., and Takeda, T., Translation camera, Proceedings Fourteenth International Conference on Pattern Recognition, pp. 613--617, 1998.]] Google ScholarDigital Library
- Wong, E. K. and Chen, M., A Robust Algorithm for Text Extraction in Color Video, Proceedings of IEEE Int. Conference on Multimedia and Expo (ICME2000), 2000.]]Google ScholarCross Ref
- Wu, V., Manmatha, R., and Riseman, E. M., Textfinder: an automatic system to detect and recognize text in images. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 11, pp. 1224--1229, 1999.]] Google ScholarDigital Library
- Yang, J., Yang, W., Denecke, M., and Waibel, A., Smart sight: a tourist assistant system. Proceedings of Third International Symposium on Wearable Computers, pp. 73--78. 1999.]] Google ScholarDigital Library
- Yang, J., Gao, J., Yang, J., Zhang, Y., Waibel, A., Towards Automatic Sign Translation, Proceedings of Human Language Technology 2001.]] Google ScholarDigital Library
- Zhong Y., Karu, K., and Jain, A. K., Locating Text in Complex Color Images, Pattern Recognition, vol. 28, no. 10, pp. 1523--1536, 1995.]]Google ScholarCross Ref
- An automatic sign recognition and translation system
Recommendations
Towards automatic sign translation
HLT '01: Proceedings of the first international conference on Human language technology researchSigns are everywhere in our lives. They make our lives easier when we are familiar with them. But sometimes they also pose problems. For example, a tourist might not be able to understand signs in a foreign country. In this paper, we present our efforts ...
A machine translation system from Arabic sign language to Arabic
AbstractArabic sign language (ArSL) is one of the sign languages that is used in Arab countries. This language has structure and grammar that differ from spoken Arabic. Available ArSL recognition systems perform direct mapping between the recognized sign ...
Text to Sign Language Translation System: A Review of Literature
Many machine translation systems for spoken languages are available, but the translation system between the spoken and Sign Language are limited. The translation from Text to Sign Language is different from the translation between spoken languages ...
Comments