Abstract
Multimodal interfaces (MI) can well improve the interactivity between users and computers through cooperation of different interactive devices and methods to exchange information and understand their requirements or response. As a hotspot in information processing, content- based retrieval (CBR) of multimedia has intrinsic demand for multimodal interface techniques to suit for input / output of multiple media types. In this paper, different MI techniques in CBR of multimedia are introduced, which are classified into three classes, namely traditional CUI/GUI, multimedia UI and intelligent multimodal UI. The analysis and comparison of these MI techniques with corresponded media retrieval ways are also given. It is hoped that the investigation in this paper can much promote the work both in MI and CBR of multimedia for efficient and effective information interaction between human and machines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bolt, R. A.: The Human Interface. California: Lifetime Learning Press (1984)
Sheiderman, B.: Direct Manipulation. A Step Beyond Programming Languages. IEEE Computer, Vol. 16. No. 8 (1983)
Hartson, H. R., etc.: The UAN: A User-Oriented Representation for Direct Manipulation User Interfaces. ACM Trans. on Information Systems. Vol. 8, No. 3 (1990) 181–203
Card, S. K., etc.: The Psychology of Human Computer Interaction. Hillsdale, N. J. (ed.): Lawrence Erlbaum (1983)
Garve, W. W.: Auditory Icons. Using Sound in Computer Interface. Human-Computer Interface, Vol. 2 (1986)
Hauptmann, A. G., Mcavinney, P.: Gestures with Speech for Graphic Manipulation. Int. J. of Man-Machine Studies. Vol. 18. No. 2 (1993)
Burdea, G., Coiffet, P.: Virtual Reality Technology. John Wiley and Sons, Inc. New York. (1994)
Lin, Y., Chen M., etc.: An Architecture for Multimodal Agent Interactive System. Proc. of the 5th Int. con. On CAD/CG’97. Beijing. Int. Academic Press, (1997)
Wang, J.: Integration of Eye-Gaze, Voice and Manual Response in Multimodal User Interface. In Proc. of the IEEE Int. Conf. on System, Man and Cybernetics (1995)
Bolognessi, T., etc.: Introduction to the ISO Specification Language LOTOS. Computer Networks and ISDN Systems, Vol. 14. (1987) 25–59
Idris, F., Panchanatban, S.: Review of Image and Video Indexing Techniques. Univ. of Ottawa, Canada (1996)
Rui, Y., Huang, T. S., Chang, S. F.: Image Retrieval: Current Techniques, Promising Directions and Open Issues. J. of visual Communication and Image Representation, vol.10. (1999) 1–23
Chang, S. K., etc.: Reality Bites-Progressive Querying and Result Visualization in Logical and VR Spaces. http://www.unisa.it/gencos.dir/chang/365/real.htm
Chang, S. F., etc.: A Fully Automated Content-based Video Search Engine Supporting Spatial-Temp acoustical Queries. IEEE Trans. On Circuits and System for Vidio Technology, Vol. 8. No. 5. (1998)
Hauptmann, A., Witbrock, M.: Informedia: News-on-Demand Multimedia Information Acquisition and Retrieval. Intelligent Multimidia Retrival. Mark, T. Maybury, (ed.) AAAI Press (1997) 213–223
Smith, J. R., Chang, S. F.: VisualSEEK: A Fully Automated Content-based Image Query System. ACM Multimedia96, Boston, MA, Nov. 20 (1996)
Smith, M., Kanade, T.: Video Skimming and Characterization through the Combination of Image and Language Understanding. IEEE Int. Workshop ICCV98,India (1998)
Deng, Y. N., Manjunath, B. S.: Content-based Search of Video Using Color, texture and Motion. Proc. of IEEE on IP, Vol. 2. CA (1997) 534–537
Zhang H J, etc. Video Paring, Retrieval and Browsing: An Integrated and Content-based Solution. ACM Multimedia (1995) 15–24
Ren, J. C., etc: A Self-Extensible Model for Content-based Video Retrieval. Int. Workshop MMWS2000, Hong Kong (2000) 259–262
Yeo, B. L., Yeung, M. M.: Classification, Simplification and Dynamic Visualization of Scene Transition Graphs for Video Browsing. Storage and Retrieval for Image and Vidio Databases VI. SPIE Vol. 3321. Jan. (1998) 60–70
Servetto, S., etc.: A Region-based Representation of Images in Mars. Special issue on Multimedia Signal Processing, J. on VLSI Signal Processing. Oct. (1998)
IBM: QBIC-IBM’s Query by Image Content, http://wwwqbic.almaden.ibm.com/
Castagno, R.,Ebrahimi, T., Kunt, M.: Video Segmentation Based on Mu24. ltiple Features r Video 25. L’96, (1996) for Interactive Multimedia Applications, IEEE Trans. On Circuits and Systems fo Technology, Vol.8, No.5, Sep. (1998)
Rodger, J. M., etc,: Towards the Digital Music Library: Tune Retrieval from Acoustic Input. In Proceedings of DL’96, (1996)
Wiggins, etc.: A Framework for the Evaluation of Music Representation Systems. Computer Music Journal, Vol. 17, No. 3 (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ren, J., Zhao, R., Feng, D.D., Siu, Wc. (2000). Multimodal Interface Techniques in Content-Based Multimedia Retrieval. In: Tan, T., Shi, Y., Gao, W. (eds) Advances in Multimodal Interfaces — ICMI 2000. ICMI 2000. Lecture Notes in Computer Science, vol 1948. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-40063-X_82
Download citation
DOI: https://doi.org/10.1007/3-540-40063-X_82
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41180-2
Online ISBN: 978-3-540-40063-9
eBook Packages: Springer Book Archive