Abstract
The user interface is of critical importance in applications providing mapping services. It defines the visualisation and interaction modes for carrying out a variety of mapping tasks, and ease of use is essential to successful user adoption of a mapping application. This is redoubled in a mobile context, where mobile device limitations can hinder usability. In particular, interaction modes such as a pen/stylus are limited and can be quite difficult to use in a mobile context. Moreover, the majority of GIS interfaces are inherently complex and require significant user training, which can be a serious problem for novice users such as tourists. We propose an increased focus on developing multimodal interfaces for mobile GIS, allowing for two or more modes of input, as an attempt to address interaction complexity in the context of mobile mapping applications. Such interfaces allow users to choose the modes of interaction that are not only most intuitive to them, but also most suitable for their current task and environment. This chapter presents the user interaction problem and the utility of multimodal interfaces for mobile GIS. We describe our multimodal mobile GIS CoMPASS which helps to address the problem by permitting users to interact with spatial data using a combination of speech and gesture input. CoMPASS is set in the context of a representative survey across a range of comparable multimodal systems, and the effectiveness of our approach is evaluated in a user study which demonstrates that multimodal interfaces provide more intuitive and efficient interaction for mobile mapping applications.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Benoit, C., Martin, J.C., Pelachaud, C., Schomaker, L. and Suhm, B. (2000): Audio-visual and Multimodal Speech-based Systems. In Handbook of Multimodal and Spoken Dialogue Systems: Resources, Terminology and Product Evaluation, D. Gibbon, I. Mertins and R. Moore (Eds.), pp. 102-203, Kluwer.
Bertolotto, M. and Egenhofer, M. (1999): Progressive Vector Transmission. In 7th ACM Symposium on Advances in Geographic Information Systems, pp. 152-157, Kansas City, Kansas.
Bolt, R. A. (1980): Put-That-There: Voice and Gesture at the Graphics Interface. In The 7 th International Conference on Computer Graphics and Interactive Techniques, pp. 262-270, Seattle, Washington.
Cohen, P.R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L. and Clow, J. (1997): QuickSet: Multimodal Interaction for Distributed Applications. In 5th ACM Conference on Multimedia, pp. 31-40, New York, New York.
Cohen, P., McGee, D. and Clow, J. (2000): The Efficiency of Multimodal Interaction for a Map-based Task. In 6th International Conference on Applied Natural Language Processing, pp 331-338, Seattle, Washington.
Constantini, E., Pianesi, F. and Prete, M. (2005): Recognising Emotions in Human and Synthetic Faces: The Role of the Upper and Lower Parts of the Face. In 10th International Conference on Intelligent User Interfaces, pp. 20-27, San Diego, California.
Doyle, J., Weakliam, J., Bertolotto, M. and Wilson, D. (2006a): A Multimodal Interface for Personalising Spatial Data in Mobile GIS. In ICEIS – 8 th International Conference on Enterprise Information Systems, in press, Paphos, Cyprus.
Doyle, J. (2007a): Multimodal Interface Design for Mobile GIS. A PhD thesis submitted to University College Dublin, January 2007.
Doyle, J., Bertolotto, M. and Wilson, D. (2007b): Multimodal Interface Design for Geospatial LBS. Under review for the Journal of Location Based Services.
Fuhrmann, S., MacEachren, A., Dou, J., Wang, K. and Cox, A. (2005): Gesture and Speech-based Maps to Support Use of GIS for Crisis Management: A User Study. In AutoCarto 2005, Las Vegas, Nevada.
IBM ViaVoice (2006):
http://www-306.ibm.com/software/info1/websphere/index.jsp?tab=products/mobilespeech
Jost, M., Haussler, J., Merdes, M. and Malaka, R. (2005): Multimodal Interaction for Pedestrians: An Evaluation Study. In 10th International Conference on Intelligent User Interfaces, pp. 59-66, San Diego, California.
Krueger, A., Butz, A., Muller, C., Stahl, C., Wasinger, R., Steinberg, K.E. and Dirschl, A. (2004): The Connected User Interface: Realizing a Personal Situated Navigation Service. In ,bt>The 9 th International Conference on Intelligent User Interfaces (IUI 04), pp. 161-168, Madeira, Portugal, ACM press.
Lynch, D., Bertolotto, M. and Wilson, D. (2005): Spatial Annotations in a Mapping Environment. In GISRUK ‘05, GIS Research UK, pp. 524-528, Glasgow, Scotland.
Malaka, M., Haeussler, J. and Hidir, A. (2004): SmartKom Mobile – Intelligent Ubiquitous User Interaction. In 9th International Conference on Intelligent User Interfaces, pp. 310-312, Madeira, Portugal.
Nickel, K. and Stiefelhagen, R. (2003): Pointing Gesture Recognition based on 3D-tracking of Face, Hands and Head Orientation. In 5th International Conference on Multimodal Interfaces, pp. 140-146, Vancouver, Canada.
OpenMap (2006): http://openmap.bbn.com/
Oviatt, S. (1996a): Multimodal Interfaces for Dynamic Interactive Maps. In SIGCHI Conference on Human Factors in Computing Systems, pp. 95-102, Vancouver, Canada.
Oviatt, S. (1999): Mutual Disambiguation of Recognition Errors in a Multimodal Architecture. In The Conference on Human Factors in Computing Systems (CHI ’99), pp. 576-583, Pittsburgh, Pennsylvania.
Oviatt, S. (2000a): Taming Recognition Errors with a Multimodal Interface. In Communications of the ACM, Vol. 43, No. 9, pp.45-51.
Oviatt, S. (2000b): Multimodal System Processing in Mobile Environments. In 13th Annual ACM Symposium on User Interface Software and Technology,pp. 21-30, San Diego, California.
Oviatt, S. (2003): Multimodal Interfaces. In Handbook of Human-computer Interaction, J. Jacko and A. Sears (Eds.), pp. 286-304, New Jersey.
Qvarfordt, P. (2003): Spoken Feedback in Multimodal Interaction: Effects on Users Experience of Qualities of Interaction. In 1st Nordic Symposium on Multimodal Communication, pp. 21-34.
Qvarfordt, P. and Zhai, S. (2005): Conversing with the User based on Eye-Gaze Patterns. In SIGCHI Conference on Human Factors in Computing Systems, pp. 221-230, Portland, Oregon.
Rauschert, I., Agrawal, P., Sharma, R., Fuhrmann, S., Brewer, I., MacEachren, A., Wang, H. and Cai, G. (2002a): Designing a Human-Centered, Multimodal GIS Interface to Support Emergency Management. In 10th ACM International Symposium on Advances in Geographic Information Systems, pp. 119-124, Virginia.
Rauschert, I., Sharma, R., Fuhrmann, S., Brewer, I. And MacEachren, A. (2002b): Approaching a New Multimodal Interface. In 2nd International Conference on GIS (GIScience), Colorado.
Rugelbak, J. and Hamnes, K. (2003): Multimodal Interaction – Will Users Tap and Speak Simultaneously? In Telektronikk ’03, pp. 118-124.
Schapira, E. and Sharma, R. (2001): Experimental Evaluation of Vision and Speech based Multimodal Interfaces. In Workshop on Perceptive User Interfaces, pp. 1-9, Florida.
Suhm, B., Myers, B. and Waibel, A. (2001): Multimodal Error Correction for Speech User Interfaces. In ACM Transactions on Computer-Human Interaction, Vol. 8, No. 1, pp. 60-98.
Wasinger, R. and Kreuger, A. (2004): Multimodal Interaction with Mobile Navigation Systems. In Information Technology, Vol. 46, No. 6, pp. 322-331, Oldenbourg Publishers.
Wasinger, R., Stahl, C. and Kreuger, A. (2003a): M3I in a Pedestrian Navigation and Exploration System. In The 4 th International Symposium on Human Computer Interaction with Mobile Devices (Mobile HCI 03), pp. 481-485, Udine, Italyl, Springer-Verlag LNCS.
Wasinger, R., Stahl, C. and Kreuger, A. (2003b): Robust Speech Interaction through the use of Multiple and Different Media Input Types. In Eurospeech 2003, pp. 1049-1052.
Weakliam, J., Bertolotto, M. and Wilson, D. (2005a): Implicit Interaction Profiling for Recommending Spatial Content. In ACMGIS ’05, pp. 285-294, Bremen, Germany.
Weakliam, J., Lynch, D., Doyle, J., Bertolotto, M and Wilson, D. (2005b): Delivering Personalized Context-Aware Spatial Information to Mobile Devices. In W2GIS, 5 th International Workshop on Web and Wireless Geographic Information Systems, pp. 194-205, Lausanne, Switzerland.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Doyle, J., Bertolotto, M., Wilson, D. (2008). A Survey of Multimodal Interfaces for Mobile Mapping Applications. In: Meng, L., Zipf, A., Winter, S. (eds) Map-based Mobile Services. Lecture Notes in Geoinformation and Cartography. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-37110-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-540-37110-6_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37109-0
Online ISBN: 978-3-540-37110-6
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)