Antti Nurminen
ABSTRACT
Human sensory system is a complex mechanism, providing us with a wealth of data from our environment. Our nervous system constantly updates our awareness of the environment based on this multisensory input. We are attuned to cues, which may alert of a danger, or invite for closer inspection. We present the first integrated mobile platform with state-of-the-art visual, aural and haptic augmentation interfaces, supporting localization and directionality where applicable. With these interfaces, we convey cues to our users in the context of urban cultural experiences. We discuss the orchestration of such multimodal outputs and provide indicative guidelines based on our work.
- Bunt, H. 1998. Issues in multimodal human-computer communication. In Multimodal Human-Computer Communication. Springer, 1--12. Google Scholar
Digital Library
- Dúlizia, A. 2009. Exploring multimodal input fusion strategies. The Handbook of Research on Multimodal Human Computer Interaction and Pervasive Services: Evolutionary Techniques for Improving Accessibility, 34--57.Google Scholar
- et.al., D. W. 2010. Real-time panoramic mapping and tracking on mobile phones. In IEEE Virtual Reality, IEEE.Google Scholar
- Lemmelä, S., Vetek, A., Mäkelä, K., and Trendafilov, D. 2008. Designing and evaluating multimodal interaction for mobile contexts. In Proceedings of the 10th International Conference on Multimodal Interfaces, ACM, New York, NY, USA, ICMI '08, ACM, 265--272. Google ScholarDigital Library
- Mansoux, B., Nigay, L., and Troccaz, J. 2007. Output multimodal interaction: The case of augmented surgery. In People and Computers XX Engage, N. Bryan-Kinns, A. Blanford, P. Curzon, and L. Nigay, Eds. Springer London, 177--192.Google Scholar
- Nurminen, A. 2008. Mobile 3d city maps. IEEE Computer Graphics and Applications 28, 4, 20--31. Google ScholarDigital Library
- Oulasvirta, A., Tamminen, S., Roto, V., and Kuorelahti, J. 2005. Interaction in 4-second bursts: The fragmented nature of attentional resources in mobile hci. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, USA, CHI '05, ACM, 919--928. Google ScholarDigital Library
- Rousseau, C., Bellik, Y., Vernier, F., and Bazalgette, D. 2004. Architecture framework for output multimodal systems design. In Proceedings of OZCHI, vol. 4, ACM.Google Scholar
- Schomaker, L., Nijtmans, J., Camurri, A., Lavagetto, F., Morasso, P., Benot, C., Guiard-Marigny, T., Goff, B. L., Adjoudani, A., Defe, I., Mnch, S., Hartung, K., Blauert, J., and Robert-Ribes, J. 1995. A taxonomy of multimodal interaction in the human information processing system. Tech. rep., Esprit Basic Research Action 8579.Google Scholar
- Turk, M. 2014. Multimodal interaction: A review. Pattern Recognition Letters 36, 189--195. Google ScholarDigital Library
Index Terms
- Mobile multisensory augmentations with the CultAR platform
Recommendations
Mobile mixed reality interface developments
SA '12: SIGGRAPH Asia 2012 Symposium on AppsModern mobile devices can act as advanced graphical interfaces to our environment. Mobile augmented reality (AR) systems provide a direct egocentric view, where content is overlaid on top of a camera feed. This user interface is intuitive, as it is ...
Egocentric space-distorting visualizations for rapid environment exploration in mobile mixed reality
VR '10: Proceedings of the 2010 IEEE Virtual Reality ConferenceMost of today's mobile internet devices contain facilities to display maps of the user's surroundings with points of interest embedded into the map. Other researchers have already explored complementary, egocentric visualizations of these points of ...
Can Haptic Feedback on One Virtual Object Increase the Presence of Another Virtual Object?
VRST '22: Proceedings of the 28th ACM Symposium on Virtual Reality Software and TechnologyThis paper investigated whether increased presence from experiencing haptic feedback on one virtual object can transfer to another virtual object. Two similar studies were run in different environments: an immersive virtual environment and a mixed ...
Comments